Working with S3

S3 Documentation

http://aws.amazon.com/documentation/s3/

http://aws.amazon.com/s3/getting-started/

http://aws.amazon.com/s3/developer-resources/

Amazon S3 Path Deprecation Plan

See https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

Publish

To publish to S3 there are several ways:

You can do this from the AWS console. Login as your userid to the S3 console at https://console.aws.amazon.com/s3.

Follow this guide at http://docs.aws.amazon.com/AmazonS3/latest/gsg/GetStartedWithS3.html

Hostng images on S3 - http://www.labnol.org/internet/host-images-files-on-amazon-s3-storage/4923/

Third party tools for accessing your S3 bucket

Besides using the AWS console, AWS Command line tools (AWS CLI) or API's for accessing your S3 bucket there are other more GUI like tools for this.

• http://www.bucketexplorer.com/ (Multi-platform)

• http://s3browser.com/ (Windows)

• http://www.crossftp.com/ (Cross platform)

• http://www.cloudberrylab.com/ (Windows only)

• http://www.panic.com/transmit/ (Mac only)

• http://www.s3fox.net/ (Firefox plug-in) {Note as of 2017-06-05 this does not seem to work any longer]

Import/Export

There are also some import/export features available: http://aws.amazon.com/importexport/tools/

Command line tools

• S3cmd - http://s3tools.org/download

• AWS CLI S3 command - http://docs.aws.amazon.com/cli/latest/reference/s3/index.html and to install AWS CLI visit http://docs.aws.amazon.com/cli/latest/userguide/installing.html

$ aws s3 ls s3://mybucket

• https://github.com/twpayne/s3-parallel-put

Syncing

Notes about syncing data to S3 storage

Hosting your domain on S3

Please read: http://stackoverflow.com/questions/20264928/s3-static-website-index-document

IMPORTANT: This assumes the entirety of web site example.com will be hosted by S3.

http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html

Also look at http://stackoverflow.com/questions/5043616/amazon-s3-static-web-sites-custom-domain-or-subdomain

1. Now create a S3 bucket with a name of example.com. It must be exactly that. It must match your intended fully qualified domain named (FQDN).

2. Register DNS it does not need to be managed by Route 53. In this case we will assume example.com.

3. IMPORTANT! Point DNS name example.com to S3 endpoint as specified in the bucket (look in Properties in the S3 Console) you created above in Step #1 as a DNS CNAME. Do not use s3.amazonaws.com this will not fully work.

4. Under the S3 Console for your newly created bucket under the section called "Static Website Hosting" check "Enable website hosting" and set an Index Document. Usually it's called index.html but it can be anything.

5. Create a policy to allow anonymous users (the internet) to access the site. By default only the GetObject action in the policy will be used which means each object must be referenced by the index document or some other page referenced originally by the index document. In other words S3 does not support directory lstings or indexes like Apache web server does. For more on S3 Access Control have a look here: http://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html

A sample anonymous policy:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "AddPerm",
			"Effect": "Allow",
			"Principal": "*",
			"Action": [
				"s3:GetObject"
			],
			"Resource": [
				"arn:aws:s3:::example.com/*"
			]
		}
	]
}

6. Create a Group for publishers (say example.com). Attach a managed policy in that group so that members of this group can publish to the bucket. After creating the group go to Permissions->Managed Policies -> Inline Policies with the name like "example.com-group-policy".

A sample policy for the IAM Group example.com:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::example.com/*"
        }
    ]
}

7. Now using your favorite tool upload content to the S3 bucket and dont forget to update your index document.

S3 as a file-system

https://github.com/danilop/yas3fs