Moving to the Cloud: Amazon S3

Amazon Web Services

Amazon S3 is Amazon's cloud storage solution.  It's is accessible over the Internet through a web browser via Amazon's Management Console or via a web page (if you enable it) or through some of the S3 clients out there.  There is also an command line API for Amazon S3 that can be used on your local computer or it can be installed on an Amazon EC2 instance for easy transfer to and from S3.  And the best part is, it's pretty inexpensive for 'unlimited' storage.  This post is all about S3: what it does and how you can use it.
Once you log into Amazon Web Services Management Console, you default to the S3 tab.  To start using S3, you have to create an S3 bucket; it's like a domain name only for S3.  The name of your S3 bucket must be unique, so pick a name for your bucket that is not likely already taken.

After you have created an S3 bucket, you can then create folders, upload files, etc.  S3 is priced pretty reasonably for storage and data transfer.  But you have to be aware that there are still costs.  For a complete breakdown of the costs associated with S3 storage, see the following link.   http://aws.amazon.com/s3/#pricing

One very cool feature of S3 is that you can expose an S3 bucket as a website.  This feature can come in very handy if you want cheap, no frills web hosting or if you want to share files in an S3 bucket with a lot of people.  To convert an S3 bucket into a website, first select the bucket you want to expose as a website and click the Properties button.  Next, select the Website tab and check the enabled box.  The URL for accessing your S3 storage bucket on the web will be listed as the Endpoint.

There is also an S3 toolkit for command line operations.  It can be extremely handy when you want to transfer files to and from S3 and EC2 instances or if you are more comfortable with the command line.  The name of the command line toolkit is s3cmd and it is available at http://s3tools.org/s3cmd. The link provides a ton of information on how to use s3cmd, so I won't go into that in this post.  S3tools.org also has a tool called s3fuse which uses fuse to allow you to mount an S3 bucket as a filesystem in Linux - pretty cool stuff!

Finally, there are a few S3 browser applications.  One of the more popular S3 browsers is CloudBerry, available at http://cloudberrylab.com/.  On my Mac, I am actually partial to Transmit http://www.panic.com/transmit/.  Transmit has a nice simple interface and when I am not using the command line, it is typically my S3/FTP/EC2 file transfer client of choice.