Using S3 as a cache store


I’ve written a plugin that uses s3 as a cache store for you rails app. Basically, it allows you to cache content at s3 instead of your own server – continuing the theme of using s3 as an asset server.

The Amazon Web Services blog links to a site using S3 to serve up their static content. The encouraging thing was that informal tests showed no noticable differences between serving it up locally and S3. Very nice.

I noticed the same thing using my soon to be released S3Cache plugin. As I mentioned in a comment to the story that I’m looking for a few good testers to test out this plugin. S3Cache creates a new fragment cache filestore using Amazon’s Simple Storage Service (S3). Instead of your fragment caches stored at say “#{RAILS_ROOT}/cache”, your cache can now be located at one of your S3 buckets.

S3Cache also extends ActionView Helpers by adding a new cache helper called “cache_xml”. This caches xml and rss files at S3. Your rss feed can then be accessed at something like

My main goal is writing this plugin was to offload my xml and rss feeds to S3. This way my server is never touched when these files are accessed. But I found that I could extend it too to the cached elements of my site and page performance not be effected. So in effect most of my application can be served from S3, yet it is still database driven and dynamic.

So if anyone is interested in testing this plugin, before I release it into the wild, let me know and I’ll email them a copy. Preferably you’ll be already using fragment caching on your site.


10 Responses to “Using S3 as a cache store”

  1. 1 Adam Groves

    Ha! I was thinking the exact same thing about using S3 as an alternative to memcache. I’m interested!

  2. This is extremely interesting! Mind sending me a copy?

  3. I’ve done exactly the same thing on – all the static files (graphics, CSS, audio, etc) are on S3 now. Much less hassle for my server, which can now concentrate on serving the dynamic pages and running a database. And I’ve saved a bundle on bandwidth costs, too.

    (I use Feedburner for things like RSS feeds; which is just as sensible a route to take, I guess)

  4. 4 taco

    I’ve been playing with serving all of my assets on S3, but run into the issue of SSL.
    I’ve gone the virtual-hosting route with a DNS CNAME pointing to my bucket, but this totally doesn’t work with SSL.

    Any feedback on / if you’ve run into any SSL solutions / issues would be appreciated.

  5. great input men

  6. 6 Mark Bobsin

    Hi everyone,
    I am looking in creating a video content website using AWS. I read that with S3 you can incur overhead for serving many smaller files (i.e. static website images, text, etc). and that would also have more GET requests which you have to pay for…mind you only $.01 for 10,000 requests.

    What if you grabbed a whole set of files…zipped or otherwise packaged together to be stored, saved/retained and fetched from S3 as one file. Then on EC2 instance uncompress/unpackage and save on local server storage on in cache.

    Does this make sense? I am still in the planning phases of the “cloud” architecture…sure adds a twist on things. This maybe only be applicable if ever ramping you to very large thoroughput with having many users and associated file requests served up.

    Mark Bobsin
    Markham Ontario Canada

  7. I tried doing the same idea, connecting S3 instance with Process Scheduler. Initially it wasn’t working so I decided to clear cachey. To make it optimal, i installed APP SERVER. As a backup, I have Flash copy running.

  8. connecting S3 instance with Process Scheduler. Initially it wasn’t working so I decided to clear cachey. To make it optimal, i installed APP SERVER. As a backup,

  9. Amazing post. I read your posts fairly often and you always do a good job articulating the
    whatever topic you’re blogging about. Btw, I shared this on Twitter and my followers loved it.
    Keep up the great work!

  1. 1 links, ideas and geek stuff » Blog Archive » links for 2007-06-13

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: