Using S3 as a cache store

25Nov06

I’ve written a plugin that uses s3 as a cache store for you rails app. Basically, it allows you to cache content at s3 instead of your own server – continuing the theme of using s3 as an asset server.

The Amazon Web Services blog links to a site using S3 to serve up their static content. The encouraging thing was that informal tests showed no noticable differences between serving it up locally and S3. Very nice.

I noticed the same thing using my soon to be released S3Cache plugin. As I mentioned in a comment to the story that I’m looking for a few good testers to test out this plugin. S3Cache creates a new fragment cache filestore using Amazon’s Simple Storage Service (S3). Instead of your fragment caches stored at say “#{RAILS_ROOT}/cache”, your cache can now be located at one of your S3 buckets.

S3Cache also extends ActionView Helpers by adding a new cache helper called “cache_xml”. This caches xml and rss files at S3. Your rss feed can then be accessed at something like http://steveodom.s3.amazonaws.com/feeds/my_feed.rss.

My main goal is writing this plugin was to offload my xml and rss feeds to S3. This way my server is never touched when these files are accessed. But I found that I could extend it too to the cached elements of my site and page performance not be effected. So in effect most of my application can be served from S3, yet it is still database driven and dynamic.

So if anyone is interested in testing this plugin, before I release it into the wild, let me know and I’ll email them a copy. Preferably you’ll be already using fragment caching on your site.

About these ads


12 Responses to “Using S3 as a cache store”

  1. 1 Adam Groves

    Ha! I was thinking the exact same thing about using S3 as an alternative to memcache. I’m interested!

  2. This is extremely interesting! Mind sending me a copy?

  3. I’ve done exactly the same thing on http://www.mediauk.com – all the static files (graphics, CSS, audio, etc) are on S3 now. Much less hassle for my server, which can now concentrate on serving the dynamic pages and running a database. And I’ve saved a bundle on bandwidth costs, too.

    (I use Feedburner for things like RSS feeds; which is just as sensible a route to take, I guess)

  4. 4 taco

    I’ve been playing with serving all of my assets on S3, but run into the issue of SSL.
    I’ve gone the virtual-hosting route with a DNS CNAME pointing to my bucket, but this totally doesn’t work with SSL.

    Any feedback on / if you’ve run into any SSL solutions / issues would be appreciated.

  5. great input men

  6. 6 Mark Bobsin

    Hi everyone,
    I am looking in creating a video content website using AWS. I read that with S3 you can incur overhead for serving many smaller files (i.e. static website images, text, etc). and that would also have more GET requests which you have to pay for…mind you only $.01 for 10,000 requests.

    What if you grabbed a whole set of files…zipped or otherwise packaged together to be stored, saved/retained and fetched from S3 as one file. Then on EC2 instance uncompress/unpackage and save on local server storage on in cache.

    Does this make sense? I am still in the planning phases of the “cloud” architecture…sure adds a twist on things. This maybe only be applicable if ever ramping you to very large thoroughput with having many users and associated file requests served up.

    Mark Bobsin
    Markham Ontario Canada

  7. I tried doing the same idea, connecting S3 instance with Process Scheduler. Initially it wasn’t working so I decided to clear cachey. To make it optimal, i installed APP SERVER. As a backup, I have Flash copy running.

  8. connecting S3 instance with Process Scheduler. Initially it wasn’t working so I decided to clear cachey. To make it optimal, i installed APP SERVER. As a backup,

  9. I don’t know if it’s just me or if everybody else experiencing problems with your website.
    It appears as if some of the written text on your content are running off
    the screen. Can somebody else please provide feedback and let me
    know if this is happening to them too? This could be a problem with my browser because I’ve had
    this happen previously. Many thanks

  10. Asking questions are actually good thing if you are not understanding something entirely, however this post offers pleasant
    understanding even.

  11. I see a lot of interesting content here, i know writing articles is time
    consuming, but i know unlimited source of content for your website , just
    type in google – rewriter creates an unique article in a
    minute


  1. 1 links, ideas and geek stuff » Blog Archive » links for 2007-06-13

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: