A Case for EC2

13Jun07

When iLike’s Facebook application took off here is how they handled it:

“In our first 20 hours of opening doors we had 50,000 users sign up, and it is only accelerating. (10,000 users joined in the first 12 hrs. 10,000 more users in the next 3 hrs. 30,000 more users in the next 5 hrs!!)”

“We started the system not knowing what to expect, with only 2 servers, but ready with backup. Facebook’s rabid userbase chewed up our 2 servers almost instantly. We doubled our capacity to catch up. And then we doubled it again. And again. And again. Oh crap – we ran out of servers!! Although iLike.com has a very healthy level of Web traffic, and even though about half of all the servers in our datacenter were sitting unused, idle, as backup capacity, we are now completely maxed out.”

“We just emailed everybody we know across over a dozen Bay Area startups, corporations, and venture firms in a desperate plea to find spare servers so we can triple our capacity for the continued onslaught. Tomorrow we are picking up over 100 servers from different companies to have them installed just to handle the weekend’s traffic. (For those who responded to our late night pleas, thank you!)”

Seems like a good case for EC2.



10 Responses to “A Case for EC2”

  1. 1 Mike

    Would you happen to know what tools/framework they are using? (RoR?? )

  2. Good question. They are running Rails. I poked around a bit and found this in the very first blog post for iLike:

    So our company is a little bit odd for a startup, and for me (natbro) it’s odd in all ways good. On the one hand, we’ve opened offices in San Francisco and Seattle, are building a brand new site & service at iLike.com targeted at consumers, and it’s all shiny new whizzy web-2.0-running-on-linux-apache-mysql-and-ruby-on-rails goodness.

    Even better, I found this post from iLike’s blog entitled: iLike Scaling Rails:

    http://ilike.typepad.com/engineering/2007/02/ilike_scaling_r.html

    Our basic web-nodes are currently an Apache 2.2.3 mod_proxy_balancer (simple full-reverse-proxy which uses round-robin scheduling) directing traffic to different sub-sets of a 35-process mongrel cluster which runs against a single rails tree. The Apache configuration is simple and categorizes traffic based on URL into mongrel sub-sets of the cluster, which keeps certain classes of traffic from bottle-necking others. The Apache server also delivers static content without hitting the mongrels. (I’ll update this with some httpd.conf examples if anybody is interested)

  3. 3 Ed

    Interestingly, in an email they sent round desperately asking to borrow servers, they mention that they HAD offloaded as much off to EC2 as possible, and using any more was ‘not an option’.

    http://venturebeat.com/wp-content/uploads/2007/05/ali-letter.jpg (via: http://venturebeat.com/2007/05/26/facebook-users-vote-for-ilike-but-what-happened-to-audio/)

    Any ideas why that might be? On the EC2 site it seems you’re limited to 20 instances by default, but you can apply to get that limit raised.

    Do you think they would have been able to scale more with EC2 if they had all their architecture on there to begin with? Are there any large scale Rails sites running solely on EC2 which you know of?

    Sorry for all the questions, it’s just that I’m evaluating whether to use EC2 at the moment (just in case something like iLike happens with us), so am interested to hear more on how well Rails can scale on EC2.

  4. 4 josh

    I thought EC2 couldn’t really do databases since they aren’t persistent?

  5. Ed, I was not aware of the 20 instance limit. Thanks for highlighting that. It would have been really interesting to see what would have happened had they been using EC2 and didn’t have the 20 instance limit. This would have been a great illustration. I don’t really know of any large scale Rails site running on EC2 currently.

    Josh – The virtual ascpect of EC2 does not preclude you from running databases. On the contrary, the virtual aspects of EC2 forces you to think about your backup strategies more so than using physical hardware. Physical hardware does fail, but we (I) tend not to worry about it as much for some reason. I have not had an EC2 instance crash yet, though its too early to saying whether a virtual instance will fail more than a physical instance. But at least I plan for it more when using a virtual EC2 instance. In that way, I feel more confident running an EC2 instance for my databases.

  6. 6 josh

    Interesting, I’ll have to give this some more thought as I was almost considering Amazon’s web services for a project I’m working on. My judgment about EC2 not running databases was based on a really interesting IT Conversations podcast I heard awhile back:

    http://www.itconversations.com/shows/detail1728.html

    They discuss a lot of the technical aspects of developing a large, distributed web app using many of Amazon’s web services.

  7. Josh, thanks for the link. I’d be curious to know what you decide to do.

  8. Steve, just to be clear, so how exactly do you work with databases con EC2? Every time you (re)boot an instance you have to manually get a backup from S3 and reload the database? What about master-slave scenarios?

  9. Rodrigo, my understanding is that you do not lose your data when you reboot and instance. I actually have not had to reboot an instance yet, but that is my understanding. In any case, automated backup and restore scripts make it easy to get the data back.

  10. What i do not understood is in fact how you are no longer actually a lot more well-preferred than you may
    be now. You are so intelligent. You recognize therefore significantly with regards to this matter,
    produced me for my part believe it from a lot
    of varied angles. Its like men and women aren’t involved except it is one thing to do with Girl gaga! Your personal stuffs excellent. All the time care for it up!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: