Will Facebook Need to Start Its Network from Scratch?



It appears Facebook is built on a house of cards.

Cards that rely on more than 4,000 MySQL “shards” or should that be sharts?

According to GigaOm:

…Facebook has split its MySQL database into 4,000 shards in order to handle the site’s massive data volume, and is running 9,000 instances of memcached in order to keep up with the number of transactions the database must serve.

And, citing database guru Michael Stonebraker…

…Facebook is operating a huge, complex MySQL implementation equivalent to “a fate worse than death,” and the only way out is “bite the bullet and rewrite everything.”

That may sound like an exaggeration, but having lived through this nightmare myself, he may have a point.

Trackur started off using MySQL and that worked absolutely wonderfully until we got to about 20,000 users and millions of entries. At that point, things started to groan, break, or create mysterious bugs out of thin air. In the end, we had to bite the bullet and move to a NoSQL platform that was better suited to the vast amount of data we have to index, search, and serve.

Those headaches were real. And we were on a much, much, MUCH smaller scale than Facebook. I can only imagine that the MySQL team at Facebook look something like this on any given day:

  • http://www.seo-theory.com/ Michael Martinez

    I believe Google has the technology they are looking for….

    • http://www.frankthinking.com Frank Reed

      If Facebook ever sharts that’ll be the biggest “Clean up on Aisle 7!” in the history of the Internet.

      And, man, it would be fun to watch.

  • Ferg

    I wouldn’t worry about it since Google+ will be picking up the slack if not all of the clientele.

    • Alex

      LOL are you kidding? Google + is obnoxious as f*ck, I got annoyed with it after 5 minutes

      • TC

        FB annoyed me after only 3 minutes and I still use it.

  • http://www.orbitalalliance.com Simon Yohe

    I’m not an expert, but it seems that this is being pushed for his own means, to promote his database… if you read the comments on the GigaOm report, it seems like this is completely fabricated.

  • Dee Asher

    Not to be silly, but it’s working with over 500,000,000 users right? I know that seems simple, but it’s also simple to just be cynical and critical.

    • almosnow

      Guys really, much of the best NoSQL apps were created inside Facebook.

      I’ve seen a lot of amazing software coming from them. They hire really young aspiring engineers. I’ve tested their API’s myself, they have like a 20ms response time on a “complex” query, data replication happens almost instantly.

      I really, SERIOUSLY doubt that Facebook is running on MySQL.

  • Kerry Veenstra

    During a technical presentation by a Facebook employee about 6 months ago, we heard that they are using MySQL with InnoDB tables, which is consistent with wanting transactions.

    Stonebraker observed that going to NoSQL requires adding consistency functions to the business logic, and so performance improvements of eliminating transaction processing from the database engine limited.