As you may have noticed, Twitter has had some reliability issues over the past few months. Part of this was related to the World Cup, part of it is because they just continue to grow at a fast pace — 300,000 new accounts are created a day now. It has gotten to the point where Twitter needs their own warehouse for tweet storage. So they’re building one, in Salt Lake City.
While it undoubtedly won’t be as large as Apple’s forthcoming billion-dollar data center in North Carolina, Twitter says they have been working on a “custom-built” one that will be opening later this year.
These troubles have been difficult for the many third party developers and service providers who are dependent on the Twitter ecosystem for their own survival as well. Right now, no one is really very happy with Twitter’s performance and the excuses of event overload or anything else will likely have less credibility moving forward if Twitter truly wants to be counted amongst the Googles and Facebooks of the world. Of course, Google’s network of data centers is well known and Facebook announced earlier this year that they were going the private data center route as well. Having said that even now, I don’t ever experience issues with Facebook’s availability and rarely if ever with Google (unless you count their sometimes dog slow e-mail service).
The Twitter engineering blog tries to give a picture of how this will help Twitter and everyone associated with it moving forward.
First, Twitter’s user base has continued to grow steadily in 2010, with over 300,000 people a day signing up for new accounts on an average day. Keeping pace with these users and their Twitter activity presents some unique and complex engineering challenges (as John Adams, our lead engineer for application services, noted in a speech last month at the O’Reilly Velocity conference). Having dedicated data centers will give us more capacity to accommodate this growth in users and activity on Twitter.
Second, Twitter will have full control over network and systems configuration, with a much larger footprint in a building designed specifically around our unique power and cooling needs. Twitter will be able to define and manage to a finer grained SLA on the service as we are managing and monitoring at all layers. The data center will house a mixed-vendor environment for servers running open source OS and applications.
So hang in there folks, Twitter is trying hard. Unfortunately, while I believe that they are doing the best they can it will not be enough for a our world of “What have you done for me lately?” Right now, people aren’t very happy with Twitter and as they say in baseball “You’re only as good as your last at bat”. If Twitter keeps striking out that will not help the cause one bit.
Here’s to less Fail Whale appearances and some semblance of stability for Twitter in the future.