Sign in
in
   
"It is the mark of an educated mind to be able to entertain a thought without accepting it."  -Aristotle

About Me

I am a co-founder of Notches, an early stage startup currently based in NYC. We are building a free, open reviews network that anyone can participate in and anyone can build on top of. You can find out more on our official blog.

Read more about my background.

Connect with me on...

Recent Readers

Flickr Photos

 

Browse by Tags

All Tags » Twitter » Programming » Software Development (RSS)
  • The Challenges of Scaling Twitter's Follower Model

    I wrote previously that Twitter's architecture is not scalable for real-time messaging . Indeed, the Twitter development team talked about this recently . Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency's sake, Twitter was built with technologies and practices that are more appropriate to a content management system. Over the last year and a half we've tried to make our system behave like a messaging system as much as possible, but that's introduced a great deal of complexity and unpredictability. Dare Obasanjo recently looked at the issues and implications and suggested that it's not the technical architecture to blame, but perhaps rather the logical model. If Twitter was simply a micro-content publishing tool with push notifications for SMS and IM then the team wouldn't be faulted for designing it as a Content Management System (CMS). In that case you'd just need three data structures...
  • Twitter's problems are the result of architecture, XMPP may be the answer

    The problem with scaling Twitter is not the choice of framework, but the choice of architecture. In other words, abandoning Ruby On Rails probably isn't going to solve all of their problems. At the same time, I'm not sure that decentralization is necessary and comes with its own set of challenges. The real problem is that the polling model of the Twitter API doesn't scale for real-time communications. As it is today, many Twitter clients will poll (by default) every 3-5 minutes to see if there was an update. Not only are they not really participating in real-time, they are generating an enormous number of requests that - even while each payload is small - generate a lot of overhead in aggregate just in checking and responding. As I've suggested in the past , a better solution would be to move the "real-time" API around the Jabber/XMPP client instead of HTTP. The good news is that Twitter already has an IM presence so it's easy enough for third-party clients...