Inaugural: Twitter streaming with node.js and HTML5 WebSockets.

The idea of having a full javascript stack with code reuse and the removal of language impedance between client and server has always interested me.

It seemed that a full javascript stack was an utopia, waiting in the distant future, but not anymore. Now there are a few powerful javascript solutions for the server and with CouchDB I can aim for a full javascript stack.

One of those solutions is node.js. It’s an high performance non-blocking event driven framework. It is brilliantly designed, it feels right, and the performance is impressive to say the least. I’ve not been so excited with a new technology since Rails appeared on my radar a few years ago.

If you’re unfamiliar with it, go watch Ryan Dahl presentation at JSConf.

WebSockets and Twitter Real Time Streaming

Yesterday I stumbled upon Ruben Fonseca excellent blog post on a Ruby Twitter Streaming example using AMQP, EventMachine and HTML5 WebSockets (basically it streams real time tweets to the browser without the use of polling).
Ruben’s experiment was inspired by Ilya Grigorik post on WebSockets and Ruby.

I decided to follow on Ruben’s post and do the exact same thing with node.js, only IMO much simpler. Ruben’s architecture was based on the following:

Twitter Stream → Filter → RabbitMQ → AMQP → Eventmachine → WebSocket → Browser

The node.js version:

Twitter Stream → node.js → WebSocket → HTML5 Browser

Let’s look at the code:

The Server

Notice how everything looks familiar, and at first glance it seems we’re writing typical client side javascript.
I find it to be very elegant and simple solution, and with a lot less dependencies. The javascript style callbacks and closures are well suited for an evented non-blocking framework like node.js.

I’m using Jacek Becela very cool minimal web sockets library that creates the websocket server with a similar interface to the native tcp.createServer.

The Client

Here for comparison sake (and because I’m lazy) I copied Ruben’s client side script.

You can check out the project and installation instructions on the github repo.

How to run it yourself

• Download and install node.js (instructions here)
• Download the project from github.
• Start the server: node server.js <twitter_username> <twitter_password>
• Open index.html with a WebSocket compatible browser (Chrome or Webkit nightly)

Thanks to Lourens Naudé (@methodmissing) and Pedro Teixeira (@pedrogteixeira) for reading drafts of this.