Ruby Weekly is a weekly newsletter covering the latest Ruby and Rails news.

RabbitMQ – A Fast, Reliable Queuing Option for Rubyists

By Peter Cooper / April 9, 2009

rabbitmq.png When it comes to developing large systems with many interdependent parts, it’s common nowadays to use “queues.” A queue is, for the most part, just a list that you can add items to and remove items from. Apps can use queues to despatch jobs / tasks to other apps or to shuttle logs and status information around.

RabbitMQ is a fast, reliable, open source queueing option. It's not developed in Ruby, though, but in Erlang, a functional language with a reputation for distributed, high availability, fault tolerant apps. It presents itself as a separate daemon - much as Apache, Passenger, memcached, or MySQL aren’t written in Ruby. Using it from Ruby isn’t difficult, and, of course, you can use it from a multitude of languages so if you want to push in items from Ruby and pull them out to be processed with something else (or vice versa) it's ideal. RabbitMQ can be used as a hub for communication between all of your processes.

On RabbitMQ, Rany Keddo, the developer of Workling, said:

If you’re doing high volume messaging and need a high degree of reliability, you should definitely consider using RabbitMQ over Starling.

(And since I started to draft this article, this point has perhaps been proven even more with Twitter's ditching of Starling for a new, Scala-based system.)

So RabbitMQ has the power and reputation and if you want to do some serious queueing, you should be checking it out. With that in mind, Alexis Richardson (from the RabbitMQ team) and I have put together a few resources that will be useful to you (as a Rubyist) if/when you decide to check RabbitMQ out:

Official RabbitMQ Homepage - Homepage with general feature information, links to documentation, downloads and examples.

Installing RabbitMQ on OS X - A guide to installing RabbitMQ on Mac OS X by Rany Keddo. With the installation of Erlang, it's not quite the quickest process, but it's worth the wait.

AMQP "driver" for Ruby/EventMachine - A Ruby library by Aman Gupta that provides an event driven AMQP implementation for Rubyists, built around EventMachine. Works on Ruby 1.8, 1.9, and JRuby and tested directly against RabbitMQ.

Riding the Rabbit - A blog post by Dan Sinclair that demonstrates the setup of a RabbitMQ instance, and the use of the aforementioned AMQP Ruby/EventMachine library.

OH HAI RabbitMQ! Screencast - A great screencast by British Rubyist Jonathan Conway that demonstrates how to set up RabbitMQ and perform some basic operations.

Warren - A wrapper for AMQP - A library for pushing messages on to RabbitMQ queues (and receiving them at the other end) by the Brightbox crew. Authentication and filtering features can be added through custom classes.

RabbitMQ and Ruby presentation - Ben Hood (one of RabbitMQ's developers) gives a 40 minute presentation on RabbitMQ and Ruby. He looks at the background of RabbitMQ, the motivations behind using something like RabbitMQ, the approaches that have been taken in the past, AMQP, and, yes, a lot more.

Comments

  1. Ericson Smith says:

    There are so many options available now that are simpler and just as effective:

    * MemcacheQ
    * Redis

    Granted RabbitMQ has some more features that make it compelling, but as our apps become more complex, I found that more and more we need to use queues for a variety of things. The above products allow us to quickly setup a persistent queue that services a number of clients efficiently and in an atomic way.

  2. Jake mallory says:

    It's great to see messaging adopted in the ruby/rails community. Some co-workers have created a great queue API called rosetta-queue that allows you to swap out the message broker using adapters, of which rabbitmq is available. Http://rubyforge.org/projects/rosetta-queue

  3. Hubert Łępicki says:

    That looks fantastic. Again, if I knew about it 2 months ago, it would have saved the day...

  4. Ed Borasky says:

    Funny you should mention message queues. One of the Portland, Oregon bright lights in the Ruby community, Igal Koshevoy, did some research on message queue technologies and presented his results at this past Tuesday's PDX Ruby Brigade meeting. It turns out there are lots more than just RabbitMQ and Starling.

    Here's the link:

    http://github.com/igal/rubyqueues/tree/master

  5. Mel Gray says:

    Lizzy is a great library for interacting with RabbitMQ / AMQP.

    Its built on top of the AMQP gem and is MUCH easier to get started with than out and out AMQP.

    http://github.com/bmizerany/lizzy/tree/master

  6. BJ Clark says:

    We use AMQP at AboutUs.org to send 50 million+ messages a month. Very very happy with it's performance.

  7. Matt Brubeck says:

    Ericson Smith, can you explain what you mean by "simpler?" RabbitMQ Server 1.5.4 is only about 6,000 lines of code, around the same size as either Redis or MemcacheQ.

  8. naniter says:

    looks like you forgot nanite [?]

  9. Pingback: Open Sourcery

Other Posts to Enjoy

Twitter Mentions