Ruby Weekly is a weekly newsletter covering the latest Ruby and Rails news.

Ruby Implementation Shootout: A Bright Future for Ruby Performance!

By Peter Cooper / December 9, 2008

ruby-shootout-dec.png

Back in December 2007, Antonio Cangiano presented The Great Ruby Shootout, a large set of benchmarks of then-current Ruby implementations (Ruby 1.8.5, YARV - early Ruby 1.9, JRuby, Ruby.NET, Rubinius, XRuby, and Cardinal). Even then, despite Ruby 1.9's infancy and experimental nature, Ruby 1.9 came out as about three times faster than Ruby 1.8.5 - while other implementations were barely faster than 1.8.5.

Now.. for 2008: Ruby 1.9.1 is the winner..!

Now Antonio has put together an all new December 2008 Great Ruby Shootout! There's a lot of reading to do but the results are very exciting. JRuby's performance has come along in leaps and bounds and Ruby 1.9.1 is around five times faster than Ruby 1.8.7. Phusion's Ruby Enterprise Edition (basically an amended fork of 1.8.6) also comes in at a surprising two-times faster than 1.8.7! Remember that Antonio's benchmarks are by no means representative of how your application might be using Ruby, but these numbers are extremely encouraging.

As well as comparing JRuby, Ruby 1.8.7, Ruby 1.9.1, Rubinius and REE, separate benchmarks were run for Maglev, MacRuby and IronRuby - all still reasonably experimental implementations. Maglev worked out at about twice the speed of 1.8 (though some operations were far slower), MacRuby offered performance equivalent to 1.8 (at this point), and IronRuby was a little slower due to its focus on .NET integration and matching the Ruby spec more closely (versus working on performance and optimizations). Make sure to head over to the article for the full results.

Antonio sums it up pretty well:

Overall I think these are great results. Ruby 1.8 (MRI), with its slowness and memory leaks, belongs to the past. It’s time for the community to move forward and on to something better and faster - and we don’t lack interesting alternatives to do so at this stage.

Could Ruby 1.9.1 be about to beat Python, Perl or PHP?

Now all we need is for Ruby 1.9.1 to reach final release and to see how it stacks up on the Alioth Shootout. Currently Ruby 1.8 is at the very bottom and an earlyish build of Ruby 1.9 is 9th from bottom just below Python, Perl and PHP. If the performance of Ruby 1.9 can go from 28 times slower than compiled C++ to just 18 times slower, Ruby will leapfrog these three strong competitors and one of the biggest criticisms of Ruby will become extinct.

(As an aside - is anyone familiar enough with the Alioth Shootout to run the benchmarks for the latest Ruby 1.9.1 build to see how it compares so far?)

Comments

  1. Hongli Lai says:

    I was pretty surprised at the REE results. It has proven to be 25% faster on average at serving Rails apps so seeing that it scored 2x higher than MRI on this benchmark is surprising. But I guess this also means that one should always take benchmarks with a grain of salt. :)

    I'm really interested in a new Alioth shootout. In my experience, Ruby 1.9 is indeed *much* faster than 1.8. I'd also like to see Alioth's shootout being updated for the latest JRuby release. The JRuby guys have been consistently improving their performance for the past few releases.

  2. dmpk2k says:

    It's not surprising that a Rails app sees less of a performance improvement -- it does a lot of I/O too. The benchmarks used were CPU-bound while Rails isn't.

    What you see with these benchmarks is the upper bound on improvement you'll see in the "real world". In practice it'll be a lot lower.

  3. Radarek says:

    Antonio updated results (http://antoniocangiano.com/2008/12/10/reflections-on-the-ruby-shootout/). He did mistake installing it by apt-get which is known to be a lot slower that compiling ruby from source. REE is on an equal footing with 1.8.7.

  4. Daniel Berger says:

    I wonder if REE applied Darryl Gove's patch for Solaris, too:

    http://www.opensparc.net/blogs/2008-03/darryl-gove-ruby-performance-gains-on-sparc.html

  5. George McGrumble says:

    As fast, not faster. Saying something is, for example, five times faster doesn't make logical sense, in that if you carry it to its conclusion then something that is equally fast is also one times faster.

    And if not, then five times faster ought to parse to six times as fast, but obviously nobody uses it that way.

  6. Isaac Gouy says:

    > Ruby 1.9.1 is around five times faster than Ruby 1.8.7

    With Antonio's corrected measurements Ruby 1.9.1 is around 2.5 times faster than Ruby 1.8.7

    > As an aside - is anyone familiar enough with the Alioth Shootout to run the benchmarks for the latest Ruby 1.9.1 build to see how it compares so far?

    The six month old measurements show Ruby 1.9.0 is around 2.25 faster than Ruby 1.8.6

    http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=yarv

    A JRuby comparison is shown in the current benchmarks game -

    http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=jruby

Other Posts to Enjoy

Twitter Mentions