eigenclass logo
MAIN  Index  Search  Changes  PageRank  Login

Rails on 1.9: first benchmarks, YARV exposed to non-synthetic tests

update.png Take a look at the Kansai Rails meeting slides. It lists what had to be modified in order to get Rails to run on 1.9. Even if you don't understand the Japanese text, the code samples show how the new stuff in 1.9 affects real-life applications.

News from moriq again. He's on the way to benchmarking real Rails apps under 1.9.

Preliminary results

When benchmarking basic requests with a minimalistic schema (only one int column), even though YARV serviced requests 15% faster, it took much longer to load the environment (50% slower). This comes as no surprise since plain old (evil) eval is slower in YARV.

Unexpected results did come up when benchmarking direct DB requests against sqlite3, both directly and through ActiveRecord:

# sqlite3-ruby select (n=1000)
ruby-1.8.6: 2142 req/sec
ruby-1.8.6: 2153 req/sec
ruby-1.9.0: 2313 req/sec
ruby-1.9.0: 2328 req/sec
  
# activerecord -> sqlite3-ruby select (n=1000)
ruby-1.8.6: 1098 req/sec
ruby-1.8.6: 1103 req/sec
ruby-1.9.0:  801 req/sec
ruby-1.9.0:  806 req/sec

In fact, the mere fact of requiring active_record (and indirectly ActiveSupport) causes a sharp decrease in performance:

sqlite3-ruby-bench-1.jpg

(bm3-as-* is the basic sqlite3 benchmark with "require active_record")

Culprits

If you've ever read ActiveSupport, you'll have noticed that it often uses very expensive techniques, but having to pay that high a price even when you don't actually use it at all is surprising indeed.

This remarkable speed hit can be traced to a number of libs:

  • builder
  • core_ext
  • dependencies
  • reloadable
  • deprecation
  • multibyte

This seems an appropriate place to quote ko1:

Some people [are] using YARV's benchmarks I wrote. But I didn't write these codes to measure "Ruby's general benchmark test", but to measure speed-up ratio on YARV. It's means that I wrote codes what YARV optimizes. We must prepare more suitable benchmarks for "Ruby implementations".



Psyco - jacob (2007-04-29 (Sun) 21:37:47)

Is there a Psyco for Ruby?

mfp 2007-04-30 (Mon) 03:09:44

Nope; Ruby is way behind Python implementation-wise... Forget about just-in-time specialization for now, even the jump to a bytecode VM is taking longer than it should :-|



Last modified:2007/03/28 04:53:00
Keyword(s):[blog] [ruby] [frontpage] [rails] [benchmark] [yarv] [1.9]
References: