MAIN

Jens Himmelreich reported a problem with my weak hash table implementations:

You use a hash, but you only save the object_id in the WeakHash. Every time you need the hash, you get it back from the heap by _id2ref. You use a finalizer to be acknowledged, if the hash is garbage-collected.

Sometimes this concept works, sometimes I get a RangeError. I log the finalization and my theory - without reading reference-material about ruby-garbage-collection - is this: The hash is cleared on the heap, but the finalizer is not immediately called. In this gap it's possible to get a RangeError even if you are using intranet software.

I took another look at gc.c, and Jens is right on.

You can find corrected versions of the SimpleWeakHash and WeakHash classes (two weak hash tables with slightly different semantics) below, but I'll first expand on why GC and object finalization can be fairly distant in time.

The GC

Ruby uses a very simple mark&sweep GC, which as its name implies works by first marking all live*1 objects, and then reclaiming all those that remained unmarked.

gc.c is actually one of the easiest core components; it's easier to follow than eval.c, since it doesn't require remembering lots of things (node types and undocumented conventions regarding the way they nest) and it's leaps and bounds simpler than parse.y. So it only took me a few minutes to verify Jens' intuition.

Finalizers

How do finalizers fit in the picture? As the rest of gc.c, this is also very simple. The interpreter just adds the objects being swept for which a finalizer has been defined to a list of... well, objects whose finalizers need to be called. (If you want to see where this happens, read gc_sweep and look for uses of the deferred_final_list and final_list variables). At some point, later in time, Ruby follows that list and executes the finalizers one after the other.

This explains how finalizers are executed, but not when. The code could hardly be more expressive:

simplefold is a small vim script that improves (for some definition of "improve", see the description + screenshots below) on other folding methods (syntax, marker, indent, expr). It was originally written for Ruby, but now also supports Objective Caml, PHP, Perl and Java. It features:

• optimized vertical space usage
• sensible foldtext
• top-level folds: one per interesting definition. No need to open a class fold to see which methods it contains. Get a quick overview of the classes/methods/functions (whatever applies in the current filetype) with zM.
• optional nested folds for if/while and so on
• easy to adapt to other filetypes; just by setting 2-3 regexps

simplefold 0.5.0 introduces Objective Caml, Perl and PHP support, a few bugfixes and a new way to define fold boundaries.

Getting it

The last tarball is simplefold-0.5.0.tar.gz.

You can also access the darcs repository (you can explore it from your browser too) at http://eigenclass.org/repos/simplefold .

simplefold vs. fold_syntax

Both images correspond to the same files; I've timed them carefully to show equivalent views (top-level, class-level, method-level) at a time (you might have to reload the page to make sure they play synchronously). The first one is fdm=syntax, the second is simplefold's

evil.rb is an experimental library that makes it possible to manipulate internal structures from the Ruby interpreter*1.

If you're interested in the internals of the interpreter, evil.rb can prove an excellent educational tool, since it allows you to peek into the object model, method dispatching...

In the past, some people have been using evil.rb to *2

• change the class of an object
• manipulate the inheritance chain for fun and profit
• grab instance variables or singleton methods
• swap objects (ever heard of Object#become?)
• change the self context of a Proc (the way 1.9's instance_exec does)
• mess with the flags of an object (frozen, tainted...)

Here's a list of articles where evil.rb is used to extend Ruby or illustrate how the interpreter works (please drop a comment if you have additional pointers of interest):

There's seemingly no end to the drivel coming from these Neo-Rails sects, or is it people pretending to be Java supporters posing as Railers?

Summing up, lgenerator.rb.

Here's a simple example (see the PostRailsMonkey Manifesto for a complex one):

require 'lgenerator'
gen = Generator.new do |g|
  # default non-terminal is main, can be set with    start :whatever
  main.is :hello, "\n", :ex1, "\n", :ex2
  hello.is "Hello, ", :somebody, "!"
  somebody.is "matz" | "world"     # basic disjunction

  ex1.is "This is a more ", ("complex " | "elaborate "), :example, "."
  example.is "example" | "test"

  ex2.define do
    is "Some simple sentence."
    is "Another, involving ", ["a more complex ", :exp] | "harder stuff ", "."
    is "Yet another possibility; each one is chosen with prob. 1/3."
  end
 
  exp.is "expression" | "disjunction" | :example
end

puts gen.generate

#>> Hello, world!
#>> This is a more elaborate test.
#>> Yet another possibility; each one is chosen with prob. 1/3.

The language generator can handle recursive derivations and perform actions when a rule is followed. You can for instance make it remember the choice for a particular derivation and use it in another:

Take a look at the Kansai Rails meeting slides. It lists what had to be modified in order to get Rails to run on 1.9. Even if you don't understand the Japanese text, the code samples show how the new stuff in 1.9 affects real-life applications.

News from moriq again. He's on the way to benchmarking real Rails apps under 1.9.

Preliminary results

When benchmarking basic requests with a minimalistic schema (only one int column), even though YARV serviced requests 15% faster, it took much longer to load the environment (50% slower). This comes as no surprise since plain old (evil) eval is slower in YARV.

Unexpected results did come up when benchmarking direct DB requests against sqlite3, both directly and through ActiveRecord:

# sqlite3-ruby select (n=1000)
ruby-1.8.6: 2142 req/sec
ruby-1.8.6: 2153 req/sec
ruby-1.9.0: 2313 req/sec
ruby-1.9.0: 2328 req/sec
  
# activerecord -> sqlite3-ruby select (n=1000)
ruby-1.8.6: 1098 req/sec
ruby-1.8.6: 1103 req/sec
ruby-1.9.0:  801 req/sec
ruby-1.9.0:  806 req/sec

In fact, the mere fact of requiring active_record (and indirectly ActiveSupport) causes a sharp decrease in performance:

(bm3-as-* is the basic sqlite3 benchmark with "require active_record")

Culprits