eigenclass logo
MAIN  Index  Search  Changes  PageRank  Login

FastRI 0.2.0: full-text searching, smarter search strategies

FastRI is an alternative to the ri command-line tool (used for browsing Ruby's stdlib and third-party API docs). It is much faster, and also allows you to offer RI lookup services over DRb. FastRI is smarter than ri, and can find classes anywhere in the hierarchy without specifying the "full path". FastRI can perform full-text searching. Its RubyGems support is better than ri's, and it knows which gem a method/class definition came from.

0.2.0 introduces full-text searching and method lookup in superclasses for core classes and modules. In case you're interested, I've described how FastRI's (custom) full-text search engine works here (the core takes a couple hundred lines of pure Ruby).

Getting it

FastRI can be downloaded here. You can also install it with RubyGems:

 gem install fastri

Examples

Intelligent searches

 $ fri File.inject
 ------------------------------------------------------ Enumerable#inject
      enum.inject(initial) {| memo, obj | block }  => obj
      enum.inject          {| memo, obj | block }  => obj
 ------------------------------------------------------------------------
      Combines the elements of enum by applying the block to an 
      accumulator value (memo) and each element in turn.
 ...

Full-text search

 $ fri -S HTTP request
 42 hits
 ========================================================================
 Found in system  Net::HTTP
   
 ... HTTP Request Classes
 ...       Net::HTTP::Delete
 ... jects belong to its own response class which indicate HTTP result
 status. For details of response classes, see section 'HTTP Response
 Classes'.
 [...]
 $ fri -S open file mode
 8 hits
 ========================================================================
 Found in system  IO
  
 ... , it should be one of the operating system specific constants
 (O_RDONLY, O_WRONLY, O_RDWR, O_APPEND and so on). See man open(2)
 for more information.
 ... perating system conventions if possible. For instance, on a Windows
 system the filename ``/gumby/ruby/test.rb'' will be opened as
 ``\gumby\ruby\test.rb''. When specifying a Windows-style filename in
 a Ruby string, remember to escape the back
 ... I/O ports may be opened in any one of several different modes, which
 are shown in this section as mode. The mode may either be a Fixnum
 or
 ... ning of file.
 ...   'r'  |  Read-only, starts at beginning of file  (default mode).
 ... mand line (or STDIN if no files are mentioned). ARGF provides the
 methods #path and #filename to access the name of the file currently
 being read.
 ...   'a+' |  Read-write, starts at end of file if file exists,
 ...   'a'  |  Write-only, starts at end of file if file exists,
 ...   'w+' |  Read-write, truncates existing file to zero length
 ...   'w'  |  Write-only, truncates existing file
 ...    'b' |  (DOS/Windows only) Binary file mode (may appear with
 ...   'r+' |  Read-write, starts at beginning of file.
 ... ill convert pathnames between different operating system conventions
 if possible. For instance, on a Windows system the filename
 ``/gumby/ruby/test.rb'' will be opened as ``\gumby\ruby\test.rb''.
 When specifying a Windows-style filename in a
 [...] 

Summary of changes since version 0.1.1 (2006-11-10)

Features

  • fri can do full-text search (-S, --full-text); try fri -S byte order
  • fri can now determine where a method actually came from for core classes e.g.
   fri File.inject   -> docs for Enumerable#inject
  • you can specify which ports the DRb services must bind to:
   fastri-server -s 192.168.1.2:54321
   fri -s 192.168.1.2:12345
  • new search methods: "anywhere" (a) and "anywhere, case-indep." (A)

Acknowledgements

Tomasz Wegrzanowski

  • contributed the code that allows fri to find a method in the ancestors for core classes

core dump? - chris (2006-11-16 (Don) 13:20:11)

Hi,

First let me say thanks for your great work. I really like it.

I just gem-installed fri, but I get

 $ fastri-server -B
 Indexing RI docs for ZenTest version 3.4.1.
 [...]
 Indexing RI docs for wirble version 0.1.2.
 /usr/lib/ruby/1.8/yaml.rb:133: [BUG] Segmentation fault
 ruby 1.8.4 (2005-12-24) [i486-linux]
 
 zsh: abort (core dumped)  fastri-server -B

If it helps, I can provide the output of

  $ strace -o tmp/fastri-server.log fastri-server -B

mfp 2006-11-16 (Thr) 17:08:53

I got a similar report on win32 (also segfault in yaml.rb:133) with ruby 1.8.4, and it went away after upgrading to 1.8.5.

Several bugs in syck were fixed in Jan/Feb 2006 (shortly after 1.8.4 got released on 2005-12-24). This might have been the one that bit you (but there are more):

Tue Jan 31 21:24:54 2006  Hirokazu Yamamoto  <ocean@m2.ccsnet.ne.jp>

        * ext/syck/rubyext.c (syck_resolver_transfer): workaround for SEGV.
          ex: ruby -ryaml -e 'YAML.load("!map:B {}")' [ruby-core:7217]

I don't think I can do much on FastRI's side to avoid the bug, so you can either

  • upgrade to 1.8.5, or
  • avoid using the full-text search :)

I'm not sure full-text searching is that useful in practice (but coding it was fun :). It'll soon get somewhat better when I sort results by relevance. And, following matz's lead, I have some "wild ideas" I might try to implement later.


Extending fastri for gem rdoc - Saimon Moore (2006-11-20 (Mon) 11:45:16)

How about adding indexing for gem rdocs as well :)

mfp 2006-11-20 (Mon) 16:54:07

?

fastri indexes RI documentation generated from the same documentation (embedded in the code) as the HTML Rdoc. So it effectively already indexes rdoc documentation for gems too:

$ fri -T -S ActiveRecord connect | head -15
7 hits
========================================================================
Found in activerecord 1.14.4  ActiveRecord::Base

...   class Song < ActiveRecord::Base
...   User < ActiveRecord::Base
...   class User < ActiveRecord::Base
... This feature is implemented by keeping a connection pool in
ActiveRecord::Base that is a Hash indexed by the class. If a
connection is requested, the retrieve_connection method will
... ugh ActiveRecord::Base.establish_connection and retrieved by
ActiveRecord::Base.connection. All classes inheriting from
ActiveRecord::Base will use this connection. But you can also set a
class-specific connection. For example, if Course is 
... iveRecord::Base will use this connection. But you can also set  a

Saimon 2006-11-21 (Tue) 01:28:58

Hi Mauricio,

The -T option doesn't seem to exist in fastri-0.2.0.1

See this.

Also in my gem doc directory I have this. Am I missing something?

mfp 2006-11-21 (Tue) 04:18:00

-T (--no-pager), -P (--pager) and --pager-cmd are only in HEAD at the moment, but they will be in the next release.

As for the missing docs, the RI support was added to RubyGems only in 0.9.0, so FastRI will not index gems installed with previous versions.

You can either:

  • install RubyGems 0.9.0 and reinstall the gems whose docs you want to obtain, or
  • generate the ri documentation manually by running
rcov --ri -o appropriate_dir lib/

in the gem directory, where appropriate_dir is a dir "parallel to" the rdoc dir for that gem (e.g. $prefix/lib/ruby/gems/1.8/doc/somegem-0.1.1/ri)

Once you have the required "ri" directories, e.g.:

usr/lib/ruby/gems/1.8/doc$ ls -l builder-2.0.0/*
builder-2.0.0/rdoc:
total 36
drwxr-xr-x 8 batsman batsman 4096 2006-11-04 16:38 classes
-rw-r--r-- 1 batsman batsman   31 2006-11-04 16:38 created.rid
drwxr-xr-x 4 batsman batsman 4096 2006-11-04 16:38 files
-rw-r--r-- 1 batsman batsman 1244 2006-11-04 16:38 fr_class_index.html
-rw-r--r-- 1 batsman batsman 1428 2006-11-04 16:38 fr_file_index.html
-rw-r--r-- 1 batsman batsman 2410 2006-11-04 16:38 fr_method_index.html
-rw-r--r-- 1 batsman batsman  762 2006-11-04 16:38 index.html
-rw-r--r-- 1 batsman batsman 4314 2006-11-04 16:38 rdoc-style.css

builder-2.0.0/ri:
total 20
drwxr-xr-x 4 batsman batsman 4096 2006-11-04 16:38 Builder
-rw-r--r-- 1 batsman batsman   31 2006-11-04 16:38 created.rid
drwxr-xr-x 2 batsman batsman 4096 2006-11-04 16:38 Fixnum
drwxr-xr-x 2 batsman batsman 4096 2006-11-04 16:38 Object
drwxr-xr-x 2 batsman batsman 4096 2006-11-04 16:38 String

fastri-server will pick up the gems:

$ fastri-server -b
Indexing RI docs for ParseTree version 1.5.0.
Indexing RI docs for Rubilicious version 0.2.0.
Indexing RI docs for ZenTest version 3.4.2.
Indexing RI docs for actionmailer version 1.2.5.
Indexing RI docs for actionpack version 1.12.5.
Indexing RI docs for actionwebservice version 1.1.6.
[...]
Building index.
Indexed:
* 11226 methods
* 2315 classes/modules
Needed 1.425515 seconds

$ fastri-server -B
Indexing RI docs for ParseTree version 1.5.0.
Indexing RI docs for Rubilicious version 0.2.0.
Indexing RI docs for ZenTest version 3.4.2.
[...]

Saimon 2006-11-21 (Tue) 05:01:09

Installing rubygems 0.9.0 did the trick though I had to delete the gem source cache (via sudo rm -fR `gem env gemdir`/source_cache).

Many thanks and looking forward to the next release.



Last modified:2006/11/16 09:53:52
Keyword(s):[blog] [ruby] [fastri] [0.2.0] [release]
References: