eigenclass logo
MAIN  Index  Search  Changes  PageRank  Login

rcov 0.4.0: more accurate code coverage, scriptability, unrotten internals

update.png rcov 0.5.0 to be released imminently, with a few major features like cross-referenced reports, a load of bugfixes, and a binary RubyGems package for win32.

rcov 0.4.0 is really feature-packed, and (hopefully) less buggy than previous releases. This can be ascribed in large part to the use of rcov on itself to achieve better test coverage. Download it from rcov: code coverage for Ruby.

The looks

Starting with the most obvious changes, the XHTML output looks much better now with the new color-scheme. Plain text output formats have also improved considerably. Take a look at this sample report and compare the following screenshots to those from version 0.2.0: typo-summary.png typo-detailed.png

Scriptability and automation with Rake

rcov 0.4.0 is much easier to use from external scripts, and invoking rcov from your Rakefile takes no effort with the included Rcov::RcovTask; it can take as little as:

   require 'rcov/rcovtask'
   Rcov::RcovTask.new

Really! This creates a task named "rcov" which will run the tests matching test/test*.rb and create a XHTML report. And a nice textual report on stdout). And a clobber_rcov task, which the main clobber target will depend on, to delete the former.

However, sometimes that lone Rcov::RcovTask.new will not suffice; this would be a more realistic example:

   # if you don't like the default description
   desc "Analyze code coverage of the unit tests." 
   Rcov::RcovTask.new do |t|
     # change the glob pattern in the next line
     t.test_files = FileList['test/test*.rb'] 
     t.rcov_opts << "--sort coverage"   # sort by increasing coverage rate
   end

By default, RcovTasks will show a text report resembling

+-----------------------------------------------------+-------+-------+--------+
|                  File                               | Lines |  LOC  |  COV   |
+-----------------------------------------------------+-------+-------+--------+
|app/controllers/application.rb                       |    39 |    29 |  31.0% |
|app/helpers/application_helper.rb                    |   147 |   119 |  23.5% |
|app/models/aggregations/tada.rb                      |    75 |    45 |  31.1% |
|app/models/aggregations/upcoming.rb                  |    78 |    48 |  29.2% |
|app/models/article.rb                                |   109 |    78 |  67.9% |
|app/models/category.rb                               |    30 |    23 |  60.9% |
|app/models/sidebar.rb                                |    36 |    27 |  40.7% |
|components/plugins/sidebars/archives_controller.rb   |    35 |    27 |  29.6% |
|components/plugins/sidebars/category_controller.rb   |    20 |    16 |  50.0% |
|components/plugins/sidebars/delicious_controller.rb  |    20 |    16 |  50.0% |
|components/plugins/sidebars/flickr_controller.rb     |    20 |    16 |  50.0% |
|components/plugins/sidebars/fortythree_controller.rb |    20 |    16 |  50.0% |
|...s/plugins/sidebars/fortythreeplaces_controller.rb |    20 |    16 |  50.0% |
|components/plugins/sidebars/static_controller.rb     |    27 |    24 |  29.2% |
|components/plugins/sidebars/tada_controller.rb       |    20 |    16 |  50.0% |
|components/plugins/sidebars/technorati_controller.rb |    20 |    16 |  50.0% |
|components/plugins/sidebars/upcoming_controller.rb   |    20 |    16 |  50.0% |
|components/plugins/sidebars/xml_controller.rb        |    16 |    13 |  53.8% |
|components/sidebars/sidebar_controller.rb            |   110 |    80 |  32.5% |
|lib/html_engine.rb                                   |    29 |    23 |  78.3% |
|lib/login_system.rb                                  |    85 |    34 |  23.5% |
|lib/migrator.rb                                      |    28 |    22 |  40.9% |
|lib/renderfix.rb                                     |    32 |    25 |  16.0% |
|lib/xmlrpc_fix.rb                                    |    13 |    12 |  25.0% |
+-----------------------------------------------------+-------+-------+--------+
|Total                                                |  1754 |  1251 |  60.1% |
+-----------------------------------------------------+-------+-------+--------+
60.1%   42 file(s)   1754 Lines   1251 LOC

(note that only files with coverage under 80% were shown in the above report, that's why you don't see all the 42 files)

See README.rake in the distribution tarball (or the RDoc documentation, which includes it) for more information about Rcov::RcovTask.

Library reuse

Parts of the code have been moved into a reusable library that allows you to build on top of rcov's coverage analysis capabilities. This change allowed for much better testing of rcov's internals, which helped squash many bugs and add features while making the overall code look better thanks to some long-needed refactoring.

Short summary of user-visible changes

  • easy automation via Rake using the bundled Rcov::RcovTask
  • better heuristics: supports heredocs at last, including variants your editor is probably unable to handle, and =begin/=end comments
  • --rails option to ignore files under vendor/, enviroment/ and config/
  • parts of the runtime exposed and documented for external use
  • new color scheme, with alternating shades of the base colors
  • -w to set $VERBOSE=true (turns on warnings)
  • --text-report and --text-summary
  • --sort and --sort-reverse for the summaries and reports
  • --threshold and --only-uncovered
  • --replace-progname

Backwards incompatible changes

  • renamed --text to --text-counts and --rich-text to --text-coverage: they were misnamed to begin with
  • changed the meaning of -t and -T (--text-summary and --text-report)
  • $0 is not changed by default for each loaded file anymore; the old behavior (modulo a small bugfix) can be reproduced with --replace-progname

Thanks

(do tell if I forgot to list your name)

Robert Feldt
  • helped to refine the color scheme
  • very valuable suggestions; a large percentage of the new functionality in 0.4.0 is a consequence of his requests
Andre Nathan
  • identified a couple bugs in the heuristics that eluded my testing

rcov 0.4.0 installing and using - DeeJay (2006-05-23 (Tue) 15:05:07)

I'm very new to Ruby and testing, so apologies if I am missing something very obvious...

My environment is WinXPpro-SP2, Ruby installed via the OneClick Installer.

1] after installing rcov by using

    ruby setup.rb all --without-ext

I get told rcov is not recognised as a command. Renaming rcov to rcov.rb seems to fix this. (But does it break anything else?)

2] cd'ing into test and running

    rcov some_sample.rb

produces an html report in /coverage as expected.

BUT rcov test/*.rb does NOT create /coverage or generate any reports. (is that because the *.rb notation is a shell expansion that is not honoured on Windows?)

3] installing as a gem? since rcov will run in the absence of rcovrt.so, is it possible to ship a basic gem that does not rely on the existence of a compilation facility?

Thanks for your efforts - DJ

mfp 2006-05-23 (Tue) 15:23:37

(2) is a stupid bug I introduced shortly before the release --- I broke the pure-Ruby tracing module and didn't realize it because my tests were being run with rcovrt! It's fixed in the source repository, mirrored at http://eigenclass.org/repos/rcov/ (the file you want is lib/rcov/lowlevel.rb, the fix took 3 lines).

You can find a rcovrt.so binary for win32 in rcov: code coverage for Ruby.

But the answer you want is the following :) The RubyGems packages (both platform-independent and the one built for win32, which includes rcovrt.so and can be used with the one-click installer) are ready. I've sent them to a few (coerced :) testers, and they will be released officially when I get positive feedback.

In few words, the platform-specific package for win32, which will get released in short (I'm basically waiting for a couple "k, works for me") is the answer to all three points :-)

mfp 2006-05-23 (Tue) 15:31:14

Actually, if you drop an email to mfp at acm dot org, I'll be happy to send you the .gem for win32. (put "rcov" in the subject to make sure it gets past the spam filters)

Michael 2006-05-28 (Sun) 18:35:22

So what are the "3 lines" that need to be fixed?

mfp 2006-05-29 (Mon) 05:20:12

If you can wait for a few hours/days (I'm waiting for the approval of the rcov project on Rubyforge so I have a release mirror), this and much more will be in 0.5.0 (whose darcs repository is at http://eigenclass.org/repos/rcov by the way).

Otherwise:

   Mon May 22 23:14:43 CEST 2006  Mauricio Fernandez <mfp@acm.org>
     * lib/rcov/lowlevel.rb: fixed brown paper bug in the pure-Ruby low-level tracing module.
     It didn't work at all with the differential semantics of
     Rcov::CodeCoverageAnalyzer.
   diff -rN -u old-head/lib/rcov/lowlevel.rb new-head/lib/rcov/lowlevel.rb
   --- old-head/lib/rcov/lowlevel.rb	2006-05-29 13:16:53.000000000 +0200
   +++ new-head/lib/rcov/lowlevel.rb	2006-05-29 13:16:53.000000000 +0200
   @@ -52,11 +52,12 @@
    
 def self.install_hook # :nodoc:
   set_trace_func lambda {|event, file, line, id, binding, klass|
   +        next unless SCRIPT_LINES__.has_key? file
     case event
     when 'c-call', 'c-return', 'class'
       return
     end
   -        COVER[file] ||= []
   +        COVER[file] ||= Array.new(SCRIPT_LINES__[file].size, 0)
     COVER[file][line - 1] ||= 0
     COVER[file][line - 1] += 1
   }
   @@ -71,7 +72,7 @@
 end
    
 def self.generate_coverage_info # :nodoc:
   -      COVER
   +      Marshal.load(Marshal.dump(COVER))
 end
      end
    end # RCOV__

Michael 2006-05-30 (Tue) 06:48:22

Thanks!

Michael 2006-05-30 (Tue) 06:56:04

0.5.0 works like a charm!


Last modified:2006/05/22 04:03:27
Keyword(s):[blog] [ruby] [frontpage] [rcov] [0.4.0] [release] [code] [coverage]
References:[rcov: code coverage for Ruby]