Rcov 0.2.0 - code coverage tool for Ruby


#1

Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

Overview

rcov is a tool for simple code coverage analysis in Ruby. It improves on
the better known coverage[1] in a number of ways:

  • 20-300 times faster: typically, the program being inspect runs only ~3
    times slower than without rcov (i.e. not 200 times slower as with
    some other tools)
  • more convenient interface
  • prettier output (since 0.2.0)
  • more accurate coverage information through code linkage inference
    using
    simple heuristics
  • additional features like bogo-profile mode, colorblind-friendliness…

Code coverage shouldn’t be abused (in few words, C0 coverage guarantees
nothing) but it’s still useful for testing: it will at least tell you
when
your tests need more work, and most importantly where.

What’s new in 0.2.0

A change summary is available at
http://eigenclass.org/hiki.rb?rcov+0.2.0

Visible changes since 0.1.0:

  • improved HTML output (cleaner and more readable) — using xx under
    the hood
  • textual output: execution counts or code decorated with ANSI sequences
  • more convenient interface: run several scripts/tests in one go,
    argument
    propagation…

Downloading

The last version is available at
http://eigenclass.org/hiki.rb?rcov

How do I use it? What does it look like?

Just use rcov to run your program (instead of ruby), and a number of
XHTML
files with the code coverage information will be generated, e.g.

rcov test/*.rb

will execute all the .rb files under test/ and generate the code
coverage report
under coverage/. rcov can also operate in “bogo-profiling mode”
and output the relevant information in alternative formats (for the time
being, plain text with execution count annotations and colored text).
Use
rcov -h
for more information.

Sample output

See http://eigenclass.org/hiki.rb?rcov (once again) for screenshots.
The (undecorated) textual output with execution count information looks
like this:

$ rcov --no-html -t b.rb

./b.rb

                                                                   | 

2
a, b, c = (1…3).to_a |
2
10.times do |
1
a += 1 |
10
20.times do |i| |
10
b += i |
200
b.times do |
200
c += (j = (b-a).abs) > 0 ? j : 0 |
738800
end |
0
end |
0
end |
0

License

rcov is released under the terms of Ruby’s license.
rcov includes xx 0.1.0, which is subject to the following conditions:

ePark Labs Public License version 1
Copyright © 2005, ePark Labs, Inc. and contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification,
are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright
    notice, this
    list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright
    notice,
    this list of conditions and the following disclaimer in the
    documentation
    and/or other materials provided with the distribution.
  3. Neither the name of ePark Labs nor the names of its contributors
    may be
    used to endorse or promote products derived from this software
    without
    specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS
IS” AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

[1] http://freshmeat.net/projects/rubycoverage


#2

Thanks Mauricio, this looks great and arrives just at the right momen
when the students in my testing course goes into structural testing of
their programs. I’ll point them to this!

Keep up the good work!

Thanks,

Robert F.


#3

On 2/28/06, Mauricio F. removed_email_address@domain.invalid wrote:

Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

[snip]

very nice :slight_smile:


#4

many thanks for rcov, batsman!!
i was just trying it out. what a great tool. i will be using it heavily
for
sure!
– henon


#5

Hi,

Mauricio F. removed_email_address@domain.invalid wrote:
Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

On Windows XP (HOME SP2) using MSVC ++ 6.0 and latest one-click
installer:

C:\extract_ruby\rcov>ruby setup.rb
—> bin
<— bin
—> ext
—> ext/rcovrt
C:/ruby/bin/ruby.exe C:/extract/_ruby/rcov/ext/rcovrt/extconf.rb
creating Makefile
<— ext/rcovrt
<— ext
—> bin
updating shebang: rcov
<— bin
—> ext
—> ext/rcovrt
nmake
Microsoft ® Program Maintenance Utility Version 6.00.8168.0
Copyright © Microsoft Corp 1988-1998. All rights reserved.
cl -nologo -MD -Zi -O2b2xg- -G6 -I.
-IC:/ruby/lib/ruby/1.8/i386-mswin32 -IC:/ruby/lib/ruby/1.8/i386-mswin32
-IC:/extract/_ruby/rcov/ext/rcovrt -I. -I./… -I./…/missing -c
-Tcrcov.c
rcov.c
rcov.c(21) : error C2146: syntax error : missing ‘)’ before identifier
‘event’
rcov.c(21) : error C2061: syntax error : identifier ‘event’
rcov.c(21) : error C2059: syntax error : ‘;’
rcov.c(21) : error C2059: syntax error : ‘,’
rcov.c(22) : error C2059: syntax error : ‘)’
rcov.c(70) : error C2065: ‘coverage_event_hook’ : undeclared identifier
rcov.c(71) : error C2065: ‘RUBY_EVENT_ALL’ : undeclared identifier
rcov.c(71) : error C2065: ‘RUBY_EVENT_C_CALL’ : undeclared identifier
rcov.c(72) : error C2065: ‘RUBY_EVENT_C_RETURN’ : undeclared identifier
rcov.c(72) : error C2065: ‘RUBY_EVENT_CLASS’ : undeclared identifier
NMAKE : fatal error U1077: ‘cl’ : return code ‘0x2’
Stop.
setup.rb:655:in command': system("nmake") failed (RuntimeError) from setup.rb:664:inmake’
from setup.rb:1258:in setup_dir_ext' from setup.rb:1532:insend
from setup.rb:1532:in traverse' from setup.rb:1530:indive_into’
from setup.rb:1530:in traverse' from setup.rb:1534:intraverse’
from setup.rb:1533:in each' ... 7 levels... from setup.rb:996:inexec_setup’
from setup.rb:813:in invoke' from setup.rb:773:ininvoke’
from setup.rb:1578
C:\extract_ruby\rcov>


#6

Hi ,

Shashank D. removed_email_address@domain.invalid wrote:
Mauricio F. wrote:
Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

On Windows XP (HOME SP2) using MSVC ++ 6.0 and latest one-click
installer:

I was wrong … this error only occurs if I use the older version of
one-click installer.
Works fine with the latest version.

However, now after successful compile and install, when I do this:

C:\atest>rcov --no-html -t tst_rcov.rb

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb --no-html -t
tst_rcov.rb
c:/ruby/lib/ruby/1.8/optparse.rb:1445:in complete': invalid option: --no-html (OptionParser::Invalid from c:/ruby/lib/ruby/1.8/optparse.rb:1443:incomplete’
from c:/ruby/lib/ruby/1.8/optparse.rb:1278:in order!' from c:/ruby/lib/ruby/1.8/optparse.rb:1271:inorder!’
from c:/ruby/lib/ruby/1.8/optparse.rb:1351:in permute!' from c:/ruby/lib/ruby/1.8/optparse.rb:1378:inparse!’
from c:/ruby/bin/rcov.rb:627

C:\atest>rcov -h

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb -h
rcov 0.1.0 2006-01-23
Usage: rcov [options] <script.rb>
Options:
-o, --output PATH Destination directory.
-I, --include PATHS Prepend PATHS to $: (colon
separated list)
-n, --no-color Create colorblind-safe output.
-x, --exclude PATTERNS Don’t generate info for the files
matching any
of the given patterns
(comma-separated regexp list)
-p, --profile Generate bogo-profiling info.
-r, --range Color scale range for profiling
info (dB).
-h, --help Show this message
–version Show version
C:\atest>


#7

Hello,

It seems like you are using the old rcov eg. rcov 0.1 instead of 0.2

  • Ville

#8

Hi,

It works … if I use the correct version !

Sorry for the noise …

And now, I have the Windows binary in case anybody wants to try it out
with the latest one click installer.

– Shashank

Shashank D. removed_email_address@domain.invalid wrote:
Hi ,

Shashank D. wrote:
Mauricio F. wrote:
Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

On Windows XP (HOME SP2) using MSVC ++ 6.0 and latest one-click
installer:

I was wrong … this error only occurs if I use the older version of
one-click installer.
Works fine with the latest version.

However, now after successful compile and install, when I do this:

C:\atest>rcov --no-html -t tst_rcov.rb

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb --no-html -t
tst_rcov.rb
c:/ruby/lib/ruby/1.8/optparse.rb:1445:in complete': invalid option: --no-html (OptionParser::Invalid from c:/ruby/lib/ruby/1.8/optparse.rb:1443:incomplete’
from c:/ruby/lib/ruby/1.8/optparse.rb:1278:in order!' from c:/ruby/lib/ruby/1.8/optparse.rb:1271:inorder!’
from c:/ruby/lib/ruby/1.8/optparse.rb:1351:in permute!' from c:/ruby/lib/ruby/1.8/optparse.rb:1378:inparse!’
from c:/ruby/bin/rcov.rb:627

C:\atest>rcov -h

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb -h
rcov 0.1.0 2006-01-23
Usage: rcov [options]
Options:
-o, --output PATH Destination directory.
-I, --include PATHS Prepend PATHS to $: (colon separated list)
-n, --no-color Create colorblind-safe output.
-x, --exclude PATTERNS Don’t generate info for the files matching any
of the given patterns (comma-separated regexp list)
-p, --profile Generate bogo-profiling info.
-r, --range Color scale range for profiling info (dB).
-h, --help Show this message
–version Show version
C:\atest>


Relax. Yahoo! Mail virus scanning helps detect nasty viruses!


#9

On Wed, Mar 01, 2006 at 02:27:37PM +0900, Shashank D. wrote:

It works … if I use the correct version !

I think I might be to blame for that; I didn’t really think about
win32 issues like whether the executable is associated to the Ruby
interpreter. I’ll try to do better next time.


#10

On Wed, Mar 01, 2006 at 01:31:11AM +0900, Mauricio F. wrote:

Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

Fabulous!

I would like to get coverage for my test cases too. Could this be
accomodated (without tweaking @ignore_files in the source)?

many, many thanks for this tool,
dave


#11

On Wed, Mar 15, 2006 at 02:23:04AM +0900, Mauricio F. wrote:

It will be possible in the next release; in the meantime, could you use the
following patch?

I am doing, and it looks good.

Thank you!
dave


#12

On Tue, Mar 14, 2006 at 09:27:14AM +0900, David H. wrote:

On Wed, Mar 01, 2006 at 01:31:11AM +0900, Mauricio F. wrote:

Source code, additional information, screenshots… available at

http://eigenclass.org/hiki.rb?rcov

Fabulous!

I would like to get coverage for my test cases too. Could this be
accomodated (without tweaking @ignore_files in the source)?

It will be possible in the next release; in the meantime, could you use
the
following patch?

— old-rcov/bin/rcov 2006-03-14 18:12:47.000000000 +0100
+++ new-rcov/bin/rcov 2006-03-14 09:53:07.000000000 +0100
@@ -790,6 +790,11 @@
regexps = list.split(/,/).map{|x| Regexp.new x}
options.skip += regexps
end

  • opts.on("–exclude-only PATTERNS",
  •        "Skip info generation only for the files matching the",
    
  •        "given patterns (i.e. discard the default regexp list).") 
    

do |list|

  •            options.skip = list.split(/,/).map{|x| Regexp.new(x) }
    
  •        end
    
    opts.on("-p", “–profile”, “Generate bogo-profiling info.”) do
    options.profiling = true
    options.destdir ||= “profiling”

After applying: (note that in rcov 0.2.0 -n == --no-color is ignored in
-T
rich-text mode, so you’ll get ANSI-decorated output instead of the
following)

$ rcov --exclude-only lib/ruby --no-html -n -T tc_foo.rb
Loaded suite tc_foo
Started
6
.
Finished in 0.009056 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

./tc_foo.rb

require ‘test/unit’
require ‘foo’

class TC_Foo < Test::Unit::TestCase
def setup; @foo = Foo.new end

 def test_foo
   assert_equal(8, @foo.foo(2, 4))
 end

end

./foo.rb

class Foo
def foo(a,b)
if b > 3
puts a + b

else

puts a - b

end

   a * b
 end

end

class Bar
def bar

1

end

end


#13

On Fri, Mar 17, 2006 at 06:30:14AM +0900, Patrick H. wrote:

Just want to join the chorus of thank you. This is perfect. I found a
blog entry (http://asplake.blogspot.com/2006/01/revert-that-rakefile.html)
and worked it over a little into the file below which I put in
lib/tasks/rcov.rake in a rails project. It works great.

[rcov.rake]

Thank you! May I distribute it with rcov (under Ruby’s license)? It
seems many people will want to use it that way…


#14

On 3/16/06, Mauricio F. removed_email_address@domain.invalid wrote:

On Fri, Mar 17, 2006 at 06:30:14AM +0900, Patrick H. wrote:

Just want to join the chorus of thank you. This is perfect. I found a
blog entry (http://asplake.blogspot.com/2006/01/revert-that-rakefile.html)
and worked it over a little into the file below which I put in
lib/tasks/rcov.rake in a rails project. It works great.

[rcov.rake]

Thank you! May I distribute it with rcov (under Ruby’s license)? It
seems many people will want to use it that way…

Please do! I’ve got a much uglier/less comprehensive set of Rake
tasks that I use, I’d love to have a cleaner version. (My tasks build
the standard html output for files in my repository or create the ascii
‘poor mans profiling’.


#15

“Patrick H.” removed_email_address@domain.invalid writes:

task :rcov do
rm_rf RCOV_OUT
sh “#{RCOV} test/unit/.rb test/functional/.rb”
end

task :units do
  sh "#{RCOV} test/unit/*.rb"
end
task :coverage_functional do
  sh "#{RCOV} test/functional/*.rb"
end

Wouldn’t it be better if you used rake’s “ruby” method instead of “sh”
to call another Ruby script?


#16

On 3/14/06, David H. removed_email_address@domain.invalid wrote:

accomodated (without tweaking @ignore_files in the source)?
http://david.holroyd.me.uk/

Just want to join the chorus of thank you. This is perfect. I found a
blog entry
(http://asplake.blogspot.com/2006/01/revert-that-rakefile.html)
and worked it over a little into the file below which I put in
lib/tasks/rcov.rake in a rails project. It works great.

Thanks
pth

require ‘rake/clean’

RBINDIR = Config::CONFIG[‘bindir’]
RBIN = “#{RBINDIR}/rcov”
if File.exists?(RBIN)

output goes to subdirectories of this one

RCOV_OUT = “coverage”

“rake clobber” to remove output directory (along with other

generated stuff)
CLOBBER.include(RCOV_OUT)

don’t report coverage on these files

RCOV_EXCLUDE = %w(boot.rb environment.rb vendor/rails
test/functional test/unit).join(’,’)

RCOV command, run as though from the commandline. Amend as

required or perhaps move to config/environment.rb?
RCOV = “#{RBINDIR}/ruby #{RBIN} --exclude #{RCOV_EXCLUDE} --output
#{RCOV_OUT}”

desc “generate a coverage report for unit and functional tests
together in #{RCOV_OUT}/index.html”
task :rcov do
rm_rf RCOV_OUT
sh “#{RCOV} test/unit/.rb test/functional/.rb”
end

namespace :rcov do
desc “generate a unit test coverage report in
#{RCOV_OUT}/index.html”
task :units do
sh “#{RCOV} test/unit/*.rb”
end

desc "generate a functional test coverage report in 

#{RCOV_OUT}/index.html"
task :coverage_functional do
sh “#{RCOV} test/functional/*.rb”
end
end

end


#17

On Sun, May 07, 2006 at 05:26:09AM +0900, Rusty Geldmacher wrote:

Christian N. wrote:

Wouldn’t it be better if you used rake’s “ruby” method instead of “sh”
to call another Ruby script?

I think I’ve gone a step further by integrating Rcov using Rake’s
TestTask class. The full description is here:

http://www.soapboxsoftware.org/articles/2006/05/06/another-rake-task-for-rcov

I took your idea (looking into Rake::TestTask) to implement a
Rcov::RcovTask
for rake. It will be in rcov 0.4.0, due out soon.

Thank you!


#18

Christian N. wrote:

Wouldn’t it be better if you used rake’s “ruby” method instead of “sh”
to call another Ruby script?

I think I’ve gone a step further by integrating Rcov using Rake’s
TestTask class. The full description is here:

http://www.soapboxsoftware.org/articles/2006/05/06/another-rake-task-for-rcov

Comments appreciated!

rusty