Forum: Ruby rcov 0.4.0 (code coverage) - scriptability, accuracy, unrott

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Mauricio F. (Guest)
on 2006-05-22 14:29
(Received via mailing list)
Source code, additional information, screenshots... available at
    http://eigenclass.org/hiki.rb?rcov
Release information:
    http://eigenclass.org/hiki.rb?rcov+0.4.0

You can find a sample code coverage report generated by rcov at
    http://eigenclass.org/static/rcov-sample-report/

If you're on win32, you can also find a pre-built rcovrt.so (which makes
code coverage analysis >100 times faster) in the above-mentioned pages.

Overview
========
rcov is a tool for simple code coverage analysis in Ruby. It features:
* fast execution: 20-300 times faster than previous tools
* multiple analysis modes
* fairly accurate coverage information through code linkage inference
using
  simple heuristics
* XHTML and several kinds of text reports
* easy automation with Rake via a RcovTask
* colorblind-friendliness

What's new in 0.4.0
===================
See http://eigenclass.org/hiki.rb?rcov+0.4.0 for the detailed change
summary.

rcov 0.4.0 is really feature-packed, and (hopefully) less buggy than
previous
releases. This can be ascribed in large part to the use of rcov on
itself to
achieve better test coverage. Nearly 100 changesets went into rcov
0.4.0.

Starting with the most obvious changes, the XHTML output looks much
better
now with the new color-scheme. Plain text output formats have also
improved
considerably.

rcov 0.4.0 is much easier to use from external scripts, and invoking
rcov
from your Rakefile takes no effort with the included Rcov::RcovTask; it
can
take as little as:

    require 'rcov/rcovtask'
    Rcov::RcovTask.new

Really! This creates a task named "rcov" which will run the tests
matching
test/test*.rb and create a XHTML report. And a nice textual report on
stdout). And a clobber_rcov task, which the main clobber target will
depend
on, to delete the former.

However, sometimes that lone Rcov::RcovTask.new will not suffice; this
would
be a more realistic example:

    # if you don't like the default description
    desc "Analyze code coverage of the unit tests."
    Rcov::RcovTask.new do |t|
      # change the glob pattern in the next line
      t.test_files = FileList['test/test*.rb']
      t.rcov_opts << "--sort coverage"   # sort by increasing coverage
rate
    end

See README.rake in the distribution tarball (or the RDoc documentation,
which
includes it) for more information about Rcov::RcovTask.


Parts of the code have been moved into a reusable library that allows
you to
build on top of rcov's coverage analysis capabilities. This change
allowed for
much better testing of rcov's internals, which helped squash many bugs
and add
features while making the overall code look better thanks to some
long-needed
refactoring.


Short summary of user-visible changes
--------------------------------------

* easy automation via Rake using the bundled Rcov::RcovTask
* better heuristics: supports heredocs at last, including variants your
editor
  is probably unable to handle, and =begin/=end comments
* --rails option to ignore files under vendor/, enviroment/ and config/
* parts of the runtime exposed and documented for external use
* new color scheme, with alternating shades of the base colors
* -w to set $VERBOSE=true (turns on warnings)
* --text-report and --text-summary
* --sort and --sort-reverse for the summaries and reports
* --threshold and --only-uncovered
* --replace-progname

Backwards incompatible changes
------------------------------
* renamed --text to --text-counts and --rich-text to --text-coverage:
they
  were misnamed to begin with
* changed the meaning of -t and -T (--text-summary and --text-report)
* $0 is not changed by default for each loaded file anymore; the old
  behavior (modulo a small bugfix) can be reproduced with
--replace-progname

Thanks (do tell if I forgot to list your name)
======

Robert F.:
* helped to refine the color scheme
* very valuable suggestions; a large percentage of the new functionality
in
  0.4.0 is a consequence of his requests

Andre N.:
* identified a couple bugs in the heuristics that eluded my testing

Downloading
===========

The last version is available at
  http://eigenclass.org/hiki.rb?rcov


How do I use it?
================

Just use rcov to run your program (instead of ruby), and a number of
XHTML
files with the code coverage information will be generated, e.g.

 rcov test/*.rb

will execute all the .rb files under test/ and generate the code
coverage report
under coverage/. rcov can also operate in "bogo-profiling mode"
and output the relevant information in alternative formats. Use
 rcov -h
for more information.


Sample output
=============

See http://eigenclass.org/hiki.rb?rcov (once again) for screenshots.
A sample code coverage report generated by rcov is available at
    http://eigenclass.org/static/rcov-sample-report/

Text reports (also used by default in RcovTasks) resemble

+-----------------------------------------------------+-------+-------+--------+
|                  File                               | Lines |  LOC  |  COV   |
+-----------------------------------------------------+-------+-------+--------+
|app/controllers/application.rb                       |    39 |    29 |  31.0% |
|app/helpers/application_helper.rb                    |   147 |   119 |  23.5% |
|app/models/aggregations/tada.rb                      |    75 |    45 |  31.1% |
|app/models/aggregations/upcoming.rb                  |    78 |    48 |  29.2% |
|app/models/article.rb                                |   109 |    78 |  67.9% |
|app/models/category.rb                               |    30 |    23 |  60.9% |
|app/models/sidebar.rb                                |    36 |    27 |  40.7% |
|components/plugins/sidebars/archives_controller.rb   |    35 |    27 |  29.6% |
|components/plugins/sidebars/category_controller.rb   |    20 |    16 |  50.0% |
|components/plugins/sidebars/delicious_controller.rb  |    20 |    16 |  50.0% |
|components/plugins/sidebars/flickr_controller.rb     |    20 |    16 |  50.0% |
|components/plugins/sidebars/fortythree_controller.rb |    20 |    16 |  50.0% |
|...s/plugins/sidebars/fortythreeplaces_controller.rb |    20 |    16 |  50.0% |
|components/plugins/sidebars/static_controller.rb     |    27 |    24 |  29.2% |
|components/plugins/sidebars/tada_controller.rb       |    20 |    16 |  50.0% |
|components/plugins/sidebars/technorati_controller.rb |    20 |    16 |  50.0% |
|components/plugins/sidebars/upcoming_controller.rb   |    20 |    16 |  50.0% |
|components/plugins/sidebars/xml_controller.rb        |    16 |    13 |  53.8% |
|components/sidebars/sidebar_controller.rb            |   110 |    80 |  32.5% |
|lib/html_engine.rb                                   |    29 |    23 |  78.3% |
|lib/login_system.rb                                  |    85 |    34 |  23.5% |
|lib/migrator.rb                                      |    28 |    22 |  40.9% |
|lib/renderfix.rb                                     |    32 |    25 |  16.0% |
|lib/xmlrpc_fix.rb                                    |    13 |    12 |  25.0% |
+-----------------------------------------------------+-------+-------+--------+
|Total                                                |  1754 |  1251 |  60.1% |
+-----------------------------------------------------+-------+-------+--------+
60.1%   42 file(s)   1754 Lines   1251 LOC

This was run with --threshold 80, meaning that only files under 80%
coverage
are shown in the table, but the overall statistics include all analyzed
files.


The textual output with execution count information looks like this:

 $ rcov --no-html --text-counts b.rb
 ================================================================================
 ./b.rb
 ================================================================================
                                                                       |
2
 a, b, c = (1..3).to_a                                                 |
2
 10.times do                                                           |
1
   a += 1                                                              |
10
   20.times do |i|                                                     |
10
     b += i                                                            |
200
     b.times do                                                        |
200
       c += (j = (b-a).abs) > 0 ? j : 0                                |
738800
     end                                                               |
0
   end                                                                 |
0
 end                                                                   |
0


License
-------
rcov is released under the terms of Ruby's license.
rcov includes xx 0.1.0, which is subject to the following conditions:

ePark Labs Public License version 1
Copyright (c) 2005, ePark Labs, Inc. and contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification,
are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright
notice, this
     list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright
notice,
     this list of conditions and the following disclaimer in the
documentation
     and/or other materials provided with the distribution.
  3. Neither the name of ePark Labs nor the names of its contributors
may be
     used to endorse or promote products derived from this software
without
     specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Jeff R. (Guest)
on 2006-05-22 20:12
(Received via mailing list)
This is awesome.  It's enlightening and sad to see the coverage of some
of my unit tests, but without a doubt I'm hooked.  Visually looking at
the test coverage (or lack thereof) is really much more effective than I
thought.  If anything this will help to be a lot more conscious about
creating solid test suites, rather than cherry picking the easy/obvious
ones that most likely don't do the job like they should.

When do we get a gem?

Thanks a lot,
Jeff
Zed S. (Guest)
on 2006-05-23 00:21
(Received via mailing list)
On Tue, 2006-05-23 at 01:09 +0900, Jeff R. wrote:
> This is awesome.  It's enlightening and sad to see the coverage of some
> of my unit tests, but without a doubt I'm hooked.  Visually looking at
> the test coverage (or lack thereof) is really much more effective than I
> thought.  If anything this will help to be a lot more conscious about
> creating solid test suites, rather than cherry picking the easy/obvious
> ones that most likely don't do the job like they should.
>
> When do we get a gem?

Mauricio, I'll hook you up with gem builds if you want.  I use rcov a
lot and having gems makes it easier for me to get other developers to
use it.
Mauricio F. (Guest)
on 2006-05-23 02:06
(Received via mailing list)
On Tue, May 23, 2006 at 05:17:47AM +0900, Zed S. wrote:
> Mauricio, I'll hook you up with gem builds if you want.  I use rcov a
> lot and having gems makes it easier for me to get other developers to
> use it.

The one thing that has been blocking (or at least pushing down the TODO
list)
the RubyGems package was how to combine it with cross-compilation for
win32,
since I figured a RubyGems package would be of most value to win32 users
who
have no access to a C toolchain. There are a few other problems I'd have
liked
to solve, but they're much more general and probably addressable at
RubyGems'
level (C runtime compatibility, ABI checking...).

A "normal" RubyGems package (i.e. one that requires a C compiler) should
be
very easy, so I should be able to get one out fairly quickly.
This topic is locked and can not be replied to.