Hello,
Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?
On Aug 4, 2006, at 12:12 PM, Tomasz W. wrote:
Hello,
Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?–
Tomasz W. [ http://t-a-w.blogspot.com/ ]
Check out rcov. This is exactly what rcov was created for. It makes
these awesome HTML coverage reports.
Tomasz W. wrote:
Hello,
Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?
I don’t know of any mechanism built into RubyUnit or a code coverage
tool. But this is something you could do: use set_trace_func to trace
all methods invoked, then at the end of a single class’s test case or
when all tests are done go through your classes and check their
#instance_methods to verify that all of them were called. But that
still does not give you info whether you had a proper test case for the
method or the method was called by some other method etc. Maybe you can
extend the tracing to record only methods called from a unit test.
Still you would not know whether the method was simply called or whether
return values were checked or whether they were checked properly.
Kind regards
robertOn 8/4/06, Robert K. [email protected] wrote:
#instance_methods to verify that all of them were called. But that
still does not give you info whether you had a proper test case for the
method or the method was called by some other method etc. Maybe you can
extend the tracing to record only methods called from a unit test.
Still you would not know whether the method was simply called or whether
return values were checked or whether they were checked properly.
Well, one cannot have everything :-). rcov is pretty useful for finding
missing tests. My only problem with it so far is its inability to test
coverage of metaprogrammed code, but I guess that can’t be helped.
On 8/4/06, Logan C. [email protected] wrote:
Check out rcov. This is exactly what rcov was created for. It makes
these awesome HTML coverage reports.
Thanks, it was exactly what I needed. 
By the way how are people usually testing the convenience functions
that print something to STDOUT.
On Aug 4, 2006, at 1:40 PM, Tomasz W. wrote:
By the way how are people usually testing the convenience functions
that print something to STDOUT.
Use the standard StringIO library for this. You can swap out the
STDOUT object for the test and replace it when you are done but I
prefer to modify the method to be more testable:
def meth_to_test(io = STDOUT)
# use io here…
end
Then you can just pass the StringIO object when testing.
James Edward G. II
Mauricio F. wrote:
You can do that in rcov with the --test-unit-only option.
[snip]
Mauricio, thanks for the detailed explanation! Learn something new
every day…
Kind regards
robertOn Sat, Aug 05, 2006 at 03:50:35AM +0900, Tomasz W. wrote:
On 8/4/06, Robert K. [email protected] wrote:
I don’t know of any mechanism built into RubyUnit or a code coverage
tool. But this is something you could do: use set_trace_func to trace
all methods invoked, then at the end of a single class’s test case or
when all tests are done go through your classes and check their
#instance_methods to verify that all of them were called. But that
still does not give you info whether you had a proper test case for the
method or the method was called by some other method etc. Maybe you can
extend the tracing to record only methods called from a unit test.
[…]
You can do that in rcov with the --test-unit-only option.
Well, one cannot have everything :-). rcov is pretty useful for finding
missing tests. My only problem with it so far is its inability to test
coverage of metaprogrammed code, but I guess that can’t be helped.
Are you referring to this?
$ rcov --no-html --text-coverage --no-color a.rb
a.rb
module StupidMeta
def ugly_make_meth(meth)
module_eval <<-EOF, FILE, LINE+1
def #{meth}(a,b)
a + #{meth.to_s.inspect} + b
end
EOF
end
def make_meth(meth)
define_method(meth) do |a,b|
!! a + meth.to_s + b
!! end
end
end
class X
extend StupidMeta
ugly_make_meth :foo
make_meth :bar
end
we don’t actually call them, so they should both be uncovered
#x = X.new
#x.foo("method “, " here”)
#x.bar("method “, " here”)
As you can see, Module#define_method will work better, if you can use it
at
all (that is, if you don’t need a block argument, under 1.8, and don’t
care
about the closure and the speed hit). Another advantage is that it will
tell
you if the code is well formed (syntactically correct) much earlier.
The problem with the string forms of eval is that I’ve had to go to
great
lengths to make sure rcov marks heredocs correctly, and the above
annotation
is correct.
Indeed, in the above snippet the heredoc in ugly_make_meth has been
“executed”, in some sense (this interpretation gains some weight due to
the
interpolation going on; there’s actually something being evaluated
there).
But there are two interpretations for that string: as a mere heredoc,
and as
executable code that defines another method. There are thus also two
ways to
think about it as far as coverage is concerned.
I’m thinking that maybe some sort of pragma would make sense:
def ugly_make_meth(meth)
module_eval <<-EOF, FILE, LINE+1 # rcov:noheredoc
def #{meth}(a,b)
a + #{meth.to_s.inspect} + b
end
EOF
end
As for other limitations, I think I know how to handle generated code
that
cannot be meaningfully associated to your sources, but I feel it’s not
worth
the effort.
Is there anything else you’d like to see addressed?