Forum: Ruby asserting in code vs unit tests

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Esad H. (Guest)
on 2006-03-05 20:04
(Received via mailing list)
Hi!

I talked to few fellow ruby programmers, and most of them agreed that
unit tests are the place where the asserts should be placed, however in
my own opinion, unit tests mostly never cover everything and asserts in
development versions ($DEBUG) are very handy. What is your opinion?

Esad
Gareth R. (Guest)
on 2006-03-05 20:37
Esad H. wrote:
> Hi!
>
> I talked to few fellow ruby programmers, and most of them agreed that
> unit tests are the place where the asserts should be placed, however in
> my own opinion, unit tests mostly never cover everything and asserts in
> development versions ($DEBUG) are very handy. What is your opinion?
>
> Esad

Personally, I like the asserts in the test case.

It doesnt clutter the code, you can test each circumstance and it helps
with design because you have to treat the code as a unit with inputs and
outputs.

Gareth
Meinrad R. (Guest)
on 2006-03-06 00:20
(Received via mailing list)
On 3/5/06, Esad H. <removed_email_address@domain.invalid> wrote:
>
asserts in code are good to ensure preconditions and postconditions of
methods. this is "design by contract".
asserts in unittests can often only be used to check postconditions of
the
tested methods.

i prefer precondition asserts in debug code that get filtered out by a
preprocessor in the release version (hence no performance cost).
i often have the assertions followed by errorhandling code that tries to
handle the same issues gracefully in release code.

-- henon
E. Saynatkari (Guest)
on 2006-03-06 00:33
Esad H. wrote:
> Hi!
>
> I talked to few fellow ruby programmers, and most of them agreed that
> unit tests are the place where the asserts should be placed, however in
> my own opinion, unit tests mostly never cover everything and asserts in
> development versions ($DEBUG) are very handy. What is your opinion?

If your unit tests do not cover everything, how can
any other kinds? :)

> Esad


E
Chiaro S. (Guest)
on 2006-03-06 01:06
(Received via mailing list)
On 3/5/06, E. Saynatkari <removed_email_address@domain.invalid> wrote:
> Esad H. wrote:
> If your unit tests do not cover everything, how can
> any other kinds? :)
>
> > Esad

These are two different issues.

Your asserts/contracts should check that the methods are being passed
what they expect to be passed to them.  Postconditions will probably
check some invariants on the results of the methods.

Your test should not test what the contracts already check.  The tests
should check that if you pass correct values you get the expected
results.

Eg: assume that you have a method that takes only positive numbers.
you will have a contract/assertion checking that you don't pass any
negatives and a test suite that checks that, given a set of positive
inputs, you get the correct outputs.
E. Saynatkari (Guest)
on 2006-03-06 01:23
Chiaro Scuro wrote:
> On 3/5/06, E. Saynatkari <removed_email_address@domain.invalid> wrote:
>> Esad H. wrote:
>> If your unit tests do not cover everything, how can
>> any other kinds? :)
>>
>> > Esad
>
> These are two different issues.
>
> Your asserts/contracts should check that the methods are being passed
> what they expect to be passed to them.  Postconditions will probably
> check some invariants on the results of the methods.

Yes, I agree, and this was partly the point I was making.
There are situations where functional tests are required
and there are some things that just require real-time
constraints (any non-predetermined input, usually).

> Your test should not test what the contracts already check.  The tests
> should check that if you pass correct values you get the expected
> results.
>
> Eg: assume that you have a method that takes only positive numbers.
> you will have a contract/assertion checking that you don't pass any
> negatives and a test suite that checks that, given a set of positive
> inputs, you get the correct outputs.

Or, instead, you could in many circumstances ascertain that the
method that produces the input does not produce negative numbers;
this is something that a unit test would do and it would eliminate
the need of a constraint :)


E
Chiaro S. (Guest)
on 2006-03-06 01:42
(Received via mailing list)
On 3/6/06, E. Saynatkari <removed_email_address@domain.invalid> wrote:
> Or, instead, you could in many circumstances ascertain that the
> method that produces the input does not produce negative numbers;
> this is something that a unit test would do and it would eliminate
> the need of a constraint :)

We are on something interesting here.  I am tempted to agree with you,
but for the sake of discussion and to explore the idea with you I will
say:

 * a test doesn't prove that it will never produce a negative number,
it simply checks that it doesn't do it under certain very specific
conditions
 * a postcondition will always check that it doesn't produce a
negative number and it will warn you if it does.

In this sense pre and post conditions seems to serve different purposes.

the precondition defines a scope of valid input and checks that that
input is well formed.
tests check a method, passing it valid input and expecting a 'specific'
result.
a postcondition defines a scope of valid output and checks that the
output is well formed.

pre/post alone are not enough because they just check the domain of
variables.

you need tests to verify that at least under some conditions the
method yields meaningful results.

tests alone however are never exhaustive and cannot prove correctness.
assertions can catch some types of 'wrongness'.
Pat M. (Guest)
on 2006-03-06 01:51
(Received via mailing list)
On 3/5/06, chiaro scuro <removed_email_address@domain.invalid> wrote:
>  * a test doesn't prove that it will never produce a negative number,
> a postcondition defines a scope of valid output and checks that the
> output is well formed.
>
> pre/post alone are not enough because they just check the domain of variables.
>
> you need tests to verify that at least under some conditions the
> method yields meaningful results.
>
> tests alone however are never exhaustive and cannot prove correctness.
> assertions can catch some types of 'wrongness'.

Isn't that what exceptions are for?

def foo(a)
  raise 'Argument can not be negative' if a < 0

  b = something_that_should_always_be_positive

  raise 'Something strange happened and b was negative' if b < 0
  b
end

foo expects a to be positive, so when a negative number comes in
that's an exceptional case.  It also should always return a positive
number, so if b is negative then something really went wrong and it
should terminate.

Pat
Chiaro S. (Guest)
on 2006-03-06 02:00
(Received via mailing list)
I would get my assertions to raise exceptions too.  They are just a
specific kind of exception, related to the domain of inputs and
outputs, different from exceptions generated from something that goes
wrong in the middle.

What do other people think about the relationship between assertions
and exceptions?
E. Saynatkari (Guest)
on 2006-03-06 02:13
Chiaro Scuro wrote:
> On 3/6/06, E. Saynatkari <removed_email_address@domain.invalid> wrote:
>> Or, instead, you could in many circumstances ascertain that the
>> method that produces the input does not produce negative numbers;
>> this is something that a unit test would do and it would eliminate
>> the need of a constraint :)
>
> We are on something interesting here.  I am tempted to agree with you,
> but for the sake of discussion and to explore the idea with you I will
> say:
>
>  * a test doesn't prove that it will never produce a negative number,
> it simply checks that it doesn't do it under certain very specific
> conditions
>  * a postcondition will always check that it doesn't produce a
> negative number and it will warn you if it does.
>
> In this sense pre and post conditions seems to serve different purposes.
>
> the precondition defines a scope of valid input and checks that that
> input is well formed.
> tests check a method, passing it valid input and expecting a 'specific'
> result.
> a postcondition defines a scope of valid output and checks that the
> output is well formed.
>
> pre/post alone are not enough because they just check the domain of
> variables.
>
> you need tests to verify that at least under some conditions the
> method yields meaningful results.
>
> tests alone however are never exhaustive and cannot prove correctness.
> assertions can catch some types of 'wrongness'.

Sure, but you could also just have a stick of bad RAM :)

The goal is to be reasonably certain that the program behaves
as defined by the tests. This is aided by the granular and
gradual evolution of the tests. You can never really be sure
of anything.


E
John C. (Guest)
on 2006-03-06 02:24
(Received via mailing list)
Unit Tests and DbC assertions are complementary, not exclusive. I always
do both.

I tend to put the "post condition" assertions in the Unit Tests and
invariant and pre-condition assertions in the code.

Why?

1) The pre-condition assertions are "tests of the tests". Test's are
code
    too, therefore have bugs.

2) Post-conditions are effectively the asserts found in the unit test.
ie.
    If you have strong test coverage, don't bother with post-condition
    asserts, refactor them into the unit test code.

2) Client code using my code has bugs and the precondition asserts catch
a
    lot of them very rapidly and at the right point. A the client code
Unit
    test  postcondition assert says the client (or maybe the lower
layer)
    code did vaguely the wrong thing  somewhere someplace sometime. A
    Precondition assert says the client code violated a pre-condition
    RIGHT HERE.

3) Invariant code are Object Sanity Checks. Stroustrup (author of C++)
   recommends quite sanely that you should only spin a class if and only
if
   you have an invariant to protect. Invariant checking method is a
handy
   way of checking at run time that your class is doing what it was
   designed to do.

4) I always throw vanilla Exceptions on assertion failure. It meshes
nicely with the stacktrace mechanism of Ruby, but so far I can't think
of
any good reason to create a new exception class for them.



On Mon, 6 Mar 2006, chiaro scuro wrote:

>>> assertions can catch some types of 'wrongness'.
>> end
>
> --
> -- Chiaroscuro --
> Liquid Development: http://feeds.feedburner.com/LiquidDevelopment
>



John C.                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : 
removed_email_address@domain.invalid
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
Chiaro S. (Guest)
on 2006-03-06 02:30
(Received via mailing list)
Using assertions I could be pretty much sure that certain conditions
would never never ever verify :)  I couldn't do that with tests
though.

tests make sure that something happens.
assertions make sure that something doesn't happen.
Chiaro S. (Guest)
on 2006-03-06 02:33
(Received via mailing list)
Interesting.  Could you expand on "recommends quite sanely that you
should only spin a class if and only if you have an invariant to
protect"? it feels right but I am not sure I get it completely.

Another thing.. one big problems that I have with assertions is
duplication.  Do you assert any method? do you assert on class public
methods? It's very to sprinkle the same assertions all over the place.
John C. (Guest)
on 2006-03-06 04:09
(Received via mailing list)
On Mon, 6 Mar 2006, chiaro scuro wrote:

> Interesting.  Could you expand on "recommends quite sanely that you
> should only spin a class if and only if you have an invariant to
> protect"? it feels right but I am not sure I get it completely.

Don't forget he was talking C++ naturally, so he was basically saying if
you need to protect an invariant use "class" otherwise use "struct".

In Ruby/Perl this may translate to if you need to protect an invariant
use
"class", otherwise this is just an unrelated bag of "stuff" so maybe use
a
Hash.

> Another thing.. one big problems that I have with assertions is
> duplication.  Do you assert any method? do you assert on class public
> methods? It's very to sprinkle the same assertions all over the place.

I tend to think of levels of sanity / maturity.

The code in any one class tends to be equally buggy or unbuggy.

Thus placing asserts within private methods tend to leave you wondering,
is the public method that invoked this private method buggy? Or is it
the
assert in the private method wrong?

Private methods, being private tend to be very fluid and change often,
thus putting an "anchor", a "stake in the ground", an "assert" in a
private method tends just to slow me down without much return.

So doing Test Driven development the stages (for me) are thus...

1) Write Test Driving code for Class A (This test code is very unstable
/
    immature / buggy).

2) Write class A API, put in a smattering of preconditions. These
    preconditions test the test. Run test (I run the tests at almost
every
    step)

3) Fix test / preconds.

4) Write postcondition assertions in Unit Test of class A.

5) They fail, so start writing code for class A.

6) Usually I think of some more preconditions that are in important
while
    writing so I might add one or two more.

7) Often the test still fails because I'm stupid. So I start printing
    things out. Ah. That value is wrong. How could it possibly be that?
    Start going back up the call chain and pushing in pre-cond asserts
in
    the code and post-cond tests in the unit test until my mistake is
obvious.

8) Tests all pass, the code is written. I understand what and why it
does,
    I write an invariant check method and slap calls to it in a bunch of
    places (start and end of most public methods).

9) Start on a class B that uses class A, now test A and class A are
    reasonably mature, test B and class B are very immature.

10) Write Test Driving code for Class B (This test code is very unstable
/
     immature / buggy).

11) Write class B API, put in a smattering of preconditions. These
     preconditions test the test.

12) Fix test B / preconds for B.

13) Write postcondition assertions in Unit Test B of class B.

14) They fail, so start writing code for class B.


>>>--WAKE UP! THIS IS THE IMPORTANT BIT!<<<
15) _Already_ the precond assertions within class A are starting to tell
     me about bugs in class B!

16) Often the test still fails because I'm stupid. So I start printing
    things out. Ah. That value is wrong. How could it possibly be that?
    Start going back up the call chain and pushing in both pre-cond
asserts
    in the code (in both classes) and post-cond asserts (in both tests)
    until my mistake is obvious.

17) I put in an invariant check method of class B, which invokes the
     invariant check method of class A.

18) By the time test B and class B is written and tests pass my pre-cond
     and post-cond coverage of class A is substantially better and
several
     bugs / design issues have been ousted from class A.

19) Start write class C that re-uses class A, class A is _very_ mature
     now, very trustworthy, and right from the first invocation the
     pre-cond and invariant checks in class A catch bugs in test C and
     class C very early.

20) Code is a little kludgy, so refactor and clean up. Unit Test's and
     assertions catch breakage. If functional tests catch breakage
     undetected by unit tests & precond, insert test code / precond /
unit
     test postcond until breakage is caught then fix it.

21) Code is feature complete, but slow. Too many checks. Run profile and
     weed out a few checks that are in inner loops. Code is now fast,
safe
     clean and bug free.


>> 1) The pre-condition assertions are "tests of the tests". Test's are code
>>     Precondition assert says the client code violated a pre-condition
>> any good reason to create a new exception class for them.
>>> What do other people think about the relationship between assertions
>>>>
>>
>>
>> From this principle, all of life and physics may be deduced.
>>
>>
>
>
> --
> -- Chiaroscuro --
> Liquid Development: http://feeds.feedburner.com/LiquidDevelopment
>



John C.                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : 
removed_email_address@domain.invalid
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
This topic is locked and can not be replied to.