Wrong: an introspecting assert library

For WhyDay, I took the time to clean up and document Wrong, a project
Steve C. and I have been working on. It’s inspired by assert{2.0}
(http://assert2.rubyforge.org/) but rewritten from scratch. Major
kudos to Phlip for starting us down this path with assert{2.0}, and
to Ryan D. for his RubyParser and Ruby2Ruby libraries which made
it technically possible.

Before I say more, let me first say that this is totally not ready
yet! We would love to get some users, but there are some big caveats,
and there are plenty of things left to be done to make the results
look uniformly clean and beautiful. We want your feedback, and
especially to give us cases where either it blows up or the output is
ugly or uninformative.

Wrong provides a general assert method that takes any Ruby block.
Assertion failure messages are rich in detail. The Wrong idea is to
replace all those countless assert_this, assert_that library methods
which only exist to give a more useful failure message than “assertion
failed”. Wrong replaces all of them in one fell swoop, since if you
can write it in Ruby, Wrong can make a sensible failure message out of
it.

Examples:

require “wrong”
include Wrong::Assert

assert {1==1}
==> nil

assert {2==1}
==> Expected (2 == 1), but 2 is not equal to 1

x = 7; y = 10; assert { x == 7 && y == 11 }
==>
Expected ((x == 7) and (y == 11)), but
(x == 7) is true
x is 7
(y == 11) is false
y is 10

assert { ‘hand’.include?(‘bird’) }
==>
Expected “hand”.include?(“bird”), but “hand” does not include “bird”

age = 24
name = “Gaga”
assert { age >= 18 && [“Britney”, “Snooki”].include?(name) }
==>
Expected ((age >= 18) and [“Britney”, “Snooki”].include?(name)), but
(age >= 18) is true
age is 24
[“Britney”, “Snooki”].include?(name) is false
name is “Gaga”

So how do we do it? Doesn’t Ruby have poor support for AST
introspection (see
http://blog.zenspider.com/2009/04/parsetree-eol.html)? Well, yes, it
does, so we cheat: we figure out what file and line the assert block
is defined in, then open the file, read the code, and parse it
directly using Ryan D.’ amazing RubyParser and Ruby2Ruby. You can
bask in the kludge by examining chunk.rb and assert.rb. If you
find some code it can’t parse, please send it our way.

Before you get your knickers in a twist about how this is totally
unacceptable because it doesn’t support this or that use case, here
are our caveats and excuses:

  • It works! Tested in 1.8.6, 1.8.7, 1.9.1, and 1.9.2-rc2. (Thank you,
    rvm!)
  • Your code needs to be in a file. That means it doesn’t work in IRB.
    (If you’re developing Ruby code without saving it to a mounted disk,
    then sorry, Wrong is not right for you.)
  • It’s a development-time testing library, not a production runtime
    library, so there are no security or filesystem issues.
  • eval isn’t evil, it’s just misunderstood.
  • It makes a few assumptions about the structure of your code, leading
    to some restrictions:
  • You can’t have more than one call to assert per line. (This
    should not be a problem since even if you’re nesting asserts for some
    bizarre reason, we assume you know where your Return key is. And
    actually, technically you can put two asserts on a line, but it always
    describes the first one it sees, which means that if the second one
    executes, its failure message will be incorrect or broken.)
  • You can’t use metaprogramming to write your assert blocks.
  • All variables and methods must be available in the binding of the
    assertion block.

Please feel free to challenge any of the above claims in a reply :-).
I’d love to know if Ryan and the rest of the ruby-talk gang think this
is an acceptable or an outrageous use of a parsing library. In any
event, I think it constitutes a pretty compelling use case for adding
Proc#to_sexp or equivalent to the MRI runtime.

The gem’s been released on RubyGems, so “gem install wrong” should
work, but you might be better off following my github project.

  • Alex

On 3 September 2010 22:42, Alex C. [email protected] wrote:

For WhyDay, I took the time to clean up and document Wrong, a project
Steve C. and I have been working on. It’s inspired by assert{2.0}
(http://assert2.rubyforge.org/) but rewritten from scratch. Major
kudos to Phlip for starting us down this path with assert{2.0}, and
to Ryan D. for his RubyParser and Ruby2Ruby libraries which made
it technically possible.

require “wrong”
include Wrong::Assert

assert {1==1}
==> nil

assert {2==1}
==> Expected (2 == 1), but 2 is not equal to 1

x = 7; y = 10; assert { x == 7 && y == 11 }
==>
Expected ((x == 7) and (y == 11)), but
(x == 7) is true
x is 7
(y == 11) is false
y is 10

  • Alex

Neat ! I am indeed surprised you made possible to parse 1.9.

The output is still a bit looking like a stack trace of an error, but
the idea is excellent.

Thanks for this cool whyday code, you played well with a limit of the
language.

B.D.

On Sep 4, 2010, at 06:49 , Benoit D. wrote:

Neat ! I am indeed surprised you made possible to parse 1.9.

He prolly didn’t… It still prolly won’t parse 1.9isms, but luckily
those usually aren’t used in tests.

On Sep 3, 2010, at 13:42 , Alex C. wrote:

So how do we do it? Doesn’t Ruby have poor support for AST
introspection (see
http://blog.zenspider.com/2009/04/parsetree-eol.html)? Well, yes, it
does, so we cheat: we figure out what file and line the assert block
is defined in, then open the file, read the code, and parse it
directly using Ryan D.’ amazing RubyParser and Ruby2Ruby. You can
bask in the kludge by examining chunk.rb and assert.rb. If you
find some code it can’t parse, please send it our way.

HAHAHA!.. you SICK SICK BASTARDS… HAHAHAHA!

good work!

Please feel free to challenge any of the above claims in a reply :-).
I’d love to know if Ryan and the rest of the ruby-talk gang think this
is an acceptable or an outrageous use of a parsing library. In any
event, I think it constitutes a pretty compelling use case for adding
Proc#to_sexp or equivalent to the MRI runtime.

Are you kidding? that output is freakin’ beautiful!

On 4 September 2010 23:51, Ryan D. [email protected] wrote:

On Sep 4, 2010, at 06:49 , Benoit D. wrote:

Neat ! I am indeed surprised you made possible to parse 1.9.

He prolly didn’t… It still prolly won’t parse 1.9isms, but luckily those usually aren’t used in tests.

Ah, indeed.
But as it needs only to parse the body of the assert/deny block, it
will likely be rarely a big problem.

On Fri, Sep 3, 2010 at 2:28 PM, Ryan D. [email protected]
wrote:

HAHAHA!.. you SICK SICK BASTARDS… HAHAHAHA!

good work!

blush<

You just made my day.

Doesn’t Ruby have poor support for AST
introspection (see
http://blog.zenspider.com/2009/04/parsetree-eol.html)? Well, yes, it
does, so we cheat: we figure out what file and line the assert block
is defined in, then open the file, read the code, and parse it
directly

Hey, I just had a thought – maybe this technique should be called
extrospection.

Alex C. - [email protected] - http://alexch.github.com
Stalk me: Redirecting... | http://twitter.com/alexch |
http://alexch.tumblr.com

On Sep 3, 2:28 pm, Ryan D. [email protected] wrote:

HAHAHA!.. you SICK SICK BASTARDS… HAHAHAHA!

good work!

Ahem. My version doesn’t need to rip the source… C-: