Verify, a very basic testing tool

[flatekmonkey]

Hi List

The Verify tool is a KISS verification and test tool, it is inspired
by Ara T. Howard’s testy.rb.
Ara however has still implemented a BDD framework, the purpose of
Verify is to provide a Ruby P.mer
with a very simple yet effective tool to test her assumptions about
code.

Verify was motivated by a reply to Ara’s release post made by Phlip.
Phlip has made an excellent point about assert.
And that very often that is (almost?) all what is needed.
The Verify tool tries to implement exactly that idea and is the
simplest possible tool to do the job (for me).

Verify has the following goals

• Almost nothing to learn, one wrapper method and three test
  methods with almost no parameters all save one
  optional.

  At the end a resume of the verifications with an error message

for each failed verification
# will be printed to stderr or a mocking object.
# Strings and Arrays are fine for that purpose, as the reporter
uses #<< exclusively.

• No metaprogramming, no at_exit hook, execution of the
  verification inside a wrapper object, no global namespace pollution,
  except the Verify method.

• Wrapper behavior inference to the testee is limited to #verify,
  #verify_not and #verify_exceptions.

• Very simple output

• “Last Line Says It All” feature “================” → all tests
  ok, “***************” → not all tests ok.

USAGE EXAMPLES:

    require 'verify'

    Verify do
      verify do 42 end
      verify_not do nil end
      verify_exceptions NameError do abcdefghijk end
    end

    Verify "that the universe is ok" do
      verify "this only shows in error reports" do
        1 / 0
      end
    end

LICENCE:
BSD

Enjoy
Robert

Ah yeah BTW if someone really wants to download this

http://rubyforge.org/frs/?group_id=3824&release_id=30476

[flatekmonkey]

On Apr 3, 2009, at 13:02 , Robert D. wrote:

   Verify do
     verify do 42 end
     verify_not do nil end
     verify_exceptions NameError do abcdefghijk end
   end

refute: same length as verify!
raises: ditto!
(assert: I like assert)

[flatekmonkey]

On Fri, Apr 3, 2009 at 10:56 PM, Ryan D. [email protected]
wrote:

On Apr 3, 2009, at 13:02 , Robert D. wrote:

  Verify do
    verify do 42 end
    verify_not do nil end
    verify_exceptions NameError do abcdefghijk end
  end

refute: same length as verify!
Never heard of that word before, just checked it on Websters, thanx
Ryan great idea.
raises: ditto!
(assert: I like assert)
So do I, but assert is normally inserted into production code, and
Verify is still a testing tool, but you know it is so easy to alias
the methods in the anonymous module that creates the wrapper object in
the Verify method ;).
Cheers
Robert

[flatekmonkey]

Thanks to Ryan I have changed #verify_not to #refute.
But not #alert and #raises because I am afraid that these method names
might shadow too much testee code :(.

Cheers
Robert

http://rubyforge.org/frs/?group_id=3824 (verify-0.1a)

[flatekmonkey]

Robert D. wrote:

oops I forgot something important
Verify is Ruby1.9 only, do not look back ;).
R.

Damn. I love this sudden burst of creativity around the topic of
testing. A Good Thing. Would love to try this new thing of yours.
Already successfully using Ara’s.

But this 1.9 nonsense frustrates me. Much as I want to dive in (it’s
installed and ready to go), I simply don’t have time to move code I
depend upon into an environment where I don’t know whether it will fly
or not, or whether it will splat after 3 days of investment and use. I
wish there was somewhere I could go to find out what gems run with 1.9
and what do not. But, that seems not to be the ruby way. That’s plain
crazy, say I.

Maybe in a year it’ll be different. Meanwhile, 1.87 works for me. Sad.

t.

–

Tom C., MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< [email protected] >> (email)
<< TomCloyd.com >> (website)
<< sleightmind.wordpress.com >> (mental health weblog)

[flatekmonkey]

oops I forgot something important
Verify is Ruby1.9 only, do not look back ;).
R.

[flatekmonkey]

Tom C. wrote:

Damn. I love this sudden burst of creativity around the topic of
testing. A Good Thing.

Oookay. Here’s a sneak preview of assert{ 2.0 } 0.4.8.

You know how Ajax works by generating JavaScript, and slinging it at
your web
browser? And you know how Rails purportedly tests it with “assert_rjs”?
Here’s a
sample:

 assert_rjs :replace_html, "advanced_filter", ""

Too cute, right?

Wrong! That expands to nothing but a big Regexp, like
/Element.update.*advanced_filter/. So a payload of “advanced_filter”, or
even a
subsequent Element.update(‘advanced_filter’), could fool it.

Further, at work we do a lot of in-house Ajax, so we are at liberty to
render
entire partials at whim. We require our teeming minions to use only
Firefox. But
all assert_rjs does with its third argument is drop it into
assert_match. That
is not powerful enough to constrain our apps!

Just now while writing this post, I got Aaron Paterson’s rKelly working
in an
assert_rjs clone. rKelly uses racc to parse and evaluate JavaScript.
This
matches our goal of unit testing soft targets. Watir, Selenium, etc.
are all
great, and they introduced a generation to testing in general. Buuuuuut
they
work thru the browser. We are not inventing Ajax itself; we just need to
accurately spot-check that our own data go into the correct slots in our
JavaScript payloads.

So here’s a test that simulates a Rails functional test with xhr :get :

 @response = OpenStruct.new(:body => "Element.update(\"label_7\", 

“<input
checked=\“checked\” id=\“Top_Ranking\”
name=\“Top_Ranking\” type=\“checkbox\”
value=\“Y\”
\/>I want a pet < than a chihuahua<input
id=\“cross_sale_1\”
name=\“cross_sale_1\” type=\“hidden\” value=\“7\”
\/>”);")

 assert_rjs :replace_html, :label_7

K, so far that looks like the original assert_rjs. But under the hood,
it
actually lexed the Element.update() call:

 ast.pointcut('Element.update()').matches.each do |updater|
   updater.grep(RKelly::Nodes::ArgumentsNode).each do |thang|
     div_id, html = thang.value

     if target and html
       div_id = eval(div_id.value)
       html   = eval(html.value)
       if div_id == target.to_s
         assert_match matcher, html

(Open question to Aaron - is that the best way to run the query?)

The test actually determines we really got hold of the
Element.update(‘label_7’,
…). No other JavaScript line will match.

Here’s the assertions to match the text payload:

 assert_rjs :replace_html, :label_7, /Top_Ranking/
 assert_rjs :replace_html, :label_7, /pet < than a chihuahua/

Ho hum; so far assert_rjs Classic could have done all that. But…

Because I have an exact string, not a rough match, I can now treat it as
pure
HTML, and I can drop it into the mighty assert_xhtml()! Now the
assertion looks
like this:

 assert_rjs :replace_html, :label_7 do
   input.Top_Ranking! :type => :checked, :value => :Y
   input.cross_sale_1, :type => :hidden, :value => 7
 end

From here, no matter how complex that rendered partial, the assertion
can keep
up with it, and help make it safe to refactor and upgrade.

BTW Verify and Testy can get on board if they A> import
Test::Unit::Assertions
(like certain other test rigs we could mention should), and B> implement
flunk(). Both of those are all a custom assertion should ever need…

[flatekmonkey]

On Sat, Apr 4, 2009 at 1:14 AM, Tom C. [email protected] wrote:

Robert D. wrote:

But this 1.9 nonsense frustrates me. Much as I want to dive in (it’s
installed and ready to go), I simply don’t have time to move code I depend
upon into an environment where I don’t know whether it will fly or not, or
whether it will splat after 3 days of investment and use. I wish there was
somewhere I could go to find out what gems run with 1.9 and what do not.
But, that seems not to be the ruby way. That’s plain crazy, say I.
Maybe, indeed I had the very strong intention to write Lab419 to make
my code Ruby1.9 and 1.8 compatible. Too much work? Not enough time? I
dunno, finally I got tired of it and wanted my closure based Verify to
work. This goes without pain in 1.9.
But I agree with what you say.

R.

[flatekmonkey]

hi tom!

Tom C. [2009-04-04 01:14]:

I wish there was somewhere I could go to find out what gems run
with 1.9 and what do not.
there is: http://isitruby19.com/

cheers
jens

[flatekmonkey]

Without proper announce, I’d like to show that I had something similar
for quite some time now, as part of my knock project:

% cat t.tb
require ‘knock’
include Knock

testing “various expectances” do
expect(42) { 6 * 7 }
expect(nil) { nil }
expect(NameError) { abcdefghijk }

is 42, 67
ok 42 == 67
end

testing “that the universe is ok” do
testing “this only shows in error reports” do
1 / 0
end
end

% ruby t.rb
ok - various expectances - expecting value 42
ok - various expectances - expecting value nil
ok - various expectances - expecting exception NameError
ok - various expectances
ok - various expectances
not ok - /Users/chris/mess/current/t.rb:14 - that the universe is ok -
this only shows in error reports - RAISED: divided by 0

/Users/chris/mess/current/t.rb:15:in `/’

/Users/chris/mess/current/t.rb:15

/Users/chris/mess/current/t.rb:14

/Users/chris/mess/current/t.rb:13

% ruby t.rb | kn-agg
not ok - 6 tests, 1 failed (83.3333% succeeded)

The expect API is a rip-off from http://expectations.rubyforge.org/,
but simplified to its core:

% wc -l knock.rb kn-agg
61 knock.rb
10 kn-agg

Sources can be found at

GitHub - leahneukirchen/knock: Knock is a simplification of the Test Anything...

There are implementations for C, Ruby, sh/bash/zsh, and it is trivial
to implement in any language that supports “puts” in some way.

[flatekmonkey]

Robert D. wrote:

Maybe, indeed I had the very strong intention to write Lab419 to make
my code Ruby1.9 and 1.8 compatible. Too much work? Not enough time? I
dunno, finally I got tired of it and wanted my closure based Verify to
work. This goes without pain in 1.9.
But I agree with what you say.

Here’s a snip of my Rakefile, tuned for TDD:

task :default do
sh ‘ruby186 test/assert2_rjs_suite.rb’
sh ‘ruby187 test/assert2_rjs_suite.rb’
sh ‘ruby190 test/assert2_rjs_suite.rb’
sh ‘ruby191 test/assert2_rjs_suite.rb’

sh ‘ruby186 test/rubynode_reflector_suite.rb’
sh ‘ruby187 test/rubynode_reflector_suite.rb’
sh ‘ruby190 test/ripper_reflector_suite.rb’
sh ‘ruby191 test/ripper_reflector_suite.rb’

sh ‘ruby186 test/assert2_suite.rb’
sh ‘ruby187 test/assert2_suite.rb’
sh ‘ruby190 test/assert2_suite.rb’
sh ‘ruby191 test/assert2_suite.rb’
…

I’m implying that, after every couple of edits, I run every test in
every Ruby.
This system works great to avoid the “Too much work? Not enough time?”
quandary.

Actually, it doesn’t. I lied. Most of those are commented out. I cannot
figure
out how to get my peesashit Ubuntu to install all of those at the same
time, and
I otherwise don’t want to give up on the package manager. Debian’s
packaging for
Ruby sure sucks, huh?

[flatekmonkey]

Robert D. wrote:

This is a very valuable and helpful thing, but you still got lots of
work to do for making these tests pass if you are a closure maniac
like me :(.

in theory, if it hurts, do it more often. If I wrote all those tests and
code
after setting up the Rakefile to hammer several Ruby versions, then I
simply
don’t notice the overhead. If something breaks I fix it or revert and
try again.

You seem to think of adding the Ruby versions last…

[flatekmonkey]

On Sat, Apr 4, 2009 at 2:49 PM, Phlip [email protected] wrote:

Robert D. wrote:

sh ‘ruby187 test/rubynode_reflector_suite.rb’
sh ‘ruby190 test/ripper_reflector_suite.rb’
sh ‘ruby191 test/ripper_reflector_suite.rb’

sh ‘ruby186 test/assert2_suite.rb’
sh ‘ruby187 test/assert2_suite.rb’
sh ‘ruby190 test/assert2_suite.rb’
sh ‘ruby191 test/assert2_suite.rb’
…

This is a very valuable and helpful thing, but you still got lots of
work to do for making these tests pass if you are a closure maniac
like me :(.
R.

[flatekmonkey]

Jens W. wrote:

jens

Oh double flippin’ damn! That’s exactly what I had in mind. Ya just
gotta love this list. Ask and it shall be given, I may yet get religion.
Failing that…my pathetic code running wholly on Ruby 1.9, after which
I do lust.

Thank you Jens. You da man.

And thanks to all others for an entertaining thread (except Philip, that
“sneak preview of assert{ 2.0 } 0.4.8.” stiff is so far beyond me it was
easily this week’s personal low spot! However, knowing your intense
cleverness, I’m sure it is doing a number of far more rubyish souls a
ton of good. You’re a major force in the universe, as I’ve long known.)

t.

t.

–

Tom C., MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< [email protected] >> (email)
<< TomCloyd.com >> (website)
<< sleightmind.wordpress.com >> (mental health weblog)

[flatekmonkey]

Not to be contentious, but you use both define_method and *_eval to
generate code. Unless I don’t know what that term means, I believe
you do use metaprogramming.

Though, I do have a question/concern/confusion/something. I
completely and utterly fail to see the point of these new closure
obsessed frameworks or tools. Nearly every bit of their functionality
could be wrapped up in a few simple helpers like this:

class Test::Unit::TestCase
alias old_assert assert

def assert(result = nil, &block)
result = block.call if block_given?
old_assert result
end
end

Unless I’m totally missing something? I realize it’s a different
syntax or “DSL,” and that’s fine if that’s your reasoning, but I don’t
see how they’re all that revolutionary. I like closures as much as
the next guy, but I don’t see how they revolutionize my testing life.

Aimed much less at you, and more at the 4-5 of these I’ve seen in the
past few days.

–Jeremy

On Fri, Apr 3, 2009 at 3:02 PM, Robert D. [email protected]
wrote:

And that very often that is (almost?) all what is needed.
# will be printed to stderr or a mocking object.

     verify do 42 end

LICENCE:

There are some people who begin the Zoo at the beginning, called
WAYIN, and walk as quickly as they can past every cage until they get
to the one called WAYOUT, but the nicest people go straight to the
animal they love the most, and stay there. ~ A.A. Milne (from
Winnie-the-Pooh)

–
http://jeremymcanally.com/
http://entp.com/

Omg GIFs | GIFDB.com

My books:

Ruby in Practice

http://humblelittlerubybook.com/ (FREE!)

[flatekmonkey]

Jens W. wrote:

jens

Oh double flippin’ damn! That’s exactly what I had in mind. Ya just
gotta love this list. Ask and it shall be given, I may yet get religion.
Failing that…my pathetic code running wholly on Ruby 1.9, after which
I do lust.

Thank you Jens. You da man.

And thanks to all others for an entertaining thread (except Philip, that
“sneak preview of assert{ 2.0 } 0.4.8.” stiff is so far beyond me it was
easily this week’s personal low spot! However, knowing your intense
cleverness, I’m sure it is doing a number of far more rubyish souls a
ton of good. You’re a major force in the universe, as I’ve long known.)

t.

t.

–

Tom C., MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< [email protected] >> (email)
<< TomCloyd.com >> (website)
<< sleightmind.wordpress.com >> (mental health weblog)

[flatekmonkey]

Jeremy McAnally wrote:

Though, I do have a question/concern/confusion/something. I
completely and utterly fail to see the point of these new closure
obsessed frameworks or tools.

Expensive setup is a design smell.

That said, sometimes your setups are expensive. Consider this outline:

describe ‘frob_controller’ do

 setup do
   @p_frobs = assemble_some_paid_up_frobs()
 end

 specify ... specify ... specify 'behaviors of paid-up frobs'

 describe 'delinquent frobs'
   setup do
     @d_frobs = assemble_some_delinquent_frobs()
   end

   specify 'how paid-up and delinquent frobs interact' do
     mess_with_both @p_frobs, @d_frobs
   end
 end

end

The first setup builds ideal frobs, and the specify blocks (elided) test
them.

The second setup is nested, so its specifications run with both setups.
The
nested specify blocks can use both kinds of frobs, together. You get
this by
directly illustrating how the concepts nest. You don’t need to write
each setup
once and then somehow re-use it into various test suites.

A ‘def test_frob’ is a legacy of Java and its block-free programming
styles. Yet
a test case is not a method. You should not want to call it.

Treating blocks of testing code as objects makes weaving them together
easier.
Contrast the “abstract test pattern”, which Test::Unit::TestCase makes
an
absolute pain in the 'nads. It’s probably just as much fun in Java…

[flatekmonkey]

On Apr 4, 2009, at 05:49 , Phlip wrote:

I’m implying that, after every couple of edits, I run every test in
every Ruby. This system works great to avoid the “Too much work? Not
enough time?” quandary.

Actually, it doesn’t. I lied. Most of those are commented out. I
cannot figure out how to get my peesashit Ubuntu to install all of
those at the same time, and I otherwise don’t want to give up on the
package manager. Debian’s packaging for Ruby sure sucks, huh?

sudo gem install ZenTest
multiruby_setup the_usual

task :default do
sh ‘multiruby test/assert2_rjs_suite.rb’

sh ‘multiruby test/rubynode_reflector_suite.rb’

sh ‘multiruby test/assert2_suite.rb’
…

[flatekmonkey]

Robert D. wrote:

Ryan great idea.
Drat, I didn’t explain it well, and a quick check shows that refute is
“to disprove or to prove false”, according to most dictionaries. I guess
I just don’t like it as an opposite of “assert”. All the thesauruses
I’ve checked list the antonyms of “assert” as “deny” and “reject”, and
the antonyms of “refute” as “demonstrate”, “prove”, “accept”, or
“embrace”. There’s a disconnect there I can’t seem to remedy.

• Charlie

[flatekmonkey]

Robert D. wrote:

Ryan great idea.
I still contend that “refute” means to prove something is not true, not
to prove it false. refute do false end is meaningless.

• Charlie