On Saturday 09 August 2008 23:26:28 Ryan D. wrote:
On Aug 9, 2008, at 06:42 , Farrel L. wrote:
Rubinius and JRuby use a shared RSpec suite which they use to test
compatiability. It’s not the Official Spec™ but I think it’s the
closest to one.
that is so far from an actual specification it isn’t funny.
Erm, how? Is there something particularly deficient about the actual
specs?
Or are you just uncomfortable with having a test suite be a
specification?
If it’s the latter, read on.
My thoughts: If you’ve ever filed a decent bug report, you’ve already
done
this. Even for an “enhancement”, even for older trackers like Bugzilla,
there’s an expectation that you’ll write something like this:
Steps to reproduce:
- Log in
- Go to your profile
- Click on ‘foo’
Expected result:
It should ‘bar’ when I click ‘foo’.
Actual result:
I got an “Internal Server Error: 500” error. Log pasted below.
Now, actually parse that out. Sounds exactly like a regression test, to
me. “Steps to reproduce” are steps to setup the test. “Expected result”
is
your assertions.
Again, who says it has to be a bug? Test-driven development suggests
that you
should code by writing the test first, then fix it. Shouldn’t the same
hold
true for enhancements, also? All that would change is that “Expected
result”
would be some feature that doesn’t exist yet.
Behavior-driven development (behaviour-driven.org) and supporting
software
(rspec.info) only make this relationship more explicit. Given a
properly-formatted feature specification (similar to above), you can
actually
generate skeleton integration tests (called “stories”).
It works the other way around, too – Rspec will generate
English-language
versions of its tests, which can be used as a specification.
When you think about it, any open standard should have at least an open
implementation. If it is meant to allow multiple implementations, it
should
also have an automated test suite – for the Web, we have
validator.w3.org,
as well as the Acid2 (and now Acid3) tests, as an easy way to measure a
particular implementation’s adherence to the standard.
We all agree that a suite of compatibility tests is a good thing, right?
Here are those links again, in case your client didn’t make them into
links:
http://behavior-driven.org