Unusual challenges speccing external software

This is something I’ve come up against twice. First, I have an
interactive editor which calls vi, emacs, or TextMate (etc.) via Unix,
and then loads the file handed to the editor back into IRB after it’s
edited and saved. I guess that makes it a special case of a more
general question, which is how do you spec external processes, with a
slightly more complicated version of the standard answer, mocks.

The second instance is more complicated. I’m writing a MIDI code
generator and using it to drive Propellerhead Reason, which is
third-party closed-source commercial software which will run on my OS
X box but which is not actually transparent to Unix in any meaningful
way that I’m aware of. (The code generator can very probably also
drive any arbitrary MIDI consumer.) I’m using code from Topher Cyll’s
book “Practical Ruby Projects” to create Ruby wrappers to C methods in
Apple’s CoreMIDI library. The trap here is I’m calling those C methods
to generate MIDI elsewhere in the OS in some unknown way. That makes
speccing weird.

I’ve been able to refactor this code in a kind-of spec-first way. I
write scripts which use the API successfully and actually generate
music I can hear. Then I change the internals of the API and run the
scripts again to ensure they still generate the same music. In that
sense I’m basically doing the right thing, even though it’s not
particularly systematic or automated. I’m comparing the output with my
ears rather than in a way that code can grok, but it still has the
feel of a spec-first workflow.

For developing the API itself, especially the parts which generate
MIDI, I set up a layer of abstraction. Instead of directly driving the
code which sends MIDI to external software, the code which generates
the music drives an output adapter, and the output adapter sends MIDI
to external software - or, in the case of the TextOutputAdapter, just
text-dumps the music. Since it all comes back as text, it’s very easy
to write specs against.

But I’ve ended up with three artifacts that are either byproducts of
bad design or bad products of a compromised spec process. I created
three mock classes (as opposed to mock objects). For example, one is
the ArrayGenerator. In this system Generators return notes. The
ArrayGenerator “generates” an array of notes. It’s a silly
implementation, but it’s useful if you want to spec that some code can
take an arbitrary generator, which generates arbitrary notes, and then
process those notes correctly.

Sometimes I think my design has too many abstract parts. Sometimes I’m
happy with it. I want to improve my speccing approach so I can operate
with greater certainty. I think probably this approach is a good
direction, and that I should change the specs by making them more
specific. (For instance, there are places where I check that a
Generator subclass can do stuff, even though Generator itself isn’t
fully specced.) I’m not quite sure though.


Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

At the risk of sounding a bit silly, what’s your question? I couldn’t
find a question mark in the whole email…?

Daniel

At the risk of sounding a bit silly, what’s your question? I couldn’t
find a question mark in the whole email…?

OK, well, am I on the right track? Should I do it differently? If so,
in what particular way?

If you were speccing external software, would you do it this way? Do I
sound like a crazy person? If yes, more so than usual? Or only as much
as I usually do? Is this a sensible way to spec software which
interacts with but has no direct access to external software, or
should I just isolate my dependencies on external software and write
it off as unspeccable while speccing everything else? Is there a good
rule of thumb for a BDD workflow when you can’t programmatically
capture some of the output? Because that’s really what the problem is.
I built a whole set of spec objects to capture the output as text
because the system’s real output isn’t capturable, and if you can’t
capture it, you can’t check it against the spec. My code has flaws but
with good specs it’s pretty easy to isolate the flaws. In this case I
can’t tell yet if the flaws are just flaws in my weird semi-BDD
approach or in the code itself. I’ve probably done enough question
marks but basically, is there a good set of guiding principles I can
use to apply RSpec well in unconventional contexts?


Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

On Jan 17, 2008 10:24 AM, Giles B. [email protected] wrote:

At the risk of sounding a bit silly, what’s your question? I couldn’t
find a question mark in the whole email…?

OK, well, am I on the right track? Should I do it differently? If so,
in what particular way?

Giles - I think you’ll find more people with more experience in this
area on the [email protected] list. That said, and
only having glanced at your mail, here’s a thought or two:

The TDD line on this, as I understand it, is that you should write
your main application code exactly as you want to write it, imagining
that you had the perfect API to talk to and stubbing that API in your
tests. Then, when hooking up to the real services, you write thin
adapters that simply convert messages to the right format. I think
that’s what you said you are doing.

HTH,
David

Giles - I think you’ll find more people with more experience in this
area on the [email protected] list. That said, and
only having glanced at your mail, here’s a thought or two:

The TDD line on this, as I understand it, is that you should write
your main application code exactly as you want to write it, imagining
that you had the perfect API to talk to and stubbing that API in your
tests. Then, when hooking up to the real services, you write thin
adapters that simply convert messages to the right format. I think
that’s what you said you are doing.

Yes. Sorry about the Great American Novel - that is pretty much what I’m
doing.


Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

On Jan 17, 2008 11:33 AM, Giles B. [email protected] wrote:

Yes. Sorry about the Great American Novel - that is pretty much what I’m doing.

Then that’s sane :slight_smile:

It lets you keep changes to your internal app separate from changes to
the service - even allows you to easily swap in new services - and
lets you “talk in your language” within the app rather than being
subject to an API that may or may not make sense to you most of the
time.