# Ruby Test Frameworks

**URL:** <https://www.ruby-forum.com/t/ruby-test-frameworks/155224>\
**Category:** Ruby\
**Created:** [December 9, 2008, 4:38pm UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224 "2008-12-09T16:38:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mifi](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/7993a0/32.png) [@mifi](https://www.ruby-forum.com/u/mifi)\
**Post date:** [December 9, 2008, 4:38pm UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224/1 "2008-12-09T16:38:47Z")

</div>

I’m looking for a Ruby Framework that I can use to aid in testing a  
filter driver, we currently use a Perl based set of scripts that has  
some wrapping and does system events for files, directories and  
registry changes (it’s multi-platform for Windows and \*nixes).

Ideally, I want to start capturing data in a database and give a nice  
simple framework around it, which is what led me to Rails and from that  
to Ruby - and I thought being able to make our Harness more OO to allow  
for more reuse and remove duplication (which there is a lot of in our  
scripts). Plus one language to handle test results and the test harness  
seemed more efficient. Most of the Ruby Test Frameworks that I have  
looked through are all Web or GUI test frameworks, Test::Unit is nice  
but I would like to know if there is a Framework for application and  
system testing that I have missed. Has anyone used a framework for  
application and system testing they could recommend, or even an existing  
one that they have used successfully?

Some of my initial requirements are:

- Runs on Windows and Linux
- Supports calling external code/scripts, we have Java and C code that  
is called for memory utilization and socket testing
- Can generate system events for create/modify/delete of files and  
directories
- Has some hooks into Windows Registry, IIS and Linux RPM (Nice to  
haves but not required)
- Can integrate with an IDE (Visual Studio would be nice)

Any help is appreciated.

Thanks!

---

<div class="post-metadata">

**Author:** ![mifi](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/7993a0/32.png) [@mifi](https://www.ruby-forum.com/u/mifi)\
**Post date:** [December 9, 2008, 4:49pm UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224/2 "2008-12-09T16:49:29Z")

</div>

Michael Furmaniuk wrote:

> Most of the Ruby Test Frameworks that I have  
> looked through are all Web or GUI test frameworks, Test::Unit is nice  
> but I would like to know if there is a Framework for application and  
> system testing that I have missed. Has anyone used a framework for  
> application and system testing they could recommend, or even an existing  
> one that they have used successfully?

I don’t know of anything specific to ‘application and system testing’.  
If I were you, I’d start with Test::Unit or its bigger brothers Shoulda  
or Rspec, and see if it does what you actually need.

> Some of my initial requirements are:
> 
> - Runs on Windows and Linux
> - Supports calling external code/scripts, we have Java and C code that  
> is called for memory utilization and socket testing

You can call external programs using backticks, system() or IO.popen:  
e.g.

def test\_foo  
result = `foo`  
assert\_equal 0, $? # exit code  
assert\_match /success/, result # output on stdout  
end

You can call actual C using RubyInline or FFI.

> - Can generate system events for create/modify/delete of files and  
> directories

You can do those actions using FileUtils. Or do you mean you want to be  
_notified_ of those sorts of events happening in the filesystem? I’m not  
sure about that.

> - Has some hooks into Windows Registry, IIS and Linux RPM (Nice to  
> haves but not required)

Not sure, I believe there is Win32 API (but I’m a Linux person).

> - Can integrate with an IDE (Visual Studio would be nice)

I think Test::Unit has some graphical runners (but I’m a CLI person 🙂

---

<div class="post-metadata">

**Author:** ![mifi](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/7993a0/32.png) [@mifi](https://www.ruby-forum.com/u/mifi)\
**Post date:** [December 9, 2008, 9:59pm UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224/3 "2008-12-09T21:59:12Z")

</div>

On Tue, Dec 9, 2008 at 3:42 PM, Brian C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> or Rspec, and see if it does what you actually need.  
> For the simple stuff, dust is my tool of choice. Its a very simple  
> wrapper around  
> Test::Unit, so there is a shallow learning curve, but you get  
> meaningful test names,  
> and it is very easy to generate large blocks of tests from arrays of  
> inputs. Handy  
> for black box testing, testing regexes etc.

Shoulda and RSpec I think are tackling different needs. The learning  
curve is greater  
and there is more magic involved, but I suspect that test  
organisation, and consolidation  
of reporting is probably a greater priority than natural language test  
description.

> assert\_equal 0, $? # exit code  
> assert\_match /success/, result # output on stdout  
> end
> 
> You can call actual C using RubyInline or FFI.

In addition, check out JtestR for plain old Java testing (it works  
great).

> Not sure, I believe there is Win32 API (but I’m a Linux person).
> 
> > - Can integrate with an IDE (Visual Studio would be nice)

Theres a team integrating Ruby into visual studio, I can’t recall  
offhand.

> I think Test::Unit has some graphical runners (but I’m a CLI person 🙂

I haven’t found a good solution for out of the box test reporting in  
Ruby.  
RSpec has some great reporting, but I need something a bit more turnkey.

I currently use dust to test java code through JtestR on top of an  
existing  
Java build system (so our test reporting is done through JUnit reports  
and  
cobertura works for our code coverage of the java code).

---

<div class="post-metadata">

**Author:** ![mifi](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/7993a0/32.png) [@mifi](https://www.ruby-forum.com/u/mifi)\
**Post date:** [December 9, 2008, 11:27pm UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224/4 "2008-12-09T23:27:51Z")

</div>

Richard C. wrote:

> Shoulda and RSpec I think are tackling different needs. The learning  
> curve is greater  
> and there is more magic involved, but I suspect that test  
> organisation, and consolidation  
> of reporting is probably a greater priority than natural language test  
> description.

For Rspec I agree - I found it too contrived - but shoulda fitted my  
needs much better. It gives you the meaningful test names, and also  
setup/teardown contexts which avoids having to make a new class for  
every set of fixtures. But you keep good old-fashioned assertions at the  
bottom.

class Foo \< Test::Unit::TestCase  
context “Piping in FOO” do  
setup do  
File.delete(“output.txt”)  
@res = `echo "FOO" | ./wibble`  
end

```
should "generate output BAR" do
  assert_match /BAR/, @res
end

should "create file output.txt" do
  assert File.exist?("output.txt")
end

# etc

```

end  
end

---

<div class="post-metadata">

**Author:** ![mifi](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/7993a0/32.png) [@mifi](https://www.ruby-forum.com/u/mifi)\
**Post date:** [December 10, 2008, 12:42am UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224/5 "2008-12-10T00:42:40Z")

</div>

Richard C. wrote:

> On Tue, Dec 9, 2008 at 3:42 PM, Brian C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
> wrote:  
> Shoulda and RSpec I think are tackling different needs. The learning  
> curve is greater  
> and there is more magic involved, but I suspect that test  
> organisation, and consolidation  
> of reporting is probably a greater priority than natural language test  
> description.

Yes, natural language test description is nice, but if it comes at the  
expense of reporting and organization I can deal with the odd  
description. I had looked a little at RSpec but not in depth, and  
Shoulda I saw referenced in a page somewhere but it did not seem a fit  
for what I was looking for, I will check them more in depth.

> > You can call actual C using RubyInline or FFI.
> 
> In addition, check out JtestR for plain old Java testing (it works  
> great).

Thanks for these, and the other ideas, I will check them out!

---

<div class="post-metadata">

**Author:** ![mifi](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/7993a0/32.png) [@mifi](https://www.ruby-forum.com/u/mifi)\
**Post date:** [December 10, 2008, 1:08pm UTC](https://www.ruby-forum.com/t/ruby-test-frameworks/155224/6 "2008-12-10T13:08:36Z")

</div>

I’m guessing probably the IronRuby guys are the ones getting Visual  
Studio working with Ruby. 😛

– Yanik Magnan • [http://r-ch.net](http://r-ch.net)