# Optimizing testing

**URL:** <https://www.ruby-forum.com/t/optimizing-testing/115955>\
**Category:** Ruby\
**Created:** [September 30, 2007, 11:44pm UTC](https://www.ruby-forum.com/t/optimizing-testing/115955 "2007-09-30T23:44:10Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [September 30, 2007, 11:44pm UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/1 "2007-09-30T23:44:10Z")

</div>

I have a rake task that lunches all my unit tests. The execution of  
those tests may take over 20 mins. I would like to have two rake task,  
one that executes a set of quick tests and one that executes all the  
tests that will be executed in a nightly build.

For the moment I create a test suite that run all the TestCase methods  
that its name finishes by \_long.

Exemple:  
class MyTest \< Test::Unit::TestCase

# This test is executed only in the nightly build

def test\_one\_long  
sleep(3600)  
end

# This test will be executed with the quick test task and in the

nightly build  
def test\_two  
sleep(0.4)  
end  
end

Does anybody have a better idea? Is there some tool that can optimize  
the tests covering ?

---

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [October 1, 2007, 12:20am UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/2 "2007-10-01T00:20:27Z")

</div>

Martin M. wrote:

> I have a rake task that lunches all my unit tests. The execution of  
> those tests may take over 20 mins. I would like to have two rake task,  
> one that executes a set of quick tests and one that executes all the  
> tests that will be executed in a nightly build.

> For the moment I create a test suite that run all the TestCase methods  
> that its name finishes by \_long

To run only the long ones, I’d do this:

rake test TESTOPTS=-n/\_long$/

Note that’s selects for long cases, and we need another Regexp that only  
selects fast tests. One bug in all Regexps is they are infernally hard  
to  
negate. Regexps rely on their host language to call !~, instead of  
simply  
providing a negator. So you could put together some negator for \_long$  
from  
primitives, but I suspect you can’t just call something simple, like  
!(\_long$). I hope to be corrected here!

Next, the point of test _suites_ (the things that inherit \<  
Test::Unit::TestCase) is to categorize your cases by their test  
situations.  
_Not_ necessarily to categorize them by their targets. You should put  
all  
the long tests into a big suite, SlowSuite, and then flag it like this:

if ENV.has\_key? ‘slow’  
class SlowSuite \< Test::Unit::TestCase  
…  
end  
end

Then you just call ‘slow=true rake test’, or ‘rake slow=true test’, to  
also  
get the slow ones.

Next, long tests are a symptom of insufficient decoupling. If your  
tests, in  
TDD terms, are helping force your dependencies apart, then new tests  
should  
usually be narrower and faster than old ones. Not flabbier and slower. I  
get  
back to you when I fully obey this advice myself!

---

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [October 1, 2007, 9:05am UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/3 "2007-10-01T09:05:51Z")

</div>

On Sep 30, 2007, at 14:44 , Martin M. wrote:

> I have a rake task that lunches all my unit tests. The execution of  
> those tests may take over 20 mins. I would like to have two rake  
> task,  
> one that executes a set of quick tests and one that executes all the  
> tests that will be executed in a nightly build.

If your tests take twenty minutes, they aren’t unit tests. Unit  
tests are fast because they only test a unit.

What is it that makes them take 20 minutes or more?

---

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [October 1, 2007, 2:59am UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/4 "2007-10-01T02:59:43Z")

</div>

Thank you for your advices, it will be of a great help.

---

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [October 1, 2007, 2:11pm UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/5 "2007-10-01T14:11:06Z")

</div>

Eric H. wrote:

> On Sep 30, 2007, at 14:44 , Martin M. wrote:
> 
> > I have a rake task that lunches all my unit tests. The execution of  
> > those tests may take over 20 mins. I would like to have two rake  
> > task,  
> > one that executes a set of quick tests and one that executes all the  
> > tests that will be executed in a nightly build.
> 
> If your tests take twenty minutes, they aren’t unit tests. Unit  
> tests are fast because they only test a unit.
> 
> What is it that makes them take 20 minutes or more?  
> In fact I use unit testing to test a Ruby wapper over a C++ tool that  
> that takes a long time to execute. In fact it is not for testing the C++  
> app but to test the wrapper.

I also use it to test our VC++ build and packaging system. Even if I  
made small unit test, the whole process takes up to 13 mins.

---

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [October 1, 2007, 10:59pm UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/6 "2007-10-01T22:59:06Z")

</div>

On Oct 1, 2007, at 05:11 , Martin M. wrote:

> > tests are fast because they only test a unit.
> > 
> > What is it that makes them take 20 minutes or more?  
> > In fact I use unit testing to test a Ruby wapper over a C++ tool that  
> > that takes a long time to execute. In fact it is not for testing  
> > the C++  
> > app but to test the wrapper.
> 
> I also use it to test our VC++ build and packaging system. Even if I  
> made small unit test, the whole process takes up to 13 mins.

If you only want to test the wrapper you should stub out the C++  
app’s behaviors and use those stubs instead of the real thing. You  
don’t specify if this is a command-line wrapper or a ruby extension,  
the former will be mork work than the latter 😕

For a full integration test you can use the same tests with the real  
app (which will take the full amount of time).

---

<div class="post-metadata">

**Author:** ![martinos](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/57b2e6/32.png) [@martinos](https://www.ruby-forum.com/u/martinos)\
**Post date:** [October 1, 2007, 11:25pm UTC](https://www.ruby-forum.com/t/optimizing-testing/115955/7 "2007-10-01T23:25:37Z")

</div>

Eric H. wrote:

> If you only want to test the wrapper you should stub out the C++  
> app’s behaviors and use those stubs instead of the real thing. You  
> don’t specify if this is a command-line wrapper or a ruby extension,  
> the former will be mork work than the latter 😕
> 
> For a full integration test you can use the same tests with the real  
> app (which will take the full amount of time).

In fact it is a command line application. I think it is a good idea to  
stub it. I think I will do a small ruby script that generates an output  
that is similar to c++ application.

Martin