# Run test scripts concurrently

**URL:** <https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512>\
**Category:** Ruby\
**Created:** [October 25, 2006, 12:07am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512 "2006-10-25T00:07:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 12:07am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/1 "2006-10-25T00:07:17Z")

</div>

I’ve been futzing with this for some time and I am out of ideas.  
I have a number of files named “test\_something.rb” that each contain  
one test like so:

class TC\_foo \< Test::Unit::TestCase  
def test\_foo  
#do some stuff  
#assert some things  
end  
end

I have a little harness that runs them one after another like so:

topdir = File.join(File.dirname( **FILE** ))  
Dir.chdir topdir do  
tests = Dir[“test\*”]  
tests.each{|x| require x}  
end

But I would like to be able to run each individual script file in it’s  
own process simultaneously. I have been fooling around with  
Thread.new, system(), exec(), and it seems that no matter what I try,  
each script file has to finish before the next one will run.

Any suggestions for running all of my test files at the same time from  
one controller/harness?

---

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 12:31am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/2 "2006-10-25T00:31:20Z")

</div>

On Wed, 25 Oct 2006, Chris McMahon wrote:

> Thread.new, system(), exec(), and it seems that no matter what I try,  
> each script file has to finish before the next one will run.
> 
> Any suggestions for running all of my test files at the same time from  
> one controller/harness?

threads = []

Dir.chdir topdir do  
tests = Dir[“test\*”]  
tests.each{|x| threads \<\< Thread.new{ system x or raise x } }  
end

threads.each{|t| t.join}

-a

---

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 2:11am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/3 "2006-10-25T02:11:30Z")

</div>

removed\_email\_address@domain.invalid wrote:

> threads.each{|t| t.join}

In my expereince, if you join all the spawned threads to the current  
thread,  
this guarantees that they will run sequentially. They will certainly  
run,  
which is a good thing, but they won’t run concurrently.

BTW I am not saying this about threads in general, I am saying it about  
Ruby  
threads.

---

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 2:39am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/4 "2006-10-25T02:39:34Z")

</div>

Paul L. wrote:

> removed\_email\_address@domain.invalid wrote:
> 
> > threads.each{|t| t.join}
> 
> In my expereince, if you join all the spawned threads to the current thread,  
> this guarantees that they will run sequentially. They will certainly run,  
> which is a good thing, but they won’t run concurrently.
> 
> BTW I am not saying this about threads in general, I am saying it about Ruby  
> threads.

This seems concurrent to me, or did you mean something different:

threads = (0…9).map {|i| Thread.new {sleep rand(5); puts i}}  
threads.each {|t| t.join}

**END**

Output:

4  
8  
9  
0  
7  
5  
2  
1  
6  
3

---

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 8:25am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/5 "2006-10-25T08:25:56Z")

</div>

Joel VanderWerf wrote:

> > Ruby threads.  
> > 4  
> > 8  
> > 9  
> > 0  
> > 7  
> > 5  
> > 2  
> > 1  
> > 6  
> > 3

I tried to think of an explanation apart from the obvious one, but it  
seems  
I am wrong. I think the fact that they are all sleeping their time away  
may  
partly explain this outcome, but the threads are clearly running  
concurrently.

Threads that are not sleeping may not live up to the promise of this  
example. My threads tend not to run concurrently, but your example is an  
excellent refutation.

---

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 8:48pm UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/6 "2006-10-25T20:48:29Z")

</div>

Charles Oliver N. wrote:

> > excellent refutation.
> 
> Threads that are 100% Ruby code will yield and timeslice correctly.  
> However they still will never run concurrently, other than timeslicing.  
> Threads that make system calls will run sequentially, since system calls  
> can’t be scheduled by Ruby’s thread scheduler. If you have a thread make  
> a system call, that call must complete before the thread will yield.

That’s true at the C API level, but not always true at the Ruby API  
level, as you say…

> There’s some trickery with IO in some cases, but for the general case  
> this is how it works. You may try JRuby, which has fully concurrent  
> native thread support, but not everything in normal Ruby is 100%  
> supported yet…and Kernel#system isn’t quite perfect yet.

Is select() really trickery? Anyway, in addition to the IO trickery,  
there is also a concurrent #system, as this example shows:

$ cat x.rb  
t = Thread.new do  
system “sleep 1; echo SYSTEM; sleep 1; echo SYSTEM; sleep 1; echo  
SYSTEM”  
end

3.times do  
sleep 1  
puts “RUBY”  
end

t.join

$ ruby x.rb  
RUBY  
SYSTEM  
RUBY  
SYSTEM  
RUBY  
SYSTEM

For some purposes (networking) ruby’s threads are pretty good, to a  
point. For some other purposes, we can use #fork plus drb. There are  
cases where one is SOL though.

We’re all looking forward to native threads in future ruby VMs and  
JRuby.

---

<div class="post-metadata">

**Author:** ![Chris\_McMahon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/74df32/32.png) [@Chris\_McMahon](https://www.ruby-forum.com/u/Chris_McMahon)\
**Post date:** [October 25, 2006, 9:06am UTC](https://www.ruby-forum.com/t/run-test-scripts-concurrently/73512/7 "2006-10-25T09:06:22Z")

</div>

Paul L. wrote:

> I tried to think of an explanation apart from the obvious one, but it seems  
> I am wrong. I think the fact that they are all sleeping their time away may  
> partly explain this outcome, but the threads are clearly running  
> concurrently.
> 
> Threads that are not sleeping may not live up to the promise of this  
> example. My threads tend not to run concurrently, but your example is an  
> excellent refutation.

Threads that are 100% Ruby code will yield and timeslice correctly.  
However they still will never run concurrently, other than timeslicing.  
Threads that make system calls will run sequentially, since system calls  
can’t be scheduled by Ruby’s thread scheduler. If you have a thread make  
a system call, that call must complete before the thread will yield.

There’s some trickery with IO in some cases, but for the general case  
this is how it works. You may try JRuby, which has fully concurrent  
native thread support, but not everything in normal Ruby is 100%  
supported yet…and Kernel#system isn’t quite perfect yet.