No prob… but I’m not sure it’s quite what you’re looking for. I’m
refactoring code I can’t get to work instead of finding the root cause
in the threading. (BTW - any thread insight would be appreciated based
on this write-up - cuz there’s still problems! I’m starting to consider
that a Threaded Load Tester Gem might be handy…)
After staring at the screen for too long, I took a break to ponder
whether the organization of my threads was fundamentally flawed. It
occurred to me that one of the problems I was having - a deadlock -
could be avoided by instantiating the threads only when needed.
If you review my original code snippet, I spin up all the threads w/ a
proc object in initialize(), then .stop them, then perform via method
a .call on each element of the thread’s api array, then a .wakeup
followed by the .join.
…
1.upto @count do
@threads << Thread.new do
Thread.current[:hi5] = Hi5fbapi.new ‘redacted params’
Thread.current[“api_calls”] = []
apis.each do |api|
Thread.current[“api_calls”] << get_call(api) #pushes a proc obj
end
Thread.stop
end
end
…
@threads.each do |thr|
thr[:api_calls].each do |api|
p api.call thr[:hi5], @session
end
end
@threads.each {|t| t.wakeup.join}
…
On 1.8.7 w/ a single core processor, this is a highly deterministic
sequence, and would not deadlock. Once deployed to a multi-core VM
running 1.9.2-p0 (selected specifically for concurrency), not so much.
There I encountered more deadlocks and also "NoMethodError"s from
Sinatra (undefined method `bytesize’ for #<Thread:0xa36ae0c dead>).
This would occur during the .each where I would .join. So it’s trying
to join a ‘dead’ thread. Except that if I add anything to prevent that,
such as
… unless t.status == ‘dead’…
it would still deadlock or NoMethodError.
But further testing showed that adding any operation to the main thread,
prior to calling .join, would prevent the deadlock:
…
@threads.each do |thr|
p thr.inspect
…
Anyway, I rewrote things where the threads are created in the method,
not initialize(), so ‘@api_calls’ is already populated by procs, and it
works fine at small scale:
…
thr = []
1.upto @count do
thr << Thread.new do
@api_calls.each do |api|
p api.call @hi5, @session
Thread.pass
end
end
end
thr.each {|t| t.join}
…
This runs fine on 1.8.7/single and 1.9.2/multi at like 5-10 threads.
But when I ramp up to, say, 5,000 (it is a load test!), 1.8.7 is fine
but 1.9.2 segfaults.
Even 500 threads on 1.9.2 is segfaulting right now (but not 1.8.7). I
get the handy output:
[NOTE]
You may have encountered a bug in the Ruby interpreter or extension
libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html
I’ll write it up tomorrow.
Some open questions:
-
When .join is called on a multi-core system, what qualifies as the
calling thread? The .main on the main processor, or the thread which
instantiates my ThreadedLoadTester object? (i.e. what if
ThreadedLoadTester is created from a sinatra thread which itself isn’t
main?)
-
Sinatra regularly reports a ‘NoMethodError’ for ‘bytesize’ when the
last thread is dead but joined to the main thread. But only when the
main thread originates w/in sinatra, and not an inline call.
-
Is there a theoretical maximum to the number of concurrent threads
which can be created which all access a network interface? This is
admittedly a poor theory - what might really cause a segfault in 1.9.2
when 500 threads all try to access the network?
Thanks for asking 
-Alex
