I’ve built a little thread pool library which creates a bunch of threads
using Thread.new and has them being blocked on a ruby Queue via
Queue#pop (or deq).
I was hoping someone could verify that I’m fully using the jvm threading
system when I use jruby to run this. Whilst the answer might be an
obvious ‘yes’, I’m not sure if there are any snags or gotchas or issues
etc
Also noticed that when I encountered a few deadlock errors whilst
building a join function for the threadpool, jruby just hung whereas mri
ruby 1.8 and 1.9 terminated with fatal deadlock errors.
On Sat, Oct 10, 2009 at 10:20 PM, Daniel B. [email protected]
wrote:
I’ve built a little thread pool library which creates a bunch of threads
using Thread.new and has them being blocked on a ruby Queue via
Queue#pop (or deq).
I was hoping someone could verify that I’m fully using the jvm threading
system when I use jruby to run this. Â Whilst the answer might be an
obvious ‘yes’, I’m not sure if there are any snags or gotchas or issues
etc
The answer is ‘yes’, and I can’t think of any gotchas. The deadlocking
thing you mention below is an obvious one…
Also noticed that when I encountered a few deadlock errors whilst
building a join function for the threadpool, jruby just hung whereas mri
ruby 1.8 and 1.9 terminated with fatal deadlock errors.
Because the JVM is more than just the few threads you’ve started for
it, there’s always a chance another thread will be started or some
external input will cause a thread to be interrupted. 1.8/1.9 are able
to detect deadlocks because they actually do know there’s only X
number of threads and no new ones will be started, so they raise an
error.
The JVM will raise deadlock errors in some cases, but usually only
when it’s an obvious race between two threads. We can look into any
ways to improve this, but in most cases (and this is not unique to
JRuby) you’ll need to be diligent about managing locks and ensuring
lock ordering.
Oh, and I must inform you about the most useful key combination for
threaded development on the JVM: Ctrl+\ (or Ctrl+Break on windows).
This key combination dumps the current execution stack of all threads
in the system. It also initiates the deadlock checker I mentioned in
my other email. That deadlock checker is described briefly here:
And there’s sample output here:
Obviously deadlocks are program flaws, so if you use this tool when
they happen you should be able to eventually eliminate them all.
On Sat, Oct 10, 2009 at 10:20 PM, Daniel B. [email protected]
wrote:
The answer is ‘yes’, and I can’t think of any gotchas. The deadlocking
thing you mention below is an obvious one…
Awesome!
Also noticed that when I encountered a few deadlock errors whilst
building a join function for the threadpool, jruby just hung whereas mri
ruby 1.8 and 1.9 terminated with fatal deadlock errors.
Because the JVM is more than just the few threads you’ve started for
it, there’s always a chance another thread will be started or some
external input will cause a thread to be interrupted. 1.8/1.9 are able
to detect deadlocks because they actually do know there’s only X
number of threads and no new ones will be started, so they raise an
error.
The JVM will raise deadlock errors in some cases, but usually only
when it’s an obvious race between two threads. We can look into any
ways to improve this, but in most cases (and this is not unique to
JRuby) you’ll need to be diligent about managing locks and ensuring
lock ordering.
Thanks Charlie, appreciate the response (and uhm… jruby for that
matter).
Oh, and I must inform you about the most useful key combination for
threaded development on the JVM: Ctrl+\ (or Ctrl+Break on windows).
Funnily enough I did hit that key after having exhausted the control-c
option and got a whole bunch of stuff that scared me. From vague memory
it sometimes has a darker purpose on unix systems.
This key combination dumps the current execution stack of all threads
in the system. It also initiates the deadlock checker I mentioned in
my other email. That deadlock checker is described briefly here:
On the Java side, there is a nice book I refer to for concurrency
audits almost daily, Java Concurrency In Practice. It covers the
issues of multiple-core systems and how they are guaranteed to behave
under the ‘new’ Java Memory Model in modern JVMs. It also introduces
an annotation system which lets you make assertions about intended
concurrency behavior (@ThreadSafe, @GuardedBy and such), which allows
FindBugs and other static analysis tools to tell when something isn’t
as safe as you think. Since I started using it religiously, I haven’t
had a single concurrency bug pop up in my production Java code.
Since that kind of static analysis isn’t really feasible in a dynamic
language, I’ve been having a lot of trouble running down concurrency
issues in other people’s dynamic codebases. I hadn’t touched Ctrl-
swizzle for years until I started working a lot with dynamic languages
on the JVM!
Despite a dynamic language culture committed to testing, there seems
to be a lot of finger crossing when it comes to concurrency issues -
they’re hard to test.
Scala’s Actor model takes away most of the d’oh pitfalls by using
messages instead of shared objects and locks. Is there a good Actor
type framework that plays well with JRuby that I can canonize?
Thanks for the source link, Daniel, I’ll have a look today!
This key combination dumps the current execution stack of all threads
they happen you should be able to eventually eliminate them all.
To unsubscribe from this list, please visit:
Despite a dynamic language culture committed to testing, there seems to be a
lot of finger crossing when it comes to concurrency issues - they’re hard to
test.
Well, I think this is in large part because all the mainstream
dynlangs haven’t had actual concurrently-executing threads. I believe
it was Steven P. who said something like “if you’re not testing
your threading code on JRuby, you’re not testing your threading code.”
JRuby has ironically forced Rubyists to think about threading and
shared-memory concurrency for the first time.
Scala’s Actor model takes away most of the d’oh pitfalls by using messages
instead of shared objects and locks. Â Is there a good Actor type framework
that plays well with JRuby that I can canonize?
On the Java side, there is a nice book I refer to for concurrency
audits almost daily, Java Concurrency In Practice. It covers the
issues of multiple-core systems and how they are guaranteed to behave
under the ‘new’ Java Memory Model in modern JVMs. It also introduces
an annotation system which lets you make assertions about intended
concurrency behavior (@ThreadSafe, @GuardedBy and such), which allows
FindBugs and other static analysis tools to tell when something isn’t
as safe as you think. Since I started using it religiously, I haven’t
had a single concurrency bug pop up in my production Java code.
Since that kind of static analysis isn’t really feasible in a dynamic
language, I’ve been having a lot of trouble running down concurrency
issues in other people’s dynamic codebases. I hadn’t touched Ctrl-
swizzle for years until I started working a lot with dynamic languages
on the JVM!
Despite a dynamic language culture committed to testing, there seems
to be a lot of finger crossing when it comes to concurrency issues -
they’re hard to test.
Scala’s Actor model takes away most of the d’oh pitfalls by using
messages instead of shared objects and locks. Is there a good Actor
type framework that plays well with JRuby that I can canonize?
Thanks for the source link, Daniel, I’ll have a look today!
I was reading the wikipedia article on the Grand Central Dispatch
pattern and I’m interested in being able to combine something like this
with rack and jruby - haven’t got round to working out if it makes sense
though.
On the whole actor model thing I was also interested in erlang . I know
people are combining ruby with erlang but haven’t looked into that…
There was a good stack overflow interview (episode 59; on ‘IT
conversations’) where Damien Katz (of couchdb fame) was talking about
why he chose erlang (memory fragmentation problem, concurrency and error
handling etc).