I am new to Ruby. I am somewhat surprised that I was not able to find
much info on this question via google or searching of forums. I was
able to find a similar question posted a while back but no one answered
it. Perhaps threads aren’t such an important topic in Ruby culture?
Anyway, to the point. I’d like to use green threads (NOT kernel
threads) in a Ruby program I’m writing. There is no reason for me to
use kernel threads, and every reason to use green ones.
Are green threads still available? If so, how do I access/use them
(Thread class says nothing about whether it is green or kernel - I
suspect kernel)? If not, why not?
If not available, are there any alternatives? I checked into fibers,
but they are not appropriate for what I want because if an exception
happens, it happens in the “main thread” I am executing, which I don’t
want.
Are you saying that green threads are used regardless of what is
happening with kernel threads and the GIL? The link posted by fxn
seems to indicate this.
This looks like a mistake. Ruby 1.8 has green threads, Ruby 1.9 and
JRuby have native threads (though Ruby 1.9 has a GIL).
Are you saying that green threads are used regardless of what is
happening with kernel threads and the GIL? The link posted by fxn
seems to indicate this.
Are you saying that green threads are used regardless of what is
happening with kernel threads and the GIL? The thread posted by fxn
seems to indicate this.
I don’t know enough to give a definitive answer (or plainly put, to
answer), but here’s my interpretation of the post:
I believe the green threads in Ilya’s post are the Ruby-land thread
objects, they are not necessarily green in the sense of who is doing the
scheduling. This is my interpretation a posteriori because of the way
the
JVM is drawn.
They are definitely green threads in MRI 1.8 in the sense that MRI did
scheduling (you already know that).
Then, behind the scenes, they are mapped to native threads in MRI 1.9
and
JRuby. One-to-one. In the case of MRI there’s a GIL. I guess technically
a
GIL does not mean that MRI is doing the scheduling, it means MRI is
holding
a lock, but the kernel schedules the native thread (because it is
native).
Maybe there’s a gray area in the definition of green thread here, I
don’t
know.
That is my interpretation, but I don’t know enough to be certain. Please
someone correct me if that is wrong!
TCL
version of the same above, which I know uses kernel threads. It only
spawned a couple heavyweights per second.
I guess the results are indeterminate? Hopefully someone else will
chime in who knows definitively.
Ruby 1.9 uses kernel threads and does not have green threads.
There is nothing like Erlang’s lightweight processes in Ruby (or, for
that matter, in most of the languages).
Ruby 1.9 uses kernel threads and does not have green threads.
There is nothing like Erlang’s lightweight processes in Ruby (or, for
that matter, in most of the languages).
That is clear, but the definition of green thread depends on who
schedules
them.
Is the GIL acting as a scheduler? Has MRI logic about what to lock next
therefore scheduling in practice among the threads in its process? Or
is
my conjecture right that locking is mostly dumb and that ultimately the
kernel is the scheduler because the thread is native?
Is the GIL acting as a scheduler? Has MRI logic about what to lock
next
therefore scheduling in practice among the threads in its process?
Or is
my conjecture right that locking is mostly dumb and that ultimately
the
kernel is the scheduler because the thread is native?
The OP’s problem is not the scheduler itself but the associated system
resources.
You can create 20k Erlang processes (“threads”). You cannot create 20k
pthreads. Just that.
If jacques1 is right, then there are no green threads, just kernel
threads, and I’m out of luck (anyone know why they would do away with
green threads instead of keeping them and also adding kernel threads?).
I did a quick-and-dirty test here (ruby 1.9.3) to try to shed some
light. My assumption is that green threads should be incredibly cheap
to create (on the order of at least thousands per second), where as
kernel threads should be much, much slower.
(1…5000).each {Thread.new{}}
This takes around 12 seconds to execute on my machine. It seems far too
slow for green threads (I can create hundreds of thousands of green
processes in erlang per second). However, someone out there correct me
if I am wrong, but I would also expect kernel threads to be much slower
than the above, i.e. on the order of a few per second. I coded up a TCL
version of the same above, which I know uses kernel threads. It only
spawned a couple heavyweights per second.
I guess the results are indeterminate? Hopefully someone else will
chime in who knows definitively.
There is nothing like Erlang’s lightweight processes in Ruby (or, for
that matter, in most of the languages).
Well, for the purposes here, if Ruby’s threads were green threads, yes
they would have been comparable to Erlang green processes. My emphasis
is far more on the “green” and far less on whether it is a process or a
thread.
As far as the definition of green and whether it has anything to do with
a scheduler or not, my definition of green is simply whether or not it
is a ‘real’ thread or simply a virtual thread which lives inside of the
vm.
Well, for the purposes here, if Ruby’s threads were green threads, yes
they would have been comparable to Erlang green processes. My emphasis
is far more on the “green” and far less on whether it is a process or a
thread.
Ruby 1.8 has green threads. It’s explained in Xavier’s link, and I
also mentioned it above.
You seem to misread “some implementations have native threads” as “all
have native threads”.
Well, for the purposes here, if Ruby’s threads were green threads, yes
they would have been comparable to Erlang green processes. My emphasis
is far more on the “green” and far less on whether it is a process or a
thread.
Ruby 1.8 has green threads. It’s explained in Xavier’s link, and I
also mentioned it above.
You seem to misread “some implementations have native threads” as “all
have native threads”.
The title of this post is “green threads in 1.9.* ?” Furthermore, I
have this edit in one of my posts here:
EDIT: 1.9.x of course.
So I have misread nothing. I am ONLY talking about 1.9.*. I could
care less about 1.8 or any other implementation.
I’d like to use green threads (NOT kernel threads) in a Ruby program I’m
writing. There is no reason for me to
use kernel threads, and every reason to use green ones.
If you want to use 1.9, you’re out of luck. That said, you aren’t
stating
your reasoning for why you would prefer green threads over native ones.
Native threads have a longer creation time and use marginally more
memory
(they use ~20kB per thread last I measured), however they generally
perform
better, particularly around I/O. On platforms like Linux scheduling is
O(1)
so there’s really no reason you can’t create large numbers of threads.
There are definitely many reasons to prefer native threads over green
threads. This is the reason why you have seen a shift from green threads
to
native threads in MRI, Rubinius, and the JVM.
So I have misread nothing. I am ONLY talking about 1.9.*. I could
care less about 1.8 or any other implementation.
Couldn’t. Unless your level of caring is greater than your minimum. Or
you can have negative carativitiy.
I’m pretty sure Xavier (fxn) has answered this thread pretty well. My
question is: do you really care about the greenness of the threads
(academically), or do you actually want to know how best to utilise
them since they seem to have unusual profiling characteristics?
If the former, I don’t know what to suggest. Research, code scouring,
reverse engineering (or read the link posted earlier). If the latter,
I say explore and research, and if you find something that works best
for your application use it, and maybe share it with us in case we
need to do it again some day.
As an aside, and I’m not pointing the finger at anyone in this thread
when I say this, but a week or two ago I commented to a colleague of
mine about how polite the ruby-talk forum was, and how people seemed
to consciously avoid flaming and trolling, and how noobs got useful
help even when they asked “dumb noob” questions, and how people said
“thank you” . . . but in the past couple of days there’ve been a few
discussions (usually related to optimisation or performance) that have
developed a very negative tone, if not outright ALL CAPS TYPING and
rage. What’s going on? I want a return to the happy, friendly
ruby-talk of yesterweek.
Then, behind the scenes, they are mapped to native threads in MRI 1.9 and
JRuby. One-to-one. In the case of MRI there’s a GIL. I guess technically a
GIL does not mean that MRI is doing the scheduling, it means MRI is holding
a lock, but the kernel schedules the native thread (because it is native).
Maybe there’s a gray area in the definition of green thread here, I don’t
know.
MRI is still doing the scheduling in 1.9. It runs a timer thread that
interrupts running threads after a certain amount of time so waiting
threads can run. This way a single thread doing some computationally
intensive operation doesn’t monopolize the entire VM.
<<There are definitely many reasons to prefer native threads over green
threads. This is the reason why you have seen a shift from green threads
to
native threads in MRI, Rubinius, and the JVM.
Why do you want green threads?>>
There are many situations to prefer green threads over kernel
threads. It’s why I’m somewhat surprised that Ruby is just ditching
green threads, vs. keeping them and deploying kernel threads along side
them. Perhaps it’s a cultural thing. Rubyists might be more concerned
with things like web servers (Rails development) and what not, and
perhaps haven’t found want or need for green threads so much (just a
guess).
One reason to prefer green threads is if your application doesn’t need
speed up through concurrency/parallelism, rather you want a threaded
solution for algorithmic purposes, convenience purposes, or other
purposes. This is a crappy example, but say you need to do some
calculation which might fail or even crash, but you don’t want (or need)
to check the inputs to the calculation, or wrap the thing up in an error
catching mechanism, or anything of this nature. You just want to
calculate it, and if it works, fine, and if it doesn’t (or even crashes)
fine, your main program continues to run. So spawn a green thread to
run the calculation. If it works, great. If it crashes, who cares?
You might say “do that with a kernel thread.” But what if I want to
spawn this thread over and over again in a tight loop? I don’t want or
need the monstrous overhead of a kernel thread here. I just want a
green thread.
Again, there’s a whole host of situations where one might want to have
green threads vs. kernel threads. But I wasn’t here to debate that - if
Rubyists want to throw green threads in the garbage, that’s their
prerogative. I was just here to ask whether there were green threads or
not (in 1.9.*).