Why UBF() is called even if Ruby traps signals?

Ruby
1

Hi, according to the doc of rb_thread_blocking_region() (or
rb_thread_call_without_gvl() which is the same):

  • If another thread interrupts this thread (Thread#kill, signal
    delivery,
  • VM-shutdown request, and so on), ubf()' is called (ubf()’ means
  • “un-blocking function”). ubf()' should interruptfunc()’
    execution.

Ok, in my case I’m coding a wrapper for UV (a C reactor library based
on EV when running on Linux). Let me explain a simple use case I’m
using for developming/testing purposes:

  • In my script I trap SIGTERM signal:

    trap(:TERM) { puts “SIGTERM signal ignored” }

  • I call to my C extension run() method which calls to:

    rb_thread_call_without_gvl(run_uv_without_gvl, NULL, my_ubf, NULL);

  • run_uv_without_gvl() basically runs uv_run() function of UV library.
    UV then blocks until events occur. In this experiment there are not
    loaded events.

  • So there is a single Ruby thread and now it’s blocked in run_uv().
    If there was another Ruby thread it would be running with the GVL, but
    that’s not the case.

  • Then I send a SIGTERM signal to the process. UV ignores signals by
    design, sure.

  • So the problem is that: at this moment I see my_ubf() funcion called
    !!! WHY? If I plan to trap/ignore SIGTERM signal, why is my_ubf()
    ALWAYS called? why should I “interrupt my blocking function” (uv_run)
    if I don’t want since SIGTERM will be ignored in Ruby land?

I hope the description of the issue is clear, if not please ask to me,
I’ve spent days with this stuff and my conclusion is that the UBF()
function is always called.

Thanks a lot.

2

2012/5/21 Iñaki Baz C. [email protected]:

  • So the problem is that: at this moment I see my_ubf() funcion called
    !!! WHY? If I plan to trap/ignore SIGTERM signal, why is my_ubf()
    ALWAYS called? why should I “interrupt my blocking function” (uv_run)
    if I don’t want since SIGTERM will be ignored in Ruby land?

After experiments, the ubf() is called in those cases:

  1. The thread running the blocking region is killed with Thread#kill.

  2. An interrupt for which trap(SIGNAL_NAME, “SIG_INT”) has NOT been
    set arrives while the process is blocked in the blocking region. If
    the signal has a handler in Ruby land then ubf() is always called(
    regardless the trap handler will terminate the program or not !!!).

Badly, in any of those cases, if
rb_thread_interrupted(rb_thread_current()) is called within the ubf()
function it returns false, so honestly I have no idea of how to react
when my ubf() is called. If it’s due to a signal I want it to be
treated in Ruby land. If it’s a Thread#kill then I need to do my stuff
within the ubf() function. I see no way.

3

Iñaki Baz C. [email protected] wrote:

  1. An interrupt for which trap(SIGNAL_NAME, “SIG_INT”) has NOT been
    set arrives while the process is blocked in the blocking region. If
    the signal has a handler in Ruby land then ubf() is always called(
    regardless the trap handler will terminate the program or not !!!).

Yes, otherwise for a single-threaded Ruby application, signal handlers
have only one thread to fire in (so it needs to interrupt uv_run()).

Given what you say about uv_run() not being interruptable by signals,
you should probably use the self-pipe trick documented here:
http://cr.yp.to/docs/selfpipe.html

MRI 1.9.3+ itself uses a self-pipe itself, as does unicorn.

Badly, in any of those cases, if
rb_thread_interrupted(rb_thread_current()) is called within the ubf()
function it returns false, so honestly I have no idea of how to react
when my ubf() is called. If it’s due to a signal I want it to be
treated in Ruby land. If it’s a Thread#kill then I need to do my stuff
within the ubf() function. I see no way.

With the self-pipe trick, your ubf() will probably just be:

/* write one \0 byte */
write(pipe_wfd, “”, 1);

Where the other end of that pipe is being watched by uv_run()
(installed like any other IO object you’re watching for).

4

2012/5/22 Eric W. [email protected]:

Iñaki Baz C. [email protected] wrote:

have only one thread to fire in (so it needs to interrupt uv_run()).
Therefore I cannot use the ubf() for nothing. Typicall my code will be
single thread (see explanation below please).

   write(pipe_wfd, "", 1);

Where the other end of that pipe is being watched by uv_run()
(installed like any other IO object you’re watching for).

But that’s not my problem. In fact, I have a mechanism to communicate
with UV loop at any time safely from any thread:

uv_async_send() (thread safe)

So from the ubf() function I have no problem in telling UV “please do
this now or in your next instant iteration”. My problem is: what to
say?

  1. If ubf() has been called from Thread#kill then I need to run a
    function that closes all the active UV handlers so uv_run() exits.

  2. If ubf() has been called due to a signal receipt, I don’t want
    uv_run() to exit. Instead the signal wakeups the UV loop and generates
    an iteration in which I call to rb_check_interrupts() for handling the
    signal in Ruby land and I’m happy. But the ubf() has been called !!!

So the problem is that when a signal is received, it makes the ubf()
function to be called and, at that point, I have no way to know,
within the ubf() function, whether it has been called by Thread#kill
or by a received signal.

Thanks a lot.

5

On Mon, May 21, 2012 at 4:47 PM, Eric W. [email protected]
wrote:

http://cr.yp.to/docs/selfpipe.html
With the self-pipe trick, your ubf() will probably just be:

   /* write one \0 byte */
   write(pipe_wfd, "", 1);

Where the other end of that pipe is being watched by uv_run()
(installed like any other IO object you’re watching for).

Or use pselect(2). Not sure how portable that is though.

Jos

6

Jos B. [email protected] wrote:

On Mon, May 21, 2012 at 4:47 PM, Eric W. [email protected] wrote:

With the self-pipe trick, your ubf() will probably just be:

Or use pselect(2). Not sure how portable that is though.

Iñaki’s using libuv/uv_run() as a wrapper to avoid using
epoll/poll/kqueue/select/etc… directly, so I don’t think pselect()
would work for him.

Normally, regular select() will just stop with EINTR and that’s all Ruby
needs to reacquire the GVL. IO.select internally retries on EINTR, but
only after running signal handlers and processing Thread#raise/#kill
requests.

7

Iñaki Baz C. [email protected] wrote:

But that’s not my problem. In fact, I have a mechanism to communicate
with UV loop at any time safely from any thread:

uv_async_send() (thread safe)

OK, perfect. You can probably just call in your ubf(), nothing else.

So from the ubf() function I have no problem in telling UV “please do
this now or in your next instant iteration”. My problem is: what to
say?

  1. If ubf() has been called from Thread#kill then I need to run a
    function that closes all the active UV handlers so uv_run() exits.

Can you run those cleanup handlers in the killer and not the killee?
You should Thread#join the killee, and then cleanup in the killer.

On the other hand, with uv_async_send(), you can probably avoid
Thread#kill entirely. Thread#kill can be dangerous/unreliable,
as ensure clauses will still fire, and ensure clauses can be
broken, too.

  1. If ubf() has been called due to a signal receipt, I don’t want
    uv_run() to exit. Instead the signal wakeups the UV loop and generates
    an iteration in which I call to rb_check_interrupts() for handling the
    signal in Ruby land and I’m happy. But the ubf() has been called !!!

So the problem is that when a signal is received, it makes the ubf()
function to be called and, at that point, I have no way to know,
within the ubf() function, whether it has been called by Thread#kill
or by a received signal.

You’re right, there’s no way for the ubf() to know why it’s called.

The purpose of the ubf() is only to wake up a thread and have it
reacquire the GVL. That’s it.

Once a thread reacquires the GVL, it can:

  • run Ruby signal handlers (automatically done by VM)
  • die (respond to Thread#kill, again automatically done by VM)
  • give up the GVL again and resume blocking (your choice)
8

2012/5/22 Eric W. [email protected]:

Iñaki Baz C. [email protected] wrote:

But that’s not my problem. In fact, I have a mechanism to communicate
with UV loop at any time safely from any thread:

uv_async_send() (thread safe)

OK, perfect. You can probably just call in your ubf(), nothing else.

Right, I’ve got it :slight_smile: (more below).

So from the ubf() function I have no problem in telling UV “please do
this now or in your next instant iteration”. My problem is: what to
say?

  1. If ubf() has been called from Thread#kill then I need to run a
    function that closes all the active UV handlers so uv_run() exits.

Can you run those cleanup handlers in the killer and not the killee?
You should Thread#join the killee, and then cleanup in the killer.

The problem is that I don’t know if my library user will do
Thread#kill or Thread#raise or whatever :slight_smile:
But don’t worry, Finally I’ve it working (more below).

On the other hand, with uv_async_send(), you can probably avoid
Thread#kill entirely. Thread#kill can be dangerous/unreliable,
as ensure clauses will still fire, and ensure clauses can be
broken, too.

I don’t want to use it, but the user could do it :slight_smile:

You’re right, there’s no way for the ubf() to know why it’s called.

The purpose of the ubf() is only to wake up a thread and have it
reacquire the GVL. That’s it.

Once a thread reacquires the GVL, it can:

  • run Ruby signal handlers (automatically done by VM)
  • die (respond to Thread#kill, again automatically done by VM)
  • give up the GVL again and resume blocking (your choice)

Perfect. Once this is fully understood by me I’ve redesigned my code
so when my ubf() function is called:

  • I don’t know if it has been called due to a signal or due to a
    Thread#kill, so in my ubf() function I just create a uv_async handle
    (thread safe), and send it to the UV reactor.

  • The reactor is then wakeup, acquires the GVL and runs
    rb_check_interrupts(), so in case I’ve received a signal it is handled
    in Ruby land.

And just it :slight_smile:

In the other side, now I don’t assume that my ubf() must terminate my
blocking function (uv_run) so what I do is basically:

def MyLibrary.run

@running = true

uv_run_terminated_by_itself = false
begin
uv_run()
uv_run_terminated_by_itself = true
ensure
@running = false
unless uv_run_terminated_by_itself
destroy_handles()
end
end

And that’s all. The ensure block is executed in any case (due to a
signal not trapped by Ruby, a Thread#kill, …).

:slight_smile: