Forum: Ruby-core at_fork callback API

398856ea967f3cab2dbe3df99d732069?d=identicon&s=25 tmm1 (Aman Gupta) (Guest)
on 2013-11-30 22:55
(Received via mailing list)
Issue #5446 has been updated by tmm1 (Aman Gupta).


Simple implementation of an after_fork{} hook:
https://github.com/tmm1/ruby/commit/711a68b6599d17...

Any objection?
----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-43290

Author: normalperson (Eric Wong)
Status: Assigned
Priority: Low
Assignee: kosaki (Motohiro KOSAKI)
Category:
Target version: next minor


It would be good if Ruby provides an API for registering fork()
handlers.

This allows libraries to automatically and agnostically reinitialize
resources
such as open IO objects in child processes whenever fork() is called by
a user
application.  Use of this API by library authors will reduce
false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and
system()).

Ruby already has the internal support for most of this this to manage
mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.
I can
prepare a patch if an API is decided upon.
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2013-12-01 00:05
(Received via mailing list)
"tmm1 (Aman Gupta)" <ruby@tmm1.net> wrote:
> Simple implementation of an after_fork{} hook:
https://github.com/tmm1/ruby/commit/711a68b6599d17...
>
> Any objection?

I think there needs to be separate hooks for prepare/parent/child
(like pthread_atfork(3)).  The parent may need to release resources
before forking (prepare hook), and perhaps reacquire/reinitialize
them after forking (parent hook).

The prepare hook is important for things like DB connections;
the parent hook might be less useful (especially for apps which
fork repeatedly).
02da662c083396641da96c1d32fc86ed?d=identicon&s=25 KOSAKI Motohiro (Guest)
on 2013-12-05 23:38
(Received via mailing list)
2013/11/30 tmm1 (Aman Gupta) <ruby@tmm1.net>:
>
> Issue #5446 has been updated by tmm1 (Aman Gupta).
>
>
> Simple implementation of an after_fork{} hook:
https://github.com/tmm1/ruby/commit/711a68b6599d17...
>
> Any objection?

??!?!?

1. Why no before_fork?
2. zk has :after_child nad :after_parent hook and your patch don't. Why?
3. Why should the new method return proc?
4. When rb_daemon() is used, some platform call after_fork once and
    the other call twice. It seems useless.
5. Why do hook fire at rb_thread_atfork() make a lot of new array?
6. Your patch doesn't aim to remove the hooks and I'm sure it is
required.

You said the new method should be created for killing the gem specific
hooks.
But your patch seems not to be able to.
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2013-12-06 22:32
(Received via mailing list)
Eric Wong <normalperson@yhbt.net> wrote:
> the parent hook might be less useful (especially for apps which
> fork repeatedly).

I take that back.  It can be useful for for setting up pipes/sockets for
IPC
between parent and child.

Example without wrapper class and explicit IO#close:

    r, w = IO.pipe
    pid = fork do
      w.close # could be atfork_child
      ...
    end
    r.close # could be atfork_parent
    ...

However, I want to do this via callback, example with Worker class:

    class Worker
      attr_writer :pid

      def initialize
        @r, @w = IO.pipe
        Process.atfork_parent { @r.close unless @r.closed? }
        Process.atfork_child { @w.close unless @w.closed? }
      end
    end

    worker = Worker.new # IO.pipe
    worker.pid = fork { ... }
    ...

    # No need to remember what to close in parent/child
B11f10c4cd9d53970e7be20caa43f940?d=identicon&s=25 Tanaka Akira (Guest)
on 2013-12-07 05:42
(Received via mailing list)
2013/12/7 Eric Wong <normalperson@yhbt.net>:

>     end
>
>     worker = Worker.new # IO.pipe
>     worker.pid = fork { ... }
>     ...
>
>     # No need to remember what to close in parent/child

I think it doesn't work with multiple thread.

2.times {
  Thread.new {
    worker = Worker.new # IO.pipe
    worker.pid = fork { ... }
    ...
  }
}

If fork for worker 1 is called between IO.pipe and fork for worker 2,
pipes for worker 2 is leaked for the process for worker 1 and
not inherited to the process for worker 2.

I feel it is not a good example for this issue.
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2013-12-08 07:49
(Received via mailing list)
Tanaka Akira <akr@fsij.org> wrote:
> >       end
> >     end
> >
> >     worker = Worker.new # IO.pipe
> >     worker.pid = fork { ... }
> >     ...
> >
> >     # No need to remember what to close in parent/child
>
> I think it doesn't work with multiple thread.

True, but I wasn't intending this example to be used for an MT
application, but a single-threaded, multi-process HTTP server.

Generally, I do not use fork after I've spawned threads (unless
followed immediately with exec).
This topic is locked and can not be replied to.