Forum: Ruby-core mvm branch status?

18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2011-04-07 21:58
(Received via mailing list)
Hello, I noticed the "mvm" branch in SVN hasn't been updated in over a
year.
Is it abandoned or moved somewhere else?  Thanks.
9d2f78236e45a335301ba1195026105d?d=identicon&s=25 Urabe Shyouhei (Guest)
on 2011-04-07 23:11
(Received via mailing list)
Hi Eric.

(2011年04月08日 04:49), Eric Wong wrote:
> Hello, I noticed the "mvm" branch in SVN hasn't been updated in over a year.
> Is it abandoned or moved somewhere else?  Thanks.

I have its successor rebased onto 1.9.2 branch.  So it might be
"somewhere
else" pattern.  Not sure if mine is the canonical one though.
540cb3b3712ffe045113cb03bab616a2?d=identicon&s=25 Evan Phoenix (Guest)
on 2011-04-07 23:38
(Received via mailing list)
Has there been any thought on solving the C extension problem in MVM? In
the present state, I've stopped working on it in Rubinius because there
is no workable solution if there are C extensions in the mix.

--
Evan Phoenix // evan@fallingsnow.net
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2011-04-08 01:19
(Received via mailing list)
Evan Phoenix <evan@fallingsnow.net> wrote:
> Has there been any thought on solving the C extension problem in MVM?
> In the present state, I've stopped working on it in Rubinius because
> there is no workable solution if there are C extensions in the mix.

Is it just avoiding redundant load of dynamic libs and remembering to
call Init_foo() on a per-VM basis?

I suppose the other problem is C-level global variables and
rb_global_variable(), but not all extensions use those...

I think a useful subset of C extensions can be made to work with MVM,
but I haven't been thinking of this problem for long (only since reading
your email :)

Perhaps you or someone can elaborate or point me to a writeup on the
problem(s).
F1d37642fdaa1662ff46e4c65731e9ab?d=identicon&s=25 Charles Nutter (headius)
on 2011-04-11 01:00
(Received via mailing list)
On Thu, Apr 7, 2011 at 6:18 PM, Eric Wong <normalperson@yhbt.net> wrote:
>
> I think a useful subset of C extensions can be made to work with MVM,
> but I haven't been thinking of this problem for long (only since reading
> your email :)
>
> Perhaps you or someone can elaborate or point me to a writeup on the
> problem(s).

Almost all extensions store classes, symbols, constant values, and
more in C globals. If they do that, they can't be made safe with MVM,
since those globals all indirectly reference VM-specific data. And
since we can't determine that they do this on load, ko1 originally
thought to introduce a new Init entry point that's a "promise" by the
extension that it doesn't do MVM-unsafe things. I believe they had a
few extensions working that way when the primary MVM dev push tailed
off.

Of course we haven't even started to discuss concurrency with C
extensions, which is damn near impossible to do safely.

In JRuby, where we've had MVM support forever, C extensions will error
out almost immediately if loaded into a second VM. Different VMs can
load different extensions, but the same extension loaded twice will
not work.

We also have a global lock around extension calls, so only one can be
executing at a given time.

I strongly believe Ruby needs to have its C API overhauled, or at
least someone needs to seriously explore what APIs can be safe with
MVM and which APIs need fixing/replacing.

FWIW, none of this applies to JRuby extensions written in Java; they
are bound per VM and do not use any "global" state.

- Charlie
9d2f78236e45a335301ba1195026105d?d=identicon&s=25 Urabe Shyouhei (Guest)
on 2011-04-11 06:28
(Received via mailing list)
Hi,

(04/11/2011 08:00 AM), Charles Oliver Nutter wrote:
>> rb_global_variable(), but not all extensions use those...
> since those globals all indirectly reference VM-specific data. And
> out almost immediately if loaded into a second VM. Different VMs can
> FWIW, none of this applies to JRuby extensions written in Java; they
> are bound per VM and do not use any "global" state.
>
> - Charlie

C extension APIs are old (design of 1990s).  At least some kind of
binary
compatibility breakage is unavoidable for Ruby to introduce MVM.  On my
branch, old style C extensions still work as long as you don't touch any
MVM
features.  But once your ruby script creates a new VM, that VM cannot
touch or
see those already-required old extensions.  To require a C extension
form
inside that VM, that library should use different APIs (MVM-safe set) as
Charles said.  I think this is a reasonable trade-off.

What C APIs are safe to use with MVM is not fully certain yet.  For
instance
there are process-global states other than C global variables like
current
directory, signal masks, ... We are still exploring.
F1d37642fdaa1662ff46e4c65731e9ab?d=identicon&s=25 Charles Nutter (headius)
on 2011-04-12 23:57
(Received via mailing list)
On Sun, Apr 10, 2011 at 11:28 PM, Urabe Shyouhei
<shyouhei@ruby-lang.org> wrote:
> What C APIs are safe to use with MVM is not fully certain yet.  For instance
> there are process-global states other than C global variables like current
> directory, signal masks, ... We are still exploring.

For current directory, JRuby maintains that separately and any calls
that are sensitive to cwd are modified to use this per-runtime value.
Because we share the JVM, we can't safely chdir the entire process, so
this was really the only way.

I'm not sure how MRI would do the same, since it is much more
dependent on low-level C APIs and their expectation of per-process
cwd.

~/projects/jruby ➔ jirb
>> Dir.chdir 'src'
=> 0
>> Dir.pwd
=> "/Users/headius/projects/jruby/src"
>> require 'java'
=> true
>> mvm = org.jruby.Ruby.new_instance
=> #<Java::OrgJruby::Ruby:0x2c7ac5>
>> mvm.eval_scriptlet('Dir.pwd')
=> "/Users/headius/projects/jruby"
>> mvm.eval_scriptlet('Dir.chdir "src"')
=> 0
>> Dir.chdir '..'
=> 0
>> mvm.eval_scriptlet('Dir.pwd')
=> "/Users/headius/projects/jruby/src"
>> Dir.pwd
=> "/Users/headius/projects/jruby"
>>


- Charlie
18813f71506ebad74179bf8c5a136696?d=identicon&s=25 Eric Wong (Guest)
on 2011-04-13 00:38
(Received via mailing list)
Charles Oliver Nutter <headius@headius.com> wrote:
> I'm not sure how MRI would do the same, since it is much more
> dependent on low-level C APIs and their expectation of per-process
> cwd.

POSIX.1-2008 has openat(), renameat(), chownat(), etc.., which can be
used for implementing per-thread/VM working directories.  It might take
a while for more platforms to support them, but Linux has had them
for a few years.
6c06915d9aa00cd5c7d4acfb27cdd4e9?d=identicon&s=25 Hiroshi Nakamura (Guest)
on 2011-04-14 06:57
(Received via mailing list)
Hi,

On Wed, Apr 13, 2011 at 06:56, Charles Oliver Nutter
<headius@headius.com> wrote:
> On Sun, Apr 10, 2011 at 11:28 PM, Urabe Shyouhei <shyouhei@ruby-lang.org> wrote:
>> What C APIs are safe to use with MVM is not fully certain yet. For instance
>> there are process-global states other than C global variables like current
>> directory, signal masks, ... We are still exploring.
>
> For current directory, JRuby maintains that separately and any calls
> that are sensitive to cwd are modified to use this per-runtime value.
> Because we share the JVM, we can't safely chdir the entire process, so
> this was really the only way.

Shyouhei's branch works as the same.
% ./ruby-mvm -e 'p Dir.pwd; 2.times { RubyVM.new("ruby", "-e",
"Dir.chdir %q(.ext); p Dir.pwd").start }; sleep 1'
"/home/nahi/git/shyouhei-ruby/build"
"/home/nahi/git/shyouhei-ruby/build/.ext"
"/home/nahi/git/shyouhei-ruby/build/.ext"

https://github.com/shyouhei/ruby/tree/shyouhei%2Fm...

Shyouhei's branch introduced per VM initialization (InitVM_*) and some
ext/* extensions changed to use it. The remaining issue is that there
should be more exts which depends on C's static thing. libssl has
global error stack so ext/openssl must handle this... Is it worth
trying?

Regards,
// NaHi
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2011-04-15 00:46
(Received via mailing list)
Hi,

On 2011/04/13, at 7:37, Eric Wong <normalperson@yhbt.net> wrote:
> POSIX.1-2008 has openat(), renameat(), chownat(), etc.., which can be
> used for implementing per-thread/VM working directories.  It might take
> a while for more platforms to support them, but Linux has had them
> for a few years.

"Thread local working directory" has been implemented already, in MVM.
This topic is locked and can not be replied to.