Sandbox 0.0.11 -- taking the i out of eval

[why_the_lucky_stiff]

Oh rapture! Oh, delicious zesty snacks! Oh, steamed uncles in a
cabbage
display! OH, LAMB MEDALLIONS!

For serious: I am outstripped with zeal. The sandbox is open. While
still
in primitive fashion, it’s pretty hot for being eleven hours old.

gem install sandbox --source code.whytheluckystiff.net

This module swaps in a new symbol table, so you can basically manage
multiple
interpreters at once. There are so many possibilities for using this.
For
emulating selector namespaces, for jailing code.

s = Sandbox.new
s.eval(“module Hidden; end”)
=> nil
Hidden
NameError: uninitialized constant Hidden

There’s more explanation elsewhere[1], but if you want to see the hack
for
yourself, I’d hit ext/sand_table/sand_table.c[2]. Thanks go to
MenTaLguY for
helping me conceive the hack and hope there is some among you who will
see something
good in this and help me batter it into warm, submissive donuts.

_why

[1]
http://redhanded.hobix.com/inspect/theThrillingFreakyFreakySandboxHack.html
[2]
http://code.whytheluckystiff.net/svn/sandbox/trunk/ext/sand_table/sand_table.c

[why_the_lucky_stiff]

On 7/19/06, why the lucky stiff [email protected] wrote:

This module swaps in a new symbol table, so you can basically manage
multiple
interpreters at once. There are so many possibilities for using
this. For
emulating selector namespaces, for jailing code.

This could be the basis of a great “live” class browser, which I think
is
about the only way to really tell what methods, etc. are available at
any
given point in your code. It could allow any class to be loaded and run,
while not allowing them to execute “destructive” behavior (like, delete
all
your rows in a database or similar). It could also allow the reloading
of
the environment to start “fresh” - something people have asked for in
other
context too.

I can’t build it on Windows tho - I think my version of VC++ is
incorrect.
Any chance we can get a precompiled binary? Plllleease?

Justin

[why_the_lucky_stiff]

On Thu, Jul 20, 2006 at 03:18:05AM +0900, Justin B. wrote:

I can’t build it on Windows tho - I think my version of VC++ is incorrect.
Any chance we can get a precompiled binary? Plllleease?

I’ll have binaries for Hpricot and Sandbox both out in August. They
still lack
stability too much to warrant that kind of release. If someone else
finds time
to compile and release their own gems, that would be a great relief!

_why

[why_the_lucky_stiff]

On 7/19/06, why the lucky stiff [email protected] wrote:

This module swaps in a new symbol table, so you can basically manage multiple
interpreters at once. There are so many possibilities for using this. For
emulating selector namespaces, for jailing code.

This is pretty freaking cool.

Oh rapture! Oh, delicious zesty snacks! Oh, steamed uncles in a cabbage
display! OH, LAMB MEDALLIONS!

This is even better.

[why_the_lucky_stiff]

On Jul 19, 2006, at 7:20 PM, why the lucky stiff wrote:

Putting this up on my gem server right away.

_why

This is completely off topic, but I just figured out who _why reminds
me of. The King of All Cosmos from Katamari Damacy. Can’t just hear
the music playing in the background whenever you read his stuff?

[why_the_lucky_stiff]

On Thu, Jul 20, 2006 at 07:14:43AM +0900, Mauricio F. wrote:

Could you have a look at eigenclass.org ?
I applied my standard cross-compilation treatment with MinGW.
Sandbox will follow if the above works and it’s OK by you.

Oh blam! That was really unexpected, thankyou, how sweet of you,
Mauricio!
Isn’t Mauricio sweet everyone? Yes, let’s get that checked in.

Putting this up on my gem server right away.

_why

[why_the_lucky_stiff]

Hi,

In message “Re: [ANN] sandbox 0.0.11 – taking the i out of eval”
on Thu, 20 Jul 2006 01:22:00 +0900, why the lucky stiff
[email protected] writes:

|This module swaps in a new symbol table, so you can basically manage multiple
|interpreters at once. There are so many possibilities for using this. For
|emulating selector namespaces, for jailing code.

I’d like to have it in the standard distribution once it become
stable. It looks much nicer than still unreliable $SAFE=4.

						matz.

[why_the_lucky_stiff]

On Thu, Jul 20, 2006 at 11:36:55AM +0900, Logan C. wrote:
}
} On Jul 19, 2006, at 7:20 PM, why the lucky stiff wrote:
}
} >On Thu, Jul 20, 2006 at 07:14:43AM +0900, Mauricio F. wrote:
} >>Could you have a look at eigenclass.org
} >>mswin32.gem ?
} >>I applied my standard cross-compilation treatment with MinGW.
} >>Sandbox will follow if the above works and it’s OK by you.
} >
} >Oh blam! That was really unexpected, thankyou, how sweet of you,
} >Mauricio!
} >Isn’t Mauricio sweet everyone? Yes, let’s get that checked in.
} >
} >Putting this up on my gem server right away.
} >
} >_why
} >
}
} This is completely off topic, but I just figured out who _why reminds
} me of. The King of All Cosmos from Katamari Damacy. Can’t just hear
} the music playing in the background whenever you read his stuff?

+1

Also, awesome!

–Greg

[why_the_lucky_stiff]

On Thu, Jul 20, 2006 at 03:00:23PM +0900, Yukihiro M. wrote:

I’d like to have it in the standard distribution once it become
stable. It looks much nicer than still unreliable $SAFE=4.

To be stable, I might need to enhance Thread. For now is there a way to
prevent
thread calls? To prevent crashes, I might need to pause threads until
the
Sandbox is complete. Then I will give Sandbox.timeout which gives a
maximum
execution time.

_why

[why_the_lucky_stiff]

On Thu, Jul 20, 2006 at 04:11:09AM +0900, why the lucky stiff wrote:

On Thu, Jul 20, 2006 at 03:18:05AM +0900, Justin B. wrote:

I can’t build it on Windows tho - I think my version of VC++ is incorrect.
Any chance we can get a precompiled binary? Plllleease?

I’ll have binaries for Hpricot and Sandbox both out in August. They still lack
stability too much to warrant that kind of release. If someone else finds time
to compile and release their own gems, that would be a great relief!

Could you have a look at
eigenclass.org ?
I applied my standard cross-compilation treatment with MinGW.
Sandbox will follow if the above works and it’s OK by you.

FYI, here’s the patch to the Rakefile and related stuff I applied:

Thu Jul 20 00:04:00 CEST 2006 Mauricio F. [email protected]

• Rakefile, mingw-rbconfig.rb: added rubygems_win32 and related
  targets.
  Cross-compiles with MinGW.
  diff -rN -u old-hpricot.mingw/Rakefile new-hpricot.mingw/Rakefile
  — old-hpricot.mingw/Rakefile 2006-07-20 00:05:32.000000000 +0200
  +++ new-hpricot.mingw/Rakefile 2006-07-20 00:05:32.000000000 +0200
  @@ -8,6 +8,7 @@

NAME = “hpricot”
VERS = “0.3”
+PKG_REVISION = “.0”
CLEAN.include [‘ext/hpricot_scan/.{bundle,so,obj,pdb,lib,def,exp}',
‘ext/hpricot_scan/Makefile’,
'**/..sw?’, ‘*.gem’, ‘.config’]

@@ -19,7 +20,7 @@
if Dir.glob(File.join(“lib”,“hpricot_scan.*”)).length == 0
STDERR.puts “!!!”
STDERR.puts “Gem actually failed to build. Your system is”

• STDERR.puts “NOT configured properly to build Mongrel.”

• STDERR.puts “NOT configured properly to build hpricot.”
  STDERR.puts “!!!”
  exit(1)
  end
  @@ -27,7 +28,7 @@
  task :hpricot_scan => [:ragel]

desc “Packages up Hpricot.”
-task :package => [:clean]
+task :package => [:clean, :ragel]

desc “Run all the tests”
Rake::TestTask.new do |t|
@@ -39,7 +40,7 @@
spec =
Gem::Specification.new do |s|
s.name = NAME

•    s.version = VERS
  

•    s.version = VERS + PKG_REVISION
     s.platform = Gem::Platform::RUBY
     s.has_rdoc = false
     s.extra_rdoc_files = ["README", "CHANGELOG", "COPYING"]
  

@@ -49,12 +50,13 @@
s.email = ‘[email protected]’
s.homepage = ‘http://code.whytheluckystiff.net/hpricot/’

•    s.files = %w(COPYING README Rakefile) +
  

•    s.files = %w(COPYING README Rakefile mingw-rbconfig.rb) +
       Dir.glob("{bin,doc,test,lib,extras}/**/*") +
  

•      Dir.glob("ext/**/*.{h,c,rb}")
  

•      Dir.glob("ext/**/*.{h,c,rb,rl}") +
  

•      %w[ext/hpricot_scan/hpricot_scan.c] # needed because it's 
  

generated later

     s.require_path = "lib"

•    s.autorequire = "hpricot"
  

•    #s.autorequire = "hpricot"  # no no no this is tHe 3v1l
     s.extensions = FileList["ext/**/extconf.rb"].to_a
     s.bindir = "bin"
  

  end
  @@ -70,6 +72,7 @@
  ext_files = FileList[
  “#{ext}/.c",
  "#{ext}/.h”,
• “#{ext}/*.rl”,
  “#{ext}/extconf.rb”,
  “#{ext}/Makefile”,
  “lib”
  @@ -95,9 +98,61 @@

desc “Generates the scanner code with Ragel.”
task :ragel do

• sh %{/usr/local/bin/ragel ext/hpricot_scan/hpricot_scan.rl |
  /usr/local/bin/rlcodegen -G2 -o ext/hpricot_scan/hpricot_scan.c}

• sh %{ragel ext/hpricot_scan/hpricot_scan.rl | rlcodegen -G2 -o
  ext/hpricot_scan/hpricot_scan.c}
  end

+PKG_FILES = FileList[

• “test/**/*.{rb,html,xhtml}”,
• “lib/**/*.rb”,
• “ext/**/*.{c,rb,h,rl}”,
• “CHANGELOG”, “README”, “Rakefile”, “COPYING”,
• “mingw-rbconfig.rb”, “lib/hpricot_scan.so”]

+Win32Spec = Gem::Specification.new do |s|

• s.name = NAME
• s.version = VERS + PKG_REVISION
• s.platform = Gem::Platform::WIN32
• s.has_rdoc = false
• s.extra_rdoc_files = [“README”, “CHANGELOG”, “COPYING”]
• s.summary = “a swift, liberal HTML parser with a fantastic library”
• s.description = s.summary
• s.author = “why the lucky stiff”
• s.email = ‘[email protected]’
• s.homepage = ‘http://code.whytheluckystiff.net/hpricot/’
• s.files = PKG_FILES
• s.require_path = “lib”
• #s.autorequire = “hpricot” # no no no this is tHe 3v1l
• s.extensions = []
• s.bindir = “bin”
  +end

+WIN32_PKG_DIR = “hpricot-” + VERS + PKG_REVISION
+
+file WIN32_PKG_DIR => [:package] do

• sh “tar zxf pkg/#{WIN32_PKG_DIR}.tgz”
  +end

+desc “Cross-compile the hpricot_scan extension for win32”
+file “hpricot_scan_win32” => [WIN32_PKG_DIR] do

• cp “mingw-rbconfig.rb”,
  “#{WIN32_PKG_DIR}/ext/hpricot_scan/rbconfig.rb”
• sh “cd #{WIN32_PKG_DIR}/ext/hpricot_scan/ && ruby -I. extconf.rb &&
  make”
• mv “#{WIN32_PKG_DIR}/ext/hpricot_scan/hpricot_scan.so”,
  “#{WIN32_PKG_DIR}/lib”
  +end

+desc “Build the binary RubyGems package for win32”
+task :rubygems_win32 => [“hpricot_scan_win32”] do

• Dir.chdir(“#{WIN32_PKG_DIR}”) do
• Gem::Builder.new(Win32Spec).build
• verbose(true) {

•  mv Dir["*.gem"].first, "../pkg/hpricot-#{VERS + 
  

PKG_REVISION}-mswin32.gem"

• }
• end
  +end

+CLEAN.include WIN32_PKG_DIR
+
task :install do
sh %{rake package}
sh %{sudo gem install pkg/#{NAME}-#{VERS}}
diff -rN -u old-hpricot.mingw/mingw-rbconfig.rb
new-hpricot.mingw/mingw-rbconfig.rb
— old-hpricot.mingw/mingw-rbconfig.rb 1970-01-01 01:00:00.000000000
+0100
+++ new-hpricot.mingw/mingw-rbconfig.rb 2006-07-20 00:05:32.000000000
+0200
@@ -0,0 +1,174 @@
+
+# This rbconfig.rb corresponds to a Ruby installation for win32
cross-compiled
+# with mingw under i686-linux. It can be used to cross-compile
extensions for
+# win32 using said toolchain.
+#
+# This file assumes that a cross-compiled mingw32 build (compatible
with the
+# mswin32 builds) is installed under $HOME/ruby-mingw32.
+
+module Config

• #RUBY_VERSION == “1.8.5” or

• raise "ruby lib version (1.8.5) doesn’t match executable version

(#{RUBY_VERSION})"
+

• TOPDIR = File.dirname(FILE).chomp!(“/lib/ruby/1.8/i386-mingw32”)
• DESTDIR = ‘’ unless defined? DESTDIR
• CONFIG = {}
• CONFIG[“DESTDIR”] = DESTDIR
• CONFIG[“INSTALL”] = “/usr/bin/install -c”
• CONFIG[“prefix”] = (TOPDIR || DESTDIR +
  “#{ENV[“HOME”]}/ruby-mingw32”)
• CONFIG[“EXEEXT”] = “.exe”
• CONFIG[“ruby_install_name”] = “ruby”
• CONFIG[“RUBY_INSTALL_NAME”] = “ruby”
• CONFIG[“RUBY_SO_NAME”] = “msvcrt-ruby18”
• CONFIG[“SHELL”] = “/bin/sh”
• CONFIG[“PATH_SEPARATOR”] = “:”
• CONFIG[“PACKAGE_NAME”] = “”
• CONFIG[“PACKAGE_TARNAME”] = “”
• CONFIG[“PACKAGE_VERSION”] = “”
• CONFIG[“PACKAGE_STRING”] = “”
• CONFIG[“PACKAGE_BUGREPORT”] = “”
• CONFIG[“exec_prefix”] = “$(prefix)”
• CONFIG[“bindir”] = “$(exec_prefix)/bin”
• CONFIG[“sbindir”] = “$(exec_prefix)/sbin”
• CONFIG[“libexecdir”] = “$(exec_prefix)/libexec”
• CONFIG[“datadir”] = “$(prefix)/share”
• CONFIG[“sysconfdir”] = “$(prefix)/etc”
• CONFIG[“sharedstatedir”] = “$(prefix)/com”
• CONFIG[“localstatedir”] = “$(prefix)/var”
• CONFIG[“libdir”] = “$(exec_prefix)/lib”
• CONFIG[“includedir”] = “$(prefix)/include”
• CONFIG[“oldincludedir”] = “/usr/include”
• CONFIG[“infodir”] = “$(prefix)/info”
• CONFIG[“mandir”] = “$(prefix)/man”
• CONFIG[“build_alias”] = “i686-linux”
• CONFIG[“host_alias”] = “i586-mingw32msvc”
• CONFIG[“target_alias”] = “i386-mingw32”
• CONFIG[“ECHO_C”] = “”
• CONFIG[“ECHO_N”] = “-n”
• CONFIG[“ECHO_T”] = “”
• CONFIG[“LIBS”] = "-lwsock32 "
• CONFIG[“MAJOR”] = “1”
• CONFIG[“MINOR”] = “8”
• CONFIG[“TEENY”] = “4”
• CONFIG[“build”] = “i686-pc-linux”
• CONFIG[“build_cpu”] = “i686”
• CONFIG[“build_vendor”] = “pc”
• CONFIG[“build_os”] = “linux”
• CONFIG[“host”] = “i586-pc-mingw32msvc”
• CONFIG[“host_cpu”] = “i586”
• CONFIG[“host_vendor”] = “pc”
• CONFIG[“host_os”] = “mingw32msvc”
• CONFIG[“target”] = “i386-pc-mingw32”
• CONFIG[“target_cpu”] = “i386”
• CONFIG[“target_vendor”] = “pc”
• CONFIG[“target_os”] = “mingw32”
• CONFIG[“CC”] = “i586-mingw32msvc-gcc”
• CONFIG[“CFLAGS”] = "-g -O2 "
• CONFIG[“LDFLAGS”] = “”
• CONFIG[“CPPFLAGS”] = “”
• CONFIG[“OBJEXT”] = “o”
• CONFIG[“CPP”] = “i586-mingw32msvc-gcc -E”
• CONFIG[“EGREP”] = “grep -E”
• CONFIG[“GNU_LD”] = “yes”
• CONFIG[“CPPOUTFILE”] = “-o conftest.i”
• CONFIG[“OUTFLAG”] = "-o "
• CONFIG[“YACC”] = “bison -y”
• CONFIG[“RANLIB”] = “i586-mingw32msvc-ranlib”
• CONFIG[“AR”] = “i586-mingw32msvc-ar”
• CONFIG[“NM”] = “i586-mingw32msvc-nm”
• CONFIG[“WINDRES”] = “i586-mingw32msvc-windres”
• CONFIG[“DLLWRAP”] = “i586-mingw32msvc-dllwrap”
• CONFIG[“OBJDUMP”] = “i586-mingw32msvc-objdump”
• CONFIG[“LN_S”] = “ln -s”
• CONFIG[“SET_MAKE”] = “”
• CONFIG[“INSTALL_PROGRAM”] = “$(INSTALL)”
• CONFIG[“INSTALL_SCRIPT”] = “$(INSTALL)”
• CONFIG[“INSTALL_DATA”] = “$(INSTALL) -m 644”
• CONFIG[“RM”] = “rm -f”
• CONFIG[“CP”] = “cp”
• CONFIG[“MAKEDIRS”] = “mkdir -p”
• CONFIG[“LIBOBJS”] = " fileblocks$(U).o crypt$(U).o flock$(U).o
  acosh$(U).o win32$(U).o"
• CONFIG[“ALLOCA”] = “”
• CONFIG[“DLDFLAGS”] = " -Wl,–enable-auto-import,–export-all"
• CONFIG[“ARCH_FLAG”] = “”
• CONFIG[“STATIC”] = “”
• CONFIG[“CCDLFLAGS”] = “”
• CONFIG[“LDSHARED”] = “i586-mingw32msvc-gcc -shared -s”
• CONFIG[“DLEXT”] = “so”
• CONFIG[“DLEXT2”] = “dll”
• CONFIG[“LIBEXT”] = “a”
• CONFIG[“LINK_SO”] = “”
• CONFIG[“LIBPATHFLAG”] = " -L"%s""
• CONFIG[“RPATHFLAG”] = “”
• CONFIG[“LIBPATHENV”] = “”
• CONFIG[“TRY_LINK”] = “”
• CONFIG[“STRIP”] = “strip”
• CONFIG[“EXTSTATIC”] = “”
• CONFIG[“setup”] = “Setup”
• CONFIG[“MINIRUBY”] = “ruby -rfake”
• CONFIG[“PREP”] = “fake.rb”
• CONFIG[“RUNRUBY”] = “$(MINIRUBY) -Icd $(srcdir)/lib; pwd”
• CONFIG[“EXTOUT”] = “.ext”
• CONFIG[“ARCHFILE”] = “”
• CONFIG[“RDOCTARGET”] = “”
• CONFIG[“XCFLAGS”] = " -DRUBY_EXPORT"
• CONFIG[“XLDFLAGS”] = " -Wl,–stack,0x02000000 -L."
• CONFIG[“LIBRUBY_LDSHARED”] = “i586-mingw32msvc-gcc -shared -s”
• CONFIG[“LIBRUBY_DLDFLAGS”] = " -Wl,–enable-auto-import,–export-all
  -Wl,–out-implib=$(LIBRUBY)"
• CONFIG[“rubyw_install_name”] = “rubyw”
• CONFIG[“RUBYW_INSTALL_NAME”] = “rubyw”
• CONFIG[“LIBRUBY_A”] = “lib$(RUBY_SO_NAME)-static.a”
• CONFIG[“LIBRUBY_SO”] = “$(RUBY_SO_NAME).dll”
• CONFIG[“LIBRUBY_ALIASES”] = “”
• CONFIG[“LIBRUBY”] = “lib$(LIBRUBY_SO).a”
• CONFIG[“LIBRUBYARG”] = “$(LIBRUBYARG_SHARED)”
• CONFIG[“LIBRUBYARG_STATIC”] = “-l$(RUBY_SO_NAME)-static”
• CONFIG[“LIBRUBYARG_SHARED”] = “-l$(RUBY_SO_NAME)”
• CONFIG[“SOLIBS”] = “$(LIBS)”
• CONFIG[“DLDLIBS”] = “”
• CONFIG[“ENABLE_SHARED”] = “yes”
• CONFIG[“MAINLIBS”] = “”
• CONFIG[“COMMON_LIBS”] = “m”
• CONFIG[“COMMON_MACROS”] = “”
• CONFIG[“COMMON_HEADERS”] = “windows.h winsock.h”
• CONFIG[“EXPORT_PREFIX”] = “”
• CONFIG[“MINIOBJS”] = “dmydln.o”
• CONFIG[“MAKEFILES”] = “Makefile GNUmakefile”
• CONFIG[“arch”] = “i386-mingw32”
• CONFIG[“sitearch”] = “i386-msvcrt”
• CONFIG[“sitedir”] = “$(prefix)/lib/ruby/site_ruby”
• CONFIG[“configure_args”] = “‘–host=i586-mingw32msvc’
  ‘–target=i386-mingw32’ ‘–build=i686-linux’
  ‘–prefix=#{ENV[“HOME”]}/ruby-mingw32’ ‘build_alias=i686-linux’
  ‘host_alias=i586-mingw32msvc’ ‘target_alias=i386-mingw32’”
• CONFIG[“NROFF”] = “/usr/bin/nroff”
• CONFIG[“MANTYPE”] = “doc”
• CONFIG[“LTLIBOBJS”] = " fileblocks$(U).lo crypt$(U).lo flock$(U).lo
  acosh$(U).lo win32$(U).lo"
• CONFIG[“ruby_version”] = “$(MAJOR).$(MINOR)”
• CONFIG[“rubylibdir”] = “$(libdir)/ruby/$(ruby_version)”
• CONFIG[“archdir”] = “$(rubylibdir)/$(arch)”
• CONFIG[“sitelibdir”] = “$(sitedir)/$(ruby_version)”
• CONFIG[“sitearchdir”] = “$(sitelibdir)/$(sitearch)”
• CONFIG[“topdir”] = File.dirname(FILE)
• MAKEFILE_CONFIG = {}
• CONFIG.each{|k,v| MAKEFILE_CONFIG[k] = v.dup}
• def Config::expand(val, config = CONFIG)
• val.gsub!(/$$|$(([^()]+))|${([^{}]+)}/) do |var|

•  if !(v = $1 || $2)
  

• ‘$’

•  elsif key = config[v = v[/\A[^:]+(?=(?::(.*?)=(.*))?\z)/]]
  

• pat, sub = $1, $2
• config[v] = false
• Config::expand(key, config)
• config[v] = key
• key = key.gsub(/#{Regexp.quote(pat)}(?=\s|\z)/n) {sub} if pat
• key

•  else
  

• var

•  end
  

• end
• val
• end
• CONFIG.each_value do |val|
• Config::expand(val)
• end
  +end
  +RbConfig = Config # compatibility for ruby-1.9
  +CROSS_COMPILING = nil unless defined? CROSS_COMPILING

[why_the_lucky_stiff]

On Fri, Jul 21, 2006 at 12:30:14AM +0900, why the lucky stiff wrote:

On Thu, Jul 20, 2006 at 03:00:23PM +0900, Yukihiro M. wrote:

I’d like to have it in the standard distribution once it become
stable. It looks much nicer than still unreliable $SAFE=4.

To be stable, I might need to enhance Thread. For now is there a way to prevent
thread calls? To prevent crashes, I might need to pause threads until the
Sandbox is complete. Then I will give Sandbox.timeout which gives a maximum
execution time.

I don’t want to diss your work, since it is obviously a significant
accomplishment. But aren’t these problems symptoms of the fact that the
approach (swapping out global variables) is sort of a hack? Wouldn’t
all these problems disappear if you did this the “right” way, by making
these globals into members of a structure that get passed to every
function (eg. make the parser re-entrant)?

I’m sure you considered that for a moment, but dismissed it because it
would be far too intrusive (affecting the entire interpreter, and all
extensions). I can’t think of any practical way around it (I thought of
a few impractical ones though: one used the preprocessor, another used
C++ :).

Is a re-entrant interpreter coming anytime soon? Ruby 2.0? That would
make a sandboxing extension the realm of mere mortals, instead of taking
the black magic that why has pulled off.

Josh

[why_the_lucky_stiff]

On Thu, Jul 20, 2006 at 06:14:06PM +0200, Mauricio F. wrote:

On Fri, Jul 21, 2006 at 12:51:12AM +0900, Joshua H. wrote:

On Fri, Jul 21, 2006 at 12:30:14AM +0900, why the lucky stiff wrote:
This is exactly what was done in Sydney. I know know of any plan to bring
that to HEAD though (and there’s no way it could get in ruby_1_8).

YARV is doing (or will do, haven’t checked for a while) native threads, but I
don’t know if it’s reentrant or just using a global interpreter lock like
Python.

After a quick glance at the code, it seems to be using a global VM lock
(GVL) -> model 2:

YARV Thread Desgin

model 1: Userlevel Thread
Same as traditional ruby thread.

model 2: Native Thread with Giant VM lock
Using pthread (or Windows thread) and Ruby threads run concurrent.

model 3: Native Thread with fine grain lock
Using pthread and Ruby threads run concurrent or parallel.

---

model 2:
A thread has mutex (GVL: Global VM Lock) can run. When thread
scheduling, running thread release GVL. If running thread
try blocking operation, this thread must release GVL and another
thread can continue this flow. After blocking operation, thread
must check interrupt (YARV_CHECK_INTS).

Every VM can run parallel.

Ruby threads are scheduled by OS thread scheduler.

---

model 3:
Every threads run concurrent or parallel and to access shared object
exclusive access control is needed. For example, to access String
object or Array object, fine grain lock must be locked every time.
*/
<<<<

I think I read somewhere that model 3 wouldn’t be done before 2007.

[why_the_lucky_stiff]

On Fri, Jul 21, 2006 at 12:51:12AM +0900, Joshua H. wrote:

these globals into members of a structure that get passed to every
function (eg. make the parser re-entrant)?

This is exactly what was done in Sydney. I know know of any plan to
bring
that to HEAD though (and there’s no way it could get in ruby_1_8).

YARV is doing (or will do, haven’t checked for a while) native threads,
but I
don’t know if it’s reentrant or just using a global interpreter lock
like
Python.

[why_the_lucky_stiff]

On Fri, Jul 21, 2006 at 12:51:12AM +0900, Joshua H. wrote:

I don’t want to diss your work, since it is obviously a significant
accomplishment. But aren’t these problems symptoms of the fact that the
approach (swapping out global variables) is sort of a hack? Wouldn’t
all these problems disappear if you did this the “right” way, by making
these globals into members of a structure that get passed to every
function (eg. make the parser re-entrant)?

You are welcome to diss it, I would encourage it and I hope to (fingers
crossed)
one day be publicly, formally defamed. I feel just like you, though. I
hope
this hack will go away one day. That day will break all the C
extensions, but,
as you say: mere mortals, right?

For the rest of my head-nodding, see struct sandkit in
ext/sand_table/sand_table.c.

_why