Merge YARV

SASADA_Koichi · December 31, 2006, 8:49am

Hi,

I’ll commit YARV to Ruby Subversion repository.
Sorry for being late.

Current implementation (/trunk) will be copied to /branches/matzruby
ChangeLog file of YARV will be copied to doc/ChangeLog-YARV
I have never checked ext/tk, ext/openssl.
Please check it > nagai-san, yuzo-san
Ripper doesn’t work, so I fixed ripper/extconf.rb to skip build.
Please check it > minero-san
make test failed.
make test-all output many errors.
Please check it and teach me the minimum set of error.
(current status: http://www.atdot.net/sp/view/len4bj)

SASADA_Koichi · December 31, 2006, 11:24pm

On Dec 30, 2006, at 23:49, SASADA Koichi wrote:

make test-all output many errors.
Please check it and teach me the minimum set of error.
(current status: http://www.atdot.net/sp/view/len4bj)

On OS X, exec is a victim of the merge too:

$ svn up
At revision 11447.
$ make
[…]
make[1]: ruby19' is up to date. $ ./ruby19 -e 'exec "/bin/ls"' -e:1:in exec’: Operation not supported - /bin/ls (Errno::EOPNOTSUPP)
from -e:1:in `’

–
Eric H. - [email protected] - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

SASADA_Koichi · January 1, 2007, 12:26am

e$B!!$5$5$@$G$9!#e(B

e$B!!:rHUe(B YARV e$B$r%^!<%8$7$^$7$?!#e(B

e$B!!8=>u$NJs9p$G$9!#e(B

ext:

openssl, tk
e$B$O%3%s%Q%$%k$G$-$k$^$G$O$7$F$$$k$N$G$9$,!"A4A3F0$+$7$F$*$je(B
e$B$^$;$s!#$43NG’D:$1$^$9$G$7$g$&$+!#e(B

make test-all:

WEBrick e$B$Ne(B timeout e$B$G8G$^$j$^$9!#e(B
drb
e$B$N%F%9%H$,F0$+$J$+$C$?$?$a6/0z$K%9%-%C%W$9$k$h$&$KJQ99$7$F$"$j$^e(B
e$B$9!#e(BThread.critical
e$B$rL5;k$9$k$?$a$K$3$s$J$U$&$K$J$C$F$$$^$9!#e(B
generator e$B$bF1$8$G$9!#e(B
e$B%(%i!<$,BgNL$G$9$,!“e(B__sned__, send e$B$”$?$j$,860x$G$9!#e(B
e$B$5$C$-$d$C$?$i$3$s$J46$8!'e(B
e$B!!e(Bhttp://www.atdot.net/sp/view/7ef5bj

e$B;EMM!'e(B
e$B!!e(Bsend/send/__send e$B$G!“8F$VB&$,e(B private
e$B$@$C$?$i!J!)!K!“e(Bfuncall e$B$,8Fe(B
e$B$P$l$?$3$H$HF1$8$h$&$&$K$9$k!”$H$$$&;EMM$,$h$/$o$+$j$^$;$s!#e(Bcaller
e$B$,e(B
NODE_VCALL/FCALL e$B$G8F$P$l$F$$$?>l9g!”$b$7$/$Oe(B toplevel
e$B$O!“8F$S=P$7;~$be(B
e$B$=$l$r7Q>5$9$k!”$H$$$&;EMM$G$7$g$&$+!#e(B
e$B!J4X?t8F$S=P$7;~!"$=$s$J>pJs$O<h$C$F$$$J$$$N$G$I$&$7$h$&$+G:$s$G$$$?$j!Ke(B

matzrubye$B!'e(B
e$B!!e(Bdrbrain e$B$,e(B matzruby
e$B$K%3%_%C%H$7$F$$$^$7$?$,!“e(Bmatzruby e$B$N%a%s%F%J%s%9e(B
e$B$O$I$&$9$k$Y$-$G$7$g$&$+!#=PMh$l$P!”$^$D$b$H$5$s$+$i%"%J%&%s%9$r=P$7$F$be(B
e$B$i$($l$P!#;d$O!"$^$D$b$H$5$s$,;EMM8!F$$9$k$?$a$N%V%i%s%A!"$H2r<a$7$F$$$^e(B
e$B$9!#e(Bminiruby e$B$^$G;H$($l$P==J,!"$H$$$&46$8!#e(B

e$B!!$"$1$^$7$F$*$a$G$H$&$4$6$$$^$9!#:#G/$b$I$&$>59$7$/$*4j$$$7$^$9!#e(B
e$B!!$5$5$@$3$&$$$AGRe(B

SASADA_Koichi · January 1, 2007, 1:38am

e$B31$H$$$$$^$9!#e(B

On 2007/01/01, at 8:25, SASADA Koichi wrote:

drb e$B$N%F%9%H$,F0$+$J$+$C$?$?$a6/0z$K%9%-%C%W$9$k$h$&$KJQ99$7$F$"$j$^e(B
e$B$9!#e(BThread.critical e$B$rL5;k$9$k$?$a$K$3$s$J$U$&$K$J$C$F$$$^$9!#e(B

YARVe$B$G$$$^;H$($kGSB>@)8f$N;EAH$_$K$O$J$K$,$"$j$^$9$+e(B?

e$BA4BN$rDd$a$kI,MW$O$J$$$N$G!"e(BMutexe$B$J$je(BMonitore$B$J$j$K=q$-D>$7$F$be(B
e$BBg>fIW$G$9$h!#e(B

SASADA_Koichi · January 1, 2007, 11:53am

wonderful, YARV is Ruby!

Please check it and teach me the minimum set of error.
the benchmark doesn’t work, here’s the fix:

benchmark/run_rite.rb:

prog = File.read(file).map{|e| e.rstrip}.join("\n")

prog = File.read(file).lines.map{|e| e.rstrip}.join("\n")

So I “make benchmark”:

Pentomino yields a strange error:
yarv./benchmark/bm_app_pentomino.rb:21:in clone': can't modify array during iteration (RuntimeError) from ./benchmark/bm_app_pentomino.rb:21:inblock in piece’
from ./benchmark/bm_app_pentomino.rb:16:in each' from ./benchmark/bm_app_pentomino.rb:16:inpiece’
from ./benchmark/bm_app_pentomino.rb:47:in mkpieces' from ./benchmark/bm_app_pentomino.rb:126:in’

Exceptions seem to be slower then with the old interpreter. But
everything else is much faster.

Thank you!
[murphy]

SASADA_Koichi · January 1, 2007, 1:59pm

I’ve written a little script to turn the benchmark output into a HTML
table:

http://www.rubychan.de/share/yarv_speedups.html

This makes it a little easier to see how great YARV is ^^

The script is here: http://www.rubychan.de/share/yarv_bench
[murphy]

SASADA_Koichi · January 2, 2007, 3:38am

In message [email protected],
`SASADA Koichi [email protected]’ wrote:

e$B!!:rHUe(B YARV e$B$r%^!<%8$7$^$7$?!#e(B

NetBSD 4.99.7e$B$G<!$N$h$&$K$J$j$^$7$?!#e(B

gcc -g -O2 -fPIC -pipe -I. -I…/ruby-1.9 -DRUBY_EXPORT -c
…/ruby-1.9/thread.c
In file included from …/ruby-1.9/thread.c:116:
…/ruby-1.9/thread_pthread.ci: In function ‘native_thread_create’:
…/ruby-1.9/thread_pthread.ci:184: error: ‘PTHREAD_STACK_MIN’ undeclared
(first use in this function)
…/ruby-1.9/thread_pthread.ci:184: error: (Each undeclared identifier is
reported only once
…/ruby-1.9/thread_pthread.ci:184: error: for each function it appears
in.)
…/ruby-1.9/thread_pthread.ci: In function
‘rb_thread_create_timer_thread’:
…/ruby-1.9/thread_pthread.ci:419: error: ‘PTHREAD_STACK_MIN’ undeclared
(first use in this function)
*** Error code 1

Stop.
make: stopped in /home/gotoyuzo/src/ruby/build-ruby-1.9

make test-all:

WEBrick e$B$Ne(B timeout e$B$G8G$^$j$^$9!#e(B

e$B$3$l$O!"e(Bmatzrubye$B$G$bF1$8$G!"e(BArray#eache$B$N<B9TCf$O$=$Ne(BArraye$B$re(B
e$BJQ99$G$-$J$/$J$C$?$;$$$G$9!#0J2<$N%Q%C%A$G$I$&$G$7$g$&$+!#e(B

Index: lib/webrick/utils.rb

— lib/webrick/utils.rb (revision 11451)
+++ lib/webrick/utils.rb (working copy)
@@ -124,7 +124,7 @@
while true
now = Time.now
@timeout_info.each{|thread, ary|

```
         ary.each{|info|
```

         ary.dup.each{|info|
           time, exception = *info
           interrupt(thread, info.object_id, exception) if time <

now
}

SASADA_Koichi · January 2, 2007, 5:02am

e$B$3$s$K$A$O!"$J$+$`$ie(B(e$B$&e(B)e$B$G$9!#e(B

In message “[ruby-dev:30061] Re: merge YARV”
on Jan.01,2007 08:25:24, [email protected] wrote:
| ext:
| * openssl, tk e$B$O%3%s%Q%$%k$G$-$k$^$G$O$7$F$$$k$N$G$9$,!"A4A3F0$+$7$F$*$je(B
| e$B$^$;$s!#$43NG’D:$1$^$9$G$7$g$&$+!#e(B

VC6e$B$Ge(Btk(e$B$H$$$&$+e(Btcltklib)e$B$,%j%s%/$G$-$^$;$s!#e(B
e$B%j%s%/;~$K%7%s%%k$NITB-$r%A%'%C%/$7$J$$4D6-$J$i!"DL$C$A$c$&e(B
e$B$s$G$9$+$M!#e(B
e$B$3$s$J46$8!#e(B

tcltklib.obj : error LNK2001: e$B30It%7%s%%ke(B
“_is_ruby_native_thread” e$B$OL$2r7h$G$9e(B
tcltklib.obj : error LNK2001: e$B30It%7%s%%ke(B “_rb_proc_new”
e$B$OL$2r7h$G$9e(B

e$B$3$NN><T$O$I$&$9$l$P$$$$$s$G$7$g$&e(B?
e$BFC$K8e<T$Oe(Bintern.he$B$K@k8@$O$"$k$N$KDj5A$,$J$$$h$&$J!#e(B

e$B$=$l$G$O!#e(B

SASADA_Koichi · January 3, 2007, 3:26pm

e$B!!$5$5$@$G$9!#e(B

GOTOU Yuuzou wrote:

NetBSD 4.99.7e$B$G<!$N$h$&$K$J$j$^$7$?!#e(B
…
…/ruby-1.9/thread_pthread.ci:419: error: ‘PTHREAD_STACK_MIN’ undeclared (first use in this function)
*** Error code 1

e$B!!$9$_$^$;$s!#BP1~$7$?$H;W$C$?$N$G$9$,!"$J$s$+BP1~$G$-$F$^$;$s$G$7$?!#e(B

make test-all:

WEBrick e$B$Ne(B timeout e$B$G8G$^$j$^$9!#e(B

e$B$3$l$O!"e(Bmatzrubye$B$G$bF1$8$G!"e(BArray#eache$B$N<B9TCf$O$=$Ne(BArraye$B$re(B
e$BJQ99$G$-$J$/$J$C$?$;$$$G$9!#0J2<$N%Q%C%A$G$I$&$G$7$g$&$+!#e(B

e$B!!$J$k$[$I!#e(B

e$B!!$=$&$$$($P!"!V%3%_%C%H$7$F2<$5$$!W$H$$$&$^$D$b$H$5$s$N$*JV;v$O$I$&$9$ke(B
e$B$Y$-$J$s$G$7$g$&$+!#e(B

SASADA_Koichi · January 3, 2007, 3:26pm

Hi,

murphy wrote:

I’ve written a little script to turn the benchmark output into a HTML
table:

http://www.rubychan.de/share/yarv_speedups.html

This makes it a little easier to see how great YARV is ^^

The script is here: http://www.rubychan.de/share/yarv_bench
[murphy]

Thanks!

notes:

vm1_* and vm2_* results should reduce loop_whileloop and
loop_whileloop2 results.
these tests are only toy programs. they can’t measure good
performance. can anyone add pragmatic programs?

regards,

SASADA_Koichi · January 3, 2007, 3:24pm

e$B!!$5$5$@$G$9!#e(B

Masatoshi SEKI wrote:

YARVe$B$G$$$^;H$($kGSB>@)8f$N;EAH$_$K$O$J$K$,$"$j$^$9$+e(B?

e$BA4BN$rDd$a$kI,MW$O$J$$$N$G!"e(BMutexe$B$J$je(BMonitore$B$J$j$K=q$-D>$7$F$be(B
e$BBg>fIW$G$9$h!#e(B

e$B!!e(BMutex
e$B%/%i%9$,AH$9~$$K$J$C$F$$$k$N$G!"$=$A$i$rMxMQ$7$F$$$?$@$1$l$P$He(B
e$B;W$$$^$9!#62$i$/!"e(BMutex#synchronize
e$B$GMQ$OB-$j$k$N$G$O$J$$$+$H;W$$$^$9!#e(B

e$B!!e(BThread.exclusive e$B$r!"e(B

$global_lock = Mutex.new
class Thread
def self.exclusive
$global_lock.synchronize{
yield
}
end
end

e$B$H$$$&$U$&$K!"!Ve(BThread.exclusivee$B$G<B9T$7$F$$$k%9%l%C%I$O$?$+$@$+0l8D$7$+e(B
e$B$J$$!W$H$$$&0UL#$K$9$k$N$O%"%j$G$7$g$&$+$M!)!!F1$8L>A0$G0c$&5sF0$K$9$k$Ne(B
e$B$O:.Mp$N85$+$J!#e(B

SASADA_Koichi · January 3, 2007, 3:28pm

Hi,

murphy wrote:

benchmark/run_rite.rb:

prog = File.read(file).map{|e| e.rstrip}.join("\n")

prog = File.read(file).lines.map{|e| e.rstrip}.join("\n")

I’ll fix it.

Exceptions seem to be slower then with the old interpreter. But
everything else is much faster.

It’s seems affect of last enum spec change. we can’t modify an array
during iteration:

ary = [1, 2]
ary.each{
p ary.clone #=> same error
}

… why Array#clone is prohibited? In rb_ary_initialize(), cloned array
is checked by ary_iter_check(), but a flag ARY_ITERLOCK should not be
set by Object#clone.

SASADA_Koichi · January 3, 2007, 3:28pm

Hi,

Eric H. wrote:

does anyone has a solution? now, I can’t access MacOS.

SASADA_Koichi · January 3, 2007, 6:29pm

Hi,

does anyone has a solution? now, I can’t access MacOS.

At the end of the compilation there are warnings:

gcc -g -O2 -pipe -fno-common -L. main.o -lruby-static -lpthread
-ldl
-lobjc -o ruby
/usr/bin/ld: warning multiple definitions of symbol _setregid
./libruby-static.a(process.o) definition of _setregid in section
(__TEXT,__text)
/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/…/…/…/libpthread.dylib(setregid.So)
definition of _setregid
/usr/bin/ld: warning multiple definitions of symbol _setreuid
./libruby-static.a(process.o) definition of _setreuid in section
(__TEXT,__text)
/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/…/…/…/libpthread.dylib(setreuid.So)
definition of _setreuid

Then RDoc documentation generation fails:

./miniruby ./runruby.rb --extout=.ext – “./bin/rdoc” --all --ri --op
“/usr/local/test/share/ri/1.9/system” “.”
“./ruby ./bin/rdoc --all --ri --op /usr/local/test/share/ri/1.9/system
.”
./runruby.rb:62:in exec': Operation not supported - ./ruby ./bin/rdoc --all --ri --op /usr/local/test/share/ri/1.9/system . (Errno::EOPNOTSUPP) from ./runruby.rb:62:in’
make: *** [do-install-doc] Error 1

I am running Mac OS X 10.4.8 with Xcode 2.4.1 (i.e. gcc 4.0.1).

Regards,
Julian

SASADA_Koichi · January 3, 2007, 3:28pm

e$B!!$5$5$@$G$9!#e(B

U.Nakamura wrote:

VC6e$B$Ge(Btk(e$B$H$$$&$+e(Btcltklib)e$B$,%j%s%/$G$-$^$;$s!#e(B
e$B%j%s%/;~$K%7%s%%k$NITB-$r%A%’%C%/$7$J$$4D6-$J$i!"DL$C$A$c$&e(B
e$B$s$G$9$+$M!#e(B
e$B$3$s$J46$8!#e(B

tcltklib.obj : error LNK2001: e$B30It%7%s%%ke(B “_is_ruby_native_thread” e$B$OL$2r7h$G$9e(B
tcltklib.obj : error LNK2001: e$B30It%7%s%%ke(B “_rb_proc_new” e$B$OL$2r7h$G$9e(B

e$B$3$NN><T$O$I$&$9$l$P$$$$$s$G$7$g$&e(B?
e$BFC$K8e<T$Oe(Bintern.he$B$K@k8@$O$"$k$N$KDj5A$,$J$$$h$&$J!#e(B

e$B!!FbMF$NOC$+$i8@$&$H!“A0<T$NItJ,$O!”$=$b$=$be(BRubye$B$N%9%l%C%I%b%G%k$,JQ$o$Ce(B
e$B$F$$$k$N$G:,K$+$i8+D>$5$J$$$H$$$1$J$$$h$&$J5$$,$7$F$$$^$9!#$?$@!"$H$j$"e(B
e$B$($:%M%$%F%#%V%9%l%C%I$r;H$C$F$O5o$^$9!#e(B

e$B!!8e<T$OMQ0U$7$?$$$H;W$$$^$9!#e(BProce$BItJ,$O%=!<%9$,$^$@@0M}$G$-$F$$$^$;$s!#e(B
proc.c e$B$G$bMQ0U$7$F!"$=$A$i$K2!$79~$`46$8$r9M$($F$$$^$9!#e(B

e$B!!$H$3$m$G!“e(Bintern.h
e$B$N4X?t$H!”<B:]$K<BAu$7$F$$$k$+!J$=$7$F!“Cf?H$,6ue(B
e$B$8$c$J$$$+!K!”$H$$$&$N$O$-$A$s$H@0M}$7$F$$j$^$;$s!#2?$+!"5!3#E$KD4$Y$ke(B
e$BJ}K!$C$F$"$k$G$7$g$&$+!#:$$C$?$iBP=h!"$G$$$$$N$+$J!#e(B

SASADA_Koichi · January 3, 2007, 10:16pm

Hi,

    from -e:1:in `<main>'
does anyone has a solution? now, I can’t access MacOS.

I think I found the reason: (simply googling mac os EOPNOTSUPP execv)
Mac OS X has an undocumented behavior concerning the execve() system
call
inside a threaded process. If a process tries to call execve() and has
more
than one active thread, the kernel returns the error EOPNOTSUPP. After a
closer
look at kernexec.c in the Darwin XNU source code, it becomes apparent
that for
shellcode to function properly inside a threaded process, it will need
to call
either fork() or vfork() before calling execve().

I did an ugly test to check and by doing fork and running execv in the
child does indeed work in Ruby’s source code.

Cheers,
Vincent ISAMBART

SASADA_Koichi · January 3, 2007, 7:31pm

On Jan 3, 2007, at 09:29, Julian Christopher Schnidder wrote:

definition of _setregid
/usr/bin/ld: warning multiple definitions of symbol _setreuid
./libruby-static.a(process.o) definition of _setreuid in section
(__TEXT,__text)
/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/…/…/…/libpthread.dylib
(setreuid.So)
definition of _setreuid

These are not compilation errors. They are by design.

–
Eric H. - [email protected] - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

SASADA_Koichi · January 6, 2007, 7:30pm

e$B1J0fe(B@e$BCNG=!%6e9)Bg$G$9!%e(B

From: SASADA Koichi [email protected]
Subject: [ruby-dev:30068] Re: merge YARV
Date: Wed, 3 Jan 2007 23:24:30 +0900
Message-ID: [email protected]

e$B!!FbMF$NOC$+$i8@$&$H!“A0<T$NItJ,$O!”$=$b$=$be(BRubye$B$N%9%l%C%I%b%G%k$,JQ$o$Ce(B
e$B$F$$$k$N$G:,K$+$i8+D>$5$J$$$H$$$1$J$$$h$&$J5$$,$7$F$$$^$9!#$?$@!"$H$j$"e(B
e$B$($:%M%$%F%#%V%9%l%C%I$r;H$C$F$O5o$^$9!#e(B

e$B$^$@D4$Y$F$O$$$J$$$N$G$9$,!$e(Btcltklib.c e$B$He(B multi-tk.rb
e$B$H$Oe(B
e$BBgI}$K=q$-49$($,I,MW$H$$$&$3$H$K$J$j$=$&$G$9$M!%e(B(;_
Ruby/Tk e$B$Ne(B YARV e$BBP1~$K$O>/$7;~4V$,$+$+$k$+$b$7$l$^$;$s!%e(B
e$B%9%l%C%I4X78$G$O$$$/$D$+<ALd$5$;$F$$$?$@$/$3$H$Ke(B
e$B$J$k$+$b$7$l$^$;$s$N$G!$$=$N:]$K$O$h$m$7$/$*4j$$CW$7$^$9!%e(B

SASADA_Koichi · January 6, 2007, 7:30pm

e$B$J$+$@$G$9!#e(B

At Mon, 1 Jan 2007 08:25:24 +0900,
SASADA Koichi wrote in [ruby-dev:30061]:

e$B:rHUe(B YARV e$B$r%^!<%8$7$^$7$?!#e(B

bisone$B$+$i7Y9p$,=P$k$N$,5$$K$J$C$?$N$G$9$,!“e(Bf_largliste$B$N$”$H$Ne(B
vtable_size()e$B$OITMW$G$9$+e(B?

e$B$b$&0lE@!"$G$-$l$Pe(B.ince$B$be(B.SUFFIXESe$B$KDI2C$7$F$*$$$F$/$@$5$$!#e(B

Index: common.mk

— common.mk (revision 11475)
+++ common.mk (working copy)
@@ -3,4 +3,6 @@ lib: $(LIBRUBY)
dll: $(LIBRUBY_SO)

+.SUFFIXES: .inc
+
RUBYOPT =

Index: parse.y

— parse.y (revision 11475)
+++ parse.y (working copy)
@@ -3218,21 +3218,21 @@ lambda : {
%*/
}

```
             {
```

```
   {
```

   /*%%%*/
 $<num>$ = lpar_beg;
  lpar_beg = ++paren_nest;

```
 }
```
```
 f_larglist
```
```
   {
```

       $<num>$ = vtable_size(lvtbl->dvars);

```
   /*%
```
```
   %*/
   }
```

 f_larglist
 lambda_body
   {
   /*%%%*/
 $$ = $3;

                   $$->nd_body = block_append($$->nd_body, $5);

 $$->nd_body = block_append($$->nd_body, $4);
 dyna_pop($<vars>1);
 lpar_beg = $<num>2;
   /*%

```
     $$ = dispatch2(lambda, $3, $5);
```

     $$ = dispatch2(lambda, $3, $4);
   %*/
   }

@@ -6102,6 +6102,6 @@ parser_yylex(struct parser_params *parse
if (c == ‘>’) {
lex_state = EXPR_ARG;

return tLAMBDA;
```
 }
```

```
 return tLAMBDA;
```

SASADA_Koichi · January 6, 2007, 7:30pm

e$B$J$+$@$G$9!#e(B

At Mon, 1 Jan 2007 08:25:24 +0900,
SASADA Koichi wrote in [ruby-dev:30061]:

e$B:rHUe(B YARV e$B$r%^!<%8$7$^$7$?!#e(B

cygwine$B$G$Oe(Bthread_win32.cie$B$,;H$o$l$k0lJ}$Ge(Bfork()e$B$,B8:_$9$k$N$G!"e(B
rb_thread_reset_timer_thread()e$B$,$J$$$H$$$&%(%i!<$K$J$j$^$7$?!#e(B

Index: thread_win32.ci

— thread_win32.ci (revision 11475)
+++ thread_win32.ci (working copy)
@@ -308,3 +308,10 @@ rb_thread_create_timer_thread(void)
}

+void
+rb_thread_reset_timer_thread(void)
+{

timer_thread_handle = 0;
rb_thread_create_timer_thread();
+}

#endif /* THREAD_SYSTEM_DEPENDENT_IMPLEMENTATION */