Memory profiler for test-all

okkezSS · October 28, 2010, 7:25pm

$B!!(Btest-all
$B$,%a%b%j$r?)$$$9$.$k$C$F$$$&$s$G!"3F%F%9%H8e$K$I$NDxEY%a%b%j(B
$B$r?)$&$N$+D4$Y$k%W%m%U%!%$%i$r:n$C$F$_$^$7$?!#(B

$B!!$3$l!"%3%_%C%H$7$F$$$$$G$9$+!)(B

$B!!$G!"(Btest/runner.rb $B$K2?$i$+$N;X<($rM?$($?$i!J(Bmake test-all
$B$G2?$i$+$N(B
$B;X<($rM?$($?$i!K$3$l$r;H$&$h$&$K$7$F$_$?$$$s$G$9$,!“2?$+J}K!$O$”$k$G$7$g(B
$B$&$+!#(Btest/unit $B$,(B OptionParser
[email protected]$7$A$c$&$s$G!"%*%W%7%g%s$O(B
$BEO$C$F$3$J$$$s$G$9$h$M!#(B

$B!!4D6-JQ?t$GM?$($k$C$F$N$b<[email protected]$1$I!#(B

$B!!$A$J$$K!"<j85$G$3$l$r;H$C$F$$k$H!"%a%b%jITB-$G<:GT$7$F$$$?(B
test-all
$B$,@.8y$9$k$h$&$K$J$C$F$7$^$$$^$7$?!#(Blazy sweep
[email protected]$H!"%a%b%jITB-$,82Cx$K(B
$B$J$k$N$+$b$7$l$J$$!#(Blazy sweep
[email protected]$+$i$$$C$F!"$=$j$c$=$&$J$s$G$9$,!#(B

$B!!(Bfiddle $B$N2U=j$G(B SEGV $B$7$^$9!#[email protected]!"(Bfiddle
[email protected]$1$G<B9T$9$k$H(B SEGV $B$7$J(B
$B$$!#$-$A$s$HD4$Y$J$$$HL5M}$C$]$$$G$9!#(BObjectSpace.each_objects
$BAjEv$r$7(B
$B$F$$$k$N$G!“C/$+$,JQ$J%*%V%8%’%/%[email protected]@.$7$F$$$k!”[email protected]$H;W$$$^$9!#(B

SASADA_Koichi · October 28, 2010, 8:15pm

[email protected]$G$9!#(B

$B!!$9$$^$;$s!"%%9$,$"$C$?$N$GD>$7$^$7$?!#(B

$B!!%%s!&%%U$O4D6-JQ?t$GM?$($k$h$&$K$7$F$_$^$7$?!#(B

(2010/10/28 18:25), SASADA Koichi wrote:

$B!!(Bfiddle $B$N2U=j$G(B SEGV $B$7$^$9!#[email protected]!"(Bfiddle [email protected]$1$G<B9T$9$k$H(B
SEGV $B$7$J(B
$B$$!#$-$A$s$HD4$Y$J$$$HL5M}$C$]$$$G$9!#(BObjectSpace.each_objects $BAjEv$r$7(B
$B$F$$$k$N$G!“C/$+$,JQ$J%*%V%8%’%/%[email protected]@.$7$F$$$k!”[email protected]$H;W$$$^$9!#(B

$B!!$3$l$O!"$3$N;EAH$_$,$&$^$/[email protected]$1$G!"2~D{HG$G$O$-$C$A$j(B
test-all TESTS=fiddle $B$G(B SEGV $B$7$^$7$?!#(B

$B!!(Btest-all TESTS=’-x fiddle’
[email protected]$i$l$?7k2L$G$9!J2#<4!’;~4V!"C10L$O<B9T(B
$B$7$?%F%9%H?t!K!#(B

http://www.atdot.net/fp_store/f.m4i0bl/file.graph.png

$B!!$H$3$m$I$3$m$G%a%b%j$r$I$+$C$H<h$C$F$$$k$N$,$o$+$j$^$9!#(B

SASADA_Koichi · October 28, 2010, 11:11pm

2010/10/28 SASADA Koichi [email protected]:

$B!!$3$l$O!"$3$N;EAH$_$,$&$^$/[email protected]$1$G!"2~D{HG$G$O$-$C$A$j(B
test-all TESTS=fiddle $B$G(B SEGV $B$7$^$7$?!#(B

$B$?$C$?:#%a!<%k$r8+$^$7$?!#[email protected]$1Aa$/$K$3$NLdBj$rD>$9$D$b$j$G$9$,!"(B
$B:#F|%V%i%8%k$+$i%&%k%0%"%$$KE~Ce$7$?;v$H!"(Bwifi$B$N%3%M%/%7%g%s$,$I$3$K(B
$B9T$C$F$b0-$$$N$G!">/!9;~4V$r2<$5$$!#%&%k%0%"%$;[email protected]|0JFb$K(B
[email protected]=PMh$k$H;W$$$^$9!#(B

$B:{ED$5$s$N$4M}2r$K46<U$7$^$9!#(B

SASADA_Koichi · October 29, 2010, 12:41am

$B$^$D$b$H(B $B$f$-$R$m$G$9(B

In message “Re: [ruby-dev:42484] Re: memory profiler for test-all”
on Fri, 29 Oct 2010 05:57:33 +0900, Aaron P.
[email protected] writes:

|> $B!!$3$l$O!"$3$N;EAH$_$,$&$^$/[email protected]$1$G!“2~D{HG$G$O$-$C$A$j(B
|> test-all TESTS=fiddle $B$G(B SEGV $B$7$^$7$?!#(B
|
|$B$?$C$?:#%a!<%k$r8+$^$7$?!#[email protected]$1Aa$/$K$3$NLdBj$rD>$9$D$b$j$G$9$,!”(B
|$B:#F|%V%i%8%k$+$i%&%k%0%"%$$KE~Ce$7$?;v$H!"(Bwifi$B$N%3%M%/%7%g%s$,$I$3$K(B
|$B9T$C$F$b0-$$$N$G!">/!9;~4V$r2<$5$$!#%&%k%0%"%$;[email protected]|0JFb$K(B
|[email protected]=PMh$k$H;W$$$^$9!#(B
|
|$B:{ED$5$s$N$4M}2r$K46<U$7$^$9!#(B

$B$C$F$$$&$+!"F|K\8l$G%V%D%V%[email protected][email protected]$1$J$N$K$A$c$s$H%U%)%m!<(B
$B$7$A$c$&(BAaron$B$NEXNO$rB:7I$7$^$9!#(B

SASADA_Koichi · October 29, 2010, 1:44am

(2010/10/28 23:19), Yukihiro M. wrote:

$B$C$F$$$&$+!"F|K\8l$G%V%D%V%[email protected][email protected]$1$J$N$K$A$c$s$H%U%)%m!<(B
$B$7$A$c$&(BAaron$B$NEXNO$rB:7I$7$^$9!#(B

$B!!$A$c$s$H1Q8l$G$b=q$-$^$7$?$h!*!d(B[ruby-core:32942]

$B;d$bB:7I$7$^$9!#(B

SASADA_Koichi · October 29, 2010, 12:31pm

nagachika $B$H?=$7$^$9!#(B

$B$I$N%a!<%k(B/ticket $B$KBP$9$k%3%a%s%H$K$9$k$+LB$$$^$7$?$,(B
$B$3$N%a%b%j%W%m%U%!%$%i$G(B ext/objspace
$B$rMxMQ$7$F$$$?$N$G$3$A$i$K!#(B

r29602 $B$+$i$N(B trunk $B$G$O(B rb_classext_t
$B$N9=B$$,JQ99$K$J$C$?$?$a(B
ObjectSpace.memsize_of $B$O(B Class/Module $B$NDj?t$N%F!<%V%k$V$s$N(B
$B%a%b%j%5%$%:$r2C;;$7$=$3$M$F$$$^$9!#(B

r29602 $B$O(B r29604 $B$N(B private constant $B$N$?$a$NA0=`Hw$J$N$G!"(B
$B$3$A$i$b$A$g$C$HBT$C$?>uBV$+$b$7$l$^$;$s$,!"0l1~Js9p$^$G!#(B

ex)
require “objspace”
cls = Class.new
10.times do |i|
p [i, ObjectSpace.memsize_of(cls)]
cls.send(:const_set, :“A#{i}”, nil)
end

@trunk
[0, 80]
[1, 80]
[2, 80]
[3, 80]
[4, 80]
[5, 80]
[6, 80]
[7, 80]
[8, 80]
[9, 80]

@[email protected](B
[0, 80]
[1, 148]
[2, 148]
[3, 148]
[4, 148]
[5, 148]
[6, 292]
[7, 316]
[8, 340]
[9, 364]

Index: ext/objspace/objspace.c

— ext/objspace/objspace.c ($B%j%S%8%g%s(B 29626)
+++ ext/objspace/objspace.c ($B:n6H%3%T!<(B)
@@ -69,6 +69,9 @@
if (RCLASS(obj)->ptr->iv_tbl) {
size += st_memsize(RCLASS(obj)->ptr->iv_tbl);
}

```
  if (RCLASS(obj)->ptr->const_tbl) {
```

      size += st_memsize(RCLASS(obj)->ptr->const_tbl);

  }
  size += sizeof(rb_classext_t);
  break;
 case T_STRING:

SASADA_Koichi · October 29, 2010, 12:34pm

nagachika $B$H?=$7$^$9!#(B

$B$9$_$^$;$s!"%j%S%8%g%sHV9f$r4V0c$($F$^$7$?!#(B

r29602 -> r29600
r29604 -> r29603

$B$G$7$?!#(B

SASADA_Koichi · October 29, 2010, 4:25pm

[email protected]$G$9!#(B

At Fri, 29 Oct 2010 03:05:07 +0900,
SASADA Koichi wrote in [ruby-dev:42481]:

$B!!(Bfiddle $B$N2U=j$G(B SEGV $B$7$^$9!#[email protected]!"(Bfiddle [email protected]$1$G<B9T$9$k$H(B
SEGV $B$7$J(B

$B$$!#$-$A$s$HD4$Y$J$$$HL5M}$C$]$$$G$9!#(BObjectSpace.each_objects
$BAjEv$r$7(B

$B$F$$$k$N$G!“C/$+$,JQ$J%*%V%8%’%/%[email protected]@.$7$F$$$k!”[email protected]$H;W$$$^$9!#(B

$B!!$3$l$O!"$3$N;EAH$_$,$&$^$/[email protected]$1$G!"2~D{HG$G$O$-$C$A$j(B
test-all TESTS=fiddle $B$G(B SEGV $B$7$^$7$?!#(B

Fiddle::Closure#allocate$B$,(Bcif$B$r=i4|2=$7$F$$$J$$$+$i$N$h$&$G$9!#(B
$B$I$3$+$G(Binitialize$B$5$l$J$$$^$^$N%$%s%9%?%s%9$,$"$k$s$G$7$g$&!#(B
$B$3$l$G(BSEGV$B$7$^$9!#(B

10000.times{ObjectSpace.memsize_of(Fiddle::Closure.allocate)}

allocate()$B$K(B MEMZERO(closure->cif, ffi_cif, 1) [email protected]$1$G$b(B
$BMn$A$J$/$J$j$^$9$,!"$=$b$=$b(Bcif$B$r%]%$%s%?$K$9$kI,MW$b$J$$$h$&$K(B
$B8+$($^$9!#(B

diff --git a/ext/fiddle/closure.c b/ext/fiddle/closure.c
index 5e9ede7…157b97c 100644
— a/ext/fiddle/closure.c
+++ b/ext/fiddle/closure.c
@@ -5,7 +5,7 @@ VALUE cFiddleClosure;
typedef struct {
void * code;
ffi_closure *pcl;

ffi_cif * cif;

ffi_cif cif;
int argc;
ffi_type **argv;
} fiddle_closure;
@@ -23,7 +23,6 @@ dealloc(void * ptr)
#else
munmap(cls->pcl, sizeof(cls->pcl));
#endif

xfree(cls->cif);
if (cls->argv) xfree(cls->argv);
xfree(cls);
}
@@ -37,7 +36,7 @@ closure_memsize(const void * ptr)
if (ptr) {
size += sizeof(*cls);
#if !defined(FFI_NO_RAW_API) || !FFI_NO_RAW_API
size += ffi_raw_size(cls->cif);

size += ffi_raw_size(&cls->cif);
#endif
size += sizeof(*cls->argv);
size += sizeof(ffi_closure);
@@ -148,7 +147,6 @@ allocate(VALUE klass)
closure->pcl = mmap(NULL, sizeof(ffi_closure), PROT_READ |
PROT_WRITE,
MAP_ANON | MAP_PRIVATE, -1, 0);
#endif

closure->cif = xmalloc(sizeof(ffi_cif));

return i;
}
@@ -185,7 +183,7 @@ initialize(int rbargc, VALUE argv[], VALUE self)
rb_iv_set(self, “@ctype”, ret);
rb_iv_set(self, “@args”, args);
cif = cl->cif;

cif = &cl->cif;
pcl = cl->pcl;

result = ffi_prep_cif(cif, NUM2INT(abi), argc,

SASADA_Koichi · October 29, 2010, 5:25pm

(2010/10/29 11:31), nagachika wrote:

r29602 $B$+$i$N(B trunk $B$G$O(B rb_classext_t $B$N9=B$$,JQ99$K$J$C$?$?$a(B
ObjectSpace.memsize_of $B$O(B Class/Module $B$NDj?t$N%F!<%V%k$V$s$N(B
$B%a%b%j%5%$%:$r2C;;$7$=$3$M$F$$$^$9!#(B

$B!!$"$j$,$H$&$4$6$$$^$9!#<h$j9~$_$^$9!#(B

SASADA_Koichi · October 29, 2010, 4:37pm

[email protected]$G$9!#(B

(2010/10/28 18:25), SASADA Koichi wrote:

$B!!(Btest-all $B$,%a%b%j$r?)$$$9$.$k$C$F$$$&$s$G!"3F%F%9%H8e$K$I$NDxEY%a%b%j(B
$B$r?)$&$N$+D4$Y$k%W%m%U%!%$%i$r:n$C$F$_$^$7$?!#(B

$B!!H?BP0U8+$b$J$$$7!"$^$!!“1F6A$b>/$J$+$m$&!”$H$$$&[email protected]$7$FFM$C9~$_(B
$B$^$7$?!#(B

SASADA_Koichi · October 29, 2010, 10:57pm

2010/10/29 Nobuyoshi N. [email protected]:

Fiddle::Closure#allocate$B$,(Bcif$B$r=i4|2=$7$F$$$J$$$+$i$N$h$&$G$9!#(B
$B$I$3$+$G(Binitialize$B$5$l$J$$$^$^$N%$%s%9%?%s%9$,$"$k$s$G$7$g$&!#(B
$B$3$l$G(BSEGV$B$7$^$9!#(B

10000.times{ObjectSpace.memsize_of(Fiddle::Closure.allocate)}

allocate()$B$K(B MEMZERO(closure->cif, ffi_cif, 1) [email protected]$1$G$b(B
$BMn$A$J$/$J$j$^$9$,!"$=$b$=$b(Bcif$B$r%]%$%s%?$K$9$kI,MW$b$J$$$h$&$K(B
$B8+$($^$9!#(B

$B==J,$KG<[email protected]=PMh$kM}M3$G$9$M!#%Q%C%A$K$bLdBj$J$$$H;W$$$^$9!#AaB.;[email protected]$KLdBj$rH/8+$7$F$/$l$FK\Ev$K46<U$7$^$9!#(B
$B$N$V$5$s$,:#2s$N%Q%C%A$r%3%_%C%H$7$F$/$l$?$i!"<!2s;d$,K,F|$9$k:]$K%S!<%k$r$*$4$k$3$H$r$*LsB+$$$?$7$^$9!#(B