Run this program and observe memory usage.
def leaker
a = “”
10000.times do
a = a + “buble”
end
a = “”
end
loop do
leaker
end
On 16.02.2008 13:12, Piotr S. wrote:
end
loop do
leaker
end
Your point being?
robert
Robert K. wrote:
Your point being?
robert
I run this on ruby-1.8.6 (Linux and cygwin [WinXP]).
Why memory used by ruby process grow?
Is that normal behaviour ?
I wrote the same progam on python and perl and I have discover that that
this problem occur only in ruby.
On Feb 16, 2008, at 12:39 PM, Piotr S. wrote:
Robert K. wrote:
Your point being?
robert
I run this on ruby-1.8.6 (Linux and cygwin [WinXP]).
Why memory used by ruby process grow?
Is that normal behaviour ?
I ran that code via irb and compared the memory usage to running
‘loop {}’ in irb.
Your code used more memory but it oscillated around a fixed point
that was about 10 megabytes larger than the empty loop process, it
didn’t grow.
This was on Mac OS X 10.4, ruby 1.8.6.
Guys, it’s worth to send it to ruby-core and ask there. I checked
ruby1.9 and it has the same issue :/.
–
Rados³aw Bu³at
http://radarek.jogger.pl - mój blog
Piotr S. wrote:
Run this program and observe memory usage.
Slightly modified program, with option to GC each cycle:
def leaker
a = “”
10000.times do
a = a + “buble”
end
a = “”
end
gc = ARGV[0] == “gc”
300.times do |i|
printf “%6d %10s”, i, ps h -o rss #{$$}
leaker
GC.start if gc
end
With ruby-1.8.6p111 (linux), memory usage (RSS) seems to grow without
bound. With the extra GC, it varies between a low of about 4Mb and a
high point that’s on the same curve as without the extra GC.
Grain of salt and all…
I have simple, but ugly solution for this problem.
class String
alias plus +
def +(a)
res = self.plus(a)
GC.start
return res
end
end
2008/2/16, Piotr S. [email protected]:
Robert K. wrote:
Your point being?
I run this on ruby-1.8.6 (Linux and cygwin [WinXP]).
Why memory used by ruby process grow?
Is that normal behaviour ?I wrote the same progam on python and perl and I have discover that that
this problem occur only in ruby.
Well, concluding from the postings not all versions of Ruby on all
platforms suffer this phenomenon. As far as I can see so far you have
proven that
(1) on cygwin on Windows XP and on Linux
(2) Ruby version 1.8.6
(3) will allocate and free memory in a way that
(4) the OS shows continuous increasing size of used memory
(5) for a particular program, namely one that allocates memory at a high
rate.
Whether that’s a bug or not is probably really a question for ruby
core. There might be multiple causes for this observed behavior
including but not limited to Ruby’s GC.
Kind regards
robert
2008/2/18, Piotr S. [email protected]:
I have simple, but ugly solution for this problem.
class String
alias plus +
def +(a)
res = self.plus(a)
GC.start
return res
end
end
That’s not a solution. That’s a bad workaround which might degrade
performance of other applications.
Cheers
robert
To perhaps underscore Robert’s point, I did this in IRB on the OCI
version on WinXP, and its behavior appeared (just watching memory
usage in the Windows task manager) very similar to the with-gc variant
on cygwin, even though it was run without explicit GC calls.
(Interestingly, about 27 MB of the excess over what IRB was using
before running this remained in use afterward until an explicit
GC.start, and the explicit call to GC only dropped that by about 21
MB.)
On Feb 16, 4:12 am, Piotr S. [email protected] wrote:
end
loop do
-- Posted viahttp://www.ruby-forum.com/.
leaker
end
Not a bug in any way. It’s simply that the garbage collector has not
been trigger. It runs only when it needs to. As you realized you, you
can force it to run by doing ‘GC.start’. If you run ‘GC.start’ at the
end of your program, you’ll see the memory reclaimed.
This is a typical behavior for a mark/sweep garbage collector. Perl
and Python are reference count, freeing memory the instant it goes out
of scope. Ruby waits to free memory until it needs memory to free some.
Piotr S. wrote:
Obviously this is a just an example to illustrate the issue, but
def leaker
a = “”
10000.times do
a << “buble”
end
a = “”
end
loop do
leaker
end
does not exhibit the same memory usage as the OP.
-Justin
[email protected] wrote:
Not a bug in any way. It’s simply that the garbage collector has not
been trigger. It runs only when it needs to. As you realized you, you
can force it to run by doing ‘GC.start’. If you run ‘GC.start’ at the
end of your program, you’ll see the memory reclaimed.This is a typical behavior for a mark/sweep garbage collector. Perl
and Python are reference count, freeing memory the instant it goes out
of scope. Ruby waits to free memory until it needs memory to free some.
Yes, I agree. I analyzed source code of gc.c. GC was triggered each time
when malloc_increase reach malloc_limit. Variable malloc_limit was
increased by factor proportional to (malloc_increase - malloc_limit). In
each loop cycle malloc_increase > malloc_limit because malloc_increase
is sum of arithmetic series of string length between GC calls.
On Feb 19, 2008 8:59 AM, [email protected] [email protected] wrote:
Not a bug in any way. It’s simply that the garbage collector has not
been trigger. It runs only when it needs to. As you realized you, you
can force it to run by doing ‘GC.start’. If you run ‘GC.start’ at the
end of your program, you’ll see the memory reclaimed.This is a typical behavior for a mark/sweep garbage collector. Perl
and Python are reference count, freeing memory the instant it goes out
of scope. Ruby waits to free memory until it needs memory to free some.
Basically it’s true. But notice that this program should use constant
of maxium memory (maybe I’m wrong). When method ‘leaker’ is finished
all objects created in it are candidates to free (but GC isn’t called
yet) because there is no reference to them (in particular binding or
scope reference). When method ‘leaker’ is called again, Ruby tries
create new objects and instead of increasing heap capacity it should
run GC first and frees all unnecessary objects.
Calling manually GC is not solution because… it isn’t work in this
case. Calling GC.start after ‘leaker’ frees some memory but heap
capacity is constanly increased so it leads to using big amount of
memory. Joel VanderWerf showed it in his graph.
Second, GC.start shouldn’t be used in typical case. It could affect
whole program (specially slow down program ;)).
def leaker
a = Array.new(1_000_000) {|i| "blah"}
end
loop do
printf “%10s”, ps h -o rss #{$$}
leaker
end
Why this program doesn’t cause memory leaks?
btw, great story about GC from why the lucky stiff:
http://whytheluckystiff.net/articles/theFullyUpturnedBin.html
Rados³aw Bu³at
http://radarek.jogger.pl - mój blog
<No. of cycle> * “buble”.length + 1 (‘\0’)
in my program:
n = 10000
“buble”.size = 5
S = 10000 * (5 + 1 + 50000 + 1)/2 = 5000 * 50007 = 250_035_000 bytesIn your program the strings occupy only
1_000_000 * (“blah”.length + 1) = 6_000_000 bytes
You refer to cycles inside method “leaker”. But this method always
creates the same amount of objects so I don’t see reason why ruby eats
more and more memory for each “leaker” call.
loop do
leaker # call leaker, it creates some amount of objects
here all objects are possibly to free (there is no reference to
them)
so when “leaker” is called one more time and it tries to
increase heap capacity it should first frees all these objects
end
My point is that this simple program should use constant amount of
maximum (like mine simple program). memory but it doesn’t (even if
we call GC manually it frees memory but heap capacity increases so
each time maximum used memory is greater than before).
–
Rados³aw Bu³at
http://radarek.jogger.pl - mój blog
“R” == =?ISO-8859-2?Q?Rados=B3aw Bu=B3at?= writes:
R> My point is that this simple program should use constant amount of
R> maximum (like mine simple program). memory but it doesn’t (even if
R> we call GC manually it frees memory but heap capacity increases so
R> each time maximum used memory is greater than before).
Write it like this
def leaker
GC.disable
a = “”
10000.times do
a = a + “buble”
end
GC.enable
a = “”
end
Guy Decoux
Radosław Bułat wrote:
def leaker a = Array.new(1_000_000) {|i| "blah"} endloop do
printf “%10s”,ps h -o rss #{$$}
leaker
end
Why this program doesn’t cause memory leaks?
Beacuse in this program
a = ""
10000.times do
a = a + "buble"
end
in each cycle ruby creates a new string with length:
<No. of cycle> * “buble”.length + 1 (’\0’)
No. of cycle Heap
-
a = "buble" -
a = "bublebuble" , a' = "buble" -
a = "bublebublebuble", a' = "bublebuble", a'' = "buble"
etc.
so after n-cycles we have
S = n * ((“buble”.length + 1) + ((“buble”.length * n + 1)) / 2
in my program:
n = 10000
“buble”.size = 5
S = 10000 * (5 + 1 + 50000 + 1)/2 = 5000 * 50007 = 250_035_000 bytes
In your program the strings occupy only
1_000_000 * (“blah”.length + 1) = 6_000_000 bytes
Piotr S. wrote:
malloc(size)
malloc_increase = 0
}
In my day they were called libraries. Now its called “wikipedia”.
But efore you read up, ask yourself how you think it might work. One
hint. Its paradoxical. To first collect garbage you could identify
everything you don’t wan to throw away.
HTH
On 2/20/08, Eliot M. [email protected] wrote:
In my day they were called libraries. Now its called “wikipedia”.
I hadn’t seen this article before, a quick skim and I’d say it looks
like a decent survey, although I’d like to see a little more credit
given to some of the GC innovators:
Henry Baker,
Dave Ungar
Eliot M. ?
But efore you read up, ask yourself how you think it might work. One
hint. Its paradoxical. To first collect garbage you could identify
everything you don’t wan to throw away.
Yes indeed, unless you want a potentially unpleasant surprise! Far
worse for a GC to throw away non-garbage than not to throw away all
garbage.
Rick DeNatale
My blog on Ruby
http://talklikeaduck.denhaven2.com/
Radosław Bułat wrote:
You refer to cycles inside method “leaker”. But this method always
creates the same amount of objects so I don’t see reason why ruby eats
more and more memory for each “leaker” call.
Haw gc works ? Here is pseudocode:
malloc_limit = 8_000_000
malloc_increase = 0;
malloc(size)
{
malloc_increase += size;
if (malloc_increase > malloc_limit)
garbage_collect()
}
garbage_collect()
{
malloc_limit = malloc_limit + (malloc_increase - malloc_limit) *
some_factor
malloc_increase = 0
}