I just discovered that ruby allocates memory in 8Mb chunks.
Forgive my ignorance.
Is there a way to change this behaviour? Since my app runs in an
environment with very strict memory constraints, a memory spike of 8Mb
can be lethal…
I would change it to a lower figure.
It doesn’t matter if I have to patch my ruby source files in order to
achieve this goal.
Any clues?
That may not be Ruby’s fault entirely. In many cases, the size of memory
chunks delivered by “malloc” and its friends is determined by the
compiler or operating system. I’ve worked on systems where you got the
next higher power of two, for example. If you asked for 5,000 you got
8192, if you asked for 65 you got 128, etc. And many modern systems
won’t give you less than a physical page.
A physical page on most x86 operating systems is 4096 bytes, so 8 MB is
2048 pages. There is also an option to run larger pages in Linux. What
OS and architecture are you on?
P.S.: I’m cross-posting this to ruby-core – that’s where patches are
discussed.
That may not be Ruby’s fault entirely. In many cases, the size of memory
chunks delivered by “malloc” and its friends is determined by the
compiler or operating system. I’ve worked on systems where you got the
next higher power of two, for example. If you asked for 5,000 you got
8192, if you asked for 65 you got 128, etc. And many modern systems
won’t give you less than a physical page.
A physical page on most x86 operating systems is 4096 bytes, so 8 MB is
2048 pages. There is also an option to run larger pages in Linux. What
OS and architecture are you on?
P.S.: I’m cross-posting this to ruby-core – that’s where patches are
discussed.
Thank you for answering.
My app runs in a linux gentoo environment on an arm processor embedded
device.
I cross-compile ruby with an arm-linux compiler in a gnu linux host.
The device has only 64Mb of ram and the last allocation of 8Mb happening
when the process is near 30-32Mb results in a NOEMEM error.
I just discovered that ruby allocates memory in 8Mb chunks.
Forgive my ignorance.
Is there a way to change this behaviour? Since my app runs in an
environment with very strict memory constraints, a memory spike of 8Mb
can be lethal…
I would change it to a lower figure.
Thank you for answering.
My app runs in a linux gentoo environment on an arm processor embedded
device.
I cross-compile ruby with an arm-linux compiler in a gnu linux host.
The device has only 64Mb of ram and the last allocation of 8Mb happening
when the process is near 30-32Mb results in a NOEMEM error.
Interesting … not many people I know use Gentoo embedded. I run Gentoo
on my workstations and tried the cross-development chain on my Gumstix.
But it was a lot of work and the “native” Gumstix development tools were
a lot lighter weight.
Do you absolutely have to have Ruby 1.8? You might be able to use Ruby
1.6, which has fewer “features” and is a bit lighter weight as a result.
“miniruby” might also work. And there are other lighter-weight languages
– Forth, of course, and Lua.
On 16.04.2008, at 16:50, M. Edward (Ed) Borasky wrote:
Any clues?
A physical page on most x86 operating systems is 4096 bytes, so 8 MB
is 2048 pages. There is also an option to run larger pages in Linux.
What OS and architecture are you on?
From _why’s article “The Fully Upturned Bin” 1 I see that by
default Ruby allocates a minimum of 8 MB for the object heap. Isn’t
this related to the OP’s question?
On Thu, Apr 17, 2008 at 5:35 AM, M. Edward (Ed) Borasky [email protected] wrote:
Do you absolutely have to have Ruby 1.8? You might be able to use Ruby 1.6,
I remember a thread where they used 1.6 and had problems with ruby
crashes,
it was too late for them to upgrade and they offered a bounty for a fix
(though I don’t remember what was the end of the story…)
I just discovered that ruby allocates memory in 8Mb chunks.
Forgive my ignorance.
Is there a way to change this behaviour? Since my app runs in an
environment with very strict memory constraints, a memory spike of 8Mb
can be lethal…
I would change it to a lower figure.
Thank you all for the very useful hints. I solved a problem I had for a
very long time. I thought it was a memory leak but I couldn’t find any.
Now I know that it was simply ruby allocating memory. Due to the heavy
memory constraints I have in my enviroement (an arm-linux embedded
device) an allocation of 8Mb of memory could cause a crash of my app due
to not enough memomry.
To solve the problem I simply changed #define GC_MALLOC_LIMIT 8000000 to #define GC_MALLOC_LIMIT 1000000 in gc.c. That is enough to solve my
problem and to not cause the garbage collector to be called too often.
Thank you again,
Emanuele R…
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.