Ruby try to allocate memory when forking

I am trying to run this code at a Ubuntu 13.10 64bits with 256mb:

array = []
str = “0”

20_000_000.times do |i|
puts free -m if i % 1_000_000 == 0
array << str.next!
end

But I get “Cannot allocate memory” when Ruby try to call “free -m”.

As you can see, my server still has enough memory to run free:

         total       used       free     shared    buffers

cached
Mem: 238 172 66 0 5
40
-/+ buffers/cache: 126 112
Swap: 0 0 0
testing.rb:7:in ``’: Cannot allocate memory - free (Errno::ENOMEM)
from testing.rb:7:in block in <main>' from testing.rb:6:intimes’
from testing.rb:6:in `’

Everything goes right if I remove backtick call.

It seems Ruby is invalidating memory before change process binary.

Ruby is CoW friendly when forking, isn’t it?!

What version of Ruby are you using?

I have tryied 1.9.3, 2.0.0 and 2.1.0.

Daniel L. [email protected] wrote:

I am trying to run this code at a Ubuntu 13.10 64bits with 256mb:

It seems Ruby is invalidating memory before change process binary.

Ruby is CoW friendly when forking, isn’t it?!

Ruby 2.0 and later is CoW-friendly. Even then, using 64-bit on a
256MB machine is probably too tight.

Eric W.,

I am just reproducing an error at a small hardware. Actually, it happens
at a 1.6GB RAM machine. The ruby process has 1GB at moment it is trying
to backticking a lightweight utility.

Wait, that script you posted used 1GB of memory on your system?
Or a different script used 1GB, because that script only uses 160M RSS
(266M VM) on my similar Debian 7.0 x86_64 system.

Can you try with vm.overcommit_memory=1 sysctl set? That would allow
the system to always overcommit instead of heuristic overcommit (0, the
default).

You can also save around 8KB of memory by using system(“free -m”)
instead of puts free -m.

On Sun, Jan 19, 2014 at 11:39 PM, Daniel L. [email protected]
wrote:

Eric, a different script use 1GB. It is a library calling identify at
a RoR application running under Unicorn.

Everything works fine if I use Kernel#system instead Kernel#`. I think
there is something odd at pipe_open function.

pipe_open likely needs memory for file handlers and buffers. If you do
not open a pipe (as with system) then you obviously do not need that
memory.

Kind regards

robert

Eric, a different script use 1GB. It is a library calling identify at
a RoR application running under Unicorn.

Everything works fine if I use Kernel#system instead Kernel#`. I think
there is something odd at pipe_open function.

I don’t think it needs more them 112 MB just to open a pipe.

On Tue, Jan 21, 2014 at 6:14 PM, Daniel L. [email protected]
wrote:

I don’t think it needs more them 112 MB just to open a pipe.

Very likely so, yes. But there are other reasons why malloc() can fail
AFAIK (ulimit -v comes to mind).

And: maybe a shell is started for the backtick version.

Kind regards

robert

Can I just point out that using (({ puts ... })) is a bit of an
antipattern. Why bother capturing the output and then immediately
printing it, without doing any other work on it? In this case I’d always
use (({ system ‘…’ })) anyway.

The fact that #system works is, in this case, an added bonus.

Also: have you checked your Ruby object space? You’re allocating 20
million extra strings there, each one several bytes bigger than the one
in “0”.next. Surely that adds a bit of overhead.

If you are right I will not be able to use backquote at a Virtual Box
machine with just 112 MB. But I can. I guess there is a bug.

By the way:
$ ulimit -v
unlimited

Matthew, As I have said to Eric, I am just trying to isolate a situation
that happens at my server.

I am investigating a similar “cannot allocate memory issue”.
In order to reproduce the problem I created this simple gist:

https://gist.github.com/adamniedzielski/8635539

I am running this on Ubuntu x64 with 8 GB RAM memory.

The strategy is simple: allocate much memory by creating huge array and
try to use backtick operator or system method.

You will have to adjust the big number in such a way that the Ruby
process takes more memory than the amount of free memory in your system.

In my case the script will output:

system fails
ls fails

I tested it on Ruby 1.9.3, 2.0.0, 2.1.0

Adam, I’ve created an issue at Ruby’s redmine.

https://bugs.ruby-lang.org/issues/9433

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs