Forum: Ruby Creating byte arrays

F4c2dcb24f2828cde331ba6a2c9ea10d?d=identicon&s=25 Jabari Z. (jabari_z)
on 2014-08-15 17:46
I need to create a byte array of 262144 bytes initialized to all '0'
values.

I've seen examples where they use strings to do this.

If I do:     x = Array.new(262144,0)

I will get an array of 262144 elements of '0' values, but will they be
stored as bytes (each element taking only 8-bits)?

I know you can convert and integer to bytes as:

[500000].pack("N").unpack("cccc") => [0, 7, -95, 32]

Also, what method returns the total number of bytes that an array uses.

x.size|length  returns the number of array elements but not the amount
of memory used.

Thanks
4a65f01f7ece0b720bdb0de3c3db089e?d=identicon&s=25 Dansei Yuuki (blutorange)
on 2014-08-15 20:50
Consider using C(++) for low level, memory or performance critical
tasks. You can embed C in ruby.

Just open irb, create a big array like

> (x=Array.new(3*10**8){0}).class

and watch your RAM usage. On my machine, I'm getting a bit more than 8
bytes (64bit wordsize) for each '0' ( = size of 0.object_id + array
overhead).
Using
99999999999999999999999999999999 instead of 0 results in the same RAM
usage. (Because there is only one object, see below, and try: a=999 ;
b=999 ; a.object_id == b.object_id)

As you mentioned, you could use

> (x=Array.new(3*10**7){"\x00"}).class

I get almost 54 bytes per null byte. Note that Array.new(10,[1]) will
only create one instance of the object, while Array.new(10){[1]} will
create 10 instances. To see this, try

> x = Array.new(10,[0])
> x[0][0] = 9
> p x

Even this uses 8 byte for each array entry

>(x=Array.new(3*10**8)).class

And the conclusion is, ruby needs to store at least the Object#object_id
(64bit on my machine) for
each array entry. Ruby arrays are not fit for the task you're trying to
accomplish.

Changing to strings,

> (x="\x00" * 3*10**8).class

This gives me 300 MB RAM usage, ie 1 byte for each null byte.

Getting the size of an object can be tricky in ruby; arrays store
references to objects, not the object itself. See also
https://www.ruby-forum.com/topic/156648

https://gist.github.com/camertron/2939093 is a half-guess based class to
estimate the memory used by an object. Don't expect magic.
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.