Forum: Ruby BInary Files

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
F9fe16a7636a9f2e1e8819847aa7e461?d=identicon&s=25 Gene Vincent (genev)
on 2009-04-01 21:23
Hi I'm pretty new to Ruby and writing code in general. That being said
Im trying to learn how to create and modify(by a percentage) a binary
file. Any advise or pointers would be apprieciated.
770048af205ed307b8cf35ae2282ee2f?d=identicon&s=25 Michael Malone (Guest)
on 2009-04-01 22:11
(Received via mailing list)
Gene Vincent wrote:
> Hi I'm pretty new to Ruby and writing code in general. That being said
> Im trying to learn how to create and modify(by a percentage) a binary
> file. Any advise or pointers would be apprieciated.
>
My personal view, is that binary files should be avoided unless
absolutely necessary.  Being able to look through the contents of a file
and eyeball what's wrong is incredibly useful.  That being said, if you
absolutely need to write binary, take a look at the documentation for
the Array#pack method.  That should give you a nice starting point.

Michael

=======================================================================
This email, including any attachments, is only for the intended
addressee.  It is subject to copyright, is confidential and may be
the subject of legal or other privilege, none of which is waived or
lost by reason of this transmission.
If the receiver is not the intended addressee, please accept our
apologies, notify us by return, delete all copies and perform no
other act on the email.
Unfortunately, we cannot warrant that the email has not been
 altered or corrupted during transmission.
=======================================================================
F9fe16a7636a9f2e1e8819847aa7e461?d=identicon&s=25 Gene Vincent (genev)
on 2009-04-01 22:55
Michael Malone wrote:
> Gene Vincent wrote:
>> Hi I'm pretty new to Ruby and writing code in general. That being said
>> Im trying to learn how to create and modify(by a percentage) a binary
>> file. Any advise or pointers would be apprieciated.
>>
> My personal view, is that binary files should be avoided unless
> absolutely necessary.  Being able to look through the contents of a file
> and eyeball what's wrong is incredibly useful.  That being said, if you
> absolutely need to write binary, take a look at the documentation for
> the Array#pack method.  That should give you a nice starting point.
>
> Michael
>
> =======================================================================
> This email, including any attachments, is only for the intended
> addressee.  It is subject to copyright, is confidential and may be
> the subject of legal or other privilege, none of which is waived or
> lost by reason of this transmission.
> If the receiver is not the intended addressee, please accept our
> apologies, notify us by return, delete all copies and perform no
> other act on the email.
> Unfortunately, we cannot warrant that the email has not been
>  altered or corrupted during transmission.
> =======================================================================

sadly yes I do have to use some binary files for the test im trying to
run, but thank you very much for the info.
Gene
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-04-01 23:03
(Received via mailing list)
Gene Vincent wrote:
> Hi I'm pretty new to Ruby and writing code in general. That being said
> Im trying to learn how to create and modify(by a percentage) a binary
> file. Any advise or pointers would be apprieciated.

As noted, Array#pack is useful. If you want a DSL that fills in some
gaps in pack, try this: http://redshift.sourceforge.net/bit-struct.
68a08995ddf9b3f8a0b562ed0baf63af?d=identicon&s=25 Michael Bruschkewitz (Guest)
on 2009-04-03 10:50
(Received via mailing list)
"Joel VanderWerf" <vjoel@path.berkeley.edu> schrieb im Newsbeitrag
news:49D3D66F.9020200@path.berkeley.edu...
> As noted, Array#pack is useful. If you want a DSL that fills in some
> gaps in pack, try this: http://redshift.sourceforge.net/bit-struct.

That's a really nice lib.

Is any Lib out there which realizes a bit-stream?
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-04-03 20:06
(Received via mailing list)
Michael Bruschkewitz wrote:
>
> "Joel VanderWerf" <vjoel@path.berkeley.edu> schrieb im Newsbeitrag
> news:49D3D66F.9020200@path.berkeley.edu...
>> As noted, Array#pack is useful. If you want a DSL that fills in some
>> gaps in pack, try this: http://redshift.sourceforge.net/bit-struct.
>
> That's a really nice lib.
>
> Is any Lib out there which realizes a bit-stream?

what would that do?
68a08995ddf9b3f8a0b562ed0baf63af?d=identicon&s=25 Michael Bruschkewitz (Guest)
on 2009-04-04 08:45
(Received via mailing list)
"Joel VanderWerf" <vjoel@path.berkeley.edu> schrieb im Newsbeitrag
news:49D64FD7.3000006@path.berkeley.edu...
> Michael Bruschkewitz wrote:
>> Is any Lib out there which realizes a bit-stream?
> what would that do?

Writing output bit-wise to a (ex.) string. Read back.
Convert to/from char-array.

Example:
Given:
    a,b,c,d values of different bit-size
    a is a flag is 1 bit long,
    b is 5 bit long, c is 7 bit long.
    d is 3 bit long.
It should be possible:
    bs << a << (1==a ? b : c) << d
    bs >> a >> (1==a ? b : c) >> d

After insertion, bs should contain following bits (p is padding):
    1bbb bbdd dppp pppp
or
    0ccc cccc dddp pppp.

Other example would be var-length array of 5-bit values followed by
var-length array of 13-bit structs.

Use case: ASN.1 and similar formats.
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-04-05 19:58
(Received via mailing list)
Michael Bruschkewitz wrote:
>>> Is any Lib out there which realizes a bit-stream?

It would be nice for ruby to have some bit operation primitives like
extract(start_bit, n_bits) and insert and so on. Then bit stream ops
could be built op from these. The former would make a nice C ext...
68a08995ddf9b3f8a0b562ed0baf63af?d=identicon&s=25 Michael Bruschkewitz (Guest)
on 2009-04-06 08:35
(Received via mailing list)
"Joel VanderWerf" <vjoel@path.berkeley.edu> schrieb im Newsbeitrag
news:49D8F10F.2090705@path.berkeley.edu...
> Michael Bruschkewitz wrote:
>>>> Is any Lib out there which realizes a bit-stream?
>
> It would be nice for ruby to have some bit operation primitives like
> extract(start_bit, n_bits) and insert and so on. Then bit stream ops
> could be built op from these. The former would make a nice C ext...

I did some rudimentary work (enough for my purposes) here:
http://www.bruschkewitz.de/job/ruby/CBitStream.rb

There is no optimisation at all, just made it work.

I would prefer not to use a C-Extension unless it would be part of Ruby
standard library.

IMHO, there are enough possibilities to optimize in Ruby.
For example, it is not necessary to analyze the values bit by bit.
Instead, if the current buffer byte contains already 1 bit from former
output operations, remaining 7 bits, these 7 bits could be copied at
once.
It breaks down to shift the value to be written by some bits (if even
necessary), "or"-ing current buffer byte and assinging rest of output
bytes
to buffer.
I did similar work some years before in Modula.
Depending on output buffer representation (byte, word, dword, qword,
Array,
String, Bignum), this could be very effective even in Ruby.

So, fitting into the general performance landscape, an implementation
done
in Ruby would be enough, because the values to be written will - most
probably - be prepared in Ruby too.


Regards,
Michael B.
This topic is locked and can not be replied to.