# Binary file modification

**URL:** <https://www.ruby-forum.com/t/binary-file-modification/73236>\
**Category:** Ruby\
**Created:** [October 23, 2006, 4:41am UTC](https://www.ruby-forum.com/t/binary-file-modification/73236 "2006-10-23T04:41:52Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 4:41am UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/1 "2006-10-23T04:41:52Z")

</div>

Hi

I’ve been modifying a binary file that contains various data including  
some audio. I’m trying to add my own audio (instead of the audio in the  
file) which I believe I have done but I need to modify the  
content-length header in the file which indicates the length of the  
audio sample. I’ve been reading the content-length data from the file  
using something similar to :

f=open(config[:file],“rb”)  
f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET  
length=f.read(3).unpack(“H2H2H2”).hex.to\_i  
f.close  
=\> an integer

This basically opens the file as a binary file, skips to the audio  
content header data position and then unpacks 3 bytes into a string  
(formatted as hex) which is then converted to an integer value.

I’d like to be able to reverse this process and take any an integer  
value (1024 in the case shown below) and write it to a binary file with  
some header and footer data - something like :

mydata = SOMEHEADERDATA  
mydata += [“1024”].pack(“someformat”)  
mydata += AUDIODATA  
f=open(config[:file],“wb”)  
f.write(mydata)  
f.close

However I’m a bit stuck on how to pack the data (if this is the correct  
solution). If someone could point me in the correct direction it would  
be much appreciated.

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 8:38am UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/2 "2006-10-23T08:38:55Z")

</div>

> -----Original Message-----  
> From: removed\_email\_address@domain.invalid  
> [[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Rob L.  
> […]  
> I’ve been reading the content-length data from  
> the file  
> using something similar to :
> 
> f=open(config[:file],“rb”)  
> f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET  
> length=f.read(3).unpack(“H2H2H2”).hex.to\_i

That should die because Array#hex doesn’t exist, but I get the idea. If  
you  
do this kind of thing a lot with different kinds of data then you should  
note that to\_i and to\_s both take optional base arguments, so  
foo.unpack(‘H\*’).first.to\_i(16) should work

[…]

> I’d like to be able to reverse this process and take any an integer  
> value (1024 in the case shown below) and write it to a binary  
> file with  
> some header and footer data - something like :  
> […]  
> mydata += [“1024”].pack(“someformat”)  
> […]  
> However I’m a bit stuck on how to pack the data (if this is  
> the correct  
> solution).

Here’s the problem - if you need exactly three bytes then you are going  
to  
have to apply your own padding, since the pack routines will only pack  
directly as a long or a short which will mostly be 4 and 2 bytes - both  
of  
which could cause you problems. My hacks always involve \<\<'ing a single  
byte  
integer onto a string. In your case, here is a horrible oneliner which  
you  
should not use because it is gross.

num=1024  
num.to\_s(16)[0…2].instance\_eval {(self.reverse + ‘0’ \* (6 -  
self.length)).reverse}.scan(/…/).inject(‘’) {|s,byte| s \<\<  
byte.to\_i(16)}

You could also try googling BitStruct, which is a ruby library that  
might  
help by defining these headers and footers as structure objects.

Cheers,

ben

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 10:31am UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/3 "2006-10-23T10:31:55Z")

</div>

Rob L. wrote:

> f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET  
> some header and footer data - something like :  
> be much appreciated.  
> Ben’s right, you can use BitStruct for this:

require ‘bit-struct’

class AudioData \< BitStruct

unsigned :audio\_length, 3\*8, :endian =\> :little  
rest :data

# Note: don’t use :length as the name of a field, because it will

# conflict with the #length method inherited from String.

# the :endian option can also be :big, :network (== :big), or :native

end

audio\_data = AudioData.new

data = “foo bar baz”  
audio\_data.data = data  
audio\_data.audio\_length = data.length

p audio\_data  
p audio\_data.to\_s

**END**

Output:

#  
“\v\000\000foo bar baz”

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 11:50am UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/4 "2006-10-23T11:50:30Z")

</div>

On Mon, 23 Oct 2006, Rob L. wrote:

> Hi
> 
> I’ve been modifying a binary file that contains various data including  
> some audio. I’m trying to add my own audio (instead of the audio in the  
> file) which I believe I have done but I need to modify the

You’ve had other answers that answer the question as put. Another  
approach that may be useful is to look at SNG, a textual means of  
accessing the contents of PNG graphics:

[http://www.faqs.org/docs/artu/ch06s01.html#id2910193](http://www.faqs.org/docs/artu/ch06s01.html#id2910193)

```
    HTH
    Hugh

```

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 1:05pm UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/5 "2006-10-23T13:05:49Z")

</div>

Ben N. wrote:

> > -----Original Message-----  
> > From: removed\_email\_address@domain.invalid  
> > [[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Rob L.  
> > […]  
> > I’ve been reading the content-length data from  
> > the file  
> > using something similar to :
> > 
> > f=open(config[:file],“rb”)  
> > f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET  
> > length=f.read(3).unpack(“H2H2H2”).hex.to\_i
> 
> That should die because Array#hex doesn’t exist, but I get the idea. If  
> you  
> do this kind of thing a lot with different kinds of data then you should  
> note that to\_i and to\_s both take optional base arguments, so  
> foo.unpack(‘H\*’).first.to\_i(16) should work

Thanks for pointing that out, it’s strange as the code definately works  
(it returns correct content lengths repeatedly for different files) but  
as you say Array#hex doesn’t exist - I’ll look into this …

> […]
> 
> > I’d like to be able to reverse this process and take any an integer  
> > value (1024 in the case shown below) and write it to a binary  
> > file with  
> > some header and footer data - something like :  
> > […]  
> > mydata += [“1024”].pack(“someformat”)  
> > […]  
> > However I’m a bit stuck on how to pack the data (if this is  
> > the correct  
> > solution).
> 
> Here’s the problem - if you need exactly three bytes then you are going  
> to  
> have to apply your own padding, since the pack routines will only pack  
> directly as a long or a short which will mostly be 4 and 2 bytes - both  
> of  
> which could cause you problems. My hacks always involve \<\<'ing a single  
> byte  
> integer onto a string. In your case, here is a horrible oneliner which  
> you  
> should not use because it is gross.
> 
> num=1024  
> num.to\_s(16)[0…2].instance\_eval {(self.reverse + ‘0’ \* (6 -  
> self.length)).reverse}.scan(/…/).inject(‘’) {|s,byte| s \<\<  
> byte.to\_i(16)}

I’m intrigued by this, I can see you are repacking the int value into  
three bytes but I’m still unclear as to how you would then add this to  
the file. I guess it needs to be packed as 3 bytes - my guess was

num=1024  
newpacket += [num].pack(“3C”)

However this doesn’t seem to work using the reading scheme above. Could  
you tell me where I’m going wrong ?

> You could also try googling BitStruct, which is a ruby library that  
> might  
> help by defining these headers and footers as structure objects.

I’m looking at using bitstruct for the next version of the project, but  
as I’ve almost got this working now I’d like to avoid a re-write at this  
stage if possible !

> Cheers,
> 
> ben

Thanks for all the help - you’ve saved me hours of work already !

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 1:29pm UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/6 "2006-10-23T13:29:48Z")

</div>

> Thanks for pointing that out, it’s strange as the code definately works  
> (it returns correct content lengths repeatedly for different files) but  
> as you say Array#hex doesn’t exist - I’ll look into this …

A quick reply to myself - I’ve just checked the code and I’m missing an  
Array#join call in the above example, so the example wasn’t correct …

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 3:05pm UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/7 "2006-10-23T15:05:29Z")

</div>

> -----Original Message-----  
> From: removed\_email\_address@domain.invalid  
> [[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Rob L.  
> Sent: Monday, October 23, 2006 6:06 PM  
> To: ruby-talk ML  
> Subject: Re: Binary file modification
> 
> Ben N. wrote:  
> […]
> 
> > num=1024  
> > num.to\_s(16)[0…2].instance\_eval {(self.reverse + ‘0’ \* (6 -  
> > self.length)).reverse}.scan(/…/).inject(‘’) {|s,byte| s \<\<  
> > byte.to\_i(16)}
> 
> I’m intrigued by this, I can see you are repacking the int value into  
> three bytes but I’m still unclear as to how you would then  
> add this to  
> the file.

It’s already packed. The return value of the long expression will be a 3  
byte string which you can insert as you like (print it to the IO handle,  
\<\<  
it to another string etc etc). The accumulator in the inject method is a  
string {|str,byte| … might have been more readable, sorry.

ben

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 24, 2006, 11:30am UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/8 "2006-10-24T11:30:33Z")

</div>

> It does seem to work for small values like 1024 but not larger ones. I  
> know there will be an upper limit to the integer size I can fit into the  
> three bytes but the 359447 value is one I’ve taken from an existing file  
> so I believe this is a valid integer to try and pack into these three  
> bytes.
> 
> Any advice ?

Some more information on this - the maximum value that can be  
successfully packed is 4095

Any advice ?

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 23, 2006, 3:45pm UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/9 "2006-10-23T15:45:50Z")

</div>

> It’s already packed. The return value of the long expression will be a 3  
> byte string which you can insert as you like (print it to the IO handle,  
> \<\<  
> it to another string etc etc). The accumulator in the inject method is a  
> string {|str,byte| … might have been more readable, sorry.
> 
> ben

Sorry I should have studied it a little closer …

I’ve tried the following and your re-packing doesn’t seem to work :

f = open(“testfile”,“wb”)  
num=359447  
f.write(num.to\_s(16)[0…2].instance\_eval {(self.reverse + ‘0’ \* (6 -  
self.length)).reverse}.scan(/…/).inject(’’) {|s,byte| s \<\<  
byte.to\_i(16)})  
f.close  
f = open(“testfile”,“rb”)  
f.pos=0  
puts f.read(3).unpack(‘H\*’).first.to\_i(16)  
f.close

=\> 1404

It does seem to work for small values like 1024 but not larger ones. I  
know there will be an upper limit to the integer size I can fit into the  
three bytes but the 359447 value is one I’ve taken from an existing file  
so I believe this is a valid integer to try and pack into these three  
bytes.

Any advice ?

---

<div class="post-metadata">

**Author:** ![rob](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/838e76/32.png) [@rob](https://www.ruby-forum.com/u/rob)\
**Post date:** [October 29, 2006, 3:28pm UTC](https://www.ruby-forum.com/t/binary-file-modification/73236/10 "2006-10-29T15:28:44Z")

</div>

Joel VanderWerf wrote:

> Ben’s right, you can use BitStruct for this:

Forgot the link:

[http://raa.ruby-lang.org/project/bit-struct/](http://raa.ruby-lang.org/project/bit-struct/)