# Binaryparse: length in bytes of block definition?

**URL:** <https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719>\
**Category:** Ruby\
**Created:** [June 7, 2009, 7:15pm UTC](https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719 "2009-06-07T19:15:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Markus\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@Markus\_F](https://www.ruby-forum.com/u/Markus_F)\
**Post date:** [June 7, 2009, 7:15pm UTC](https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719/1 "2009-06-07T19:15:35Z")

</div>

Hi,

is there a way to derive the size in bytes of a custom  
BinaryBlocker::Blocker definition?

class Foo \< BinaryBlocker::Blocker  
has\_one :offset, :int32  
has\_one :length, :int32  
end

Deriving the length would yield 8.

I’ve been using bit-struct for similar tasks, but it doesn’t support  
nesting class definition. However, bit-struct provides a  
round\_byte\_length() for that purpose.

Is there an equivalent for it in binaryparse?

thanks,

- Markus

---

<div class="post-metadata">

**Author:** ![Markus\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@Markus\_F](https://www.ruby-forum.com/u/Markus_F)\
**Post date:** [June 8, 2009, 8:22pm UTC](https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719/2 "2009-06-08T20:22:21Z")

</div>

Markus F. wrote:

> I’ve been using bit-struct for similar tasks, but it doesn’t support  
> nesting class definition. However, bit-struct provides a  
> round\_byte\_length() for that purpose.

Can you describe the nesting you are trying to do? Bit-struct does have  
some nesting capability, and my working copy now supports embedded  
vectors (fixed length arrays of bit-structs). This is an addition in the  
last week or so, but I can make it available.

Nesting example:

class NestedPart \< BitStruct  
unsigned :x, 5  
unsigned :y, 3  
char :s, 5\*8  
end

class Container \< BitStruct  
nest :n1, NestedPart, “Nest 1”  
nest :n2, NestedPart, “Nest 2”  
end

The more or less equivalent effect using vectors instead:

class MyEntry \< BitStruct  
unsigned :x, 5  
unsigned :y, 3  
char :s, 5\*8  
end

class Container \< BitStruct  
vector :n, MyEntry, “some entries”, :length =\> 2  
end

---

<div class="post-metadata">

**Author:** ![Markus\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@Markus\_F](https://www.ruby-forum.com/u/Markus_F)\
**Post date:** [June 9, 2009, 12:06am UTC](https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719/3 "2009-06-09T00:06:06Z")

</div>

Hi,

first, I guess I’ve to apologize in some way, because I was the OP last  
week and you kindly, very quickly, posted a suggestion on pastebin and I  
was completely ignorant. Actually, I was in limbo because I had so many  
questions and didn’t know where to start.

Joel VanderWerf wrote:

> Can you describe the nesting you are trying to do?

Currently, I’m writing a reader for the BSP file format, as described  
here: [Unofficial Quake 3 Map Specs](http://graphics.stanford.edu/~kekoa/q3/)

The week before, I was writing a MD3 model reader, which works already  
thanks to bit-struct 🙂

Back to the BSP format: since it uses some fixed arrays, and it was one  
of your post which suggested binaryparse, I tried that one and succeeded  
so far, except the following:

1. it was incredible hart to get _into_ binaryparse as you basically  
have to learn from the source, as there’s no real usable rdoc doc

2. probably due 1), I’ve no idea how to read float values with it. I  
just don’t get it, I guess ☹ It does not seem to have out of the box  
support, however it seems to provide some kind of support via custom  
packing/unpacking, but I can’t figure out how.

Due 2), I don’t support reading the lumps containing floats ATM. Right  
now I don’t need them, but reading all data is the goal.

> ```
> char :s, 5*8
> unsigned :x, 5
> unsigned :y, 3
> char :s, 5*8
> 
> ```
> 
> end
> 
> class Container \< BitStruct  
> vector :n, MyEntry, “some entries”, :length =\> 2  
> end

Now, I wouldn’t mind going back to bit-struct for the BSP reading stuff,  
as I find the documentation outstanding well! However, through  
binaryparse I discovered one thing I’m really enjoying: it supports  
direct reading from an IO-type object. So I can pass either a File IO or  
an StringIO object directory, which is really comfortable.

Maybe bit-struct supports that in some way, but I couldn’t find  
anything. Basically, with bit-struct I’ve to know how much to pass to a  
BitStruct class. Binaryparse, in contrast, uses the given structure  
information to know itself how much to consume.

class Header \< BinaryBlocker::Blocker  
has\_one …  
has\_one …  
end

class Datadir \< BinaryBlocker::Blocker  
…  
…  
end

io = File.new(“data”)

Header.new io  
Datadir.new io

That’s really nice compared to

Header.new io.read(Header.round\_byte\_length)  
Datadir.new io.read(Datadir.round\_byte\_length)

But I can see that there are different philosophies at work, given that  
it’s named round\_byte\_length, because reading single bits wouldn’t be  
really possible that way, therefore the “rest” reader …

About your upcoming addition and your example: how would a simple  
integer array look like?

C-struct example:

typedef struct {  
float normal[3];  
float dist;  
} dplane\_t;

Ruby (?):

class MyFloat \< BitStruct  
float :value, 32  
end

class Plane \< BitStruct  
vector :normal, MyFloat, :length =\> 3  
float :dist, 32  
end

This wold mean I need to access plane.normal[0].value I guess … having  
not to need the additional .value would be nice, e.g.

class Plane \< BitStruct  
vector :normal, :length =\> 3 do  
float 32  
end  
float :dist, 32  
end

Would multiple dimension be supported, too? Like C-style float[2][2] ?

class Plane \< BitStruct  
vector :normal, :length =\> 3 do  
vector :length =\> 3 od  
float 32  
end  
end  
float :dist, 32  
end

Ok, I think it’s getting out of control now 😉

thanks

- Markus

---

<div class="post-metadata">

**Author:** ![Markus\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@Markus\_F](https://www.ruby-forum.com/u/Markus_F)\
**Post date:** [June 13, 2009, 1:54pm UTC](https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719/4 "2009-06-13T13:54:47Z")

</div>

Hello Joel,

I’ve now rewritten my code to take advantage of you latest 0.12 changes  
and have to say that it was a very pleasing experience. Everything  
worked out of the box and the addition of the IO object was a real time  
safer in moving from binaryparser to be bit-struct-only.

During this rewrite-session I made some notes how my further bit-struct  
experience went. If you’re annoyed by them, simply ignore 'em 😉

- Allow block also for “nest” ?

It may sound redundant in the first place, but when it’s just for  
providing more structure without requiring an extra class definition, it  
could be useful:

class Foo \< BitStruct  
nest :coord do  
signed :x, 8  
signed :y, 8  
end  
end

Foo.new.coord.x

- Is explicit :length for vector necessary?

compared to

class Tag \< BitStruct  
signed :flags, 32  
vector :axis, Vec, 3  
end

Or am I missing any ambiguities because of the simple cases?

- Allow length not only in bits, but also bytes?

In my cases I’ve never encountered a situation where I needed a member  
of a structure to represent anything which is not on a byte boundary.

All my signed and unsigned int have to read “32” (32bit, 4bytes) all  
over the place. For strings I’m writing “64\*8” for string with 64  
characters.

How about a new default\_option :granularity which is “1” by default.  
When set to “8”, every specified length (except for vector …) is  
multiplied by that factor before used?

- Why “vector” and not just “array”?

I’m getting pretty confused all the time due the term vector 🙂 I’m  
always thinking about graphical vectors, etc. For, me arrays would also  
better describe the environment how the data is used (array of ints …  
vector of ints … ?).

Joel VanderWerf wrote:

> > class MyFloat \< BitStruct  
> > Plane.new.normal[2].value  
> > As you said, looks like one layer to much. Thanks for your block-support  
> > to vector, could this maybe be detected for simple cases?

class Plane \< BitStruct  
vector :normal, :length =\> 3 do  
signed 32  
end  
end

I.e., having only one element and deliberately skip the naming to be  
prepared for

Plane.new.normal[2]

> As you can see, the reader method is fine, but the writer has to be  
> broken down into a sequence of writes to each nested chunk… yuck.
> 
> The reason it doesn’t allow simply
> 
> md.row[2].col[7].x = 1.23  
> […]
> 
> Any other ideas…?

Unfortunately not at this time. I’m using bit-struct solely for reading  
binary data ATM and not creating them.

thanks,

- Markus

---

<div class="post-metadata">

**Author:** ![Markus\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c4cdca/32.png) [@Markus\_F](https://www.ruby-forum.com/u/Markus_F)\
**Post date:** [June 10, 2009, 2:20am UTC](https://www.ruby-forum.com/t/binaryparse-length-in-bytes-of-block-definition/169719/5 "2009-06-10T02:20:17Z")

</div>

Markus F. wrote:

> Now, I wouldn’t mind going back to bit-struct for the BSP reading stuff,  
> as I find the documentation outstanding well! However, through  
> binaryparse I discovered one thing I’m really enjoying: it supports  
> direct reading from an IO-type object. So I can pass either a File IO or  
> an StringIO object directory, which is really comfortable.

That’s a useful addition. I’ve added that to bit-struct 0.12 now (along  
with the vector stuff), you can get it at:

[http://redshift.sourceforge.net/bit-struct](http://redshift.sourceforge.net/bit-struct)

> Ruby (?):
> 
> class MyFloat \< BitStruct  
> float :value, 32  
> end
> 
> class Plane \< BitStruct  
> vector :normal, MyFloat, :length =\> 3  
> float :dist, 32  
> end

and then

Plane.new.normal[2].value

works, but yes it seems like one layer too many.

> Would multiple dimension be supported, too? Like C-style float[2][2] ?
> 
> class Plane \< BitStruct  
> vector :normal, :length =\> 3 do  
> vector :length =\> 3 od  
> float 32  
> end  
> end  
> float :dist, 32  
> end

Yes, that’s supported now:

class MD \< BitStruct  
vector :row, :length =\> 3 do  
vector :col, :length =\> 10 do  
float :x, 32  
end  
end  
end

md = MD.new  
rows = md.row  
row = rows[2]  
cols = row.col  
col = cols[7]  
col.x = 1.23  
cols[7] = col  
row.col = cols  
rows[2] = row  
md.row = rows

p md  
p md.row[2].col[7].x

As you can see, the reader method is fine, but the writer has to be  
broken down into a sequence of writes to each nested chunk… yuck.

The reason it doesn’t allow simply

md.row[2].col[7].x = 1.23

is that all those intermediate objects have to be something, either  
separate (copied) bit-structs, or proxies that delegate back to the  
original. If the latter, there is IMO the possibility of confusion  
as these proxies propagate around and the original object changes  
unexpectedly.

Any other ideas…?