# BinData (or alternative)

**URL:** <https://www.ruby-forum.com/t/bindata-or-alternative/138543>\
**Category:** Ruby\
**Created:** [May 24, 2008, 11:05am UTC](https://www.ruby-forum.com/t/bindata-or-alternative/138543 "2008-05-24T11:05:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Michael\_TSRichter](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@Michael\_TSRichter](https://www.ruby-forum.com/u/Michael_TSRichter)\
**Post date:** [May 24, 2008, 11:05am UTC](https://www.ruby-forum.com/t/bindata-or-alternative/138543/1 "2008-05-24T11:05:15Z")

</div>

Is there any documentation out there that shows the full use of this gem  
(like, for example, how to actually use the “BinData::Choice” class in a  
non-trivial example)? Alternatively, is there an  
easier/better-documented gem out there for readably parsing binary data  
in strings?

---

<div class="post-metadata">

**Author:** ![Michael\_TSRichter](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@Michael\_TSRichter](https://www.ruby-forum.com/u/Michael_TSRichter)\
**Post date:** [May 24, 2008, 12:08pm UTC](https://www.ruby-forum.com/t/bindata-or-alternative/138543/2 "2008-05-24T12:08:03Z")

</div>

This is my minimal example. I’m trying to map terms from a wire  
protocol into native Ruby types. The general structure of terms in this  
wire protocol is:

```
    byte:131 + byte:tag + (complex structure according to second tag
    type)

```

As an example of such a secondary tag type, there is one of the simplest  
structures: the “atom”. (It essentially maps to a Ruby symbol  
conceptually.) In the wire protocol an atom is: byte:100 + word:length

- length:string-data. An example would be the atom equivalent to the  
symbol :abcde. Wrapped in the full term definition it would be “\203d  
\000\005abcde” where \203 is the term tag, d is the atom tag, \000\005  
is the length of the atom and abcde is the actual atom value.

Recognizing the atom with BinData is a joy. It’s trivial, it’s  
readable, it’s generally a lot of fun. I have dozens of such sub-terms  
to write, but each one would likely literally only take me minutes (with  
a few exceptions for which c.f. below). It’s the whole wrapping it up  
together into the final term that’s causing problems.

Looking at the BinData classes, it seems to me that the Choice class is  
perfect for what I need. Sadly BinData comes with very little  
comprehensive tutorial documentation and very terse reference  
documentation. One of the gaping holes in its coverage is, exactly, the  
choice data type. As a result I’m left with this huge hole in my  
ability to implement this wire protocol with readable code.

Below I have replicated my minimal test case. Class Atom is the one  
that tears apart the atom subtype and it works like a charm. (You will  
also note how easy it is to read it.) The test string beneath it is  
easily parsed and returns exactly the kind of thing I want to be able to  
use. BinData saves the day! (You don’t want to know what my first  
round of coding this looked like!)

Class Term, on the other hand, is my attempt to add that extra 131  
wrapping. It has so far resisted any attempt to make it do anything  
other than generate errors about how nilClass doesn’t have some method I  
know nothing about (id2class). Which makes sense, of course. I can’t  
get anything but that message, however, or even weirder ones when I  
start writing desperation code to figure out how it works. Reading the  
source doesn’t help much – there’s just nothing there by way of  
explaining how things fit together that helps me find the root of the  
problem.

Now if the 131-term was the only issue facing me, I’d just hack around  
it by checking the first byte of each term for 131 and then parsing the  
rest based on the second byte. But it’s not that simple. (Of course.)  
What I’ve removed from this example are some very complex cases. There  
is, for example, the compressed payload. It has a tag, an uncompressed  
length and a string of bytes that, when uncompressed, contain another  
secondary term. There are also some term types that are containers for  
arbitrary sequences of other terms (tuples, lists, etc.). Each of these  
term types requires recursively parsing the binary string to extract  
their term by tag type. This can also get arbitrarily deep. It’s not a  
situation that’s amenable to trivial hacks. It needs robust binary  
parsing and dynamic type building.

So my question boils down to this: first, how can I get the simple case  
below working? And if I can’t is there any kind of binary  
parsing/unpacking library out there that can handle this kind of thing?  
(The ones I checked out on RubyForge didn’t seem able to handle  
arbitrary, recursive extraction like what I need.)

```
    require 'rubygems'
    require 'bindata'
    require 'stringio'
    require 'zlib'

    TAG_MAP = {
      TAG_ATOM = 100 => :atom,
      TAG_TERM = 131 => :term
    }

    class Atom < BinData::Struct
      endian :big

      uint8 :tag, :value => TAG_ATOM, :check_value
    => TAG_ATOM
      uint16 :len, :value => lambda { data.length }
      string :data, :read_length => :len
    end

    p test_string = [TAG_ATOM].pack('C') + "\0\5" + "abcde"
    p Atom.new.read(StringIO.new(test_string))

    class Term < BinData::Struct
      @choicelist = []
      TAG_MAP.each_key do |x|
        @choicelist[x] = [:uint8, {:value => TAG_MAP[x]}]
      end

      endian :big

      uint8 :tag, :value => TAG_TERM, :check_value =>
    TAG_TERM
      choice :term, :choices => @choicelist, :selection => :tag
    end

    p test_string = [TAG_TERM].pack('C') + test_string
    p Term.new.read(StringIO.new(test_string))
```