# Unpacking signed shorts and integers with specified endianne **URL:** \ **Category:** Ruby\ **Created:** [June 19, 2007, 12:37am UTC](https://www.ruby-forum.com/t/unpacking-signed-shorts-and-integers-with-specified-endianne/107189 "2007-06-19T00:37:02Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\ **Post date:** [June 19, 2007, 12:37am UTC](https://www.ruby-forum.com/t/unpacking-signed-shorts-and-integers-with-specified-endianne/107189/1 "2007-06-19T00:37:02Z")
I’m deciphering a byte-based binary file structure which can be stored in either big or little endian. (The first byte of the file describes the format.) Among others, some of the fields are UINT32 (unsigned 4- byte integers), and some are INT32 (signed 4-byte integers). I know about BitStruct, but was trying to roll my own solution using String.unpack. (The file format has nested and repeating sections in it.) Given that I know exactly how many bytes I want for each field and whether the file is big or little endian, I thought it would be easy to pick a specific unpack character for each field. However, reading through the String.unpack docs, I can’t find something that corresponds to signed 2-byte and 4-byte integers for a given endianness. Here’s what I see (ASCII art ahead): | signed | unsigned | bytes | big | little | big | little | ------±------±-------±------±-------+ 1 | c | C | 2 | ? | ? | n | v | 4 | ? | ? | N | V | 1. I see “l” (lowercase L) which is 4 bytes treated as a signed integer…but in ‘native’ endian order. Does String.unpack not provide a way to unpack a 4-byte signed integer with a specified endianness? 2. I see “s” which is 2 bytes treated as a signed integer…but in ‘native’ endian order. Does String.unpack not provide a way to unpack a 2-byte signed integer with a specified endianness? (The file format doesn’t actually use any signed 2-byte integers, but I wanted to include them for completeness.) ---
**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\ **Post date:** [June 19, 2007, 8:11pm UTC](https://www.ruby-forum.com/t/unpacking-signed-shorts-and-integers-with-specified-endianne/107189/2 "2007-06-19T20:11:30Z")
On Jun 18, 4:32 pm, Phrogz [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > 1. I see “l” (lowercase L) which is 4 bytes treated as a signed > integer…but in ‘native’ endian order. Does String.unpack not provide > a way to unpack a 4-byte signed integer with a specified endianness? > > 2. I see “s” which is 2 bytes treated as a signed integer…but in > ‘native’ endian order. Does String.unpack not provide a way to unpack > a 2-byte signed integer with a specified endianness? Alright, let’s try asking this question a different way. Given a binary file spec like the following, where the first byte tells you whether to interpret the rest of the file as little- or big-endian, and given the presence of a signed integer in the file, how would you write a parser for this? endian UINT8 (1==little, 0==big) job\_count UINT8 # of repeated binary sections following this (jobs) JOB\* ``` Each JOB section is: foo UINT8 bar UINT16 jim UINT32 jam INT32 jill UINT8 ``` gib\_mark 0xdead # 2 bytes gib\_count UINT16 # of repeated binary sections following this (gibs) GIB+ ``` Each GIB section is: gob 8 chars blurb UINT8 ``` Would you invent a new character for String.unpack, split the string around it, and use knowledge of the native endianness of the platform you’re running on to decide whether to pull out those 4 bytes independently, reverse them, and then unpack the result as a signed integer? Is there some sweet trick you could do after extracting 4 bytes as an integer to switch the implied interpreted endianness? Would you patch String.unpack in C to add options for specific-endian signed shorts and integers? Can you easily do the above (including the repeating sub-binary sections) with BitStruct? ---
**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\ **Post date:** [June 19, 2007, 8:20pm UTC](https://www.ruby-forum.com/t/unpacking-signed-shorts-and-integers-with-specified-endianne/107189/3 "2007-06-19T20:20:33Z")
On Jun 18, 4:35 pm, Phrogz [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > 1. I see “l” (lowercase L) which is 4 bytes treated as a signed > integer…but in ‘native’ endian order. Does String.unpack not provide > a way to unpack a 4-byte signed integer with a specified endianness? > > 2. I see “s” which is 2 bytes treated as a signed integer…but in > ‘native’ endian order. Does String.unpack not provide a way to unpack > a 2-byte signed integer with a specified endianness? See the ‘N’, ‘n’, ‘V’ and ‘v’ directives. There are equivalent directives for floats as well - ‘E’, ‘e’, ‘G’ and ‘g’. Regards, Dan ---
**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\ **Post date:** [June 19, 2007, 8:39pm UTC](https://www.ruby-forum.com/t/unpacking-signed-shorts-and-integers-with-specified-endianne/107189/4 "2007-06-19T20:39:36Z")
On Jun 19, 2007, at 11:19 AM, Daniel B. wrote: > directives for floats as well - ‘E’, ‘e’, ‘G’ and ‘g’. > Those handle endianness, but not signed values. I suppose you could > unpack as unsigned, then manually test for the sign bit being set and > correct the value. Even uglier, you could unpack as unsigned with > desired endianness, repack as unsigned in native order, then unpack as > signed in native order. -Mark ---
**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\ **Post date:** [June 24, 2007, 6:43am UTC](https://www.ruby-forum.com/t/unpacking-signed-shorts-and-integers-with-specified-endianne/107189/5 "2007-06-24T06:43:25Z")
Mark Day wrote: > > See the ‘N’, ‘n’, ‘V’ and ‘v’ directives. There are equivalent > > directives for floats as well - ‘E’, ‘e’, ‘G’ and ‘g’. > > Those handle endianness, but not signed values. I suppose you could > unpack as unsigned, then manually test for the sign bit being set and > correct the value. Even uglier, you could unpack as unsigned with > desired endianness, repack as unsigned in native order, then unpack as > signed in native order. What bit-struct does in these cases is the first kind of ugly: # Let’s say we start with a negative number packed in # 16 bits, big-endian: x = -123 s = [x].pack(“n”) # Note that the sign is not packed with the number. It packs to the # same chars as 2\*\*16 + x bits = 16 max\_unsigned = 2 \*\* bits max\_signed = 2 \*\* (bits - 1) to\_signed = proc { |n| (n \>= max\_signed) ? n - max\_unsigned : n } puts to\_signed[s.unpack(“n”).first] # ==\> -123 (This has come up a few times on the list – search for “to\_signed”, for example.) It’s still a hack, though, and I’d like to see Gavin’s RCR go through, if the naming issues can be resolved.