# Failure cases for assignment of binary string to AR model

**URL:** <https://www.ruby-forum.com/t/failure-cases-for-assignment-of-binary-string-to-ar-model/132659>\
**Category:** Rails\
**Created:** [March 22, 2008, 7:03pm UTC](https://www.ruby-forum.com/t/failure-cases-for-assignment-of-binary-string-to-ar-model/132659 "2008-03-22T19:03:45Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Matt\_Olson](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/8dc957/32.png) [@Matt\_Olson](https://www.ruby-forum.com/u/Matt_Olson)\
**Post date:** [March 22, 2008, 7:03pm UTC](https://www.ruby-forum.com/t/failure-cases-for-assignment-of-binary-string-to-ar-model/132659/1 "2008-03-22T19:03:45Z")

</div>

I’ve run into a strange bug that I need some help with. I have a model  
with a binary column. I’m using Postgres 8.2.4 on the backend. For the  
majority of cases, I’m able to store and retrieve binary content using  
this setup with no problem. However, I’ve found two test cases that  
fail. Consider the following test code, taken from one of my actions:

# The test file I write out here contains out the binary data as

expected, so I know that @incoming\_blob is correct.  
f = File.open(“/tmp/incoming\_blob”, “w+”)  
f.write @incoming\_blob  
f.close

# Set the value in the AR record, but don’t even save it.

@staged\_extension.value = @incoming\_blob

# Now write out the value from the AR record, and it is truncated to

the first null byte.  
f = File.open(“/tmp/from\_ar\_#{params[:extension\_local\_ref]}”, “w+”)  
f.write @staged\_extension.value  
f.close

I’ve done the same thing from the console. It seems that the act of  
assigning the binary String to the ActiveRecord model truncates it to  
the first null byte. The really weird thing is that it works for other  
blobs (jpgs and other binary data). At first my code was saving to the  
database (of course) and I thought the bug was somewhere in the  
postgres driver, but it’s not – as you can see here, there is no  
database activity going on at all.

There’s nothing fancy going on with the model – I’ve stripped out all  
validations on this field, and I still run into this problem.

The two test cases that fail with the code above can be found here:  
[http://rubycloud.com/pg/1288.1024.jpg](http://rubycloud.com/pg/1288.1024.jpg)  
[http://rubycloud.com/pg/base.tsz](http://rubycloud.com/pg/base.tsz)

I have to think given the simplicity of this code that there is a bug  
in the ActiveRecord column assignment methods. I’m looking into it,  
but any ideas/insight/help you can give would be greatly appreciated.

Thanks,  
Matt

---

<div class="post-metadata">

**Author:** ![Matt\_Olson](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/8dc957/32.png) [@Matt\_Olson](https://www.ruby-forum.com/u/Matt_Olson)\
**Post date:** [March 22, 2008, 8:48pm UTC](https://www.ruby-forum.com/t/failure-cases-for-assignment-of-binary-string-to-ar-model/132659/2 "2008-03-22T20:48:35Z")

</div>

I have discovered the problem. My logic in thinking that there was no  
interaction with the Postgres adapter simply because there was no  
actual database interaction was flawed. Turns out that setting and  
retrieving values to/from the attributes hash makes calls to type\_cast  
which in turn makes calls to string\_to\_binary and binary\_to\_string.

The problem lies in  
ActiveRecord::ConnectionAdapters::PostgreSQLColumn. The heuristic used  
to determine whether a column’s value has been previously encoded by  
escape\_bytea is not sufficient in all cases.

In binary\_to\_string, it checks whether it should unescape a value by  
looking for the tell-tale sequence \nnn. However, it the two source  
files that are causing so much trouble, these sequences appear in the  
original data. So unescape\_bytea gets called on a block of data that  
was not previously escaped.

In base.tsz, the sequence \690 appears at byte offset 64,688.  
In 1288.1024.jpg, the sequence \754 appears at byte offset 27,316.

It seems that we need a more reliable way to mark a data block as  
previously escaped by escape\_bytea. This is a somewhat tricky problem.  
We could save a marker with the data and then remove it when  
unescaping it, but this would mean that the persisted data would only  
be usable from within Rails, or that any other code that works with  
the database would have to know about this trick. Kind of ugly. Any  
other ideas?

–Matt