# Database Interaction Encoding Problem

**URL:** <https://www.ruby-forum.com/t/database-interaction-encoding-problem/148314>\
**Category:** Rails\
**Created:** [September 11, 2008, 5:37pm UTC](https://www.ruby-forum.com/t/database-interaction-encoding-problem/148314 "2008-09-11T17:37:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aawilson](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/e495f1/32.png) [@aawilson](https://www.ruby-forum.com/u/aawilson)\
**Post date:** [September 11, 2008, 5:37pm UTC](https://www.ruby-forum.com/t/database-interaction-encoding-problem/148314/1 "2008-09-11T17:37:20Z")

</div>

This is cross-posted to [railsforum.com](http://railsforum.com), so if you frequent both you may  
see it in both places.

So, my co-worker and I are working on this project, a Rails app using an  
MS SQL database with lots of legacy stuff built in. He’s stumbled  
across a strange error, and we don’t really know where to start. Here’s  
the gist of things:

This is an online place for buying things, and descriptions for each  
item are stored in  
the database as encoded strings. On a particular product, pulling a  
database object from the database with Product.find formats this string  
incorrectly, replacing encoded characters with ‘?’, in all environments  
on both servers we run this project from, EXCEPT from the CONSOLE of the  
production environment (in which case, everything comes out exactly as  
it should). In development environments, both the console and the app  
return mis-encoded strings. In the actual production application  
(diagnosed with logs, and shown on the actual page), the encoding is  
messed up. SO, the question is:

What could be giving different behavior between the console and the app  
of the same environment? Why isn’t this difference consistent among  
environments (i.e., why isn’t the output sane in the development  
environment console)? If we could nail that difference, we could  
compensate for whatever is giving this wonky output. As it is, we’re  
pretty stumped.

Thanks in advance for any response.

---

<div class="post-metadata">

**Author:** ![aawilson](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/e495f1/32.png) [@aawilson](https://www.ruby-forum.com/u/aawilson)\
**Post date:** [September 11, 2008, 6:06pm UTC](https://www.ruby-forum.com/t/database-interaction-encoding-problem/148314/2 "2008-09-11T18:06:26Z")

</div>

On 11 Sep 2008, at 16:37, Aa Wilson wrote:

> the gist of things:  
> production environment (in which case, everything comes out exactly as  
> compensate for whatever is giving this wonky output. As it is, we’re  
> pretty stumped.

Could be what your terminal is set to dispay, eg if the terminal on  
the production machine is set ti displaying stuff in ISO-Latin-1 and  
the underlying data is too but everyone else is expecting the data to  
be utf-8 then you might get what you observe. I’d check if the  
database encoding setting (in database.yml) matches what the database  
is doing and that any headers on the actual page saying what the  
content-type/charset are also match that.

Fred

---

<div class="post-metadata">

**Author:** ![aawilson](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/e495f1/32.png) [@aawilson](https://www.ruby-forum.com/u/aawilson)\
**Post date:** [September 11, 2008, 9:17pm UTC](https://www.ruby-forum.com/t/database-interaction-encoding-problem/148314/3 "2008-09-11T21:17:21Z")

</div>

Frederick C. wrote:

> On 11 Sep 2008, at 16:37, Aa Wilson wrote:
> 
> > the gist of things:  
> > production environment (in which case, everything comes out exactly as  
> > compensate for whatever is giving this wonky output. As it is, we’re  
> > pretty stumped.
> 
> Could be what your terminal is set to dispay, eg if the terminal on  
> the production machine is set ti displaying stuff in ISO-Latin-1 and  
> the underlying data is too but everyone else is expecting the data to  
> be utf-8 then you might get what you observe. I’d check if the  
> database encoding setting (in database.yml) matches what the database  
> is doing and that any headers on the actual page saying what the  
> content-type/charset are also match that.
> 
> Fred

It turned out to be a setting in our freetds.conf (client charset=utf8)  
that was missing. It took us forever to find out that was the case,  
because the damn production console behavior was throwing us off, so we  
worked our way through the entire chain before we eliminated everything  
but tds.

Thanks for your response!