Having problems with rails connectivity to postgres (libpq?)

Rails
1

I have some daemons that connect to a PG database and are pretty busy.
Every few days at least one of them starts failing with this error.

PGError: lost synchronization with server: got message type “something”

Sometimes the message type is an empty string. Other times it’s a
letter like “E” or something. It seems to vary from incident to
incident.

This is highly annoying because at this point the daemon becomes
useless. If I stop the daemon and start it again everything works
fine. A few days later some other daemon or maybe the same one will
fail with a similar message.

I am running the latest version of rails 3.0.3 on an up to date ubuntu
connected to postgres 8.4.

Any idea of what is going on?

2

I have recently deployed a similar setup, Ubuntu 10.04, pgsql 8.4,
rails 3.0.3, pg gem etc.

I have started to see this error occasionally, that request fails then
subsequent requests are OK, the pgsql log says the client
disconnected!

Sequel::DatabaseDisconnectError (PGError: server closed the connection
unexpectedly

I’ve not had problems before with older versions.

Take a look at you postgresql log and see if you can get clues there,
I’m not sure what is causing my problems either.
/var/log/posgresql/…

3

On Sat, Dec 18, 2010 at 11:05 AM, Jim M. [email protected]
wrote:

I have recently deployed a similar setup, Ubuntu 10.04, pgsql 8.4,
rails 3.0.3, pg gem etc.

I have started to see this error occasionally, that request fails then
subsequent requests are OK, the pgsql log says the client
disconnected!

In my case the application is unable to connect back to the database.

Sequel::DatabaseDisconnectError (PGError: server closed the connection
unexpectedly

I was thinking it might be an activerecord problem but if you are
having the same issue with sequel maybe it’s at a lower level than
that.

Somebody suggested it might be a connection pool or a thread pool
issue but I have no way of testing that.

4

I’ve spent the day trying to track this down. I am not seeing it with
my older rails/activerecord app, only with the newer rails 3.0.3/
sequel app.

I “think” it may be the ssl connection to postgresql. On newer Ubuntu
systems a local tcp connection uses ssl by default (kind of odd but
still).

What I did was to remove the host,username,password settings from
database.yml which forced it to use a local unix pipe to connect, so
far I have not seen this error, but it is pretty infrequent.

You can also switch off SSL in the posgresql,conf file., check to see
if ssl is set to true in that file, or when yo connect at the command
line with

psql -h localhost yourdbname

if it prints out SSL connection (cipher: DHE-RSA-AES256-SHA, bits:
256) then it is using SSL.

Let me know.

5

On Sun, Dec 19, 2010 at 7:26 PM, Jim M. [email protected] wrote:

I’ve spent the day trying to track this down. I am not seeing it with
my older rails/activerecord app, only with the newer rails 3.0.3/
sequel app.

I am seeing it using activerecord on rails 3.0.3

The problem just occured and although the client is constantly putting
out errors like this

PGError: lost synchronization with server: got message type “o”,
length 544501536
PGError: lost synchronization with server: got message type “�”,
length -65536
PGError: lost synchronization with server: got message type “�”,
length -65536
PGError: lost synchronization with server: got message type “�”,
length -65536
PGError: lost synchronization with server: got message type “o”,
length 544501536
PGError: lost synchronization with server: got message type
“o”, length 544501536
PGError: lost synchronization with server: got message type
“o”, length 544501536

there is nothing in the postgres log.

I “think” it may be the ssl connection to postgresql. On newer Ubuntu
systems a local tcp connection uses ssl by default (kind of odd but
still).

What I did was to remove the host,username,password settings from
database.yml which forced it to use a local unix pipe to connect, so
far I have not seen this error, but it is pretty infrequent.

I can’t do this because my database is on another server.

You can also switch off SSL in the posgresql,conf file., check to see
if ssl is set to true in that file, or when yo connect at the command
line with

psql -h localhost yourdbname

if it prints out SSL connection (cipher: DHE-RSA-AES256-SHA, bits:
256) then it is using SSL.

It’s definitely using SSL. I might try turning off the SSL and see if
that does any good.

6

Yea turn off ssl if you are on local network and see if that
helps…

I suspect what is happening in my case, (and maybe yours), is the SSL
connection is “renegotiating” and failing, which is documented
elsewhere (google postgresql ssl renogotiation)

So far I have not seen this issue since I switched to a local
connection. Although I have two other older rails apps running
presumably with SSL and they have not exhibited this error.

7

I suspect what is happening in my case, (and maybe yours), is the SSL
connection is “renegotiating” and failing, which is documented
elsewhere (google postgresql ssl renogotiation)

So far I have not seen this issue since I switched to a local
connection. Although I have two other older rails apps running
presumably with SSL and they have not exhibited this error.

I have turned it off. Let’s see if the problem occurs again.

8

It turns out you can also turn off ssl in the database.yml file by
adding

sslmode: disable

Although I have found no way to determine if the connection is SSL or
not.