I’ve been seeing rare malformed responses from nginx with a strange 26
byte header prepended to them.
$ od -c file.part | head
0000000 037 213 \b \0 \0 \0 \0 \0 \0 003 002 \0 \0 \0 377 377
0000020 003 \0 \0 \0 \0 \0 \0 \0 \0 \0 H T T P / 1
0000040 . 1 2 0 0 O K \r \n S e r v e
The first two bytes look suspiciously like a gzip header.
I saw a problem like this back in April '09 when I first deployed nginx
as a reverse proxy in front of Apache:
I upgraded versions at the time to 0.6.35, and the problem persisted.
Turns out, I had gzip on in apache, as well as in nginx. Disabling gzip
in apache seemed to correct the issue. We ran with that configuration,
with nginx on port 80, and apache behind it, on the same box for a year
without further issue.
Recently, we moved nginx to a separate box on the same LAN, keeping the
same version (0.6.35 – I know, it’s old, but it’s been stable).
Suddenly, we’re seeing very similar symptoms again. It doesn’t happen
reliably, and it doesn’t happen often. I had one user report 3
instances over the course of 15 minutes. After receiving a garbled
response, a second request for the same URL seems to succeed.
We still have one site going through nginx 0.6.35 on the same server as
apache without issue. Only the “remote” nginx installation is
I’m not seeing any errors in the apache or nginx error logs for these
requests, and I’m confident that apache is not gzipping in this context,
only nginx (the nginx access log shows a response size about 25% that of
apache, suggesting that apache’s bodies are infact uncompressed)
We’ve upgraded to 0.7.65, but I’m not really expecting that to resolve
the issue – if it does, I’ll certainly report back. Our sysop crawled
the nginx changelogs and much of the mailing list archives, and didn’t
find anything that sounded relevant.
So, has anyone seen anything like this before? Any leads would be