Forum: NGINX proxy_cache incorrectly returning 304 Not Modified

2940bc7d4506f3e099e3dcc32a412b98?d=identicon&s=25 Jon Leighton (jonathan_l)
on 2014-01-08 15:58
(Received via mailing list)
Hi there,

I work on a site which has nginx in front of a Rails application, and we
use proxy_cache.

For the home page, our application returns a "max-age=600, public"
Cache-Control header, and we have nginx configured to cache the response
using proxy_cache.

This generally works fine, but last night nginx started to respond with
304 Not Modified to requests that *didn't* include any caching headers
(If-Modified-Since or ETag). Our Pingdom alerts showed this issue, and
here is the request/response captured:

GET / HTTP/1.0
User-Agent: Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)
Host: loco2.com

304 Not Modified
Cache-Control: max-age=600, public
Date: Tue, 07 Jan 2014 22:59:32 GMT
ETag: "900e1f11422519337c9ed25fad299ce0"
Server: nginx/1.4.4
Status: 304 Not Modified
Strict-Transport-Security: max-age=31536000
X-Cache-Status: HIT
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: c7ee78ab-df49-4467-bced-753b2cc622ab
X-UA-Compatible: chrome=1
X-XSS-Protection: 1; mode=block
Connection: Close

Does this look like a bug? Or could it be a configuration issue? I can't
think of any reason why this should be the correct thing for the proxy
cache to do.

Many thanks,

Jon

--
http://jonathanleighton.com/
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-01-09 15:47
(Received via mailing list)
Hello!

On Wed, Jan 08, 2014 at 02:57:18PM +0000, Jon Leighton wrote:

> 304 Not Modified to requests that *didn't* include any caching headers
> ETag: "900e1f11422519337c9ed25fad299ce0"
>
> Does this look like a bug? Or could it be a configuration issue? I can't
> think of any reason why this should be the correct thing for the proxy
> cache to do.

This easily can be a result of a misconfiguration (e.g.,
proxy_set_header used incorrectly) and/or backend
problem.

Additionally, response headers looks very suspicious.  There
shouldn't be "Status: 304 Not Modified" in nginx responses, and
"Connection: Close" capitalization doesn't match what nginx uses.
Unless it's something introduced by Pingdom interface, this may
indicate that it's checking something which isn't nginx.

--
Maxim Dounin
http://nginx.org/
2940bc7d4506f3e099e3dcc32a412b98?d=identicon&s=25 Jon Leighton (jonathan_l)
on 2014-01-10 13:19
(Received via mailing list)
Hello Maxim,

Thanks for your reply.
> "Connection: Close" capitalization doesn't match what nginx uses.
> Unless it's something introduced by Pingdom interface, this may
> indicate that it's checking something which isn't nginx.

Thank you. I reviewed my configuration and made some tweaks, but I'm
still having trouble. Actually I haven't seen the 304 Not Modified
again, but now I am getting a blank response body for the cached page
("/").

For example:

GET / HTTP/1.0
User-Agent: Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)
Host: loco2.com

200 OK
Cache-Control: max-age=600, public
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Fri, 10 Jan 2014 11:49:44 GMT
ETag: "8a8ca149d65dd1343c60366876821659"
Server: nginx/1.4.4
Status: 200 OK
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Cache-Status: HIT
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 0d85f7a2-cdbf-4288-a62d-05628abefdba
X-UA-Compatible: chrome=1
X-XSS-Protection: 1; mode=block
Connection: Close

[empty response]

Do you know of any reasons why the response might be blank? I copied the
cache file so I could review it, and I checked that the X-Request-Id
matches so it's definitely the same entry. The cache file *does* contain
the response body - in gzip form. I am using "gzip off" for this
location block, so I don't think nginx is interfering there. I'm at a
complete loss about why this is happening, any ideas you have would be
much appreciated.

Thanks,
Jon
2940bc7d4506f3e099e3dcc32a412b98?d=identicon&s=25 Jon Leighton (jonathan_l)
on 2014-01-10 13:54
(Received via mailing list)
On 10/01/14 12:18, Jon Leighton wrote:
> Do you know of any reasons why the response might be blank? I copied the
> cache file so I could review it, and I checked that the X-Request-Id
> matches so it's definitely the same entry. The cache file *does* contain
> the response body - in gzip form. I am using "gzip off" for this
> location block, so I don't think nginx is interfering there. I'm at a
> complete loss about why this is happening, any ideas you have would be
> much appreciated.

I just realised the obvious - that this is a problem for clients who
don't ask for gzip. Will fix that. Don't think that explains the 304
issue though...
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-01-10 17:06
(Received via mailing list)
Hello!

On Fri, Jan 10, 2014 at 12:18:18PM +0000, Jon Leighton wrote:

> >
> For example:
> ETag: "8a8ca149d65dd1343c60366876821659"
> Connection: Close
>
> [empty response]
>
> Do you know of any reasons why the response might be blank? I copied the
> cache file so I could review it, and I checked that the X-Request-Id
> matches so it's definitely the same entry. The cache file *does* contain
> the response body - in gzip form.

It looks like your Pingdom reports are a bit confusing, and most
likely there is a body in the response, but Pingdom doesn't show it
because it's gzipped.

You may try testing with telnet / curl to see what actually goes
on.

> I am using "gzip off" for this
> location block, so I don't think nginx is interfering there. I'm at a
> complete loss about why this is happening, any ideas you have would be
> much appreciated.

The "gzip off" isn't relevant, as it won't try to compress
anything with Content-Encoding already set anyway.

Note though that nginx doesn't understand "Vary: Accept-Encoding",
and will return cached response regardless of client's
Accept-Encoding.  To make sure gzipped responses aren't sent to
clients without gzip support you should either disable caching of
such responses, or switch off gzip of your backend (e.g., with
"proxy_set_header Accept-Encoding '';"), or switch on gunzip in
nginx (http://nginx.org/r/gunzip).

Please also see here for some debugging hints:

http://wiki.nginx.org/Debugging

--
Maxim Dounin
http://nginx.org/
2940bc7d4506f3e099e3dcc32a412b98?d=identicon&s=25 Jon Leighton (jonathan_l)
on 2014-01-10 17:37
(Received via mailing list)
Hello!

On 10/01/14 16:06, Maxim Dounin wrote:
> Note though that nginx doesn't understand "Vary: Accept-Encoding",
> and will return cached response regardless of client's
> Accept-Encoding.  To make sure gzipped responses aren't sent to
> clients without gzip support you should either disable caching of
> such responses, or switch off gzip of your backend (e.g., with
> "proxy_set_header Accept-Encoding '';"), or switch on gunzip in
> nginx (http://nginx.org/r/gunzip).

Thanks, this is the conclusion I came to also. Also I agree about
Pingdom probably just not making it clear that the response was all
binary.

I am hopefully going to solve it by putting the compression method
(based on Accept-Encoding) in the cache key. Not sure if the 304 issue
will come back but it's a step in the right direction at least.

Thanks,
Jon
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.