Proxy caching

Hi all,

I would like to use nginx to cache some responses from my backends.
There are basically 2 types of responses I would like to cache.

  • public images that never expire
  • private content

The first one was easy, got that working already. Since my application
uses timestamps in the html when referring to images, changes to them
will be instantly picked up.

The second one is way harder.
Responses can become quite large (1 upto 10Mb).
My backend application is putting etags on the responses so it can
return 304s if nothing changed.
Also, it puts a 5min max-age cache-control on it,
Lastly, responses are marked private, because they are behind http basic
authentication (handled by the backend app).

What I would like nginx to do is the following:
It should store the first response for a certain url, using the current
http auth in the key (so it won’t accidentally become publicly
available).
If a second request comes in for the same url with the same http auth
within 5 minutes: serve from cache.
If a second request comes in for the same url with the same http auth
after 5 mintutes: ask backend using the if-not-modified header.

  • if modified, store the new response.
  • if 304 not modified, respond from cache, update cache to be valid for
    5 minutes again.
    If a request comes in for the same url but with other http auth: ignore
    cache and ask backend.

The http auth stuff seems solvable by using the http auth in the
cache_key.
The other 2 things I would like to solve are:

  • I can’t find clear info about nginx’s handling of etag and
    if-not-modified. Will it work as described above out-of-the-box?
    It’s especially important that nginx uses if-not-modified when querying
    the backend, even if the client request itself didn’t contain it.
    Ofcourse if the client request did include it, it should receive a nice
    304 too.

  • The backend sends Cache-Control: private for these responses. I would
    like nginx to cache them though.
    Ignoring the Cache-Control header doesn’t seem to work (because without
    cache-control at all, caching seems disabled)
    Another option might be to have the backend set cache-control to public
    and have nginx set it back to private, but that sounds a bit hacky.

Does anyone have any suggestions about how to proceed?
Thanks a lot!

Mathijs

Posted at Nginx Forum:

Hello!

On Sat, May 07, 2011 at 08:49:43AM -0400, bluescreen303 wrote:

will be instantly picked up.
It should store the first response for a certain url, using the current
cache and ask backend.
304 too.
Right now nginx can’t use conditional requests to refresh cache.

  • The backend sends Cache-Control: private for these responses. I would
    like nginx to cache them though.
    Ignoring the Cache-Control header doesn’t seem to work (because without
    cache-control at all, caching seems disabled)
    Another option might be to have the backend set cache-control to public
    and have nginx set it back to private, but that sounds a bit hacky.

Solution is to ignore Cache-Control header and enable caching
explicitly with proxy_cache_valid directive.

Maxim D.

Hi,

give varnish [1] a look, its probably better suited for such a scenario.

[1] http://www.varnish-cache.org/