Proxy_cache_bypass with ssi caches template

Hi everyone,
I’ve been struggling with this for some time now trying to drop varnish
in favor for nginx.

Server: nginx/0.8.54

quick peek on problem - SSI on; proxy_cache_bypass caches backend
response as regular html page with HTTP/1.1 200 OK inside, so next fetch
from cache will return to client a page with SSI tags inside
(unprocessed).

And now the full configuration if the problem is still not clear.
My goal: force refresh of item in cache
Item has SSI tags inside which I would like to be processed (this is
obvious)

So far I’ve found that there’s no way of refreshing cache other than
deleting a file on hdd (which could lead to strange problems in my
opinion) or using proxy_cache_bypass. Bypass goes right to backend and
the response is cached, but with ssi on it’s cached wrong.

Configuration:

  • debian6-32 (running nginx 0.8.54)
  • bypass will be used when ‘Secret’ header is set
  • backend has two files - index.php and footer.php

Step 1) first fetch (without bypass) returns this:

[20:45:01] debian6-32:~# printf "GET /num/5 HTTP/1.0\r\nHost:
debian6-32\r\n\r\n" | nc.traditional debian6-32 80 | grep '>'
>>>> FOOTER 08:46:05
>>>> INDEX 08:46:05
[20:45:08] debian6-32:~# printf "GET /num/5 HTTP/1.0\r\nHost:
debian6-32\\r\n\r\n" | nc.traditional debian6-32 80 | grep '>'
>>>> FOOTER 08:46:05
>>>> INDEX 08:46:05

We can see that cache and SSI works fine (footer line is produced by
footer.php) and the whole page is cached

Step 2) let’s refresh cache using secret header

[20:45:16] ponury@eva:~ $ printf "GET /num/5 HTTP/1.0\r\nHost:
debian6-32\r\nSecret: 1\r\n\r\n" | nc.traditional debian6-32 80 | grep
'>>'
>>>> FOOTER 08:47:09
>>>> INDEX 08:47:09
[20:45:18] ponury@eva:~ $ printf "GET /num/5 HTTP/1.0\r\nHost:
debian6-32\r\nSecret: 1\r\n\r\n" | nc.traditional debian6-32 80 | grep
'>>'
>>>> FOOTER 08:47:11
>>>> INDEX 08:47:11
[20:45:20] ponury@eva:~ $

Also bypass works great each time backend is being hit.

Step 3) fetch that page without bypass:

[20:45:11] debian6-32:~# printf "GET /num/5 HTTP/1.0\r\nHost:
debian6-32\r\nCookie: refresh=1\r\n\r\n" | nc.traditional debian6-32 80
| grep '>'
<!--# include file="/footer.php" -->
>>>> INDEX 08:47:11
[20:45:23] debian6-32:~#

And here it is… clients receives unprocessed ssi header. The good news
is - its fresh. Bad news - it shouldn’t happen.

My current nginx config:
[code]
location / {
ssi on;

        proxy_cache my-cache;
        proxy_cache_valid  200 302  1m;

        if ($remote_addr ~* "192.168.0.128") {
            set $is_backend 1;
        }
        proxy_cache_bypass $is_backend $http_secret;

        proxy_pass http://192.168.0.128$uri$is_args$args;
    }

[/code]

So, is there something I’m missing or this just can’t be done?

Thank you in advance for any help.

Posted at Nginx Forum:

Hello!

On Mon, Jan 10, 2011 at 02:55:04PM -0500, bponury wrote:

Hi everyone,
I’ve been struggling with this for some time now trying to drop varnish
in favor for nginx.

Server: nginx/0.8.54

quick peek on problem - SSI on; proxy_cache_bypass caches backend
response as regular html page with HTTP/1.1 200 OK inside, so next fetch
from cache will return to client a page with SSI tags inside
(unprocessed).

[…]

So, is there something I’m missing or this just can’t be done?

The problem is that proxy_cache_bypass doesn’t work as of now (it
usually corrupts response headers, though may corrupt response
body/cause other problems as well). It may only be used with
identical proxy_no_cache.

The problem you see with SSI is related to corrupted response
headers - response no longer has Content-Type and not processed by
SSI as a result.

Maxim D.

Maxim D. Wrote:

The problem is that proxy_cache_bypass doesn’t
work as of now (it
usually corrupts response headers, though may
corrupt response
body/cause other problems as well). It may only
be used with
identical proxy_no_cache.

Well that makes sense now. I see that other headers are corrupted.

Which brings us back to my question - what is the best way to
force-refresh a page in cache? I want my backend to invalidate some
pages as they change. And those pages could have SSI inside. I know that
deleting a file from hdd works, but my invalidation requests will happen
many times, removing and recreating files again and again could have
performance impact. And it seems just wrong. Any help here?

Posted at Nginx Forum:

Hello!

On Mon, Jan 10, 2011 at 08:08:05PM -0500, bponury wrote:

Well that makes sense now. I see that other headers are corrupted.

Which brings us back to my question - what is the best way to
force-refresh a page in cache? I want my backend to invalidate some
pages as they change. And those pages could have SSI inside. I know that
deleting a file from hdd works, but my invalidation requests will happen
many times, removing and recreating files again and again could have
performance impact. And it seems just wrong. Any help here?

Right now the only option is to remove cache files, either
directly or with cache purge module.

Maxim D.