# Cache gzipped content with proxy\_pass **URL:** \ **Category:** NGINX\ **Created:** [August 16, 2010, 2:57pm UTC](https://www.ruby-forum.com/t/cache-gzipped-content-with-proxy-pass/193357 "2010-08-16T14:57:33Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![chrisl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58f4c7/32.png) [@chrisl](https://www.ruby-forum.com/u/chrisl)\ **Post date:** [August 16, 2010, 2:57pm UTC](https://www.ruby-forum.com/t/cache-gzipped-content-with-proxy-pass/193357/1 "2010-08-16T14:57:33Z")
Hi. AFAIK nginx only stores plain (uncompressed) content when acting as a caching proxy server. With gzip=on, content for gzip-accepting clients will be compressed by the server _for_ _every_ _request_ (?). Wouldn’t it be better to have two cache files like this: /data/nginx/cache/c29b7f54b2df7773722d382f4809d65029c /data/nginx/cache/c29b7f54b2df7773722d382f4809d65029c.gz Another solution would be to add the internal variable gzip\_ok to the config variable space. Then one could add this variable to proxy\_cache\_key: proxy\_cache\_key “…:$gzip\_ok:…” Currently I’ve added http\_accept\_enconding to the cache key, but since there are hundreds of different permutations for the value this is not an optimal solution. Chris Posted at Nginx Forum: > **[cache gzipped content with proxy\_pass](https://forum.nginx.org/read.php?2,120314,120314#msg-120314)** > > Hi. AFAIK nginx only stores plain (uncompressed) content when acting as a caching proxy server. With gzip=on, content for gzip-accepting clients will be compressed by the server \_for\_ \_every\_ \_request\_ (?). Wouldn't it be better to have two cache... ---
**Author:** ![chrisl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58f4c7/32.png) [@chrisl](https://www.ruby-forum.com/u/chrisl)\ **Post date:** [August 16, 2010, 4:48pm UTC](https://www.ruby-forum.com/t/cache-gzipped-content-with-proxy-pass/193357/2 "2010-08-16T16:48:48Z")
Hello! On Mon, Aug 16, 2010 at 08:56:47AM -0400, chrisl wrote: > AFAIK nginx only stores plain (uncompressed) content when acting as a > caching proxy server. Not really. nginx caches what upstream server sent to it. If response was compressed by upstream server - it will cache compressed response. > With gzip=on, content for gzip-accepting clients > will be compressed by the server _for_ _every_ _request_ (?). Yes - unless upstream server already returned precompressed content. > Wouldn’t it be better to have two cache files like this: > > /data/nginx/cache/c29b7f54b2df7773722d382f4809d65029c > /data/nginx/cache/c29b7f54b2df7773722d382f4809d65029c.gz You may want to follow this thread: > **[\[ANNOUNCE\] gunzip filter module 0.3](http://mailman.nginx.org/pipermail/nginx/2010-March/019471.html)** It is believed that separate gzip cache is better idea. > Another solution would be to add the internal variable gzip\_ok to the > config variable space. Then one could add this variable to > proxy\_cache\_key: > > proxy\_cache\_key “…:$gzip\_ok:…” > > Currently I’ve added http\_accept\_enconding to the cache key, but since > there are hundreds of different permutations for the value this is not > an optimal solution. The problem is that nginx itself and upstream server may have different ideas on what may be compressed. And you need upstream’s one here, not nginx’s one. Though probably it’s a good idea to expose nginx’s one anyway. Maxim D. ---
**Author:** ![chrisl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58f4c7/32.png) [@chrisl](https://www.ruby-forum.com/u/chrisl)\ **Post date:** [August 21, 2010, 1:28pm UTC](https://www.ruby-forum.com/t/cache-gzipped-content-with-proxy-pass/193357/3 "2010-08-21T13:28:56Z")
Maxim, thank you for your answer. I’ve patched nginx 0.7.67 to export gzip\_ok to config file variable space (see attachement). I’m not a C guy, but it seems to work 🙂 I’ currently using it like this: location /json-data/ { set $ae “”; if ($gzip\_ok) { set $ae “gzip”; } proxy\_pass [http://10.2.3.4/json-upstream/](http://10.2.3.4/json-upstream/); proxy\_cache\_key “$gzip\_ok:$uri”; proxy\_set\_header User-Agent “nginx”; proxy\_set\_header Accept-Encoding $ae; […] } I get cache keys like this: # grep -a ^KEY /var/spool/nginx/\* /var/spool/nginx/0c341794ed57bf03d72fb9a207f43d0e:KEY: :/json-data/123/45/ /var/spool/nginx/17961a45b339647f5e8f239dde713428:KEY: OK:/json-data/123/45/ By hiding the client User-Agent and Accept-Encoding, I deactivate upstream gzip detection logic. For this specific task this patch/config works perfectly. I more general approach would be nice, though. Chris — src/http/ngx\_http\_variables.c.orig 2010-08-21 12:17:42.000000000 +0200 +++ src/http/ngx\_http\_variables.c 2010-08-21 12:19:07.000000000 +0200 @@ -66,6 +66,10 @@ ngx\_http\_variable\_value\_t \*v, uintptr\_t data); static ngx\_int\_t ngx\_http\_variable\_request\_completion(ngx\_http\_request\_t \*r, ngx\_http\_variable\_value\_t \*v, uintptr\_t data); +#if (NGX\_HTTP\_GZIP) +static ngx\_int\_t ngx\_http\_variable\_gzip\_ok(ngx\_http\_request\_t \*r, - ngx\_http\_variable\_value\_t \*v, uintptr\_t data); +#endif static ngx\_int\_t ngx\_http\_variable\_request\_body(ngx\_http\_request\_t \*r, ngx\_http\_variable\_value\_t \*v, uintptr\_t data); static ngx\_int\_t ngx\_http\_variable\_request\_body\_file(ngx\_http\_request\_t \*r, @@ -123,6 +127,9 @@ #if (NGX\_HTTP\_GZIP) { ngx\_string(“http\_via”), NULL, ngx\_http\_variable\_header, offsetof(ngx\_http\_request\_t, headers\_in.via), 0, 0 }, - - { ngx\_string(“gzip\_ok”), NULL, ngx\_http\_variable\_gzip\_ok, - ``` 0, 0, 0 }, ``` #endif #if (NGX\_HTTP\_PROXY || NGX\_HTTP\_REALIP) @@ -1538,6 +1545,41 @@ return NGX\_OK; } +#if (NGX\_HTTP\_GZIP) +static ngx\_int\_t +ngx\_http\_variable\_gzip\_ok(ngx\_http\_request\_t \*r, - ngx\_http\_variable\_value\_t \*v, uintptr\_t data) +{ - ngx\_int\_t rc; - - if (!r-\>gzip\_tested) { - ``` rc = ngx_http_gzip_ok(r); ``` - - ``` if (rc != NGX_OK && rc != NGX_DECLINED) { ``` - ``` v->not_found = 1; ``` - ``` return NGX_OK; ``` - ``` } ``` - } - - if (r-\>gzip\_ok) { - ``` v->len = 2; ``` - ``` v->valid = 1; ``` - ``` v->no_cacheable = 0; ``` - ``` v->not_found = 0; ``` - ``` v->data = (u_char *) "OK"; ``` - - ``` return NGX_OK; ``` - } - - v-\>len = 0; - v-\>valid = 1; - v-\>no\_cacheable = 0; - v-\>not\_found = 0; - v-\>data = (u\_char \*) “”; - - return NGX\_OK; +} +#endif static ngx\_int\_t ngx\_http\_variable\_request\_body(ngx\_http\_request\_t \*r, Posted at Nginx Forum: [http://forum.nginx.org/read.php?2,120314,122064#msg-122064](http://forum.nginx.org/read.php?2,120314,122064#msg-122064) ---
**Author:** ![chrisl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58f4c7/32.png) [@chrisl](https://www.ruby-forum.com/u/chrisl)\ **Post date:** [May 27, 2011, 2:29am UTC](https://www.ruby-forum.com/t/cache-gzipped-content-with-proxy-pass/193357/4 "2011-05-27T02:29:41Z")
I’ve been following this discussion and I’m anxiously waiting for some resolution. I think Maxim’s approach is right: implementing a gzip\_cache filter that could be used for caching any content. This would allow for much more flexibility than just using it for proxy\_pass/proxy\_cache. This could effectively eliminate the gzip\_static filter. If you have plain HTML served and “gzip on;” plus something like “gzip\_cache on;” all of your plain text content would be served using the cached gzip content. Similarly, you would be able to use this for proxy\_pass, proxy\_store, fastcgi\_pass, etc. It wouldn’t matter since the caching of the gzip’d content would be stored in the gzip\_cache filter and negotiated directly with the client. There’s a lot of chatter about using Varnish or Lighty or \ to give Nginx pre-gzipped data in a proxy response. This sounds absolutely horrible. The bottom line is that if Nginx is the webserver that is dealing with the client directly, it should then be Nginx’s responsibility to serve the appropriate version (gzip’d or plain text) content based on the client’s request headers. That being said, Nginx should be managing it’s own cache of filtered items. The same goes for the image\_filter. Currently, I use the image\_filter for a project and I’m bummed out that Nginx is scaling the same image over and over each time it is requested. An image\_filter\_cache would be another great feature. Am I on the right track? Posted at Nginx Forum: > **[Re: cache gzipped content with proxy\_pass](https://forum.nginx.org/read.php?2,120314,201370#msg-201370)** > > Hi. AFAIK nginx only stores plain (uncompressed) content when acting as a caching proxy server. With gzip=on, content for gzip-accepting clients will be compressed by the server \_for\_ \_every\_ \_request\_ (?). Wouldn't it be better to have two cache... ---
**Author:** ![chrisl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58f4c7/32.png) [@chrisl](https://www.ruby-forum.com/u/chrisl)\ **Post date:** [May 27, 2011, 5:49am UTC](https://www.ruby-forum.com/t/cache-gzipped-content-with-proxy-pass/193357/5 "2011-05-27T05:49:00Z")
You can cache the image filter using either lua or through a proxy pass to yourself. But yes I agree with your points.