Cache by mime type

Like the title says, can I cache images (or whatever) by mime type, not
extension? The only relevant info I found is at


However, the suggested config is not valid, because proxy directive
cannot be used inside if.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,228786,228786#msg-228786

20.07.2012, 20:57, “burn” [email protected]:

Like the title says, can I cache images (or whatever) by mime type, not
extension?

Yes.

http://nginx.org/en/docs/http/ngx_http_map_module.html
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_bypass

The only relevant info I found is at
http://stackoverflow.com/questions/6212361/nginx-cache-file-by-mime-type
However, the suggested config is not valid, because proxy directive
cannot be used inside if.

forget about “if”.


br, Denis F. Latypoff.

can you elaborate on how to implement it? For me, it seems to cache html
as well (I’m checking saved files). Here’s what I come up with:

nginx.conf

user nginx;
worker_processes 8;
worker_rlimit_nofile 100000;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
use epoll;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local]
“$request” ’
'$status $body_bytes_sent “$http_referer” ’
‘"$http_user_agent" “$http_x_forwarded_for”’;
access_log /var/log/nginx/access.log combined;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
server_tokens off;
gzip on;
gzip_static on;
gzip_comp_level 5;
gzip_min_length 1024;
keepalive_timeout 65;
limit_conn_zone $binary_remote_addr zone=addr:10m;
map $http_content_type $no_proxy {
default 0;
"~image/" 1;
}
proxy_cache_path /tmp/nginx/cache/image levels=1:1:2
keys_zone=image:1000m;
proxy_temp_path /tmp/nginx/temp;
include /etc/nginx/conf.d/
.conf;
include /etc/nginx/vhosts/*;
}

vhost:

server {
listen 8082;
server_name .forum.domain.com;
charset utf-8;
access_log /var/log/nginx/forum.domain.com-access.log2 combined;
error_log /var/log/nginx/forum.domain.com-error.log2;
location / {
root /var/www/html/forum.domain.com;
index index.php;
try_files $uri @apache;
}
location /forum {
root /var/www/html/forum.domain.com;
index index.php;
try_files $uri @apache;
}
location /gallery {
root /var/www/html/forum.domain.com;
index index.php;
try_files $uri @apache;
}
location ~ ^.*.php$ {
proxy_pass http://127.0.0.1:8081;
include proxy.cache.inc;
}
location @apache {
proxy_pass http://127.0.0.1:8081;
include proxy.cache.inc;
}
}

proxy.cache.inc:

proxy_cache image;
proxy_cache_use_stale updating;
proxy_cache_valid 7d;
proxy_cache_bypass $no_proxy;
proxy_connect_timeout 59s;
proxy_send_timeout 600;
proxy_read_timeout 600;
proxy_buffer_size 64k;
proxy_buffers 16 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_pass_header Set-Cookie;
proxy_redirect off;
proxy_hide_header Vary;
proxy_set_header Accept-Encoding ‘’;
proxy_ignore_headers Cache-Control Expires;
proxy_set_header Referer $http_referer;
proxy_set_header Host $host;
proxy_set_header Cookie $http_cookie;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,228786,228794#msg-228794

but I’m not using upstream directive. Do you mean that it’s only
possible to upstream instead of direct proxy_pass?
I tried that. I still see non-image files in cache.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,228786,228797#msg-228797

On Friday 20 July 2012 21:47:35 burn wrote:
[…]

map $http_content_type $no_proxy {
    default         0;
    "~*image/"      1;
}

$upstream_http_content_type

http://nginx.org/en/docs/http/ngx_http_upstream_module.html#variables

wbr, Valentin V. Bartenev

with “proxy_no_cache $no_proxy;”, there are css files in cache again.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,228786,228800#msg-228800

On Friday 20 July 2012 22:45:44 burn wrote:

with “proxy_no_cache $no_proxy;”, there are css files in cache again.

If you configure $no_proxy like this:

default 0;
“~*image/” 1;

Images will not be cached. All others will.

wbr, Valentin V. Bartenev

On Friday 20 July 2012 22:23:26 burn wrote:

but I’m not using upstream directive. Do you mean that it’s only
possible to upstream instead of direct proxy_pass?

Actually “proxy_pass” uses upstream module regardless of whether you’re
using the upstream directive or not.

I tried that. I still see non-image files in cache.

Also you should use “proxy_no_cache” instead of “proxy_cache_bypass”.

http://nginx.org/r/proxy_no_cache

wbr, Valentin V. Bartenev

Right. The problem is that there were images as well. And now when I
swapped values, it seems that nothing at all is cached.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,228786,228802#msg-228802

On Fri, Jul 20, 2012 at 02:45:44PM -0400, burn wrote:

Hi there,

with “proxy_no_cache $no_proxy;”, there are css files in cache again.

I think the issue is with using both proxy_no_cache and
proxy_cache_bypass
with the same $no_proxy variable.

(I think there was a bug related to this in the past – something about
if you didn’t use them with the same variable, bad things could happen.)

proxy_cache_bypass takes effect before the upstream is
contacted, so at that point $no_proxy is set based on an empty
$upstream_http_content_type. Later on, when proxy_no_cache is
considered,
$no_proxy already has a value so that is used.

The following config seems to work for me:

===
http {
map $upstream_http_content_type $no_proxy {
default proxy;
~*^image/ 0;
}
proxy_cache_path proxy_cache keys_zone=images:1m inactive=1m;

server {
    location / {
        proxy_pass  http://127.0.0.1:10080/;
        proxy_no_cache $no_proxy;
        proxy_cache_bypass 0;
        proxy_cache images;
    }
}

}

I can run the two commands

curl -i http://localhost:8000/a.png
curl -i http://localhost:8000/a.txt

and I see exactly one file in my proxy_cache directory. (And the error
log says what is going on.)

And then I can run the commands again, and from the Expires header,
I can see that the txt file came from upstream, and the png file came
from cache.

(I use the proxy_cache_bypass line just to hush a warning (in 1.2.1). I
have not tested this for the partly-remembered bug mentioned above.)

f

Francis D. [email protected]

Thanks, that seems to work. But I still don’t quite understand how. I
did use both directives with the same variable. Now when I set
“proxy_cache_bypass 0;”, it seems to correctly cache images and serve
that cache. I’d imagine that unconditional 0 in that directive would
cause nothing being served from cache ever (as that variable is
considered first). By the way, my Nginx version is 1.3.1.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,228786,228836#msg-228836

On Sun, Jul 22, 2012 at 05:27:44AM -0400, burn wrote:

Hi there,

Thanks, that seems to work. But I still don’t quite understand how. I
did use both directives with the same variable. Now when I set
“proxy_cache_bypass 0;”, it seems to correctly cache images and serve
that cache.

proxy_cache_bypass is “shall I serve this from upstream, even if I
already have a valid cached version”. Setting it to 0 (or empty) means
that everything that is valid in the cache, will be served from the
cache.

Strictly, it shouldn’t be used in this case at all. But it is needed
to hush the warning that is presumably due to the bug mentioned in
http://mailman.nginx.org/pipermail/nginx/2011-March/025557.html

(I think that that bug was fixed in 1.0.3; possibly the warning is no
longer needed?)

And now you are using proxy_cache_bypass without an identical
proxy_no_cache – although you’re not actually caring about the
proxy_cache_bypass part, so maybe you won’t have the problems referred
to.

I’d imagine that unconditional 0 in that directive would
cause nothing being served from cache ever (as that variable is
considered first).

No, unconditional 0 there means “serve from cache if possible”, and is
equivalent to the line being omitted.

proxy_no_cache is about “shall I actually save this response into the
cache, even though all other indications are that I should do so”.
That’s
the one that you want to be “no, don’t” for all of the mime types that
you don’t want cached.

All the best,

f

Francis D. [email protected]