Question about key_zone in proxy_cache_path

Hi,

The wiki (1) states:
“Zone size should be set proportional to number of pages to cache. The
size of the
metadata for one page (file) depends on the OS; currently it is 64 bytes
for
FreeBSD/i386, and 128 bytes for FreeBSD/amd64.”

So for 10.000 files i would need roundabout 1,3meg of key_zone, and for
example
1mio files would need 122meg, is it that easy?

Another question would be, if i set it to big, will it hurt performance?
I already
had the key_zone set to small so nginx complained about and it looked
like it
stopped serving new data.

Thanks and regards,
Sven

1: http://wiki.nginx.org/HttpProxyModule#proxy_cache_path

Hello!

On Sat, Jul 02, 2011 at 12:32:15PM +0200, Sven ‘Darkman’ Michels wrote:

The wiki (1) states:
“Zone size should be set proportional to number of pages to cache. The size of
the
metadata for one page (file) depends on the OS; currently it is 64 bytes for
FreeBSD/i386, and 128 bytes for FreeBSD/amd64.”

So for 10.000 files i would need roundabout 1,3meg of key_zone, and for example
1mio files would need 122meg, is it that easy?

Yes, except it’s not always trivial to calculate number of files
you’ll have in cache. Aproach which is expected to work is to set
maximum cache size and use max_size / (average file size) as a
reference point.

Note also that using proxy_cache_min_uses will consume extra keys
in keys_zone for counting uses, even if actual files won’t be
cached.

Another question would be, if i set it to big, will it hurt performance?

It will consume extra memory, and hence may hurt performance.
It’s unlikely will be noticeable though, unless you are
memory-bound.

I already had the key_zone set to small so nginx complained
about and it looked like it stopped serving new data.

It will try to drop oldest file from cache and continue, though this
may fail as well. So yes, working with full keys_zone isn’t good.

Maxim D.

Hi Maxim,

thanks for your quick reply (as usual :)!

Am 02.07.2011 15:38, schrieb Maxim D.:

Yes, except it’s not always trivial to calculate number of files
you’ll have in cache. Aproach which is expected to work is to set
maximum cache size and use max_size / (average file size) as a
reference point.

Ah, good point. I was more going to check how much file are at the
backend server and use this, but they probably (for sure) won’t be
all in the cache at the same time (at least at some setups).

Note also that using proxy_cache_min_uses will consume extra keys
in keys_zone for counting uses, even if actual files won’t be
cached.

Ah, also a good catch. Currently i don’t use the min_uses but i will
put a remark in my config about it :slight_smile:

Another question would be, if i set it to big, will it hurt performance?

It will consume extra memory, and hence may hurt performance.
It’s unlikely will be noticeable though, unless you are
memory-bound.

I don’t think i’m bound, its more like having 4GB of ram for a nginx
dedicated box which will frontend for 3-4 servers which will have
around 100meg of key_zone size each. So maybe roundabout of 1GB is
used by nginx itself and the rest will be for caching etc. (no other
stuff will run on the box).

I already had the key_zone set to small so nginx complained
about and it looked like it stopped serving new data.

It will try to drop oldest file from cache and continue, though this
may fail as well. So yes, working with full keys_zone isn’t good.

Ok, then i will try to monitor it and adjust the values if needed.
I just would like not the have the cache as another point of failures
for such things (best would be if something with caching fails to just
proxy the request…).

Thanks a lot!

Regards,
Sven