"No space left on device" for temp cache - v1.7.4

Server NGINX
1

Hi, recently we noticed that the version 1.7.3 added a feature important
to
our infrastructure: “cache revalidation now uses If-None-Match header if
possible.”.
So we changed part of our cache to the 1.7.4 version, but something
strange
started to happen, at certain point of disk usage nginx started to
return
500 to all requests with this kind of message in the error log:

[crit] 12908#0: *7209656 open() “/cache/nginx_tmp/0002938835” failed
(28: No
space left on device) while reading upstream, client: xxx.xxx.xxx.xxx,
server: , request: “GET http://xxxxxxxx.net/ HTTP/1.1”, upstream:
“http://xxx.xx.xxx.xxx:80/”, host: “xxxxxxxx.net”

It looked like that there wasn’t enough space in the temporary cache
directory. But running df -h (and after sync && df -h) the result was:

[user@nginx ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.8G 2.3G 5.4G 30% /
devtmpfs 3.7G 20K 3.7G 1% /dev
tmpfs 3.7G 0 3.7G 0% /dev/shm
/dev/xvdb 40G 12G 27G 31% 24% /cache
---------------------------------------- used for cache

I think that is important to know that a very similar configuration is
running well in a 1.7.2 server with the same specs and without the cache
revalidation directives.

This is my cache path configuration:

proxy_cache_path /cache/nginx levels=1:2 keys_zone=cache:1500m
max_size=27G
inactive=1200m;

Can you help?

Gabriel Arrais.

Posted at Nginx Forum:

2

Hello!

On Thu, Sep 11, 2014 at 10:15:43AM -0400, arraisgabriel wrote:

“http://xxx.xx.xxx.xxx:80/”, host: “xxxxxxxx.net”
---------------------------------------- used for cache
ENOSPC from open() likely means you’ve run out of inodes, not disk
space. Try looking into “df -i”, it may be helpful.

–
Maxim D.
http://nginx.org/

3

Thank you very much for the quick response. It looks like that the
cache
now stores many small files because of the revalidation feature, and
reached
the inode storage limit.

Maxim D. Wrote:

strange

---------------------------------------- used for cache
[email protected]
nginx Info Page

Posted at Nginx Forum:

4

What file system do you use for cache? Try using xfs instead ext4. Xfs
have
better inode storage than ext4.

5

Or reiserfs…

Eliezer