I’m having problem with I/O performance. I’m running nginx as caching
reverse proxy server.
When cache size on disk exceeds max_size cache manager starts working,
but
it causes two problems occur:
I/O %util reach 100% and nginx starts dropping connections
cache manager process dosen’t unlink files speed enough to delete old
file. So cache becomes bigger util the space on disk ends.
Can you give me an idea how can I solve those problems. Below are some
details.
#build on 20x 300GB SAS disks with 2 SSDs for Cachecade.
storcli64 /c0 show
VD LIST :
DG/VD TYPE State Access Consist Cache Cac sCC Size Name
1/2 RAID60 Optl RW Yes RaWBC R ON 4.357 TB
2/1 Cac0 Optl RW Yes RaWTD - ON 557.875 GB
mount
/dev/sdb1 on /cache type ext4 (rw,noatime,data=ordered)
The “proxy_cache_path” looks corrupted and incomplete. First of
all, I would suggest you to make sure you are using “levels”
parameter, see Module ngx_http_proxy_module.
The “proxy_cache_path” looks corrupted and incomplete. First of
all, I would suggest you to make sure you are using “levels”
parameter, see Module ngx_http_proxy_module.
I didn’t paste all of proxy_cache_path directive. Here you have all.
proxy_temp_path /cache/tmp;
proxy_cache_path /cache
levels=2:2
keys_zone=my-cache:20000m
max_size=3355443m
inactive=7d;
On Wed, May 21, 2014 at 09:15:16AM -0400, ixos wrote:
inactive=7d;
See no obvious problems.
Try looking into system tuning then, your disk subsystem just
can’t cope with load. There are number of ways to improve disk
i/o performance, starting from nginx tuning (aio, output_buffers
etc., see Module ngx_http_core_module) to OS tuning (in particular,
tuning vnode cache may be beneficial, not sure how to do this on
Linux), as well as using a RAID configuration which delivers
better performance. A number of recommendations can be found in
this list, see archives.
An obvious workaround is to reduce disk load by using smaller
max_size/inactive, and/or with proxy_cache_min_uses (see Module ngx_http_proxy_module).
And also nginx version if needed:
/usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.5.9
While it may be a good idea to upgrade to a recent and supported
version, there shouldn’t be a big difference from performance
point of view.