Nginx cache loader process

Server NGINX
1

We have several hundred Gs of file cached using nginx. Every time we
restarted nginx, the cache loader process will appear and server load
will
go super high and respond very slowly.

Looks like cache loader process is very I/O intensive and take a long
time
to finish. Is there anyway to get around the problem?

Thanks

Posted at Nginx Forum:

2

On Jul 4, 2013, at 4:57 AM, “badtzhou” [email protected] wrote:

We have several hundred Gs of file cached using nginx. Every time we
restarted nginx, the cache loader process will appear and server load will
go super high and respond very slowly.

Looks like cache loader process is very I/O intensive and take a long time
to finish. Is there anyway to get around the problem?

Did you try loader_* parameters for proxy_cache_path?

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path

3

On Jul 4, 2013, at 4:57 , badtzhou wrote:

We have several hundred Gs of file cached using nginx. Every time we
restarted nginx, the cache loader process will appear and server load will
go super high and respond very slowly.

Looks like cache loader process is very I/O intensive and take a long time
to finish. Is there anyway to get around the problem?

What nginx version do you use?
Cache loader runs better since 1.1.0.

–
Igor S.

4

We are running nginx 1.2.9. What will happen if cache loader is running.
The
file on disk haven’t been loaded into the cache zone yet and someone try
to
access the same file. Will it cause any issue? How will it affect cache
loader process.

If I don’t want any file on the disk to be loaded, can I simply kill the
cache loader process? Will that cause any problem?

Posted at Nginx Forum:

5

Hello,

Take a look at ‘proxy_cache_lock’ directive.

–
Vil S.

среда, 10 июля 2013 г. в 3:50, badtzhou написал:

6

Hello!

On Tue, Jul 09, 2013 at 07:50:32PM -0400, badtzhou wrote:

We are running nginx 1.2.9. What will happen if cache loader is running. The
file on disk haven’t been loaded into the cache zone yet and someone try to
access the same file. Will it cause any issue? How will it affect cache
loader process.

If the cache isn’t yet loaded, nginx will try to check if a cache
file is there by a looking into disk directly from a worker
process. That is, it will work without problems and will use
cached responses, but will be slightly less effective.

If I don’t want any file on the disk to be loaded, can I simply kill the
cache loader process? Will that cause any problem?

It’s not a good idea. In a worst case, if you’ll kill it with
kill -9 at a wrong time, shared memory zone will be corrupted,
resulting in completely non-working nginx.

If cache loader causes too high load on your server, may want to
tune it’s settings to make it less aggressive. See
Module ngx_http_proxy_module for details.

–
Maxim D.
http://nginx.org/en/donation.html