Problem with large static files

I have a strange problem with slow load of relatively large files. On a
domain without proxy (designed for serving static files), the request
time (as monitored by access log $request_time) for an simple jpg image
file of 5KB is 0.000s (seems to be shorter than 1ms). Thus, one expect
to load an image file of 15KB within 3ms, but surprisingly it takes
about 600ms. A static file of 15KB is not big at all; but there should
be a threshold which leads to this slow down upon slightly increasing
the file size.

Posted at Nginx Forum:

Hello!

On Mon, Dec 19, 2011 at 01:22:15PM -0500, etrader wrote:

I have a strange problem with slow load of relatively large files. On a
domain without proxy (designed for serving static files), the request
time (as monitored by access log $request_time) for an simple jpg image
file of 5KB is 0.000s (seems to be shorter than 1ms). Thus, one expect
to load an image file of 15KB within 3ms, but surprisingly it takes
about 600ms. A static file of 15KB is not big at all; but there should
be a threshold which leads to this slow down upon slightly increasing
the file size.

The $request_time of 0.000s isn’t something real. Instead, it
means that nginx wasn’t going through event loop while processing
the request. It was able to read the request from client, read
data from disk and send all data to socket buffer without
overflowing it. As nginx only updates it’s idea about current
time once per event loop iteration, the above results in
$request_time of 0.000s in logs. Real time spent may
significantly differ if you are disk-bound and reading from disk
blocks for a long time (and you aren’t using AIO).

Try looking into your disk subsystem stats, most likely it will
explain things.

Maxim D.

Thanks for very useful description, Maxim!
Actually, I suspected the nature of 0.000 request time; but what makes
me wonder is that it takes only 3ms to process a relatively php file
with output of 5KB. Then, why reading a static file of 15KB should take
600ms? If it is normally the case, isn’t it better to save images as
data instead of file? PHP should process a base64 string faster.

Posted at Nginx Forum:

On Dec 20, 2011, at 1:01 AM, etrader wrote:

Actually, I suspected the nature of 0.000 request time; but what makes
me wonder is that it takes only 3ms to process a relatively php file
with output of 5KB. Then, why reading a static file of 15KB should take
600ms?

It depends on the speed of your disk, filesystem, etc. Is this local
disk or something like NFS? Is this a physical machine or a vm? How many
workers are you running? We generally run 2 per cpu core – that does
“worse” than 1 per core in benchmarking, but seems to work better in
actual production use because of various places nginx may block (on
Linux).

–Brian

Thanks Brian, I always had this doubt whether it is good to have more
workers (comparing to the number of CPUs) or not.

Posted at Nginx Forum: