Uploads with nginx 1.0.12

Server NGINX
#1

Hello,

I am using file uploads with nginx 1.0.12, php5-fpm and php 5.3.10.

Currently, big file uploads (~1300 MB) do take about 30 seconds to
process
after the file is being uploaded, and the CPU load spikes. Is there a
way
to prevent that?

I am using a very simple script (just a var_dump($_FILES), nothing more,
just debugging) and the config looks like the following:
location ~ .php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_pass_request_body off;
client_body_in_file_only clean;
fastcgi_param REQUEST_BODY_FILE $request_body_file;
fastcgi_send_timeout 120;
fastcgi_read_timeout 120;
fastcgi_index index.php;
include fastcgi_params;
}

Hope that someone can shed light into this…

Thank you.

Adrian

#2

On Wed, 2012-02-29 at 20:59 +0100, Adrián Navarro wrote:

Hello,

I am using file uploads with nginx 1.0.12, php5-fpm and php 5.3.10.

Currently, big file uploads (~1300 MB) do take about 30 seconds to
process after the file is being uploaded, and the CPU load spikes. Is
there a way to prevent that?

I assume that since the spike occurs after the upload, the culprit is
PHP, not Nginx?

I am using a very simple script (just a var_dump($_FILES), nothing
more,

Assuming the spike is caused by PHP, you might take a look at the
nginx_upload module, which will handle the entire upload process for
you, and just hand your PHP script a path to the uploaded file (along
with a few other parameters).

Here’s some references:

http://blog.martinfjordvald.com/2010/08/file-uploading-with-php-and-nginx/

http://brainspl.at/articles/2008/07/20/nginx-upload-module

https://github.com/vkholodkov/nginx-upload-module/tree/2.2

Regards,
Cliff

#3

Indeed, the high load is caused by php-fpm. I was wondering if there was
any alternative config to keep fpm from spiking (like, any built-in
helper
to pass the final location without any plugin…).

I will look into the compiled module. I’m not very comfortable with
compiling it on the cluster of servers which were previously using
dotdeb,
but it’s better than nothing (uh, tips to keep the debian specific
structure when compiling?).

Thanks for pointing me out!

-Adrian

---------- Forwarded message ----------
From: Cliff W. [email protected]
Date: Wed, Feb 29, 2012 at 9:35 PM
Subject: Re: Uploads with nginx 1.0.12
To: [email protected]

On Wed, 2012-02-29 at 20:59 +0100, Adrián Navarro wrote:

Hello,

I am using file uploads with nginx 1.0.12, php5-fpm and php 5.3.10.

Currently, big file uploads (~1300 MB) do take about 30 seconds to
process after the file is being uploaded, and the CPU load spikes. Is
there a way to prevent that?

I assume that since the spike occurs after the upload, the culprit is
PHP, not Nginx?

I am using a very simple script (just a var_dump($_FILES), nothing
more,

Assuming the spike is caused by PHP, you might take a look at the
nginx_upload module, which will handle the entire upload process for
you, and just hand your PHP script a path to the uploaded file (along
with a few other parameters).

Here’s some references:

http://blog.martinfjordvald.com/2010/08/file-uploading-with-php-and-nginx/

http://brainspl.at/articles/2008/07/20/nginx-upload-module

https://github.com/vkholodkov/nginx-upload-module/tree/2.2

Regards,
Cliff

#4

On Wed, 2012-02-29 at 21:48 +0100, Adrián Navarro wrote:

Indeed, the high load is caused by php-fpm. I was wondering if there
was any alternative config to keep fpm from spiking (like, any
built-in helper to pass the final location without any plugin…).

I will look into the compiled module. I’m not very comfortable with
compiling it on the cluster of servers which were previously using
dotdeb, but it’s better than nothing (uh, tips to keep the debian
specific structure when compiling?).

If you are comfortable building .deb packages, you could just rebuild
the .deb with the additional module built in.

Alternatively, you could use the output of “nginx -V” to get a list of
compile-time options and use those when building from source.

In either case, I would make an archive of your /etc/nginx directory
just to be safe. I’d also strongly suggest doing a test build outside
of your cluster. In general, building Nginx from source is fairly
simple, but always better to be cautious.

Regards,
Cliff

#5

On Thursday 01 March 2012 00:35:51 Cliff W. wrote:

Actually,

fastcgi_pass_request_body off;
client_body_in_file_only clean;
fastcgi_param REQUEST_BODY_FILE $request_body_file;

should do approximately the same.

wbr, Valentin V. Bartenev