Dear Nginx Community,
I’m trying to setup nginx with the pylons framework via uwsgi.
Nginx-0.8.54 is compiled using:
./configure --add-module=…/nginx-upload-progress-module/
–prefix=/mynginx-bin --with-http_ssl_module
–add-module=…/nginx_accept_language_module
And nginx.conf is:
worker_processes 1;
daemon off;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
upload_progress proxied 512m;
server {
listen 5002;
server_name localhost;
charset utf-8;
root /myproj/public;
index index.html;
client_max_body_size 2048m;
location / {
uwsgi_pass unix:/myproj/uwsgi.socket;
include uwsgi_params;
uwsgi_param SCRIPT_NAME /;
track_uploads proxied 30s;
}
location ^~ /upload_progress {
report_uploads proxied;
}
}
}
You can see there I’m trying to create the ajax file uploader. UWSGI
and pylons are the latest. UWSGI is using --paste parameter. I have to
put in “daemon off;” because this is meant for development, and I’ll
have console interruption control.
I discovered that my nginx can only accept 1 connection at a time.
When a large file is uploading, all ajax calls to /upload_progress are
blocked. The requests are only answered after the upload is finished.
Opening another tab for the file upload form will also be blocked
until the upload finishes. It appears that nginx is throttling to
process only 1 request at a time.
Why does this happen? Did I somehow touched some settings to cause this?
Thanks!
Best regards,
He Shiming
Il giorno 18/gen/2011, alle ore 15.44, He Shiming ha scritto:
listen 5002;
}
I discovered that my nginx can only accept 1 connection at a time.
When a large file is uploading, all ajax calls to /upload_progress are
blocked. The requests are only answered after the upload is finished.
Opening another tab for the file upload form will also be blocked
until the upload finishes. It appears that nginx is throttling to
process only 1 request at a time.
Why does this happen? Did I somehow touched some settings to cause this?
T
How many processes/workers have you setup in uWSGI ?
It look like you are running uWSGI with only one process (or thread, or
async core)
add -p and you should solve it
–
Roberto De Ioris
http://unbit.it
On Tue, Jan 18, 2011 at 11:00 PM, Roberto De Ioris [email protected]
wrote:
Thanks but it didn’t make any difference. I’ve seen 10 worker spawned,
however, nginx isn’t dispatching requests to these uwsgi workers, if
opening more than 1 tab at a time.
Any other ideas?
–
Best regards,
He Shiming
Hello!
On Tue, Jan 18, 2011 at 11:07:20PM +0800, He Shiming wrote:
Thanks but it didn’t make any difference. I’ve seen 10 worker spawned,
however, nginx isn’t dispatching requests to these uwsgi workers, if
opening more than 1 tab at a time.
Any other ideas?
Other ideas: don’t test with browser, test with telnet/netcat. Or
at least make sure you’re monitoring what’s happening on the wire
(with tcpdump/wireshark) while doing so.
For this particular case - most likly it’s your browser which
limits connections.
Maxim D.
On Tue, Jan 18, 2011 at 11:18 PM, Maxim D. [email protected]
wrote:
Other ideas: don’t test with browser, test with telnet/netcat. Or
at least make sure you’re monitoring what’s happening on the wire
(with tcpdump/wireshark) while doing so.
For this particular case - most likly it’s your browser which
limits connections.
Maxim D.
Unfortunately telnet showed the same behavior. I really couldn’t
understand what settings could cause this behavior?
–
Best regards,
He Shiming
On Tue, Jan 18, 2011 at 11:30 PM, Sven ‘Darkman’ Michels
[email protected] wrote:
Hi,
I’m not familar with UWSGI but i think this could be the problem. If you
do not run demonized it may only be able to handle one request per time.
So for testing i would recommend to start it in deamon mode, check if
the problem still exist and if not you know whats wrong.
Regards,
Sven
Hmm… I tried daemon on. But the behavior is still the same. I start
to think I didn’t compile it right. This is really weird…
I’ve got another instance of nginx running fine, it’s the ubuntu
software repo version.
I compiled and configured exactly as my first message. I just don’t
see any reason that the number of connections is limited to 1.
–
Best regards,
He Shiming
Il giorno 18/gen/2011, alle ore 16.42, He Shiming ha scritto:
Sven
Hmm… I tried daemon on. But the behavior is still the same. I start
to think I didn’t compile it right. This is really weird…
Daemon mode is not related to this kind of problems. For example
if you start uWSGI from upstart or supervisor you do not daemonize it
and so on.
Can you post the full uWSGI command line ?
–
Roberto De Ioris
http://unbit.it
Hello!
On Tue, Jan 18, 2011 at 11:24:46PM +0800, He Shiming wrote:
Unfortunately telnet showed the same behavior. I really couldn’t
understand what settings could cause this behavior?
There are no settings in nginx which may cause such behaviour
(well, problems may happen with “sendfile on;” without
sendfile_max_chunk set while sending huge files over fast links -
due to blocking in sendfile() syscall, but it’s not your case).
This may be problem in track_uploads module though, it’s third party
one and I have no idea about it’s bugs.
You may test the following:
-
If requesting normal static file (or just sending an invalid
request) works.
-
If doing the same with track_uploads disabled works.
Maxim D.
2011/1/18 Maxim D. [email protected]:
You may test the following:
-
If requesting normal static file (or just sending an invalid
request) works.
-
If doing the same with track_uploads disabled works.
Maxim D.
UWSGI is like this: uwsgi --paste
config:/pylonsenv/ajaxupload/production.ini --socket
/pylonsenv/ajaxupload/uwsgi.socket -H /pylonsenv -p 10
I tried disabling the upload progress module. I even tried upgrading
to nginx 0.9.3, disabling all modules during compiling. Nothing works.
–
Best regards,
He Shiming
Hi,
Am 18.01.2011 15:44, schrieb He Shiming:
You can see there I’m trying to create the ajax file uploader. UWSGI
and pylons are the latest. UWSGI is using --paste parameter. I have to
put in “daemon off;” because this is meant for development, and I’ll
have console interruption control.
I’m not familar with UWSGI but i think this could be the problem. If you
do not run demonized it may only be able to handle one request per time.
So for testing i would recommend to start it in deamon mode, check if
the problem still exist and if not you know whats wrong.
Regards,
Sven
Il giorno 18/gen/2011, alle ore 17.23, He Shiming ha scritto:
UWSGI is like this: uwsgi --paste
config:/pylonsenv/ajaxupload/production.ini --socket
/pylonsenv/ajaxupload/uwsgi.socket -H /pylonsenv -p 10
I tried disabling the upload progress module. I even tried upgrading
to nginx 0.9.3, disabling all modules during compiling. Nothing works.
Are you sure you are not running pylons with debug enabled ?
It can cause some problems with multiprocess apps
–
Roberto De Ioris
http://unbit.it
On Wed, Jan 19, 2011 at 12:33 AM, Roberto De Ioris [email protected]
wrote:
Are you sure you are not running pylons with debug enabled ?
It can cause some problems with multiprocess apps
–
Roberto De Ioris
http://unbit.it
Hi, thank you all for your help. Eventually I came to the conclusion
that during the uploading, nginx gets so busy with disk i/o, it can’t
process any other requests.
It’s a local testing so I’m uploading fairly large files to simulate
the waiting. So it looks like there’s no problem after all. I just
have to figure out how to throttle upload speed to really simulate
this.
–
Best regards,
He Shiming
Hello!
On Wed, Jan 19, 2011 at 12:39:56AM +0800, He Shiming wrote:
Hi, thank you all for your help. Eventually I came to the conclusion
that during the uploading, nginx gets so busy with disk i/o, it can’t
process any other requests.
It’s a local testing so I’m uploading fairly large files to simulate
the waiting. So it looks like there’s no problem after all. I just
have to figure out how to throttle upload speed to really simulate
this.
Yep, indeed, you’re right. While reading request body nginx
doesn’t return to event loop until it’s read all the data from
socket recv buffer. This may cause it keep calling recv() if
uploading over fast link and client_body_buffer_size is smaller
than socket’s recv buffer.
Using bigger client_body_buffer_size (bigger than recv buffers
used) should help.
Maxim D.