Nginx temp file being deleted errenously

Hello everyone,

My company processes millions of requests for file uploads/downloads and
we
are looking at Nginx to replace Apache. We are running into a problem
that
I hope you can help – I have searched the web and even read Nginx HTTP
Server book by Clement Nedelcu.

At a high level, the flow of a request through our system is as follows:
user agent posts the data to upload --> Ngixn acting as a reverse
proxy
–> Object store -> post action of passing request to Uwsgi.

Problem: when uploading a large file (~ 100MB) , the post action is
failing. The error message seen in error.log is “8 sendfile() failed
(9:
Bad file descriptor) while sending request to upstream,” message. This
use
case occurs even if no load on the system,

When I enable “debug”, I see that after the Nginx streams the bytes to
the
object store, the temp file is deleted. When the post action is
executed it
fails as the temp file is gone.

*Note: this error is NOT seen when upload small files.

Details:

  • Nginx version 1.1.17

  • UWsgi vesrion 1.0

  • Cent OS 2.6.32-131.0.15.el6.x86_64

  • The relevant nginx configuration is:

    uri block to upload to object store.

    location ‘/upload/objecststorage’ {
    proxy_pass https:/objectstore_host/valut
    post_action /os/postback/ospostback.py;
    }

    location = ‘/os/postback/ospostback.py’ {
    root html/uwsgi;
    set $app ospostback;

    uwsgi_pass unix:/tmp/uwsgi.sock;
    include uwsgi_params;

    uwsgi_param SERVER_ADDR $server_addr;
    uwsgi_param SCRIPT_NAME $app;
    uwsgi_param UWSGI_MODULE $app;
    uwsgi_param UWSGI_CALLABLE “${app}_handler”;
    uwsgi_param UWSGI_PYHOME $document_root;
    uwsgi_param UWSGI_CHDIR $document_root;

    uwsgi_param UWSGI_BYTES $body_bytes_sent;
    uwsgi_param UWSGI_PATH $path;
    uwsgi_param UWSGI_PID $proc_id;
    uwsgi_param UWSGI_ID $token;
    uwsgi_param UWSGI_KEY $arg_id;
    uwsgi_modifier1 30;

    }

Any help would be great. I really want to use Nginx but this is now a
blocker issue for my company.

Regards

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,235586,235586#msg-235586

On 01/26/2013 02:36 AM, [email protected] wrote:

*Note: this error is NOT seen when upload small files.
I would store the file in some temporary space on frontend, and then
forwarded it to the storage backend.


With best regards,
Gregory E.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs