Forum: NGINX Problem with big files

2974d09ac2541e892966b762aad84943?d=identicon&s=25 jakubp (Guest)
on 2014-06-16 23:05
(Received via mailing list)
Hi

Recently I hit quite big problem with huge files. Nginx is a cache
fronting
an origin which serves huge files (several GB). Clients use mostly range
requests (often to get parts towards the end of the file) and I use a
patch
Maxim provided some time ago allowing range requests to receive HTTP 206
if
a resource is not in cache but it's determined to be cacheable...

When a file is not in cache and I see a flurry of requests for the same
file
I see that after proxy_cache_lock_timeout - at that time the download
didn't
reach the first requested byte of a lot of requests - nginx establishes
a
new connection to upstream for each client and initiates another
download of
the same file. I understand why this happens and that it's by design
but...
That kills the server. Multiple writes to temp directory basically kill
the
disk performance (which in turn blocks nginx worker processes).

Is there anything that can be done to help that? Keeping in mind that I
can't afford serving HTTP 200 to a range request and also I'd like to
avoid
clients waiting for the first requested byte forever...

Thanks in advance!

Regards,
Kuba

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,250899,250899#msg-250899
21a46ad06bed3b803d41cb2b8d6b1433?d=identicon&s=25 Justin Dorfman (Guest)
on 2014-06-16 23:13
(Received via mailing list)
>
> I use a patch
> Maxim provided some time ago allowing range requests to receive HTTP 206 if
> a resource is not in cache but it's determined to be cacheable...


Can you please link to this patch?


Regards,

Justin Dorfman <http://www.twitter.com/jdorfman>

Director of Developer Relations
MaxCDN <http://twitter.com/MaxCDNDeveloper>
2974d09ac2541e892966b762aad84943?d=identicon&s=25 jakubp (Guest)
on 2014-06-18 00:51
(Received via mailing list)
Hi Justin

Justin Dorfman Wrote:
-------------------------------------------------------
> >
> > I use a patch
> > Maxim provided some time ago allowing range requests to receive HTTP
> 206 if
> > a resource is not in cache but it's determined to be cacheable...
>
>
> Can you please link to this patch?
>
>

http://mailman.nginx.org/pipermail/nginx/2012-Apri...

Regards,
Kuba

> Regards,
>
> Justin Dorfman <http://www.twitter.com/jdorfman>
>
> Director of Developer Relations
> MaxCDN <http://twitter.com/MaxCDNDeveloper>
>

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,250899,250969#msg-250969
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-06-19 12:42
(Received via mailing list)
Hello!

On Tue, Jun 17, 2014 at 06:51:10PM -0400, jakubp wrote:

> >
> > Can you please link to this patch?
> >
> >
>
> http://mailman.nginx.org/pipermail/nginx/2012-Apri...

JFYI: improved version of this patch was committed in nginx
1.5.13:

    *) Feature: byte ranges support in the ngx_http_mp4_module and while
       saving responses to cache.

http://hg.nginx.org/nginx/rev/345e4fd4bb64

--
Maxim Dounin
http://nginx.org/
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.