Forum: NGINX Problem with big files

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
2974d09ac2541e892966b762aad84943?d=identicon&s=25 jakubp (Guest)
on 2014-06-16 23:05
(Received via mailing list)
Hi

Recently I hit quite big problem with huge files. Nginx is a cache
fronting
an origin which serves huge files (several GB). Clients use mostly range
requests (often to get parts towards the end of the file) and I use a
patch
Maxim provided some time ago allowing range requests to receive HTTP 206
if
a resource is not in cache but it's determined to be cacheable...

When a file is not in cache and I see a flurry of requests for the same
file
I see that after proxy_cache_lock_timeout - at that time the download
didn't
reach the first requested byte of a lot of requests - nginx establishes
a
new connection to upstream for each client and initiates another
download of
the same file. I understand why this happens and that it's by design
but...
That kills the server. Multiple writes to temp directory basically kill
the
disk performance (which in turn blocks nginx worker processes).

Is there anything that can be done to help that? Keeping in mind that I
can't afford serving HTTP 200 to a range request and also I'd like to
avoid
clients waiting for the first requested byte forever...

Thanks in advance!

Regards,
Kuba

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,250899,250899#msg-250899
21a46ad06bed3b803d41cb2b8d6b1433?d=identicon&s=25 Justin Dorfman (Guest)
on 2014-06-16 23:13
(Received via mailing list)
>
> I use a patch
> Maxim provided some time ago allowing range requests to receive HTTP 206 if
> a resource is not in cache but it's determined to be cacheable...


Can you please link to this patch?


Regards,

Justin Dorfman <http://www.twitter.com/jdorfman>

Director of Developer Relations
MaxCDN <http://twitter.com/MaxCDNDeveloper>
2974d09ac2541e892966b762aad84943?d=identicon&s=25 jakubp (Guest)
on 2014-06-18 00:51
(Received via mailing list)
Hi Justin

Justin Dorfman Wrote:
-------------------------------------------------------
> >
> > I use a patch
> > Maxim provided some time ago allowing range requests to receive HTTP
> 206 if
> > a resource is not in cache but it's determined to be cacheable...
>
>
> Can you please link to this patch?
>
>

http://mailman.nginx.org/pipermail/nginx/2012-Apri...

Regards,
Kuba

> Regards,
>
> Justin Dorfman <http://www.twitter.com/jdorfman>
>
> Director of Developer Relations
> MaxCDN <http://twitter.com/MaxCDNDeveloper>
>

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,250899,250969#msg-250969
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-06-19 12:42
(Received via mailing list)
Hello!

On Tue, Jun 17, 2014 at 06:51:10PM -0400, jakubp wrote:

> >
> > Can you please link to this patch?
> >
> >
>
> http://mailman.nginx.org/pipermail/nginx/2012-Apri...

JFYI: improved version of this patch was committed in nginx
1.5.13:

    *) Feature: byte ranges support in the ngx_http_mp4_module and while
       saving responses to cache.

http://hg.nginx.org/nginx/rev/345e4fd4bb64

--
Maxim Dounin
http://nginx.org/
This topic is locked and can not be replied to.