# Proxy caching large files with too many temporary files

**URL:** <https://www.ruby-forum.com/t/proxy-caching-large-files-with-too-many-temporary-files/199014>\
**Category:** NGINX\
**Created:** [November 12, 2010, 10:37am UTC](https://www.ruby-forum.com/t/proxy-caching-large-files-with-too-many-temporary-files/199014 "2010-11-12T10:37:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![okkezSS](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/o/85f322/32.png) [@okkezSS](https://www.ruby-forum.com/u/okkezSS)\
**Post date:** [November 12, 2010, 10:37am UTC](https://www.ruby-forum.com/t/proxy-caching-large-files-with-too-many-temporary-files/199014/1 "2010-11-12T10:37:12Z")

</div>

Hello community !

I’m currently using nginx as a proxy cache to a backend where large  
files are stored and I have a big blocking issue.  
I’m using the following structure:

Client — Cache1 — Cache2 — HTTP with large files

The problem I’m experiencing is in Cache1 and happens whenever a large  
file (not cached) is requested by hundreds users.

The proxy\_cache module will buffer the request to disk, writing it to  
temporary proxy cache path, but, because of the excessive number of  
requests, it will write the same file hundreds times, until is fully  
cached (and then will be served from cache).

This cause having also 2000 files open for writing 4GB at the same time,  
blocking the workers, issuing kernel panics and smashing literally the  
machine.

What I wanted to ask you guys, is : is there a way that nginx will  
buffer the file only one time and then all the workers may even use the  
same buffer to serve the uncached requests? So the temp file will be  
written only one time?

To work around this, I wrote a small perl script that I can share with  
you if you need, that basically do the following.

In nginx, with EmbeddedPerlModule, I open a file in /dev/shm and read  
each line to find the proxy\_cache\_key.  
If the key is there, then it will set a variable that will be passed to  
proxy\_no\_cache to 1. If the key is not there, proxy\_no\_cache will be set  
to 0 and the relative proxy\_cache\_key will be appended in the last line  
of the /dev/shm/cache/keys file.

This will cause nginx to use cache only for the first request and to  
fetch the file directly from Cache2 for all the successive requests.

Moreover, to avoid situations where that cache keys keep staying there  
forever, possibly blocking new file’s caching, I have another small  
script that run outside nginx (perl script) that open the  
/dev/shm/cache/keys file and scan it line by line.  
Foreach line it will compute the md5 hash and check if the file is in  
cache folder. If yes, will remove the line from the file. If not, then  
it opens all temporary cache files (I know it might sound bad, but I  
can’t find a better way right now), and read the second line that  
contains “KEY: [http://www.example.com/file.bin](http://www.example.com/file.bin)”. If the KEY match the  
key in /dev/shm/cache/keys file, then the line is keept there. If no  
temporary files contain that key, then the key is removed from the keys  
file, so that it will be cached at the next request without problems.

Do you have any better suggestion / patches that may fix the problem I  
have?

Please, consider that I get thousands connections per time (2000+),  
requesting a mix of large (mostly) and small files. I tried to play with  
all the proxy\_cache module settings but had no luck.

Thank you in advance for any help !

Kind regards,

Paolo Iannelli

Posted at Nginx Forum:

> **[Proxy caching large files with too many temporary files](https://forum.nginx.org/read.php?2,150073,150073#msg-150073)**
>
> Hello community ! I'm currently using nginx as a proxy cache to a backend where large files are stored and I have a big blocking issue. I'm using the following structure: Client --- Cache1 --- Cache2 --- HTTP with large files The problem I'm exper