# Reverse proxy and too many temp files **URL:** \ **Category:** NGINX\ **Created:** [October 27, 2010, 11:17pm UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207 "2010-10-27T23:17:32Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![okkezSS](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/o/85f322/32.png) [@okkezSS](https://www.ruby-forum.com/u/okkezSS)\ **Post date:** [October 27, 2010, 11:17pm UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/1 "2010-10-27T23:17:32Z")
hi we are using nginx as a reverse proxy in front of some webservers with media files. evrerithing works great beside when we start nginx new with an empty cache. in this situation allot of files in proxy\_temp\_path are create consuming quite fast all our diskspace, in the same time proxy\_cache\_path is only very slowly populated. i see two parameters who might help: proxy\_cache\_use\_stale proxy\_max\_temp\_file\_size a quick test with \*use\_stale was not succesfull, but maybe something else was wrong and for proxy\_max\_temp\_file\_size=0 i’m not shure if it is the right thing todo what we would like to achieve is to uses as much of the diskspace for caching. but on a restart with clean cache, not filling up the temp folder which does not count to proxy\_cache\_path max\_size. or should we just reduce the workers (16) to provent from having to many workers creating the same temp file for the same origin file? cheers andi Posted at Nginx Forum: > **[reverse proxy and too many temp files](https://forum.nginx.org/read.php?2,145097,145097#msg-145097)** > > hi we are using nginx as a reverse proxy in front of some webservers with media files. evrerithing works great beside when we start nginx new with an empty cache. in this situation allot of files in proxy\_temp\_path are create consuming quite fast all... ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 6:32am UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/2 "2010-10-28T06:32:21Z")
On Wed, Oct 27, 2010 at 4:16 PM, roadfox [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > caching. but on a restart with clean cache, not filling up the temp > folder which does not count to proxy\_cache\_path max\_size. > > or should we just reduce the workers (16) to provent from having to many > workers creating the same temp file for the same origin file? ## You probably need to increase the number and/or size of proxy\_buffers and perhaps your gzip\_buffers. The defaults are way too small and trade low memory usage for disk I/O (which is a really bad trade-off most of the time). I have “proxy\_buffers 16 32k;” and don’t see temp responses on disk at all with my applications (some of which send up to 512KB uncompressed pages. RPM ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 7:29am UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/3 "2010-10-28T07:29:22Z")
[quote]You probably need to increase the number and/or size of proxy\_buffers[/quote] Our nginx reverse proxy is only serving media files with sizes between 100KB and 2MB. As DiskI/O is a bottleneck we will try to increase proxy\_buffers and see what happens. Posted at Nginx Forum: > **[Re: reverse proxy and too many temp files](https://forum.nginx.org/read.php?2,145097,145197#msg-145197)** > > hi we are using nginx as a reverse proxy in front of some webservers with media files. evrerithing works great beside when we start nginx new with an empty cache. in this situation allot of files in proxy\_temp\_path are create consuming quite fast all... ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 8:42am UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/4 "2010-10-28T08:42:43Z")
You mentioned gzip buffers, I was of the impression that gzip was all in memory? ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 1:45pm UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/5 "2010-10-28T13:45:00Z")
Thanks all for your replies! So we go out and buy some more disks. I forgot to mention but the temp file folder grows to more than 60GB. And it must be fast disks as I/O is huge. Will increasing proxy\_buffers help in our case to reduce disk i/o? - during cache population - if cache is populatet And i was wrong above, average file size served is 140MB, largest around 2GB. regards Andi Posted at Nginx Forum: > **[Re: reverse proxy and too many temp files](https://forum.nginx.org/read.php?2,145097,145291#msg-145291)** > > hi we are using nginx as a reverse proxy in front of some webservers with media files. evrerithing works great beside when we start nginx new with an empty cache. in this situation allot of files in proxy\_temp\_path are create consuming quite fast all... ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 10:03am UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/6 "2010-10-28T10:03:13Z")
Hello! On Wed, Oct 27, 2010 at 05:16:02PM -0400, roadfox wrote: > i see two parameters who might help: > proxy\_cache\_use\_stale > proxy\_max\_temp\_file\_size > > a quick test with \*use\_stale was not succesfull, but maybe something > else was wrong proxy\_cache\_use\_stale only make sense if you already have data in cache. > and for proxy\_max\_temp\_file\_size=0 i’m not shure if it is the right > thing todo proxy\_max\_temp\_file\_size can’t (and doesn’t) limit writing to disk responses expected to be saved to cache as nginx needs them on disk to save to cache. > what we would like to achieve is to uses as much of the diskspace for > caching. but on a restart with clean cache, not filling up the temp > folder which does not count to proxy\_cache\_path max\_size. > > or should we just reduce the workers (16) to provent from having to many > workers creating the same temp file for the same origin file? No, reducing number of workers isn’t likely to help as each worker may handle many connections. Solution to this problem is so called “busy locks” which are not yet implemented. Right now there is no good solution. Some possibilities include: using limit\_zone to limit number of requests (though this will limit serving from cache as well), warming up cache before startup, adding more disk space. Maxim D. ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 5:34pm UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/7 "2010-10-28T17:34:36Z")
A large number off buffers would help 250 10mb although that’s a huge amount of memory so im not so sure! Sent from my iPod ---
**Author:** ![roadfox](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/9de053/32.png) [@roadfox](https://www.ruby-forum.com/u/roadfox)\ **Post date:** [October 28, 2010, 6:17pm UTC](https://www.ruby-forum.com/t/reverse-proxy-and-too-many-temp-files/198207/8 "2010-10-28T18:17:20Z")
Hello! On Fri, Oct 29, 2010 at 02:32:13AM +1100, Splitice wrote: > A large number off buffers would help 250 10mb although that’s > a huge amount of memory so im not so sure! No, it won’t. Responses subject to cache will go to disk anyway. Maxim D. p.s. Please do not top-post. Thank you.