# How can the number of parallel/redundant open streams/temp\_files be controlled/limited? **URL:** \ **Category:** NGINX\ **Created:** [June 24, 2014, 8:50pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575 "2014-06-24T20:50:52Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![jason\_sam](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/7993a0/32.png) [@jason\_sam](https://www.ruby-forum.com/u/jason_sam)\ **Post date:** [June 24, 2014, 8:50pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/1 "2014-06-24T20:50:52Z")
I’ve noticed that multiple (as great as 8 or more) parallel redundant streams and corresponding temp\_files are opened reading the same file from a reverse proxy backend into nginx, upon even a single request by an up-stream client, if not already cached (or stored in a static proxy’ed file) local to nginx. This seems extremely wasteful of bandwidth between nginx and corresponding reverse proxy backends; does anyone know why this is occurring and how to limit this behavior? (For example, upon receiving a request for example small 250MB mp4 video podcast video file, it’s not uncommon for 8 parallel streams to be opened, each sourcing (and competing for bandwidth) a corresponding temp\_file, where the upstream client appears to being feed by the most complete stream/temp\_file; but even upon the compete file being fully transferred to the upstream client,the remaining streams remain active until they too have finished their transfers, and then closed, and their corresponding temp\_files deleted. All resulting in 2GB of data being transferred when only 250MB needed be, not to mention that the transfer took nearly 8x longer to complete, so unless there were concerns about the integrity of the connection, it seems like a huge waste of resources?) Thanks, any insight/assistance would be appreciated. ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [June 25, 2014, 12:36am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/2 "2014-06-25T00:36:17Z")
Hello! On Tue, Jun 24, 2014 at 02:49:57PM -0400, Paul Schlie wrote: > I’ve noticed that multiple (as great as 8 or more) parallel > redundant streams and corresponding temp\_files are opened > reading the same file from a reverse proxy backend into nginx, > upon even a single request by an up-stream client, if not > already cached (or stored in a static proxy’ed file) local to > nginx. > > This seems extremely wasteful of bandwidth between nginx and > corresponding reverse proxy backends; does anyone know why this > is occurring and how to limit this behavior? [http://nginx.org/r/proxy\_cache\_lock](http://nginx.org/r/proxy_cache_lock) – Maxim D. [http://nginx.org/](http://nginx.org/) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [June 25, 2014, 1:51am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/3 "2014-06-25T01:51:45Z")
Thank you; however it appears to have no effect on reverse proxy\_store’d static files? (Which seems odd, if it actually works for cached files; as both are first read into temp\_files, being the root of the problem.) Any idea on how to prevent multiple redundant streams and corresponding temp\_files being created when reading/updating a reverse proxy’d static file from the backend? (Out of curiosity, why would anyone ever want many multiple redundant streams/temp\_files ever opened by default?) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [June 25, 2014, 2:30am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/4 "2014-06-25T02:30:38Z")
Hello! On Tue, Jun 24, 2014 at 07:51:04PM -0400, Paul Schlie wrote: > Thank you; however it appears to have no effect on reverse proxy\_store’d static > files? Yes, it’s part of the cache machinery. The proxy\_store functionality is dumb and just provides a way to store responses received, nothing more. > (Which seems odd, if it actually works for cached files; as both > are first read into temp\_files, being the root of the problem.) See above (and below). > Any idea on how to prevent multiple redundant streams and > corresponding temp\_files being created when reading/updating a > reverse proxy’d static file from the backend? You may try to do so using limit\_conn, and may be error\_page and limit\_req to introduce some delay. But unlikely it will be a good / maintainable / easy to write solution. > (Out of curiosity, why would anyone ever want many multiple > redundant streams/temp\_files ever opened by default?) You never know if responses are going to be the same. The part which knows (or, rather, tries to) is called “cache”, and has lots of directives to control it. – Maxim D. [http://nginx.org/](http://nginx.org/) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [June 25, 2014, 2:59am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/5 "2014-06-25T02:59:09Z")
Again thank you. However … (below) On Jun 24, 2014, at 8:30 PM, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Hello! > > On Tue, Jun 24, 2014 at 07:51:04PM -0400, Paul Schlie wrote: > > > Thank you; however it appears to have no effect on reverse proxy\_store’d static > > files? > > Yes, it’s part of the cache machinery. The proxy\_store > functionality is dumb and just provides a way to store responses > received, nothing more. - There should be no difference between how reverse proxy’d files are accessed and first stored into corresponding temp\_files (and below). > You may try to do so using limit\_conn, and may be error\_page and > limit\_req to introduce some delay. But unlikely it will be a > good / maintainable / easy to write solution. - Please consider implementing by default that no more streams than may become necessary if a previously opened stream appears to have died (timed out), as otherwise only more bandwidth and thereby delay will most likely result to complete the request. Further as there should be no difference between how reverse proxy read-streams and corresponding temp\_files are created, regardless of whether they may be subsequently stored as either symbolically-named static files, or hash-named cache files; this behavior should be common to both. > > (Out of curiosity, why would anyone ever want many multiple > > redundant streams/temp\_files ever opened by default?) > > You never know if responses are going to be the same. The part > which knows (or, rather, tries to) is called “cache”, and has > lots of directives to control it. - If they’re not “the same” then the tcp protocol stack has failed, which is nothing to do with ngiinx. (unless a backend server is frequently dropping connections, it’s counterproductive to open multiple redundant streams; as doing so by default will only likely result in higher-bandwidth and thereby slower response completion.) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [June 25, 2014, 4:59am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/6 "2014-06-25T04:59:23Z")
Hi, Upon further testing, it appears the problem exists even with proxy\_cache’d files with “proxy\_cache\_lock on”. (Please consider this a serious bug, which I’m surprised hasn’t been detected before; verified on recently released 1.7.2) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 2:45am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/7 "2014-07-01T02:45:03Z")
Is there any possible solution for this problem? As although proxy\_cache\_lock may inhibit the creation of multiple proxy\_cache files, it has seemingly no effect on the creation of multiple proxy\_temp files, being the true root of the problem which the description of proxy\_cache\_lock claims to solve (as all proxy\_cache files are first proxy\_temp files, so unless proxy\_cache\_lock can properly prevent the creation of multiple redundant proxy\_temp file streams, it can seemingly not have the effect it claims to)? (Further, as temp\_file’s are used to commonly source all reverse proxy’d reads, regardless of whether they’re using a cache hashed naming scheme for proxy\_cache files, or a symbolic naming scheme for reverse proxy’d static files; it would be nice if the fix were applicable to both.) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 3:33am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/8 "2014-07-01T03:33:17Z")
Hello! On Mon, Jun 30, 2014 at 09:14:06PM -0400, Paul Schlie wrote: > (Seemingly, it may be beneficial to simply replace the > sequentially numbered temp\_file scheme with hash-named scheme, > where if cached, the file is simply retained for some period of > time and/or other condition, and which may be optionally > symbolically aliased using their uri path and thereby > respectively logically accessed as a local static file, or > deleted upon no longer being needed and not being cached; and > thereby kill multiple birds with one stone per-se?) Sorry for not following your discussion with yourself, but it looks you didn’t understand what was explained earlier: […] > > > > > > (Out of curiosity, why would anyone ever want many multiple > > > > > > redundant streams/temp\_files ever opened by default?) > > > > > > > > > > You never know if responses are going to be the same. The part > > > > > which knows (or, rather, tries to) is called “cache”, and has > > > > > lots of directives to control it. > > > > > > > > - If they’re not “the same” then the tcp protocol stack has failed, which is > > > > nothing to do with ngiinx. In http, responses are not guaranteed to be the same. Each response can be unique, and you can’t assume responses have to be identical even if their URLs match. – Maxim D. [http://nginx.org/](http://nginx.org/) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 3:14am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/9 "2014-07-01T03:14:49Z")
(Seemingly, it may be beneficial to simply replace the sequentially numbered temp\_file scheme with hash-named scheme, where if cached, the file is simply retained for some period of time and/or other condition, and which may be optionally symbolically aliased using their uri path and thereby respectively logically accessed as a local static file, or deleted upon no longer being needed and not being cached; and thereby kill multiple birds with one stone per-se?) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 5:11am UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/10 "2014-07-01T05:11:38Z")
Regarding: > In http, responses are not guaranteed to be the same. Each > response can be unique, and you can’t assume responses have to be > identical even if their URLs match. Yes, but potentially unique does not imply that upon the first valid ok or valid partial response that it will likely be productive to continue to open further such channels unless no longer responsive, as doing so will most likely be counter productive, only wasting limited resources by establishing redundant channels; being seemingly why proxy\_cache\_lock was introduced, as you initially suggested. ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 1:02pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/11 "2014-07-01T13:02:38Z")
Hello! On Mon, Jun 30, 2014 at 11:10:52PM -0400, Paul Schlie wrote: > being seemingly why proxy\_cache\_lock was introduced, as you initially suggested. > Again: responses are not guaranteed to be the same, and unless > you are using cache (and hence proxy\_cache\_key and various header > checks to ensure responses are at least interchangeable), the only > thing you can do is to proxy requests one by one. If you are using cache, then there is proxy\_cache\_key to identify a resource requested, and proxy\_cache\_lock to prevent multiple parallel requests to populate the same cache node (and “proxy\_cache\_use\_stale updating” to prevent multiple requests when updating a cache node). In theory, cache code can be improved (compared to what we currently have) to introduce sending of a response being loaded into a cache to multiple clients. I.e., stop waiting for a cache lock once we’ve got the response headers, and stream the response body being load to all clients waited for it. This should/can help when loading large files into a cache, when waiting with proxy\_cache\_lock for a complete response isn’t cheap. In practice, introducing such a code isn’t cheap either, and it’s not about using other names for temporary files. – Maxim D. [http://nginx.org/](http://nginx.org/) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 3:20pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/12 "2014-07-01T15:20:54Z")
Hello! On Tue, Jul 01, 2014 at 08:44:47AM -0400, Paul Schlie wrote: > completed transfer as you’ve noted? > Your reading of the code is incorrect. A node in shared memory is created on a request start, and this is enough for proxy\_cache\_lock to work. On the request completion, the temporary file is placed into the cache directory, and the node is updated to reflect that the cache file exists and can be used. – Maxim D. [http://nginx.org/](http://nginx.org/) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 4:16pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/13 "2014-07-01T16:16:17Z")
Then how could multiple streams and corresponding temp\_files ever be created upon successive requests for the same $uri with “proxy\_cache\_key $uri” and “proxy\_cache\_lock on”; if all subsequent requests are locked to the same cache\_node created by the first request even prior to its completion? You’ve previously noted: > In theory, cache code can be improved (compared to what we > currently have) to introduce sending of a response being loaded > into a cache to multiple clients. I.e., stop waiting for a cache > lock once we’ve got the response headers, and stream the response > body being load to all clients waited for it. This should/can > help when loading large files into a cache, when waiting with > proxy\_cache\_lock for a complete response isn’t cheap. In > practice, introducing such a code isn’t cheap either, and it’s not > about using other names for temporary files. Being what I apparently incorrectly understood proxy\_cache\_lock to actually do. So if not the above, what does proxy\_cache\_lock actually do upon receipt of subsequent requests for the same $uri? ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 2:45pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/14 "2014-07-01T14:45:07Z")
As it appears a downstream response is not cached until first completely read into a temp\_file (which for a large file may require 100’s if not 1,000’s of MB be transferred), there appears to be no “cache node formed” which to “lock” or serve “stale” responses from, and thereby until the first “cache node” is useably created, proxy\_cache\_lock has nothing to lock requests to? The code does not appear to be forming a “cache node” using the designated cache\_key until the requested downstream element has completed transfer as you’ve noted? For the scheme to work, a lockable cache\_node would need to formed immediately upon the first unique cache\_key request, and not wait until the transfer of the requested item being stored into a temp\_file is complete; as otherwise multiple redundant active streams between nginx and a backend server may be formed, each most likely transferring the same information needlessly; being what proxy\_cache\_lock was seemingly introduced to prevent (but it doesn’t)? ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 6:40pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/15 "2014-07-01T18:40:24Z")
Hello! On Tue, Jul 01, 2014 at 10:15:47AM -0400, Paul Schlie wrote: > Then how could multiple streams and corresponding temp\_files > ever be created upon successive requests for the same $uri with > “proxy\_cache\_key $uri” and “proxy\_cache\_lock on”; if all > subsequent requests are locked to the same cache\_node created by > the first request even prior to its completion? Quoting documentation, [Alphabetical index of directives](http://nginx.org/r/proxy_cache_lock:) : When enabled, only one request at a time will be allowed to : populate a new cache element identified according to the : proxy\_cache\_key directive by passing a request to a proxied : server. Other requests of the same cache element will either wait : for a response to appear in the cache or the cache lock for this : element to be released, up to the time set by the : proxy\_cache\_lock\_timeout directive. So, there are at least two cases “prior to its completion” which are explicitly documented: 1. If the cache lock is released - this happens, e.g., if the response isn’t cacheable according to the response headers. 2. If proxy\_cache\_lock\_timeout expires. – Maxim D. [http://nginx.org/](http://nginx.org/) ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 10:12pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/16 "2014-07-01T22:12:26Z")
Thank you for your patience. I mistakenly thought the 5 second default value associated with proxy\_cache\_lock\_timeout was the maximum delay allowed between successive responses from the backend server is satisfaction of the reverse proxy request being cached prior to the cache lock being released, not the maximum delay for the response to be completely received and cached as it appears to actually be. Now that I understand, please consider setting the default value much higher, or more ideally set in proportion to the size of the item being cached and possibly some measure of the activity of the stream; as in most circumstances, redundant streams should never be opened, as it will tend to only make matters worse. Thank you. ---
**Author:** ![Paul\_Schlie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/22d042/32.png) [@Paul\_Schlie](https://www.ruby-forum.com/u/Paul_Schlie)\ **Post date:** [July 1, 2014, 11:04pm UTC](https://www.ruby-forum.com/t/how-can-the-number-of-parallel-redundant-open-streams-temp-files-be-controlled-limited/230575/17 "2014-07-01T23:04:15Z")
Lastly, is there any way to try to get proxy\_store to work in combination with proxy\_cache, possibly by enabling the completed temp\_file to be saved as a proxy\_store file within its uri logical path hierarchy, and the cache\_file descriptor aliased to it, or visa versa? (As it’s often nice to be able to view/access cached files within their natural uri hierarchy, being virtually impossible if stored using their corresponding hashed names alone; and not lose the benefit of being able to lock multiple pending requests to the same cache\_node being fetched so as to minimize otherwise redundant down-stream requests prior to the file being cached.)