# Dealing with buffered data with upstream generated response **URL:** \ **Category:** NGINX\ **Created:** [November 28, 2011, 5:52pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321 "2011-11-28T17:52:44Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\ **Post date:** [November 28, 2011, 5:52pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/1 "2011-11-28T17:52:44Z")
I am developing a body filter which transforms outgoing stream. As part of the function, the incoming stream is copied in new buffer and sent. This in some cases results in a condition where my filter has busy buffers but the upper layers don’t have any busy buffer. In our case the data is generated by upstream module. Since upstream module does not pay attention to connection-\>buffered, the output\_filter is never called again to flush my busy buffers and the transfer just hangs. Adding r-\>connection-\>buffered to the or condition solves the hang. ``` if (u->out_bufs || u->busy_bufs) { rc = ngx_http_output_filter(r, u->out_bufs); if (rc == NGX_ERROR) { ngx_http_upstream_finalize_request(r, u, 0); return; } ngx_chain_update_chains(&u->free_bufs, &u->busy_bufs, ``` &u-\>out\_bufs, u-\>output.tag); } Is this nginx bug or am I missing something? -Umesh ---
**Author:** ![Umesh\_Sirsiwal](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/8e8cbc/32.png) [@Umesh\_Sirsiwal](https://www.ruby-forum.com/u/Umesh_Sirsiwal)\ **Post date:** [November 29, 2011, 2:08pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/2 "2011-11-29T14:08:29Z")
Any help on this will be greatly appreciated. Thank you! -Umesh * * * I am developing a body filter which transforms outgoing stream. As part of the function, the incoming stream is copied in new buffer and sent. This in some cases results in a condition where my filter has busy buffers but the upper layers don’t have any busy buffer. In our case the data is generated by upstream module. Since upstream module does not pay attention to connection-\>buffered, the output\_filter is never called again to flush my busy buffers and the transfer just hangs. Adding r-\>connection-\>buffered to the or condition solves the hang. ``` if (u->out_bufs || u->busy_bufs) { rc = ngx_http_output_filter(r, u->out_bufs); if (rc == NGX_ERROR) { ngx_http_upstream_finalize_request(r, u, 0); return; } ngx_chain_update_chains(&u->free_bufs, &u->busy_bufs, ``` &u-\>out\_bufs, u-\>output.tag); } Is this nginx bug or am I missing something? -Umesh ---
**Author:** ![Umesh\_Sirsiwal](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/8e8cbc/32.png) [@Umesh\_Sirsiwal](https://www.ruby-forum.com/u/Umesh_Sirsiwal)\ **Post date:** [November 29, 2011, 7:31pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/3 "2011-11-29T19:31:24Z")
Hello! On Mon, Nov 28, 2011 at 11:52:38AM -0500, Umesh Sirsiwal wrote: > ``` > } > > ``` > > Is this nginx bug or am I missing something? I tend to think this is a bug, and it should probably manifest itself with e.g. gzip filter as well. (Well, as far as I see transfer shouldn’t hang completely, but it will be only resumed as soon as upstream sends some more data or closes connection.) It would be superb if you’ll be able to provide test case which catches this for our test suite ([nginx-tests: log](http://mdounin.ru/hg/nginx-tests)). I’ll look into this more closely as time permits. Maxim D. ---
**Author:** ![Umesh\_Sirsiwal](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/8e8cbc/32.png) [@Umesh\_Sirsiwal](https://www.ruby-forum.com/u/Umesh_Sirsiwal)\ **Post date:** [November 29, 2011, 8:19pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/4 "2011-11-29T20:19:07Z")
Thanks Maxim, > > ``` > > } > > > > ``` > > > > Is this nginx bug or am I missing something? > > I tend to think this is a bug, and it should probably manifest > > itself with e.g. gzip filter as well. > > (Well, as far as I see transfer shouldn’t hang completely, but it > will be only resumed as soon as upstream sends some more data or > closes connection.) > In our application we use Upstream KeepAlive module to keep the upstream > connection open. So if this happens towards the end of the last > subrequest response, the connection hangs. > > It would be superb if you’ll be able to provide test case which > catches this for our test suite ([nginx-tests: log](http://mdounin.ru/hg/nginx-tests)). > I will try to see if I can reproduce it with gzip filter. I should be > able to reproduce it with > limit-rate+gzip-filter+proxy+upstream-keepalive module combination. But, > I am not sure if the nginx-test test-suite includes upstream-keepalive. ---
**Author:** ![Umesh\_Sirsiwal](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/8e8cbc/32.png) [@Umesh\_Sirsiwal](https://www.ruby-forum.com/u/Umesh_Sirsiwal)\ **Post date:** [December 2, 2011, 6:03pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/5 "2011-12-02T18:03:23Z")
Hello! On Fri, Dec 02, 2011 at 11:13:15AM -0500, Umesh Sirsiwal wrote: > Hey Maxim, > I spent some time yesterday and today trying to reproduce this issue > with standard gzip module. But, unfortunately I could not. May be I > am doing something wrong in my module. > > My configuration looked like the following. I was using nginx > version 1.1.10. […] > ``` > proxy_buffering on; > > ``` The code in question isn’t executed with proxy\_buffering set to on, it’s only in “proxy\_buffering off” codepath. Maxim D. ---
**Author:** ![Umesh\_Sirsiwal](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/8e8cbc/32.png) [@Umesh\_Sirsiwal](https://www.ruby-forum.com/u/Umesh_Sirsiwal)\ **Post date:** [December 2, 2011, 7:39pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/6 "2011-12-02T19:39:18Z")
On 12/02/2011 12:02 PM, Maxim D. wrote: > > version 1.1.10. > > […] > > > ``` > > proxy_buffering on; > > > > ``` > > The code in question isn’t executed with proxy\_buffering set to > on, it’s only in “proxy\_buffering off” codepath. > Ouch! I changed it to “proxy\_buffering off” and retrieved 10MB file in a > loop over and over again. Still can’t reproduce it. Looking at netstat > output I can see lots of data sitting in the socket buffer. I did notice that the limit\_rate directive does not work with proxy\_buffering off. -Umesh ---
**Author:** ![Umesh\_Sirsiwal](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/8e8cbc/32.png) [@Umesh\_Sirsiwal](https://www.ruby-forum.com/u/Umesh_Sirsiwal)\ **Post date:** [December 2, 2011, 5:13pm UTC](https://www.ruby-forum.com/t/dealing-with-buffered-data-with-upstream-generated-response/214321/7 "2011-12-02T17:13:14Z")
Hey Maxim, I spent some time yesterday and today trying to reproduce this issue with standard gzip module. But, unfortunately I could not. May be I am doing something wrong in my module. My configuration looked like the following. I was using nginx version 1.1.10. ## -Umesh worker\_processes 1; daemon off; http { default\_type application/octet-stream; ``` sendfile on; keepalive_timeout 65; upstream http_backend { server 127.0.0.1:8080; keepalive 1; } server { listen 8080 sndbuf=10k; server_name localhost; proxy_buffering on; proxy_buffers 10000 4k; gzip on; gzip_proxied any; location / { proxy_pass http://http_backend/local/; proxy_http_version 1.1; proxy_set_header Connection ""; #limit_rate 100K; } location /local/ { alias /tmp/nginx-test-dL3kfZazKJ/; } } ``` ## }