# High memory consumption when proxying to a Comet server **URL:** \ **Category:** NGINX\ **Created:** [April 12, 2010, 2:17am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709 "2010-04-12T02:17:24Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 2:17am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/1 "2010-04-12T02:17:24Z")
Folks, When proxying to a Comet server, I was able to notice that Nginx is eating and not releasing a massive amount of memory. As we all know, Comet servers can have this “streaming” mode, where we receive a chunked HTTP request that never ends. Is there some problem that avoids Nginx to release memory when in proxy mode, before the connection ends at all? I am using Nginx to balance between some instances of the Comet server: upstream comet { server comet:8081; server comet:8082; … } Memory consumption for Nginx procs for each minute during the tests: 271.3 MiB + 368.0 KiB = 271.6 MiB nginx (9) 273.3 MiB + 368.0 KiB = 273.6 MiB nginx (9) 274.8 MiB + 368.0 KiB = 275.2 MiB nginx (9) 276.6 MiB + 368.0 KiB = 277.0 MiB nginx (9) 278.8 MiB + 368.0 KiB = 279.2 MiB nginx (9) 281.0 MiB + 368.0 KiB = 281.4 MiB nginx (9) 283.2 MiB + 368.0 KiB = 283.6 MiB nginx (9) 285.4 MiB + 368.0 KiB = 285.8 MiB nginx (9) 287.2 MiB + 368.0 KiB = 287.6 MiB nginx (9) 288.9 MiB + 368.0 KiB = 289.3 MiB nginx (9) I could also notice that the bigger is the message I send through the Comet server (thus, through Nginx too) the bigger is the memory allocation in the Nginx procs. Also, if this helps, Nginx only writes down the access\_log when the client closes the connection. What could I do to avoid this “leak” behaviour for long lasting proxy connections in Nginx? ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 2:50am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/2 "2010-04-12T02:50:10Z")
Hello! On Sun, Apr 11, 2010 at 09:16:33PM -0300, Rogério Schneider wrote: > When proxying to a Comet server, I was able to notice that Nginx is > eating and not releasing a massive amount of memory. > > As we all know, Comet servers can have this “streaming” mode, where we > receive a chunked HTTP request that never ends. > > Is there some problem that avoids Nginx to release memory when in > proxy mode, before the connection ends at all? […] nginx does memory allocations on per-request basis, and frees memory when request ends. Though after sending response headers possible allocations are limited by various \*\_buffers settings. For proxied requests you may want to make sure your proxy\_buffers aren’t set too high, see here: > **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)** > > ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com Maxim D. ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 3:08am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/3 "2010-04-12T03:08:55Z")
Maxim, thanks. I have tried to reduce memory usage by reducing the default values for: proxy\_buffers from “8 4k” to “2 4k”. With this one would expect the memory usage to be 4 times lower, but that was not what I saw. I saw the memory usage stay in the same growth step. May I mention that for the fact I am using Comet, I turned proxy buffering off? proxy\_buffering off; I think this is important to note and it can explain why tuning the proxy buffers down did not lower the memory consumption. Could I debug Nginx procs to see where the memory is being allocated and for what purpose? How could I debug it? Do you have some guideline? Many thanks. Regards, Rogério Schneider On Sun, Apr 11, 2010 at 9:49 PM, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > Is there some problem that avoids Nginx to release memory when in > > [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/NginxHttpProxyModule#proxy_buffers) > > Maxim D. > > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) – Rogério Schneider +55 (21) 9856 7847 MSN: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) GTalk: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Skype: stockrt [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 5:44am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/4 "2010-04-12T05:44:02Z")
> No, changing 8 buffers to 2 buffers doesn’t mean memory usage will > be 4 times less. 1) this isn’t the only allocations made, and 2) > not all requests allocate all proxy buffers, only ones with big > responses and slow clients. But see below. Great, I didn’t know that. > Next candidates is gzip\_buffers (only matters if you are using > gzip, up to 32 \* 4k by default; and keep in mind that about 200k > or so will be allocated per request anyway with gzip enabled). I am not using gzip. Nor the request come with Accept-Encoding, neither I am turning “gzip on;”, so I think this is not the case too. Another thing to notice is about the “per request” usage. I have only one request per client, but this request lasts for many minutes, still, when monitoring Nginx memory usage, it increases every minute, even with no new requests made from the client side. In this period of monitoring only the Comet server was writing data, up to the Nginx proxy. > Remaining ones are output\_buffers and fastcgi\_buffers, these > shouldn’t matter in your case anyway. I could not find where are the docs about output\_buffers. I am interested on reading about it. Also, I am using this version of Nginx: nginx version: nginx/0.7.65 built by gcc 4.3.0 20080428 (Red Hat 4.3.0-8) (GCC) configure arguments: --prefix=/opt/tr/nginx --without-select\_module –without-poll\_module --without-http\_charset\_module –without-http\_ssi\_module --without-http\_auth\_basic\_module –without-http\_autoindex\_module --without-http\_geo\_module –without-http\_map\_module --without-http\_referer\_module –without-http\_rewrite\_module --without-http\_fastcgi\_module –without-http\_memcached\_module --without-http\_limit\_zone\_module –without-http\_limit\_req\_module --without-http\_empty\_gif\_module –without-http\_browser\_module --without-http\_upstream\_ip\_hash\_module –without-mail\_pop3\_module --without-mail\_imap\_module –without-mail\_smtp\_module --with-http\_stub\_status\_module --with-debug > You may switch on debug log, it will be possible to trace big > allocations there. See here: > > [A debugging log](http://nginx.org/en/docs/debugging_log.html) I posted this log of some tests, and if you can please take a look at it, I would appreciate. It was separated in sections, the start, the small msg, the big msg, the close. Then another run, the small msg after many big msgs, and the close (with lots of “strange” free()s): > ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 9:53pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/5 "2010-04-12T21:53:00Z")
Hello! On Mon, Apr 12, 2010 at 12:43:23AM -0300, Rogério Schneider wrote: […] > Then another run, the small msg after many big msgs, and the > close (with lots of “strange” free()s): > > [nginx and comet (memory consumption behaviour) - Pastebin.com](http://pastebin.com/Fggn4Ui7) Yep, I see what’s going on. It’s chunked filter which eats memory - it allocates small buffers for chunk-size and crlf markers. They aren’t big, but they are allocated for each buffer sent, and they aren’t reused. Here is snipped from log where it causes another 4k allocation from system (note “malloc” line): 2010/04/12 00:06:04 [debug] 32748#0: \*55 recv: fd:16 1024 of 1024 2010/04/12 00:06:04 [debug] 32748#0: \*55 http output filter “/push/user1/xhrinteractive/canal2.b1?” 2010/04/12 00:06:04 [debug] 32748#0: \*55 copy filter: “/push/user1/xhrinteractive/canal2.b1?” 2010/04/12 00:06:04 [debug] 32748#0: \*55 http chunk: 1024 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096E9878:4096 2010/04/12 00:06:04 [debug] 32748#0: \*55 write new buf t:1 f:0 00000000, pos 096E985C, size: 5 file: 0, size: 0 2010/04/12 00:06:04 [debug] 32748#0: \*55 write new buf t:0 f:0 00000000, pos 096E8140, size: 1024 file: 0, size: 0 2010/04/12 00:06:04 [debug] 32748#0: \*55 write new buf t:0 f:0 00000000, pos 080A6EE5, size: 2 file: 0, size: 0 2010/04/12 00:06:04 [debug] 32748#0: \*55 http write filter: l:0 f:1 s:1031 2010/04/12 00:06:04 [debug] 32748#0: \*55 http write filter limit 0 2010/04/12 00:06:04 [debug] 32748#0: \*55 writev: 1031 As a workaround you may want to increase proxy\_buffer\_size to reduce number of such allocation (and/or just drop connections periodically). Correct fix would be to make these buffers reusable once sent to client. Maxim D. ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 10:17pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/6 "2010-04-12T22:17:35Z")
> As a workaround you may want to increase proxy\_buffer\_size to > reduce number of such allocation (and/or just drop connections > periodically). Correct fix would be to make these buffers > reusable once sent to client. Maxim, thanks for the review of the logs. Increasing proxy\_buffer\_size wouldn’t end up on malloc()ing more memory at the end of a period? I think on sending small chunks (say, 500 bytes) each minute, during hours, to the clients. For each small chunk, I think, we will be malloc()ing one buffer, or am I wrong? ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 3:58am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/7 "2010-04-12T03:58:06Z")
Hello! On Sun, Apr 11, 2010 at 10:08:26PM -0300, Rogério Schneider wrote: > Maxim, thanks. > > I have tried to reduce memory usage by reducing the default values for: > proxy\_buffers > from “8 4k” to “2 4k”. > > With this one would expect the memory usage to be 4 times lower, but > that was not what I saw. > I saw the memory usage stay in the same growth step. No, changing 8 buffers to 2 buffers doesn’t mean memory usage will be 4 times less. 1) this isn’t the only allocations made, and 2) not all requests allocate all proxy buffers, only ones with big responses and slow clients. But see below. > May I mention that for the fact I am using Comet, I turned proxy buffering off? > proxy\_buffering off; > I think this is important to note and it can explain why tuning the > proxy buffers down did not lower the memory consumption. With proxy\_buffering off; setting proxy\_buffers indeed doesn’t matter at all. Next candidates is gzip\_buffers (only matters if you are using gzip, up to 32 \* 4k by default; and keep in mind that about 200k or so will be allocated per request anyway with gzip enabled). Remaining ones are output\_buffers and fastcgi\_buffers, these shouldn’t matter in your case anyway. > Could I debug Nginx procs to see where the memory is being allocated > and for what purpose? > How could I debug it? Do you have some guideline? You may switch on debug log, it will be possible to trace big allocations there. See here: > **[A debugging log](http://nginx.org/en/docs/debugging_log.html)** Maxim D. p.s. Please do not top-post. Thank you. ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 12, 2010, 11:27pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/8 "2010-04-12T23:27:51Z")
Hello! On Mon, Apr 12, 2010 at 05:17:10PM -0300, Rogério Schneider wrote: > > As a workaround you may want to increase proxy\_buffer\_size to > > reduce number of such allocation (and/or just drop connections > > periodically).  Correct fix would be to make these buffers > > reusable once sent to client. > > Maxim, thanks for the review of the logs. > > Increasing proxy\_buffer\_size wouldn’t end up on malloc()ing more > memory at the end of a period? It’s size of single proxy buffer used for passing data with proxy\_buffering off. It will be allocated once during request start. And by allocating bigger buffer you reduce number of chunked encoding markers needed to pass big responses… > I think on sending small chunks (say, 500 bytes) each minute, > during hours, to the clients. … but once your chunks are small enough to fit into your proxy\_buffer\_size - there is no difference. > For each small chunk, I think, we will be malloc()ing one buffer, or > am I wrong? Not exactly. It allocates 2 buffer structures (52 bytes each on i386) and 18 bytes to hold chunk-size and initial CRLF (final CRLF uses static string). This gives something about 122 bytes per chunk, i.e. about 30 chunks per 4k malloc’ed from system (± alignment and allocation overheads). And here is the numbers in your logs: $ egrep ‘malloc:|recv:’ zz2 | uniq -c … 1 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096E9878:4096 31 2010/04/12 00:06:04 [debug] 32748#0: \*55 recv: fd:16 1024 of 1024 1 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096F5160:4096 30 2010/04/12 00:06:04 [debug] 32748#0: \*55 recv: fd:16 1024 of 1024 1 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096F6168:4096 … Numbers vary slightly due to process not being stable enough, but it doesn’t really matter. Maxim D. ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [May 28, 2010, 3:55pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/9 "2010-05-28T15:55:38Z")
I have the very same problem trying to proxying a MJPEG stream. I really would appreciate a solution not involving periodically resetting the connection. Many thanks, Arrigo Zanette Posted at Nginx Forum: > **[Re: High memory consumption when proxying to a Comet server](https://forum.nginx.org/read.php?2,73923,91502#msg-91502)** > > Folks, When proxying to a Comet server, I was able to notice that Nginx is eating and not releasing a massive amount of memory. As we all know, Comet servers can have this "streaming" mode, where we receive a chunked HTTP request that never... ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [April 15, 2010, 9:43am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/10 "2010-04-15T09:43:59Z")
> It’s size of single proxy buffer used for passing data with > proxy\_buffering off. It will be allocated once during request > start. And by allocating bigger buffer you reduce number of > chunked encoding markers needed to pass big responses… > … > Not exactly. It allocates 2 buffer structures (52 bytes each on > i386) and 18 bytes to hold chunk-size and initial CRLF (final CRLF > uses static string). This gives something about 122 bytes per > chunk, i.e. about 30 chunks per 4k malloc’ed from system (± > alignment and allocation overheads). > … > … > 1 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096E9878:4096 > 31 2010/04/12 00:06:04 [debug] 32748#0: \*55 recv: fd:16 1024 of 1024 > 1 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096F5160:4096 > 30 2010/04/12 00:06:04 [debug] 32748#0: \*55 recv: fd:16 1024 of 1024 > 1 2010/04/12 00:06:04 [debug] 32748#0: \*55 malloc: 096F6168:4096 > … Is there some chance in free()ing this allocations after the chunk is sent, but before the connection is finished? Since it is a long lasting connection, I need to free some space at regular interval, so I do not run out of memory. Many thanks for all the explanations. ## Sincerely, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 8, 2010, 4:16am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/11 "2010-06-08T04:16:31Z")
On Fri, May 28, 2010 at 10:54 AM, zanettea [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > I have the very same problem trying to proxying a MJPEG stream. I really would appreciate a solution not involving periodically resetting the connection. I have seen your post referring mine, here [http://forum.nginx.org/read.php?29,91967,92454](http://forum.nginx.org/read.php?29,91967,92454) and also have seen this another post related to the matter here [http://forum.nginx.org/read.php?2,84623,84623#msg-84623](http://forum.nginx.org/read.php?2,84623,84623#msg-84623) but no one seems to bother in giving a direct response if that is or isn’t possible to free memory for chunked responses BEFORE the connection ends. ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 8, 2010, 1:46pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/12 "2010-06-08T13:46:41Z")
The patch I attached actually frees memory before the connection ends but do a lot of assumptions that might not work. The cleanest solution is probably just to disable chunked transfer encoding with the option: chunked\_transfer\_encoding off; which removes the problem at the root (if it is ok for you to disable chunked transfer encoding, of if you already implement it in the proxied server). AFAIK nginx does not support memory freeing/reuse before connection ends and, with current architecture, is not a trivial job to do (requires to handle reuse of non-contiguous allocations). Regards, Arrigo Zanette ## Rogério Schneider Wrote: > [http://forum.nginx.org/read.php?2,84623,84623#msg-](http://forum.nginx.org/read.php?2,84623,84623#msg-) > [http://stockrt.github.com](http://stockrt.github.com) > > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) Posted at Nginx Forum: [http://forum.nginx.org/read.php?2,73923,95800#msg-95800](http://forum.nginx.org/read.php?2,73923,95800#msg-95800) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 8, 2010, 4:46pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/13 "2010-06-08T16:46:31Z")
> No, small allocations can’t be freed in nginx. I outlined correct > fix as soon as the problem was identified, here: > > [High memory consumption when proxying to a Comet server](http://nginx.org/pipermail/nginx/2010-April/019845.html) > > One should sit and code buffers reuse in chunked module. I have > this in my TODO list, but didn’t done this yet due to ENOTIME. Ok Maxim, now I got it correctly. Thanks for the feedback, again. ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 8, 2010, 10:35am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/14 "2010-06-08T10:35:13Z")
Hello! On Mon, Jun 07, 2010 at 11:15:20PM -0300, Rogério Schneider wrote: > On Fri, May 28, 2010 at 10:54 AM, zanettea [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > > I have the very same problem trying to proxying a MJPEG stream. I really would appreciate a solution not involving periodically resetting the connection. > > I have seen your post referring mine, here > [http://forum.nginx.org/read.php?29,91967,92454](http://forum.nginx.org/read.php?29,91967,92454) and also have seen this > another post related to the matter here > [http://forum.nginx.org/read.php?2,84623,84623#msg-84623](http://forum.nginx.org/read.php?2,84623,84623#msg-84623) but no one > seems to bother in giving a direct response if that is or isn’t > possible to free memory for chunked responses BEFORE the connection > ends. No, small allocations can’t be freed in nginx. I outlined correct fix as soon as the problem was identified, here: > **[High memory consumption when proxying to a Comet server](http://mailman.nginx.org/pipermail/nginx/2010-April/019845.html)** One should sit and code buffers reuse in chunked module. I have this in my TODO list, but didn’t done this yet due to ENOTIME. Maxim D. ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 8, 2010, 5:07pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/15 "2010-06-08T17:07:36Z")
On Tue, Jun 08, 2010 at 11:55:01AM -0300, Rogério Schneider wrote: > > The patch I attached actually frees memory before the connection ends but do a lot of assumptions that might not work. The cleanest solution is probably just to disable chunked transfer encoding with the option: > > > > chunked\_transfer\_encoding off; > > I am using 0.7.65 and it does not support this option. Try 0.7.66. – Igor S. > **[Igor Sysoev](http://sysoev.ru/en/)** ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 8, 2010, 4:55pm UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/16 "2010-06-08T16:55:53Z")
> The patch I attached actually frees memory before the connection ends but do a lot of assumptions that might not work. The cleanest solution is probably just to disable chunked transfer encoding with the option: > > chunked\_transfer\_encoding off; I am using 0.7.65 and it does not support this option. > which removes the problem at the root (if it is ok for you to disable chunked transfer encoding, of if you already implement it in the proxied server). > > AFAIK nginx does not support memory freeing/reuse before connection ends and, with current architecture, is not a trivial job to do (requires to handle reuse of non-contiguous allocations). Thanks for these pointings, Arrigo. ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com) ---
**Author:** ![RogSSSSrio\_Schneide](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/c6cbf5/32.png) [@RogSSSSrio\_Schneide](https://www.ruby-forum.com/u/RogSSSSrio_Schneide)\ **Post date:** [June 9, 2010, 1:59am UTC](https://www.ruby-forum.com/t/high-memory-consumption-when-proxying-to-a-comet-server/186709/17 "2010-06-09T01:59:10Z")
> > I am using 0.7.65 and it does not support this option. > > Try 0.7.66. I tried it and the behavior is the same. Perhaps 0.8 series solved this already? ## Regards, Rogério Schneider [http://stockrt.github.com](http://stockrt.github.com)