# Again: Workaround found for request queuing vs. num\_processors, accept/close **URL:** \ **Category:** Mongrel\ **Created:** [January 23, 2008, 8:27pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966 "2008-01-23T20:27:06Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 8:27pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/1 "2008-01-23T20:27:06Z")
Hello all. I too found out that I sometimes have some action that can take up to 10 seconds in my rails application. I’ve read all arguments Zed made about polling/and inbox strategies, and I think I just can’t work around my feeling that a “wrong” request that takes up too much time should be able to lock subsequent requests in mongrels queue. That’s what I have a balancer for, right? 🙂 So I tried Robert M.s hint[1] and used BalancerMember [http://foo](http://foo) max=1 acquire=100 to see if that helps. But I may have hit a bug[2], and this does not work for me. Does this work for someone? With what version of Apache? I am running 2.2.4. Or do all of you have only very fast requests that do not take more than a few seconds max to complete? Or do you run a different load balancer? HAProxy? I also think that the overall speed of the application would incerase even with relatively fast requests, because the balancer would forward the requests always to a free mongrel that has nothing to do, and can handle the request immediately. BTW, it looks like the acquire parameter is not in seconds, but in milliseconds, so I tried acquire=100 as well, but no difference. From[2] /\* Acquire timeout in milliseconds. \* If set this will be the maximum time to \* wait for a free connection. \*/ Maybe this belongs to the Apache ml and not to mongrel, but I think maybe some of you hit this problem and solved it somehow. Another issue: on my system, mongrel\_rails --version Mongrel Web Server 1.1.3 (installed as a gem) the --num-procs parameter does not seem to work. To debug the problem I added STDERR.puts “TEST: #{worker\_list.length} #{@num\_processors}” on line 279 of /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb, and to my surprise it looks like the parameter is ignored. If I run it $ mongrel\_rails start -a 127.0.0.1 -p 8999 --num-procs 1 & [1] 18617 \*\* Starting Mongrel listening at 127.0.0.1:8999 \*\* Starting Rails with development environment… config/boot.rb:28:Warning: require\_gem is obsolete. Use gem instead. \*\* Rails loaded. \*\* Loading any Rails specific GemPlugins \*\* Signals ready. TERM =\> stop. USR2 =\> restart. INT =\> stop (no restart). \*\* Rails signals registered. HUP =\> reload (without restart). It might not work well. \*\* Mongrel 1.1.3 available at 127.0.0.1:8999 \*\* Use CTRL-C to stop. and then: $ w3m -dump [http://localhost:8999/test](http://localhost:8999/test) \>/dev/null TEST: 0 950 $ So it looks like @num\_processors is still at the default value of 950, and not 1 as I told it on the command line. Does the deprecation warning maybe have something to do with this (I don’t think so, but you never know…)? Thank you! Simon [1][http://rubyforge.org/pipermail/mongrel-users/2007-October/004145.html](http://rubyforge.org/pipermail/mongrel-users/2007-October/004145.html) [2][[Apache-SVN] Contents of /httpd/httpd/trunk/modules/proxy/mod\_proxy.c](http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/mod_proxy.c?revision=608189&view=markup) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 9:38pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/2 "2008-01-23T21:38:10Z")
On Jan 23, 2008 12:18 PM, Simon S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Or do you run a different load balancer [for limiting the number of outstanding requests]. Many people do it with HAProxy. EngineYard built an Nginx plugin that does the same (don’t think they released it yet). – Alexey V. CruiseControl.rb [[http://cruisecontrolrb.thoughtworks.com](http://cruisecontrolrb.thoughtworks.com)] RubyWorks [[http://rubyworks.thoughtworks.com](http://rubyworks.thoughtworks.com)] ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 10:28pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/3 "2008-01-23T22:28:42Z")
On Wednesday 23 January 2008 21:36:39 Alexey V. wrote: > Many people do it with HAProxy. EngineYard built an Nginx plugin that > does the same (don’t think they released it yet). Thanks for your replay. At least now I know that someone understood my poor English 🙂 ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 10:51pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/4 "2008-01-23T22:51:35Z")
On Wed, 23 Jan 2008, Simon S. wrote: > … > So it looks like @num\_processors is still at the default value of 950, and not > 1 as I told it on the command line. Does the deprecation warning maybe have > something to do with this (I don’t think so, but you never know…)? hum… this seems like a bug. Could you please open a ticket for it? Cheers, filipe { @ [icewall.org](http://icewall.org) GPG 1024D/A6BA423E [http://filipe.icewall.org/](http://filipe.icewall.org/) } ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 11:12pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/5 "2008-01-23T23:12:13Z")
On Wednesday 23 January 2008 22:07:45 Filipe wrote: > hum… this seems like a bug. Could you please open a ticket for it? sure. [http://rubyforge.org/tracker/index.php?func=detail&aid=17428&group\_id=1306&atid=5145](http://rubyforge.org/tracker/index.php?func=detail&aid=17428&group_id=1306&atid=5145) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 11:38pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/6 "2008-01-23T23:38:01Z")
On Wednesday 23 January 2008 23:03:21 Ezra Z. wrote: > Actually the nginx fair balancer has been released for a while. You > can read up on it and get it here: > > [http://wiki.codemongers.com/NginxHttpUpstreamFairModule](http://wiki.codemongers.com/NginxHttpUpstreamFairModule) Hmmm. I can’t use a no-name web server with a load balancer that is a set of patches for that server on our intranet production server. At the first problem I would lose my job. ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 11:46pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/7 "2008-01-23T23:46:17Z")
On Jan 23, 2008, at 2:36 PM, Simon S. wrote: > problem I would lose my job. > That’s too bad then, you will have to suffer on with apache i guess. > Nginx is a lot more solid then apache is in stability and performance. > I’m personally running over 1000 nginx daemons with absolutely no > problems to speak of with it. Cheers- - Ezra Z. – Founder & Software Architect – removed\_email\_address@domain.invalid – [EngineYard.com](http://EngineYard.com) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 11:11pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/8 "2008-01-23T23:11:02Z")
On Jan 23, 2008, at 12:36 PM, Alexey V. wrote: > On Jan 23, 2008 12:18 PM, Simon S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > wrote: > > > Or do you run a different load balancer [for limiting the number of > > outstanding requests]. > > Many people do it with HAProxy. EngineYard built an Nginx plugin that > does the same (don’t think they released it yet). Actually the nginx fair balancer has been released for a while. You can read up on it and get it here: [http://wiki.codemongers.com/NginxHttpUpstreamFairModule](http://wiki.codemongers.com/NginxHttpUpstreamFairModule) Cheers- - Ezra Z. – Founder & Software Architect – removed\_email\_address@domain.invalid – [EngineYard.com](http://EngineYard.com) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 23, 2008, 11:59pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/9 "2008-01-23T23:59:47Z")
On Wednesday 23 January 2008 23:45:43 Ezra Z. wrote: > That’s too bad then, you will have to suffer on with apache i guess. Probably. > Nginx is a lot more solid then apache is in stability and performance. > I’m personally running over 1000 nginx daemons with absolutely no > problems to speak of with it. Do you have your architecture documented somewhere online? Can I read it? I need Apache for kerberos for SSO, and I don’t think nginx supports that (could not find it). I could put Apache in front of nginx balancing the mongrels, but at that point it’s maybe better to use haproxy like the the rubyworks guys suggest[1] (no need for patches). I’m open for suggestions. [1][http://rubyworks.rubyforge.org/manual/haproxy.html](http://rubyworks.rubyforge.org/manual/haproxy.html) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 24, 2008, 12:43am UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/10 "2008-01-24T00:43:07Z")
On Jan 23, 2008, at 2:57 PM, Simon S. wrote: > Do you have your architecture documented somewhere online? Can I > [1][http://rubyworks.rubyforge.org/manual/haproxy.html](http://rubyworks.rubyforge.org/manual/haproxy.html) > Hey Simon- I don’t have any whitepapers about our architecture currently available. But the 50,000 ft view is like this: Http request from client → LVS redundant load balancers → Nginx → mongrel\_cluster. The nginx and mongrel clusters run on the same Xen VM. with multiple VM’s behind the LVS load balancers. You are correct that nginx does not support kerberos so you will need to keep apache. Haproxy is probably a good solution for you in this case. Cheers- - Ezra Z. – Founder & Software Architect – removed\_email\_address@domain.invalid – [EngineYard.com](http://EngineYard.com) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 24, 2008, 1:12am UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/11 "2008-01-24T01:12:57Z")
On Jan 23, 2008 4:42 PM, Ezra Z. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > You are correct that nginx does not support kerberos so you will need > to keep apache. Haproxy is probably a good solution for you in this > case. +1 About Nginx: besides EngineYard and a lot of other Rails deployments, Nginx serves something like one-fourth of Russian internet traffic, IIRC. It’s not exactly no-name, just hasn’t been discovered by the rest of the world until recently. – Alexey V. CruiseControl.rb [[http://cruisecontrolrb.thoughtworks.com](http://cruisecontrolrb.thoughtworks.com)] RubyWorks [[http://rubyworks.thoughtworks.com](http://rubyworks.thoughtworks.com)] ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 24, 2008, 2:38am UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/12 "2008-01-24T02:38:54Z")
On Wed, 23 Jan 2008 23:36:34 +0100 Simon S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > On Wednesday 23 January 2008 23:03:21 Ezra Z. wrote: > > > Actually the nginx fair balancer has been released for a while. You > > can read up on it and get it here: > > > > [http://wiki.codemongers.com/NginxHttpUpstreamFairModule](http://wiki.codemongers.com/NginxHttpUpstreamFairModule) > > Hmmm. I can’t use a no-name web server with a load balancer that is a set of > patches for that server on our intranet production server. At the first > problem I would lose my job. Uh, but, Apache is quite literally “a patchy” version of NCSA httpd. 🙂 – Zed A. Shaw - Hate: [http://savingtheinternetwithhate.com/](http://savingtheinternetwithhate.com/) - Good: [http://www.zedshaw.com/](http://www.zedshaw.com/) - Evil: [http://yearofevil.com/](http://yearofevil.com/) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 24, 2008, 2:43am UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/13 "2008-01-24T02:43:00Z")
On Wed, 23 Jan 2008 14:45:43 -0800 Ezra Z. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > That’s too bad then, you will have to suffer on with apache i guess. > Nginx is a lot more solid then apache is in stability and performance. > I’m personally running over 1000 nginx daemons with absolutely no > problems to speak of with it. Yep, a _single_ nginx server on a 256M virtual machine with just 2 workers handled a simultaneous Redditing, TechCrunch, TechMeme, and Slashdotting without so much as a hitch in performance. That’s 170k uniq visitors in 3 days. To put this into perspective, I had people ask me to take links off my pages because the trail-off onto their pages was killing their server. Of course, I did do some special tuning, but that’s another story. – Zed A. Shaw - Hate: [http://savingtheinternetwithhate.com/](http://savingtheinternetwithhate.com/) - Good: [http://www.zedshaw.com/](http://www.zedshaw.com/) - Evil: [http://yearofevil.com/](http://yearofevil.com/) ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 24, 2008, 11:59am UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/14 "2008-01-24T11:59:05Z")
Ezra Z. wrote: > Hey Simon- > > I don’t have any whitepapers about our architecture currently > available. But the 50,000 ft view is like this: > > Http request from client → LVS redundant load balancers → Nginx → > mongrel\_cluster. > > The nginx and mongrel clusters run on the same Xen VM. with multiple > VM’s behind the LVS load balancers. Cool, good to know. This setup would be a bit overkill to me, but good to know how you set things up and how they work. > You are correct that nginx does not support kerberos so you will need > to keep apache. Haproxy is probably a good solution for you in this > case. All right. I’ve set up apache as reverse-/proxy serving haproxy balancing the mongrels. Just to help someone who would like to go down the same road, this is my apache site config: \ ServerName whatever DocumentRoot /home/myhome/MyHome/public/ ProxyPass /images ! ProxyPass /stylesheets ! ProxyPass /javascripts ! Alias /images /home/myhome/MyHome/public/images Alias /stylesheets /home/myhome/MyHome/public/stylesheets Alias /javascripts /home/myhome/MyHome/public/javascripts ProxyPass / [http://127.0.0.1:8000/](http://127.0.0.1:8000/) ProxyPassReverse / [http://127.0.0.1:8000](http://127.0.0.1:8000) ProxyPreserveHost on \ Options ExecCGI FollowSymLinks AllowOverride all Order allow,deny Allow from all and my haproxy.cfg file (took it from rubyworks): global web server maxconn 500 socket, therefore log 127.0.0.1 local0 user nobody group nogroup defaults log global mode http balance roundrobin maxconn 500 option httplog option abortonclose retries 3 redispatch clitimeout 120000 contimeout 120000 srvtimeout 120000 listen rails :8000 server rails-1 localhost:8001 maxconn 1 check server rails-2 localhost:8002 maxconn 1 check server rails-3 localhost:8003 maxconn 1 check to test I set the clitimeout, contimeout and srvtimeout to high numbers, so I can see how long actions/requests get handled. The “maxconn 1” parameter does exactly what it says (one request per mongrel). I have to test this setup a bit more, and see what haproxy does if a mongrel goes down or is disabled. Thank you all for your feedback. Simon ---
**Author:** ![simonsaid](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/b5e925/32.png) [@simonsaid](https://www.ruby-forum.com/u/simonsaid)\ **Post date:** [January 24, 2008, 2:14pm UTC](https://www.ruby-forum.com/t/again-workaround-found-for-request-queuing-vs-num-processors-accept-close/126966/15 "2008-01-24T14:14:08Z")
On 24 Jan 2008, at 01:46, Zed A. Shaw wrote: > workers handled a simultaneous Redditing, TechCrunch, TechMeme, and > Slashdotting without so much as a hitch in performance. That’s 170k > uniq visitors in 3 days. > > To put this into perspective, I had people ask me to take links off my > pages because the trail-off onto their pages was killing their server. Just a “me too” post - nginx is solid. We ran a site with a ~1Mb homepage weight (once all the graphics and video clips all over it had loaded - yeah, I know …) on nginx with 2 workers, and it took 570k PIs across 65k unique visitors in one day, and that mostly in a 5-hour period. OK, it was a dedicated dual Xeon with 4GB RAM and not much on it apart from CentOS base + nginx, but I kept a close eye and I didn’t see it go above 500Mb, and the process load was low enough to make a brave man weep. > Of course, I did do some special tuning, but that’s another story. A story you might be happy to share, perhaps? i – Igor C. // POKE // 10 Redchurch Street // E2 7DD // +44 (0)20 7749 5355 // [www.pokelondon.com](http://www.pokelondon.com)