# Long-running pages cause mongrel time-outs **URL:** \ **Category:** Mongrel\ **Created:** [December 18, 2007, 1:10am UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466 "2007-12-18T01:10:57Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 18, 2007, 1:10am UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/1 "2007-12-18T01:10:57Z")
Hi all, We use mongrel behind apache (using mod\_proxy\_balancer) to run an internal app for our business. Lately we’ve been seeing frequent errors from some of our mongrel processes. [Sun Dec 16 08:48:47 2007] [error] [client] (70007)The timeout specified has expired: proxy: error reading status line from remote server 127.0.0.1, referer: http:// [Sun Dec 16 08:48:47 2007] [error] [client] proxy: Error reading from remote server returned by /, referer: http:// [Sun Dec 16 08:49:08 2007] [error] [client] (70007)The timeout specified has expired: proxy: error reading status line from remote server 127.0.0.1 [Sun Dec 16 08:49:08 2007] [error] [client] proxy: Error reading from remote server returned by / The problem seems to be that the long-running request hogs a mongrel process, and when apache sends the next request to it (after a round-robin tour through the other 9 mongrels), that request times out too. The original action eventually comes back, but sometimes not for several minutes, so we can timeout on many requests in a row. I’ve seen problems like this mentioned in the context of file uploads (which explain some of our long-running requests), but I’ve never seen a fix posted. Maybe I haven’t looked hard enough? Does anyone know of a config fix that will skip over these blocked processes? Thanks! Mike ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 18, 2007, 1:19am UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/2 "2007-12-18T01:19:08Z")
On Dec 17, 2007, at 4:10 PM, Michael Van Wie wrote: > reading from remote server returned by /, > too. The original action eventually comes back, but sometimes not The best thing to do is to move any of your long running tasks off into a work queue and out of the mongrel/rails request response cycle. Something like Bj woudl help: [http://codeforpeople.rubyforge.org/svn/bj/trunk/README](http://codeforpeople.rubyforge.org/svn/bj/trunk/README) But I think there has been apache mod\_proxy\_balancer configs posted to this list in the past few months that will only send one request to a mongrel at a time, search the archives. or if you can use nginx instead there is a new fair proxy balancer for nginx that works well. [http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel](http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel) Cheers- - Ezra Z. – Founder & Software Architect – removed\_email\_address@domain.invalid – [EngineYard.com](http://EngineYard.com) ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 18, 2007, 7:35pm UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/3 "2007-12-18T19:35:40Z")
Hi Ezra – I found it interesting that you recommended Bj (backgroundjobs) and not BackgrounDrb. I hadn’t heard of Bj before, and I’ll check it out. Thanks for the link. Given that you initiated the development BackgrounDrb, I’m curious when and where you would recommend Bj over BackgrounDrb. Mason ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 18, 2007, 9:36pm UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/4 "2007-12-18T21:36:36Z")
There’s a library called Spawn, too, which is nice for doing asynchronous but still request-triggered background processing. It might get less nice once 1.9 comes around, though, but… bridges when we come to them. Evan ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 18, 2007, 11:41pm UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/5 "2007-12-18T23:41:19Z")
Evan W. wrote: > There’s a library called Spawn, too, which is nice for doing > asynchronous but still request-triggered background processing. It > might get less nice once 1.9 comes around, though, but… bridges when > we come to them. My Google-fu is eluding me. Can someone point me to somewhere I can learn more about Spawn? ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 18, 2007, 11:50pm UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/6 "2007-12-18T23:50:05Z")
[http://rubyforge.org/projects/spawn/](http://rubyforge.org/projects/spawn/) ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 19, 2007, 12:18am UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/7 "2007-12-19T00:18:47Z")
On Dec 18, 2007, at 1:38 PM, Jacob A. wrote: > – > Cheers, > > - Jacob A. [http://rubyforge.org/projects/spawn/](http://rubyforge.org/projects/spawn/) - Jason ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 20, 2007, 1:07pm UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/8 "2007-12-20T13:07:43Z")
On Dec 19, 2007 10:31 PM, Steve M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > Hi Ezra – > > to address this issue. > Thats just one side of the story. BackgrounDRb no longer depends > slave.rb. Its stable as hell. There is not a single bug in > BackgrounDRb that has been reported and not solved. You can checkout > activity on BackgrounDRb mailing list. BackgrounDRb is not Bj however, It can be if you want to use it like that. Please check your facts before making any such claims. > * * * > > Mongrel-users mailing list > removed\_email\_address@domain.invalid > [http://rubyforge.org/mailman/listinfo/mongrel-users](http://rubyforge.org/mailman/listinfo/mongrel-users) – Let them talk of their oriental summer climes of everlasting conservatories; give me the privilege of making my own summer with my own coals. [http://gnufied.org](http://gnufied.org) ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 20, 2007, 11:06am UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/9 "2007-12-20T11:06:13Z")
At 02:42 AM 12/19/2007, removed\_email\_address@domain.invalid wrote: > I found it interesting that you recommended Bj (backgroundjobs) and > not > BackgrounDrb. I hadn’t heard of Bj before, and I’ll check it out. > Thanks for > the link. Given that you initiated the development BackgrounDrb, I’m > curious > when and where you would recommend Bj over BackgrounDrb. > > Mason Hi Mason, I found this link which clearly illuminates your question. [http://groups.google.com/group/ruby-talk-google/browse\_frm/thread/144e9b3644ffe88e#38b99ec30098b956](http://groups.google.com/group/ruby-talk-google/browse_frm/thread/144e9b3644ffe88e#38b99ec30098b956) In short: BackgroundRb relies on slave.rb (maintained by Ara T Howard, the author of Bj), and Ara claims only one person (Eric H.) understands how to use slave.rb properly: so BackgroundRb is just too complex to maintain (too much flexibility). Bj is specifically designed to address this issue. Erza’s company EngineYard is one of the sponsors of Bj - so it’s designed to meet some business needs of theirs (I’m a customer so that includes me!). From my analysis, Bj looks good b/c it puts the jobs into a database table so things are both persistent and scalable with very little additional infrastructure. Steve ---
**Author:** ![Michael\_Van\_Wie](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/779978/32.png) [@Michael\_Van\_Wie](https://www.ruby-forum.com/u/Michael_Van_Wie)\ **Post date:** [December 20, 2007, 3:11pm UTC](https://www.ruby-forum.com/t/long-running-pages-cause-mongrel-time-outs/123466/10 "2007-12-20T15:11:19Z")
On Dec 19, 2007, at 9:01 AM, Steve M. wrote: > > Hi Ezra – > > > > I found it interesting that you recommended Bj (backgroundjobs) and > > not > > BackgrounDrb. I hadn’t heard of Bj before, and I’ll check it out. > > Thanks for > > the link. Given that you initiated the development BackgrounDrb, I’m > > curious > > when and where you would recommend Bj over BackgrounDrb. > > > > Mason Backgroundrb and Bj serve different purposes. The new backgroundrb is a complete rewrite by Hemant that throws out drb and uses an event driven styel of programming. It really is very stable now and it’s main focus is on doing jobs that require reporting status in your rails app. Like if you want to fetch some news feeds and report a progress bar via ajax polling in your app. Bdrb works very well for this and other more real time background jobs. Bj is a work queue. You just shove jobs into the queue which is a database table and the Bj daemon will run through them and call each job one at a time. But you can also have Bj daemons on each node in a cluster of servers and have them all pulling from the same queue. Bj jobs are just shell commands. So your actual jobs can be ruby scripts, shell scripts, or script/runner calls. The main idea behind Bj is that you could throw 100K jobs at it and it will just do them as fast as it can one at a time without overwhelming the system all at once. So both of these tools are very solid now, they just serve different purposes. Backgroiundrb is more for real time jobs and status reports as well as having a very nice cron like scheduler for jobs. Bj is more of a work queue that is persistent. Hope that clears things up, Both of these tools are great for different circumstances. Cheers- - Ezra Z. – Founder & Software Architect – removed\_email\_address@domain.invalid – [EngineYard.com](http://EngineYard.com)