Hi folks, First day my site goes live and it's down. Fun fun. I'm on dreamhost, and the problem seems to happen when I update the code and kill the fcgi processes to pick up the changes. It's happened twice today; first time it came back after an hour or so; this time not so far. here's the restart script: killall -USR1 dispatch.fcgi echo 'done' Here's the error in error.log: [Tue Jan 02 12:13:41 2007] [error] [client 18.104.22.168] FastCGI: comm with (dynamic) server "/home/wmnf_admin/wmnf.org/wmnf/public/dispatch.fcgi" aborted: (first read) idle timeout (60 sec) [Tue Jan 02 12:13:41 2007] [error] [client 22.214.171.124] FastCGI: incomplete headers (0 bytes) received from server "/home/wmnf_admin/wmnf.org/wmnf/public/dispatch.fcgi" Here's the last few lines of fastcgi.crash.log: [02/Jan/2007:10:58:38 :: 8165] starting [02/Jan/2007:11:55:26 :: 24649] starting [02/Jan/2007:12:03:11 :: 24649] asked to terminate ASAP [02/Jan/2007:12:03:11 :: 8165] asked to terminate ASAP [02/Jan/2007:12:03:11 :: 24649] terminated gracefully [02/Jan/2007:12:03:12 :: 8165] terminated gracefully So, it terminated but isn't restarting. I've seen recommendations elsewhere to clear out the rails session dir; I've done that with no change. Any ideas? So far no response from dreamhost support... thanks much
on 2007-01-02 22:55
on 2007-01-03 02:56
Matt Cowley wrote: > here's the restart script: > incomplete headers (0 bytes) received from server > [02/Jan/2007:12:03:12 :: 8165] terminated gracefully > > So, it terminated but isn't restarting. > > I've seen recommendations elsewhere to clear out the rails session dir; > I've done that with no change. > > Any ideas? So far no response from dreamhost support... I have the opposite problem - on the server Garvey, since Saturday, killing (-9) dispatch.fcgi processes is causing one new dispatch.fcgi process to start immediately (without waiting for an HTTP request to come in). This never happened before. For a couple of days the application wouldn't start under FastCGI, but would still run fine (but slowly) under CGI. Now it's working under FastCGI again. The only changes I have been making are the one-character changes in .htaccess to switch between FastCGI and CGI. The server has just been rebooted, but it's still resurrecting dispatch.fcgi. DreamHost support say that nothing has changed on the server, but this looks like a change in mod_fastcgi configuration to me. regards Justin Forder
on 2007-01-04 00:25
It doesn't seem like I'm getting any fcgi errors until I kill the processes. When I do, I get the errors (and the site is down) for about two hours, and then it restarts. Any ideas why this would happen? I've seen reports of dreamhost's over-zealous process-killers causing sites to return 500 errors, but I don't seem to be getting them in the normal operation; only when I try to refresh the fcgi and ruby so changes are picked up. My site's gone down 4 times in a day in a half...help would be most appreciated. Thanks
on 2007-01-04 00:43
On 1/4/07, Matt Cowley <firstname.lastname@example.org> wrote: > My site's gone down 4 times in a day in a half...help would be most > appreciated. My approach has been to killall -9 dispatch.fcgi and killall -9 ruby1.8. It seems to work.
on 2007-01-04 00:48
avocado wrote: > On 1/4/07, Matt Cowley <email@example.com> wrote: >> My site's gone down 4 times in a day in a half...help would be most >> appreciated. > > My approach has been to killall -9 dispatch.fcgi and killall -9 > ruby1.8. It seems to work. Hmm..I've been using killall -USR1 ruby killall -USR1 dispatch.fcgi I'll try your suggestion. Thanks much.
on 2007-01-04 06:49
Matt Cowley wrote: > killall -USR1 ruby > killall -USR1 dispatch.fcgi > > > I'll try your suggestion. Thanks much. > Use ps ax to see if they have gone away (or if they have resurrected themselves with different PIDs). I agree with avocado's advice, and it's consistent with this DreamHost wiki page: http://wiki.dreamhost.com/index.php/Ruby_on_Rails regards Justin Forder
on 2007-01-04 21:01
> > Use > > ps ax > > to see if they have gone away Thanks. avocado's script seems to be working better than what I had, so far. I also reduced the number of fcgi processes from 10 to 6, after getting a response from DH support that I was using too much memory (I had 10 fcgis running, each between 30 and 70mb). I've been monitoring the site today with top, and have noticed zombie fcgi processes popping up. So, I'd have 3 normal and 3 zombies. I'm looking into a script to kill those off every hour or so, which might help.
on 2007-01-05 03:01
I'm on dreamhost and when I began developing on it I had many problems with this exact nature. I had to manually kill the processes. Dreamhost gives you a free SVN service, so what I do now is just use Capistrano. It allows me to very easily deploy my applications using less time, giving me more time to develop. It also restarts the FCGI processes using the reaper script. I love it.
on 2007-09-25 22:44
Russel - Don't know if you're still looking at this thread, but here's what I found out. I think you can edit the last line in your dispatch.fcgi file to do what you're asking about. I believe that this line will limit you to 6 processes: RailsFCGIHandler.process! nil, 6 So, I'm guessing you can change it from 6 to any other number you'd like. Terry On 1/7/07, Russell Norris <firstname.lastname@example.org> wrote: > > > > anything further. > > > > Six still sounds a lot. > > > > regards > > > > Justin Forder > > > > > > > > -- Terry (TAD) Donaghe http://www.rubynoob.com http://www.tadspot.com
on 2007-09-25 22:47
Hmm. I thought I read that that line set something about the GC, I'll look into that. RSL
on 2007-09-25 22:47
Please do tell how you did this. I need nowhere near the six dispatchers that get started every time my piddlin' blog starts up. I wasn't aware we had any control over the number of dispatchers. Someone should blog this and get it on the internets if you can control them. I'll do it if no one else does [though no one would probably see it]. RSL
on 2007-09-25 22:50
Guys I have been experiencing similar frustrations for sometime and DreamHost confirmed that their procwatcher is killing my processes. The only way I have been able to get my application back up has been consistent with what I've read on these posts killall -9 ruby1.8 killall -9 dispatch.fcgi But I have been doing this manually, does anyone have a blog(like suggested) or some other systematic tips on tuning and or avoiding this problem? >From Dream Host.. "It looks like the problem is indeed your scripts being killed by our procwatcher: 2007-01-14 17:46:11 kill 5198 kreyze dispatch.fcgi "Ruby process puts user over the fcgi memory limit (170.19 used + 30.58 additional would be over 200.00 limit)" 2007-01-14 17:46:42 kill 10243 kreyze dispatch.fcgi "Ruby process puts user over the fcgi memory limit (170.19 used + 30.58 additional would be over 200.00 limit)"