Difficult process of restarting SCGI, Lighttpd, Apache, sess

I’ve managed to get Rails working using Apache->Lighttpd->SCGI.
Restarting everything is really
difficult. This appears to be the process:

  • stop apache
  • stop lighttpd
  • stop scgi
  • make sure scgi still isn’t running (it sometimes does) - if it is,
    kill it
  • delete all session files (they often cause permission errors)
  • start scgi
  • start apache
  • start lighttpd
  • cross fingers (since this process doesn’t always work and I have to
    repeat it)

When the above doesn’t work, I get ‘500’ errors on Rails’ pages, ‘Web
server may possibly be
configured wrong.’ in scgi’s log, nothing in Rails’ logs, and only proxy
connection errors in
apache’s logs if it can’t connect.

Is there an easier and less problematic way?

thanks
csn


Start your day with Yahoo! - Make it your home page!
http://www.yahoo.com/r/hs

I’ve never used the exact setup you have, but generally when you need
to perform the same tasks over and over it’s nice to write a single
script that performs all the commands. Then you can just call

./uber-restart-script

and everything is taken care of.

Pat

On Dec 1, 2005, at 12:09 PM, CSN wrote:

  • start scgi

Is there an easier and less problematic way?

thanks
csn

I don’t think you should need to restart apache or lighttpd. Since
scgi lets you rails app run on its own outside the webserver you
should just stop and restart your scgi processes. Thats one of the
advantages to scgi.

Cheers-

-Ezra Z.
Yakima Herald-Republic
WebMaster
http://yakimaherald.com
509-577-7732
[email protected]

— Ezra Z. [email protected] wrote:

  • make sure scgi still isn’t running (it sometimes does) - if it
    configured wrong.’ in scgi’s log, nothing in Rails’ logs, and only
    scgi lets you rails app run on its own outside the webserver you
    should just stop and restart your scgi processes. Thats one of the
    advantages to scgi.

That’s what I would have thought too, but after I do ‘scgi_ctrl restart’
I get ‘500 internal
error’ errors on my Rails’ pages. Restarting Lighttpd often fixes it,
but sometimes I have to go
through the complete aforementioned process.

csn


Rails mailing list
[email protected]
http://lists.rubyonrails.org/mailman/listinfo/rails


Yahoo! DSL ? Something to write home about.
Just $16.99/mo. or less.
dsl.yahoo.com

CSN wrote:

— Ezra Z. [email protected] wrote:

I don’t think you should need to restart apache or lighttpd. Since
scgi lets you rails app run on its own outside the webserver you
should just stop and restart your scgi processes. Thats one of the
advantages to scgi.

That’s what I would have thought too, but after I do ‘scgi_ctrl restart’ I get ‘500 internal
error’ errors on my Rails’ pages. Restarting Lighttpd often fixes it, but sometimes I have to go
through the complete aforementioned process.

Wow… maybe I won’t be looking into running SCGI, if that’s the case. I
run my
production apps under Lighttpd - FastCGI. There’s an Apache proxy in
front, but
I only need to restart it when I change the httpd.conf file.

For my Rails apps (0.14.3), all I need to do, to reload the code, is:

sudo ./script/process/reaper

And it just works. I suppose if SCGI gets to this stage, I might
consider
switching, but I’m not sure I really see the point. FastCGI under
Lighttpd is
fast and it works.

Incidentally, if a FastCGI guru out there knows how to run my FastCGI
listeners
as a user other than root, so I don’t need to sudo the reaper command,
that
would be a real help. Especially when I start hosting apps for multiple
users on
my server. Or is the trick to run multiple Lighttpd instances, each as a
separate user, since Lighttpd launches the FastCGI listeners…

-Brian

On 12/1/05, Brian V. Hughes [email protected] wrote:

switching, but I’m not sure I really see the point. FastCGI under Lighttpd is
fast and it works.

Incidentally, if a FastCGI guru out there knows how to run my FastCGI listeners
as a user other than root, so I don’t need to sudo the reaper command, that
would be a real help. Especially when I start hosting apps for multiple users on
my server. Or is the trick to run multiple Lighttpd instances, each as a
separate user, since Lighttpd launches the FastCGI listeners…

Or, can somone tell me how they update their production code on
TextDrive using lighttpd + fcgi?

I’m just killing the lighttpd server and restarting it, since I didn’t
know about this script/process/reaper thing.

[email protected] wrote:

  1. Start scgi, right away hit the page.

The cause? As mentioned before lighttpd will disable a processor (scgi or
fcgi) if it is down for a default time of 60 seconds. Since this means
that a quick restart of the scgi process could knock your app out for 60
seconds, this turns out to be an incredibly bad design choice.

I was wondering what all the ‘Immediate shutdown since nobody is
connected.’ messages meant in
scgi’s log.

Is this the correct syntax?

$HTTP[“host”] =~ “www.mydomain.com:8000$” {
server.document-root = “/var/www/html/rails/public/”
disable-time = 0
}

I still get ‘500’ errors after restarting scgi (Lighttpd was restarted
after changing its config),
but you’re right - it takes about a minute then pages load again.

thanks
csn


Yahoo! DSL ? Something to write home about.
Just $16.99/mo. or less.
dsl.yahoo.com

— Martin DeMello [email protected] wrote:

scgi.server = ( “dispatch.scgi” => ((
“host” => “127.0.0.1”,
“port” => 9999,
“check-local” => “disable”,
“disable-time” => 0
)) )

(someone please correct me if i’m wrong)

Ah… that appears to do the trick!

I’ve been running SCGI+Lighttpd+Apache in production for over a week
now. Aside from Lighty dying
a couple times (for unknown reasons - perhaps server load), the setup
has worked fine, and seems
to perform a lot better than my old setup (with no small thanks to
Rails’ caching, very likely).
Anybody know how to make Lighty automatically restart if it dies?

thanks
csn

martin


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

Try this:

  1. set the “disable-time” => 0 inside your SCGI lighttpd config (same
    place you put host and port).
  2. restart lighttpd.
  3. shutdown your scgi.
  4. Hit the web page.

Now, if you get a 500 error, that’s good. Then do this:

  1. Start scgi, right away hit the page.

You should get the page back immediately.

Do this about 10 times until you’re confident that yes, lighttpd has now
stopped putting a ridiculous 60 second death time on your scgi (and
fastcgi) processor.

Once you do this, you can restart any of the three servers
independently.
Extra geek points for making a special 500 page that tell people the
server is down temporarily.

The cause? As mentioned before lighttpd will disable a processor (scgi
or
fcgi) if it is down for a default time of 60 seconds. Since this means
that a quick restart of the scgi process could knock your app out for 60
seconds, this turns out to be an incredibly bad design choice.

Another alternative I’ve toyed with is “scgi flopping”. Haven’t
implemented it, but the idea is that you’d run a cluster like normal of
say 3 processors. The “flop” command would do the following:

  1. Take down processors 1 and 2 and wait for them to exit. Lighttpd
    sees
    that those are down, so starts redirecting all traffic to #3.
  2. Lighttpd disables them for 60 seconds like normal, so you now have
    about a minute to get them back up. The flop command would restart #1,
    and #2.
  3. Once #1 and #2 were up, it would shutdown #3. Lighttpd disables #3
    for 60 seconds and starts sending requests to #1 and #2. Tricky part
    here
    is figuring out when lighttpd has enabled #1 and #2. Probably look at
    the
    log files.
  4. Finally, flop would then restart #3 with the new code and you’d be
    back in business.

Again, this is ultra freaking complicated, but would probably give you a
very very graceful restart to a new code base.

Anyway, try the trick above.

Zed A. Shaw

On Thu, 01 Dec 2005 17:08:45 -0500
“Brian V. Hughes” [email protected] wrote:

CSN wrote:

Wow… maybe I won’t be looking into running SCGI, if that’s the
case. I run my production apps under Lighttpd - FastCGI. There’s an
Apache proxy in front, but I only need to restart it when I change
the httpd.conf file.

The problem is lighttpd not SCGI. Read my message to this thread
explaining what to do.

Zed

On 12/4/05, CSN [email protected] wrote:

Is this the correct syntax?

$HTTP[“host”] =~ “www.mydomain.com:8000$” {
server.document-root = “/var/www/html/rails/public/”
disable-time = 0
}

I believe it should be

scgi.server = ( “dispatch.scgi” => ((
“host” => “127.0.0.1”,
“port” => 9999,
“check-local” => “disable”,
“disable-time” => 0
)) )

(someone please correct me if i’m wrong)

martin

On Dec 7, 2005, at 3:42 PM, Joe Van D. wrote:

error’ errors on my Rails’ pages. Restarting Lighttpd often fixes
For my Rails apps (0.14.3), all I need to do, to reload the code, is:
FastCGI listeners

I’m just killing the lighttpd server and restarting it, since I didn’t
know about this script/process/reaper thing.

#!/usr/local/bin/ruby

pstab = ps axww

def kill_proc(line)
its = line.strip.split(/\s+/)
pid = its[0]
puts “KILLING #{line}”
kill -9 #{pid}
sleep(1)
end

pstab.scan(/lighttpd -f/) do |lighty|
kill_proc lighty
end

pstab.scan(/^.dispatch.fcgi\s$/) do |line|
kill_proc line
end

if pstab =~ /dispatch.fcgi/
puts “Error, zombie dispatch.fcgi’s still lurking.”
else
puts “Rogue dispatch.fcgi’s cleared, starting lighty!”
/usr/local/sbin/lighttpd -f /home/USERNAME/lighttpd/lighttpd.conf
end

That will kill lighty, then kill the fcgi's. It will then check to

make sure the fcgi’s are dead and if they are it will restart lighty
otherwise it will warn you of zombie fcgi’s and you should run it
again. This seems to work good on txtdrv. You will need to edit it
accordingly if you have more than one lighty instance running. Also
script/reaper works on textdrive and is another way of restarting a
production rails app.

Cheers-

-Ezra Z.
Yakima Herald-Republic
WebMaster
http://yakimaherald.com
509-577-7732
[email protected]

On 12/7/05, Ezra Z. [email protected] wrote:

Also
script/reaper works on textdrive and is another way of restarting a
production rails app.

Can someone post their Switchtower task that uses the reaper script to
restart Rails after a code update? I can’t seem to get it to work (I
have to rewrite it as I don’t have sudo on TxD), and just running
script/process/reaper doesn’t seem to do the trick.