Frustrated with RoR environment splintering


#1

Why isnt the RoR community focusing on robust and scalable mod_perl
style of
Apache environment, rather then splintering all over the place with
lighthttp, mongrel, WEBrick, SCGI, fcgi, etc???

Its frustrating as someone who is trying to migrate to RoR.

Its very unlikely I would ever get a contract to work on any webserver
except Apache, so it would seem logicaly to focus all our efforts to
make
Apache the ‘production environment of record’ for RoR.

Just my $.02…


#2

Yeah, choice sucks, huh? If only I had no freedom …

-Derrick


#3

I love the fact of having WEBrick installed and working w/rails. It
makes development work a breeze, not processes tying things up when
I’m not developing. I thought it was a great idea. I’ve never worked
w/SCGI or Mongrel. I do know, obviously, we need FCGI. LigHTTPD seems
to be a great server to deploy with. Not everyone loves Apache.

I also do not really see it as splintering. Why splinter and support
MySQL, PostgreSQL, Oracle, MSSQL? I only use MySQL, we don’t need
the others, focus on better MySQL support, forget the others :wink:

Jeremy


#4

lol … i just noticed you called mod_perl robust. Can anyone say mp2?


#5

Derrick S. wrote:

Yeah, choice sucks, huh? If only I had no freedom …

Actually, yes, too much choice does suck. Makes people nervous that
they’re not picking the ‘right’ one. Especially because those facing
the choice for the first time are almost by definition those most
ill-equipped to make the decision.

http://journal.dedasys.com/articles/2006/02/18/maximizers-satisficers-and-programming-languages

Although I don’t think that in this case, things are that bad… it
doesn’t take long to figure out what’s what.


David N. Welton

Linux, Open Source Consulting


#6

news.gmane,org wrote:

Just my $.02…
My impression is that FastCGI is shaping up to be that environment of
record, and it works just fine with Apache, right?


#7

On Fri, Mar 31, 2006 at 10:20:15AM -0800, news.gmane,org wrote:
} Why isnt the RoR community focusing on robust and scalable mod_perl
style of
} Apache environment, rather then splintering all over the place with
} lighthttp, mongrel, WEBrick, SCGI, fcgi, etc???

Scgi, fcgi, fcgid, and mod_ruby are not Rails-specific. Work has been
done
to allow Rails to work with any of them (except mod_ruby) because their
suitability depends on a variety of things, including the platform.
WEBrick
also predates RoR, I believe, and it is only intended for development
(for
which it is well-suited). Lighttpd exists to provide good performance
while
being more loosely integrated with (or not requiring at all) Apache than
the various CGI methods. Mongrel is the new kid on the block, and may
wind
up replacing WEBrick and/or lighthttpd.

} Its frustrating as someone who is trying to migrate to RoR.

Do your development under WEBrick. The appropriate deployment option
will
be obvious based on your particular system’s constraints.

} Its very unlikely I would ever get a contract to work on any webserver
} except Apache, so it would seem logicaly to focus all our efforts to
make
} Apache the ‘production environment of record’ for RoR.

You should certainly become knowledgeable about Apache integration,
then.
Personally, I have been happy with mod_fcgid under Apache2. Then again,
I
don’t do real deployment.

} Just my $.02…

I was pretty frustrated when I was trying to figure out deployument for
my
first app. I couldn’t figure out what I should use, nor could I
determine
how to do what I wanted to do. Eventually I did find the docs I needed,
and
all is well. Once you figure it out, you can do it again. The problem is
that there are so many choices, but those choices exist because someone
out
there needed each one of them.

It would be nice to find a single document that covered all possible
deployment options and how to go about them. AFAIK, though, it doesn’t
exist. Feel free to write one. You probably won’t have enough knowledge
to
fill in all the options, but you can fill in what you know and leave the
skeleton open for submissions. Here’s an outline you could start from:

I) Introduction
II) Choices you must make

  • What platform (Windows, *nix)?
  • Virtual host or path?
  • Integrate with Apache?
    • Apache1 or Apache2?
      ? integrate with IIS?
      III) Apache integration options
      A) mod_proxy
    1. rewrite rules
    2. lighttpd
    3. mongrel
      B) mod_scgi (I think)
      C) mod_fcgi
      D) mod_fcgid
      IV) Standalone options
      A) WEBrick
      B) lighttpd
      C) mongrel
      ???) IIS integration options???
      ???

–Greg


#8

WEBrick is really the only thing that is part of rails.

The rest of the things you mention do not ship with Rails. They are
external programs that do things like serve web requests.

Your remark is kind of like asking why Rails runs on Linux, BSD, OS/X,
and Windows. Or Why does Rails work with Oracle, Postgres, mySQL, and
other databases.

Should Rails developers concentrate on one OS or on one DB? Why should
they concentrate on one webserver (other than WEBrick)?

It works with them because they exist and people use them in their
environments.

That’s open source for you. A developer tries Rails and says ‘Gee that’s
nice, but I wish it worked with ADAbase…’ Next thing you know the
developer writes the code and Rails works with ADAbase.

Now If you are learning rails, just stick with one stack at first. use
the DB you are familiar with, and the web server you are familiar with.
Then, if you ever become disatisfied, either write code to make your
stack better, or try an alternate!

news.gmane,org wrote:

Why isnt the RoR community focusing on robust and scalable mod_perl
style of
Apache environment, rather then splintering all over the place with
lighthttp, mongrel, WEBrick, SCGI, fcgi, etc???

Its frustrating as someone who is trying to migrate to RoR.

Its very unlikely I would ever get a contract to work on any webserver
except Apache, so it would seem logicaly to focus all our efforts to
make
Apache the ‘production environment of record’ for RoR.

Just my $.02…


#9

However, to scale a mod_perl app, it’s generally
understood that you need a static+proxy front-end
passing dynamic requests back to mod_perl backends
because the mod_perl instances are so large that
you cannot handle a reasonable number of client
HTTP connections if you’re using mod_perl alone.

I have served several 10’s of millions of pageviews daily for 3 years,
currently averaging 70 Mbps network, on 4 generic dual-xeon linux
servers running mod_perl exclusively.

My average delay budget is .3 seconds per page.
I have never used this proxy method and never had a scaling problem with
mod_perl.

However, turn on Apache::Reload (which essentially converts your
mod_perl to
CGI),
and it will throttle my NetApp 720’s.

When it comes to scaling websites, practice trumps theory every time.


#10

Yeah, choice sucks, huh? If only I had no freedom …

-Derrick

Well, this was the expected argumentative response.
Why so predictable?

Variety is of course great, but for those of us who arent tinkerers, but
production engineers responsible to existing large-scale sites, existing
system administrative infrastructures, and and websites that god-forbid
require other webserver features besides RoR, we need the community to
put
its stamp of approval on a well-maintainted, industrial-strength Apache
solution for RoR with persistant interpreter and database connection
support.

The problem with the community not having this thrust is that as
engineers
we dont know in which direction to move - we dont want to end up on a
platform like FastCGI where the maintainer dissapears for 2 years. Where
will fcgi or SCGI be in 2 years?

Here’s an example - i tried to install fcgi in Apache 2.2, only to find
out
it doesnt build - turns out that someone submitted the patch to the
maintainer MONTHS ago, and it still hasnt been rolled into the fcgi
code,
nor is it mentioned on the fcgi site.

fcgi - better then fastcgi, maybe- but it looks like we cant expect it
to be
maintained much better.

With mod_perl, it was clear the community supported it as the Apache
solution of record for Perl, so after a year or two production engineers
felt safe moving in that direction. But with Ruby On Rails, the
long-term
supported solution we can invest in is still completely unclear.

And yes, there are serious reprecussions for those of us with 100,000
plus
lines of code to be invested in a commercial project that the platform
we
select does not fall out of favor in 18 months.


#11

From what I understand FastCGI has not been maintained for 2 years and that
FCGI is perferred.

“Ross” removed_email_address@domain.invalid wrote in message
news:e0k61h$ag4$removed_email_address@domain.invalid…


#12

On Mar 31, 2006, at 10:20 AM, news.gmane,org wrote:

Why isnt the RoR community focusing on robust and scalable mod_perl
style of
Apache environment, rather then splintering all over the place with
lighthttp, mongrel, WEBrick, SCGI, fcgi, etc???

mod_perl isn’t particularly scalable, in and of itself.

Now, don’t get me wrong, I love mod_perl. It gives
great performance on dynamic requests and allows Apache
mods to be written in Perl.

However, to scale a mod_perl app, it’s generally
understood that you need a static+proxy front-end
passing dynamic requests back to mod_perl backends
because the mod_perl instances are so large that
you cannot handle a reasonable number of client
HTTP connections if you’re using mod_perl alone.

So, this new community is feeling out the options.

Everything about Rails is new to most peopleat least
to some degree. MVC, ORM, Ruby, etc. I’m delighted
to have learned about Lighttpd, and squeal with
delight at the thought of having zero configuration
backend deployments with mongrel.

Eventually a consensus will emerge. Be patient.


– Tom M.


#13

Seth B. wrote:

of Apache environment, rather then splintering all over the place with

My impression is that FastCGI is shaping up to be that environment of
record, and it works just fine with Apache, right?

I meant the protocol, not a particular implementation.

But yeah, I think you’re right about which is the preffered
implementation for apache.


#14

Seth B. wrote:

The problem with the community not having this thrust is that as
engineers
we dont know in which direction to move - we dont want to end up on a
platform like FastCGI where the maintainer dissapears for 2 years. Where
will fcgi or SCGI be in 2 years?

You’re right, but… FastCGI is not Rails. fcgi and SCGI are not Rails.
The advantage to having all this flexibility is that if the maintainer
of one of those disappears for two years, you still have Rails and a
variety of other platforms to run it on.

If Rails settled on FCGI and FCGI’s maintainers all gave up, where would
we be then?


#15

On Mar 31, 2006, at 2:24 PM, Seth B. wrote:

However, to scale a mod_perl app, it’s generally
understood that you need a static+proxy front-end
passing dynamic requests back to mod_perl backends
because the mod_perl instances are so large that
you cannot handle a reasonable number of client
HTTP connections if you’re using mod_perl alone.

I have served several 10’s of millions of pageviews daily for 3 years,
currently averaging 70 Mbps network, on 4 generic dual-xeon linux
servers running mod_perl exclusively.

Seems a shame to use a very large (relatively) mod_perl instance to feed
a slow client connection. Particularly so if the request is entirely
static and requires no mod_perl usage.

Doubly so if you’re tying up a mod_perl process doing SSL!

Hey, don’t take my word for it, go directly to the source!

http://tinyurl.com/go8bp

My average delay budget is .3 seconds per page. I have never used this
proxy method and never had a scaling problem with mod_perl.

I’d humbly suggest that you’d get far better performance (with existing
hardware!) if you went to a two phased setup.

However, turn on Apache::Reload (which essentially converts your
mod_perl to
CGI), and it will throttle my NetApp 720’s.

When it comes to scaling websites, practice trumps theory every time.

Yes, I agree. That’s why I’m speaking from experience the same as you
are.


– Tom M.


#16

On 3/31/06, Seth B. removed_email_address@domain.invalid wrote:

The problem with the community not having this thrust is that as engineers
we dont know in which direction to move - we dont want to end up on a
platform like FastCGI where the maintainer dissapears for 2 years. Where
will fcgi or SCGI be in 2 years?

The questions you’ve asked are akin to, “Apache might not be
maintained in two years, so why should I use linux?” In short - it
doesn’t make a lot of sense. As others have pointed out,
FastCGI/FCGI/SCGI are not Rails.

Rails+lighttpd+fcgi has proven itself in production for quite a while.
I don’t know about SCGI, as I haven’t used it yet. So give it a
shot. If FCGI falls off the map somewhere down the road, it’s
probably due to a better replacement, and if not, there’s enough Rails
use now that we will come up with some way of making it work.

It doesn’t make sense to avoid Rails because of uncertainty with FCGI,
just as it makes no sense to avoid Linux due to uncertainty with
Apache.

Pat


#17

You’re right, but… FastCGI is not Rails. fcgi and SCGI are not Rails.
The advantage to having all this flexibility is that if the maintainer
of one of those disappears for two years, you still have Rails and a
variety of other platforms to run it on.

If Rails settled on FCGI and FCGI’s maintainers all gave up, where would
we be then?

Thats exactly my point - when I chose mod_perl it was not risky because
both
the
Apache Foundation and the Perl community were strongly behind it. It
wasnt
‘going away’.

That isnt clear with any of the current Apache implementation
persistance
solutions for RoR.

I cant put a web project in production which might need to switch either
webservers(!!) or
be sent back to CGI tommorrow because its Apache persistance solution
‘fell
out of favor’.

This should not be an argumentative concept to anyone with large-scale
production web applications.


#18

Seems a shame to use a very large (relatively) mod_perl instance to feed
a slow client connection. Particularly so if the request is entirely
static and requires no mod_perl usage.

Doubly so if you’re tying up a mod_perl process doing SSL!

Hey, don’t take my word for it, go directly to the source!

Why dont you go to Slashdot?
Slashdot runs dynamic content all day long over mod_perl, and its
links
crash production sites daily.


#19

Seems a shame to use a very large (relatively) mod_perl instance to feed
a slow client connection. Particularly so if the request is entirely
static and requires no mod_perl usage.

LOL Tom, if your website serves 70 Mbps of dynamic content over 4 linux
servers and
RADs easily and is rock stable and serves dynamic pages in .3s for 3
years, there is no ‘shame’ to be had.

This is what is called ‘a successful scalable solution’.

When I worked for Yahoo!, we were ‘real efficient’ - we developed most
dynamic content in Apache C modules!
Oh the memory we saved!!!

Mondays were especially fun. You spent 4.5 hours compiling the
production
codebase (whether YOUR site was affected by others changes or not),
checking
every so often for the inevitable error, checking CVS for the module
author,
walking to his cube (hoping to god he was there), getting an ‘oh yeah
you
gotta blahblahblsa…’, implementing blahblahblah, resume compile, and
then
RESTARTING all your webservers on 4 continents (because you cant update
C
modules without a restart!) Then go home and hope to get real work done
on
monday.

Oh but the memory we saved!!

Guess what? Yahoo! uses PHP now :wink:


#20

On Fri, Mar 31, 2006 at 03:27:06PM -0800, Seth B. wrote:

I cant put a web project in production which might need to switch either
webservers(!!) or be sent back to CGI tommorrow because its Apache
persistance solution ‘fell out of favor’.

Software doesn’t rot. If you choose a solution that works for you, then
it
will continue to work for you. Security issues you can patch, and when
you
do need to migrate to get some new feature, you’d have to do a whole
pile of
work to migrate no matter what you were switching between – sensible
upgrade paths are the exception, not the rule.

  • Matt