Forum: Ruby on Rails deployment server

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Harm de Laat (Guest)
on 2006-02-17 17:44
(Received via mailing list)
Hi all,

On theserverside.com there is an article describing an interview with
Bruce
Tate, where he calls Java dead like Cobol.
See: http://www.theserverside.com/news/thread.tss?thread_id=39066


However in the comments, some guy comes up with a good point:

He states the following:

<quote>
I couldn't come up with a viable enterprise quality/grade deployment
platform for RoR. Even in the RoR book, they describe multiple
deployment
scenerios, lighttpd with FastCGI, apache with mod_ruby or fastCGI where
the
only two recommended for production. The first is said to be unstable
sometimes (these words are directly from the book), and the second is no
longer actively supported/developed. So how comfortable would a company
feel
in deploying their app in an unstable and/or unsupported environment. So
until Ruby has a viable platform, commercial and/or open source, which
has a
lot of traction, money behind it, etc..., it's a huge risk for any
corporation to take on.
</quote>

How do you feel about the RoR deployment platform. Is lighttpd with
fastcgi
realy this unstable?
Will there be commercial vendors for RoR deployment environments. And
how
can we help to get things more stable?

Regards,

Harm de Laat
James L. (Guest)
on 2006-02-17 17:53
(Received via mailing list)
On 2/17/06, Harm de Laat <removed_email_address@domain.invalid> wrote:
> How do you feel about the RoR deployment platform. Is lighttpd with fastcgi
> realy this unstable?
> Will there be commercial vendors for RoR deployment environments. And how
> can we help to get things more stable?

Commercial or not, it's pretty cool to be able to drop a .war file
into the webapps directory in Tomcat (or JBoss for that matter) and
just have it work.  Drop in foo.war?  Now you can access it at
http://host.domain/foo/*.  No configuration required.

I'm looking for every excuse possible to dump Java in favor of Ruby,
but good production app servers is still a big reason to stay with
Java.

-- James
Ezra Z. (Guest)
on 2006-02-17 18:08
(Received via mailing list)
On Feb 17, 2006, at 7:42 AM, Harm de Laat wrote:

>
> which has a lot of traction, money behind it, etc..., it's a huge
> Harm de Laat
Lighttpd has come a long way since the book was released. I have
been using it to power a fairly large production site for the
newspaper I work at. It has been pumping out 50-80,000+ page views/
day since last august. It runs on lighttpd/fcgi and has been rock
stable for me.

	I am still working on my Rails Deployment book for the pragprogs.
Look for a beta pdf announcement sometime late march early april ;)

Cheers-
-Ezra Z.
WebMaster
Yakima Herald-Republic Newspaper
removed_email_address@domain.invalid
509-577-7732
Brian V. Hughes (Guest)
on 2006-02-17 18:14
(Received via mailing list)
Harm de Laat wrote:
> How do you feel about the RoR deployment platform. Is lighttpd with fastcgi
> realy this unstable?

Not at all. In fact, I'd say it's very stable in my experience. I've got
a
single G5 Xserve currently running 5 lighttpd instances, each listening
on
separate ports, controlling their own set of FastCGI listeners (2-4 for
each
instance) for a production Rails app. Apache, running on ports 80 and
443, acts
as the front-end for all the lighttpd instances. All of the apps run
quite well
and now that I'm getting Switchtower set up, continued deployment is
getting
even easier.

> Will there be commercial vendors for RoR deployment environments. And how
> can we help to get things more stable?

What things need to be made more stable? Obviously, there's room to make
things
easier, but I'm not sure just anyone should be deploying
production-level web
applications without some level of Webmaster experience. Just because
you know
how to build a great web application doesn't mean you know anything
about
hosting that application in a production environment.

Oh... I suppose that means there's room for a paid service to deploy and
manage
production Rails apps, so it wouldn't surprise me to see service
providers start
to emerge.

-Brian H., Webmaster
Dartmouth College
Ben M. (Guest)
on 2006-02-17 18:32
(Received via mailing list)
James L. wrote:
> just have it work.  Drop in foo.war?  Now you can access it at
> http://host.domain/foo/*.  No configuration required.
>
> I'm looking for every excuse possible to dump Java in favor of Ruby,
> but good production app servers is still a big reason to stay with
> Java.
>

YES YES YES YES! Rails needs a container! Rails needs a container! Go
Zed, go! We're
counting on you Zed!! :-D

b
Kent S. (Guest)
on 2006-02-17 18:38
(Received via mailing list)
No configuration is needed only in very simple cases. As soon as you
war application needs access to connection pools, jms queues/topic,
etc, there is as much configuration headaches as anywhere else.

Kent
---
http://www.datanoise.com
James L. (Guest)
on 2006-02-17 18:51
(Received via mailing list)
On 2/17/06, Kent S. <removed_email_address@domain.invalid> wrote:
> No configuration is needed only in very simple cases. As soon as you
> war application needs access to connection pools, jms queues/topic,
> etc, there is as much configuration headaches as anywhere else.

Yes and no.  You define things like datasources once for the entire
app server.  Then you can use them from each application with a simple
configuration.  I'm not trying to ignite a war here, but Java really
does get some things right.
Kent S. (Guest)
on 2006-02-17 19:30
(Received via mailing list)
What I meant is that if you start from scratch building a new j2ee
deployment, it will take as much of your time as deployment of a
similar rails solution, if not more. When the installation procedure
is complete and all configuration is in place, yes, you can just drop
your war  or ear file and you're done. On the other hand, if you use
switchtower (which I highly recommend), deployment of new version of
rails application is even easier and more flexible since you can
augment your deployment procedure with your custom logic.

I don't want a flame war either. With a nice process in place, rails
deployment can be as easy.

Kent
---
http://www.datanoise.com
Dylan S. (Guest)
on 2006-02-17 19:33
(Received via mailing list)
I would "love" a one drop solution to deployment... as at the current
moment, I have gems all over the place, and have no straight-forward way
of
keeping my dependencies in track.
Since not all gems are created equal, even using Geoff's
freeze_other_gems
rake task doesn't always work, and I have to do a lot of hand pushing of
files over to /vendor.
Kent S. (Guest)
on 2006-02-17 19:39
(Received via mailing list)
I just keep all dependent libraries in my vender directory and they
are all in my repository. So I don't have to think about it every time
I deploy a new version.

Kent
--
http://www.datanoise.com
Ben M. (Guest)
on 2006-02-17 19:48
(Received via mailing list)
It's simply not the same. Why does every conversation about deployment
options for rails
get derailed (har har) into talk of this is more complicated or that is
more complicated?

There is simply nothing in the rails world that works like a servlet
container, much less
an application server.

This is a fatal, deal-breaking, they're-never-gonna-take-you-seriously
problem.

I'm happy to hear about the deployment successes of Ezra, DHH, and
others. But patching
together a bunch of FastCGI processes is not the same as having a highly
flexible,
long-running container process.

I'm hoping Mongrel will change that. But if you really love rails and
want to champion
it's cause in the enterprise world, I can't possibly overstate the
importance of a
web/application container.

b
Rick B. (Guest)
on 2006-02-17 20:04
(Received via mailing list)
* Ben M. (removed_email_address@domain.invalid) [060217 12:51]:
> There is simply nothing in the rails world that works like a servlet
> container, much less an application server.
>
> This is a fatal, deal-breaking, they're-never-gonna-take-you-seriously
> problem.

I know I'm dense (I wake up with it every morning and have grown
accustomed to it after all these years), but I'm not sure I understand
why a servlet container or an application server is necessary to build
a large-scale production (/"enterprise"?) system.

Perhaps the misunderstandings on this topic are a product of saying
"Rails doesn't have the solution we use in Java therefore it can't be
solving the problem(s)" without actually going back to "the problem(s)"
to see if there are other ways to solve them and perhaps Rails might be
using other techniques.  Just a thought.

It occurs to me because over the years I have seen lots of heavy-load
production systems with strong availability guarantees which aren't
written in Java.  Outside of the Java world I have seen very few
architectures that map onto the servlet model, and not so much the
normal Java "application server" model (though there's a lot of room for
hair splitting).

It could be that Share Nothing, which wasn't invented with Rails, is
(done properly) a suitable replacement for whatever name one would apply
to the application server paradigm that people "take [...] seriously".

Rick
--
 http://www.rickbradley.com    MUPRN: 336
                       |  States. In a
   random email haiku  |  46-page decision that became
                       |  public Friday, U.S.
Kent S. (Guest)
on 2006-02-17 20:16
(Received via mailing list)
On 2/17/06, Ben M. <removed_email_address@domain.invalid> wrote:
> I'm happy to hear about the deployment successes of Ezra, DHH, and others. But patching
> together a bunch of FastCGI processes is not the same as having a highly flexible,
> long-running container process.
>
> I'm hoping Mongrel will change that. But if you really love rails and want to champion
> it's cause in the enterprise world, I can't possibly overstate the importance of a
> web/application container.
>
> b

Well, I guess I have a different background. I much better prefer to
use fine-grained tools which I can understand in isolation and combine
(or patch as you will) them the way I want it than try to manage or
even comprehend one huge process behemoth.

Kent
---
http://www.datanoise.com
Brian V. Hughes (Guest)
on 2006-02-17 20:43
(Received via mailing list)
Ben M. wrote:
> It's simply not the same. Why does every conversation about deployment
> options for rails get derailed (har har) into talk of this is more
> complicated or that is more complicated?

Because deployment of a system like rails can be difficult for people to
get
their heads around. Just because it's somewhat complex, doesn't mean
it's hard.
See my previous message about deployment sometimes requiring Webmaster
experience, not app developer experience.

> There is simply nothing in the rails world that works like a servlet
> container, much less an application server.

Why would Rails want to work like a servlet container? And as for an
"application server", that's a really loaded term. FastCGI is how you
persist
your application code in memory, so that it can quickly respond to HTTP
requests
that are handled by the web tier. The rest of the stuff that would
normally go
in the "application server" is handled by the Rails environment, so why
does
Rails need an "application server"?

> This is a fatal, deal-breaking, they're-never-gonna-take-you-seriously
> problem.

All I'm going to say here is please try to speak for yourself. If rails
doesn't
work for you as a deployment platform, then don't use it.

> I'm happy to hear about the deployment successes of Ezra, DHH, and
> others. But patching together a bunch of FastCGI processes is not the
> same as having a highly flexible, long-running container process.

One, who said we're "patching together" anything? Second, until you've
done it
yourself, you really can't comment on whether Rails can be
"highly-flexible and
long-running", container basis or no.

> I'm hoping Mongrel will change that. But if you really love rails and
> want to champion it's cause in the enterprise world, I can't possibly
> overstate the importance of a web/application container.

Maybe I'm missing something, but isn't that last sentence extremely
close-minded? Clearly a web/application container isn't the only way to
deploy a
production web application...

-Brian
Ben M. (Guest)
on 2006-02-17 21:26
(Received via mailing list)
Well, I will admit that maybe I'm so used to the "java way" that maybe
I'm not giving the
lots of little tools approach enough of a chance. I still don't think
it's going to sit
well with shops that are heavily invested in Java and .Net.

See my response to Rick for the other half of this.

b
Ben M. (Guest)
on 2006-02-17 21:26
(Received via mailing list)
That's funny Rick, cuz I feel dense at least once a day every day. :-)

As in my response on this topic to Kent, I have to admit that there is a
possibility that
my Java experience colors my view of this and I'm attaching more
significance to a servlet
container than necessary. But I really don't think so.

The very first thing I learned about Java for the web and servlets is
that this concept
was a giant leap forward for web development. Rather than having to
create a process,
execute the request, and the kill and clean up the process for every
request, servlets
gave the web programmer a single, multi-threaded, long-running process
into which she
could hook her applications. It wasn't too long before they added in
other useful features
like connection pools and the other pieces of the j2ee stack.

But really, the best thing about the container concept is that you have
a single global
memory space for your app. The multiple copies created by the
multi-threading is fairly
transparent to you (you just have to be careful of a few things).

Now, FastCGI addresses the process life-cycle problem, but it does not
give one an
application environment. Maybe the rails folks coming from the
sysadmin/PHP/perl world are
comfortable enough with the OS being their environment. Maybe that's
just as valid an
application environment. I don't really see how it can be as portable as
a container
process. I can download tomcat, unzip it, drop my .war into the webapp
folder and start
the server and it just works... on all major platforms... no muss no
fuss.

I've also gotten the (possibly incorrect) impression that the current
rails environment
doesn't allow for static application state to be maintained in memory
indefinitely. This
came up on a thread about the inability to store threads or procs in
sessions. I pointed
out that one could keep the non-serializable stuff in an in-memory hash
and store a
(serializable) string key for that non-serializble object in the
session.

But then I realized that I think -- and I still haven't gotten a
straight answer to this
-- one can't really do that with the FastCGI approach: Having a bunch of
processes running
at the end of a pipe so that you don't suffer the process life-cycle
pain is all well and
good. But each of those processes is separate... you can't have a global
in-memory storage
of stuff like my thread map or configuration or cached lookup values or
what have you. Or
can you?

I imagine there's some way of starting another ruby process that has
this common state and
each fcgi process connects to that process... er, in fact I think that
was Ezra's
solution: stick your stuff you need in a drb process and have your app
get if from there
when you need it.

I still maintain that it's not the same. I still think that a lot of
folks in the
Java/.Net world ain't gonna buy that. I'm surprised that more Java folks
aren't coming to
my aid here... maybe I'm just a crazy guy ranting to the magazine stand.
Or, I'd be happy
enough if a Java convert told me why they think this isn't an issue.

It'll be interesting to see where this is in six months; whether mongrel
has made all us
insecure container-heads happy... whether I'll just learn the tools that
exist and shut
up... whether this issue will grow as an impediment to rails adoption.

In the meantime, I'll try to shut up and learn the tools. :-)

b


PS: I think it interesting to note the volume of "help help, my fastcgi
setup isn't
working" posts on this list. On all the Java lists I've ever been on,
there's very little
"why doesn't my tomcat work"? That's cuz it just does.
Dylan S. (Guest)
on 2006-02-17 21:32
(Received via mailing list)
Ben... your PS is classic.  I remember some unstable versions of Tomcat
that
were deemed "stable", that involved starting, stopping, then starting
again
to ensure the .war was deployed properly.
We had a group of 20 people, who, each couldn't get it working correctly
through individual configs.
However, I would still recommend Tomcat over any of the other servers :)
Brian V. Hughes (Guest)
on 2006-02-17 21:35
(Received via mailing list)
Ben M. wrote:
> Well, I will admit that maybe I'm so used to the "java way" that maybe
> I'm not giving the lots of little tools approach enough of a chance. I
> still don't think it's going to sit well with shops that are heavily
> invested in Java and .Net.

You may be entirely correct about that. But, does it really need to?

If your shop is "heavily invested" in a particular architecture, that's
probably
where you should stay, unless there is a particular point of pain that's
started
to prevent work. That's really a long way of saying "if it ain't broke,
don't
fix it." :)

And that's not, at all, a Rails related thing. There's plenty of web app
deployment systems that aren't based around the servlet/container
approach. They
existed before the servlet concept was invented and they continue to
exist after...

-Brian
Ben M. (Guest)
on 2006-02-17 21:50
(Received via mailing list)
Brian V. Hughes wrote:
>
> Why would Rails want to work like a servlet container? And as for an
> "application server", that's a really loaded term. FastCGI is how you
> persist your application code in memory, so that it can quickly respond
> to HTTP requests that are handled by the web tier. The rest of the stuff
> that would normally go in the "application server" is handled by the
> Rails environment, so why does Rails need an "application server"?

I don't want rails to work like a servlet container. I want a servlet
container to run my
rails apps in. Actually, I already have one: webrick. Unfortunately, it
seems the whole
rails world has just shrugged their shoulders at the shortcomings of
webrick and said
"well, it's good for development". I'm hoping that my man Zed is gonna
fix that (albeit
with his own separate code-base).

> All I'm going to say here is please try to speak for yourself. If rails
> doesn't work for you as a deployment platform, then don't use it.

Rails isn't a deployment platform. It's a "full stack web application
framework". It's
great. There are some design choices in Rails that I quibble about, but
they all have
nothing to do with this thread. I'm just pining for a stable, fast,
powerful, flexible
container to run my rails apps in. Once it happens, it will make things
better for
everyone... I'd put money on it.

> One, who said we're "patching together" anything? Second, until you've
> done it yourself, you really can't comment on whether Rails can be
> "highly-flexible and long-running", container basis or no.

I keep getting told that we don't need a container to deploy apps in;
that I can use this
ruby package to do this, or this system tool to do that... that sounds
patched together to
me. I'm also not accusing Rails of not being flexible and long-running,
I'm arguing for
the benefits of having a ruby process in which our rails apps run... the
happy, warm,
comforting environment that has made me the happy Java junkie I am.

And the statement about not having tried something cuts both ways. If
you have done
serious Java development and want to express your opinion as to why the
container concept
simply isn't needed in the rails world, I'm all ears. I will certainly
be working my way
around to learning all the ins and outs of apache/lighty fcgi
deployment.... even if
Mongrel roared into stable full-featured release today. If nothing else,
why would anyone
not welcome more deployment options for our beloved web framework?

>> I'm hoping Mongrel will change that. But if you really love rails and
>> want to champion it's cause in the enterprise world, I can't possibly
>> overstate the importance of a web/application container.
>
>
> Maybe I'm missing something, but isn't that last sentence extremely
> close-minded? Clearly a web/application container isn't the only way to
> deploy a production web application...

Hmm, how is that close-minded? I feel like I'm trying to help the rails
community and I'm
getting tsk-tsked and told "try it... it's not so bad". That's all well
and good, but I'm
stating an opinion that I firmly believe is shared by many in the
Java/.Net world. I might
be wrong, but I'm still going to stand by my opinion until I see
evidence to the contrary.

b
David Heinemeier H. (Guest)
on 2006-02-17 22:11
(Received via mailing list)
> How do you feel about the RoR deployment platform. Is lighttpd with fastcgi
> realy this unstable?
> Will there be commercial vendors for RoR deployment environments. And how
> can we help to get things more stable?

lighttpd has come a long way since the book. We just launched Campfire
on lighttpd and are doing more than 120 requests per second without
lighty breaking a sweat. It's certainly possible to deploy large-scale
applications on Rails that stay stable. See
http://rubyonrails.org/applications for examples.
--
David Heinemeier H.
http://www.loudthinking.com -- Broadcasting Brain
http://www.basecamphq.com   -- Online project management
http://www.backpackit.com   -- Personal information manager
http://www.rubyonrails.com  -- Web-application framework
Jin L. (Guest)
on 2006-02-17 22:30
(Received via mailing list)
I've worked with Java, Tomcat, Rails, lighttpd, apache, fcgi, scgi,
windows/mac, etc etc. and I don't think that Tomcat is any better/worse
than
lighttpd with fcgi/scgi.

Dropping in a war file and having Tomcat explode it and map it to a url
is
nice, but as Kent said, once you start configuing your connection pool,
server.xml, web.xml, and the other *.xml files in your apps, its just as
much a headache as Rails. What I like about Java and Tomcat is that you
can
package other jar files with your application, so that you can deploy to
any
server with java/tomcat installed. The Rails equivalent of this would be
being able to package other gems with your rails application and having
rails reference those gems rather than looking in your ruby
installation.

However, as far as the two containers go, they are pretty similar. Both
require some configuration to get going. As far as stability, tomcat has
the
edge, but lighttpd/fcgi/scgi or apache/fcgi/scgi is gaining alot of
steam.
Zed S. and company is making that happen. SCGI is totally awesome, it
has
turned my slow and unstable windows Apache/fcgi app to super fast and
stable. Mongrel already looks like the next step up. Just remember that
Java
has Apache, jBoss, and other big OSS companies catering to their server
needs. We have DHH, Zed, and other great talents, and they are doing
things
now that took these other companies years to get right.

Finally, I think alot of software that people call "enterprise worthy"
is
not because of the actual software, but because of its supporting cast.
Tomcat has this thing called "Apache" going for them. That means
support,
regular updates, bug fixes, etc. That means one installation and some
configuration, as opposed to downloading 3 or 4 things and tieing them
together.

For a bunch of guys who believe in open source, I'd say we're not doing
so
bad. It's only been a couple months since rails even came out of beta
=)
Joe (Guest)
on 2006-02-17 22:50
A lot of the Java people I know say that Tomcat is a dog and they prefer
Resin a great deal more.

As for Lighty, I found it quite a bit easier to install and setup than
Apache.

Joe
Xavier N. (Guest)
on 2006-02-17 23:09
(Received via mailing list)
On Feb 17, 2006, at 21:29, Jin L. wrote:

> Dropping in a war file and having Tomcat explode it and map it to a
> url is nice, but as Kent said, once you start configuing your
> connection pool, server.xml, web.xml, and the other *.xml files in
> your apps, its just as much a headache as Rails. What I like about
> Java and Tomcat is that you can package other jar files with your
> application, so that you can deploy to any server with java/tomcat
> installed. The Rails equivalent of this would be being able to
> package other gems with your rails application and having rails
> reference those gems rather than looking in your ruby installation.

Isn't this what lib and vendor are meant for?

-- fxn
Rick B. (Guest)
on 2006-02-17 23:48
(Received via mailing list)
Ben,
Thanks for the time spent on your response!

I think in some ways we're in complete agreement and in others we likely
have different priorities.

For example, it would be a decided retrograde maneuver to have to be
concerned with process management, request creation, and all the cruft
from the old CGI world.  Servlets took care of that, mod_{php, perl,
ruby, etc. } took care of that, Rails takes care of that.  It's pretty
much a given these days.  In that sense servlets solve a real problem
for the developer, but it's also been solved in another half dozen ways
in other environments.

I also believe it would be nice if it were possible to pick up a little
bundle of model(s), view(s), and controller(s), with routes, drop it
into a directory and have it served off the URL root as a mini Rails
app.  That's actually not too far away -- especially if you look at
plugins with the right perspective -- given the non-trivial "if" of
having a properly configured web front-end in place.

Is making a plugin harder than making a .war file?  Hard to tell.

Starting from novice in either paradigm (Java vs. Ruby/Rails) I would
estimate they are equally difficult -- but that's actually unlikely to
be improved for anyone over time, as there will always be a novice and
things will always be somewhat difficult for him.  The more you try to
smooth away the wrinkles, the less powerful the tool he's using becomes.

For those with some experience in the paradigm, making a .war is
probably less difficult than making a Rails plugin.  That, however, is
unrelated to the validity of Rails deployment techniques:  either the
overall non-servlet/non-container methodology is sound or it isn't ­-
ease of making a drop-in package is more a question of optimizing to
developer audience.

But, on the question of whether we should be able to make drop in
"components" for pluggable deployment for Rails, I tend to side with DHH
on this one.  Much of the software world has pre-occupied itself with
the "reusable components" holy grail for well over a decade now.  The
Java economic ecosystem (not less the .Net market) seems to rely on this
reusability dream to keep its monetary Krebb's Cycle flowing
perpetually.  From where I sit, though, there still aren't reusable
components.  As David has said numerous times (I'm sure I'm
paraphrasing, so forgive me) once a component becomes big enough to be
of use, it's unwieldy enough that writing it afresh is often easier
than fitting it to your system.

That as may be, I think where I see us having different priorities (and
if we're lucky we're representative of two broader groups with similarly
differing priorities) is in the value of "sharing" at the process level.

I think you've identified a trade-off that is of relevance.  On one end
of the spectrum, to get the sort of persistence between calls or the
sharing between processes/threads that you highlight, certain constructs
must be available to the program.  This means a certain amount of
infrastructure must be present to support that type of persistence
and/or sharing.  The more important this sharing is, the more the
architecture is built around supporting sharing without burdening the
developer.

On the other end of the spectrum, to get the ability to pare down the
infrastructure, to be able to scale trivially and cheaply, to use small
and efficient specialized parts ("loosely joined"?), certain
architectural techniques are used which result in, essentially zero
sharing -- sometimes known as the Share Nothing approach.  Clearly, this
makes it more difficult for the programmer to share information between
requests or across the process/thread boundary (hence things like
storage in the session, DRb servers, memcache integration, even using
the database as a blackboard all become common techniques).

There's a sharing continuum with tradeoffs.  On the one end, while there
may be more overhead in setting up for easy sharing, it becomes easy for
the programmer to contact processes around the server (and cluster), and
easier to leave things in memory for later use.  On the other end of the
spectrum, while it may be difficult to do certain types of
communication, the overhead in setting up a project is reduced.

The fundamental problem I think people are trying to solve in building
large-scale applications is building large-scale applications.  Choosing
a point on the sharing spectrum is one part of that process of building.

All along that spectrum everyone is basically in agreement that
developer time is worth more than hardware cost.  We can always throw
more hardware at the problem.  The Share Nothing crowd says, "By sharing
nothing we can scale horizontally faster and cheaper than guys who don't
Share Nothing."  The other crowd (Share Something?) says, "Yeah, but
you're going to have to share eventually to get things done, so you
might as well plan for it.  We can still throw hardware at the problem,
maybe it costs us 15% more, but that's still cheap."

Sitting mostly in the Share Nothing world, I perceive things a certain
way:

 * The sharing people talk about doing isn't actually as common as it's
 made out to be.  I.e., optimizing for it is premature optimization,
 which is the famous Root Of All Evil.  Corollary:  the cost of sharing
 architecture is often high, and the biggest proponents of installing
 such architecture are usually the vendors who stand to make even more
 money when the architecture becomes unwieldy and requires support.

 * While it appears that sharing and messaging in the way we're talking
 about will save developer effort, often the communication mechanisms
 fail to work as intended and result, even when working properly, in
 increased coupling and complexity.  If that's not the Root, it's the
 Trunk Of All Evil.  Developer cost and support cost tends to go up
 rather than the intended down.

 * Once one starts down the road away from Share Nothing the ability to
 change storage strategies, to change how a system is accessed, to make
 decisions on where to put which data all become compromised.  It
 becomes difficult to Refactor the application deployment layout.

I tend to see the Share Nothing philosophy as the Agile Deployment
philosophy.  It stresses YAGNI.  It stresses repeatability.  It stresses
scalability both up and DOWN.

  (...try running JBoss on a 486 laptop sometime, and try running Rails
  in the same environment.  And while deploying to tomcat may be as
  simple as dropping in a .war, and setting up tomcat might require just
  a couple of commands, the stack required is as light as a
  self-assembling dump truck.  I'm not saying we should consider the 486
  laptop as a deployment target, merely that I feel the methodology
  which gives up little if anything and foregoes no option for
  deployment is more Agile than the one which closes off platforms as a
  condition of adoption)

It stresses using what works for the problems at hand.  Rails takes
things a step further and says, rather than building countless doomed
frameworks which try to predict what constructs will be useful (Big
Design Up Front), let's extract the useful pieces from applications that
already work, and refactor those pieces for use where they make sense.

If what you do a lot of is sharing information between processes, then
you should optimize your design for that.  If, on the other hand, that's
something you do rarely, that's something you shouldn't be optimizing
for.  Rails currently doesn't optimize for sharing between
process/requests, because the real world applications from which it is
extracted never do that sort of thing.  Will this kill Rails' "adoption
into the enterprise"?  Hard to tell now, but there's no point in trying
to fabricate scenarios that have no applicability to real applications
since that's the exact opposite technique to the one the Rails framework
developers took, and many believe is why Rails has been so successful.
Thinking otherwise is more BDUF than Agile -- and, while there may be
advantages to both mindsets, it's going to be hard to find kindred souls
in the Rails community who want to discard an Agile way of thinking.

Not sharing between entities in the lifecycle has led to a Share Nothing
approach which turns out to have huge benefits in Business Agility -- in
a sense Share Nothing is very much like taking Agile to where the
Enterprise Unified Process was trying to take the Rational Unified
Process.  Rather than focusing on just the hunk of code we're crafting,
we must focus on the lifecycle before and afterwards as well.  What
would Agile look like in that expanded environment?  I believe it starts
with Share Nothing.

Since Rails has started down the Share Nothing road and accrued the
benefits, while realizing that it's not seeing the need to share things
between requests and processes, it moves farther and farther away from
Share Something.  Will there come a time when the Rails developers want
rails to storm the "enterprise" castle but Share Nothing is keeping them
from there?  Hard to say (though many seem compelled to say so).

I tend to think not.  I think the Era of The Big App Server is slowly
(to put on my Bruce Tate hat: probably as slow as the never-ending
demise of COBOL, of course) coming to an end.  Rails is already storming
the "enterprise" castle, but it's doing so not because a consortium of
vendors and standards bodies are pushing it in, but rather because the
people from inside the castle are pulling it in themselves.  Deploying
in a Share Nothing fashion will get the job done, and I'm of the belief
that the lost sharing won't be missed along the way.


Anyway, this is really all to say that I see a spectrum of different
approaches to sharing in resolving the problem of building heavy-use
systems.  Many in the Java community appear to choose one end of that
spectrum, while many in the Rails community choose the other end of that
spectrum.  From experience I happen to place a higher priority on the
factors that promote Share Nothing, and believe I've seen in the Java
"enterprise" community an evolutionary history that tells me that, with
the tools available today, for the applications we're tasked to build,
Shared Something rarely ever works.  So I'm compelled to go with what
does work, which today is Shared Nothing -- and nothing enables Shared
Nothing like Rails.


But that's because of how I see my priorities.  In another world,
perhaps just a block away someone may say "that doesn't work for me."
If so, they should do what does work for them.


Best,
Rick
--
 http://www.rickbradley.com    MUPRN: 881
                       |  communicate,
   random email haiku  |  among other things, ideas,
                       |  emotions, or thoughts.
Eric K. (Guest)
on 2006-02-18 00:01
(Received via mailing list)
Hello all,

I teach Java to new programmers at a college. I use tomcat and jboss
and I have many years of "enterprise" development experience including
some very large e-commerce sites. I am really loving rails and I will
be teaching it too starting in the fall. I don't even try to teach my
java students production installation, configuration, and tuning of
java application servers. It is just too complicated and is vendor
specific. Knowing how to work with jboss doesn't really help you with
WebSphere. Every java shop that my students might work at will have
dedicated engineers for this.

I will be teaching this with rails, though. If you think about it for
a minute there really aren't that many ways to do this. All of them
get discussed often on here on the list. Ezra is even writing a book
that will most likely be definitive and help rails move forward in
lots of "enterprise" shops. With WebSphere you really need to send an
engineer to expensive training before you try deploying something
major. With JBoss you have to figure out just what the heck they are
talking about and hope for the best.

The relative simplicity of deploying a production rails application is
one of its attractions for me and I hope that doesn't get lost during
the next few years.

-Eric
Ben M. (Guest)
on 2006-02-18 00:31
(Received via mailing list)
Wow! Lots of food for thought there Rick. And I think we probably do
mostly agree. I hope
this thread hasn't been too annoying for other folks, because I've found
it really
thought-provoking.

Couple of points: For the record, I wasn't promoting the drop-in-able
.war file as a
plugin or component style device. I haven't really seen anyone do such a
thing, though I
think it would be possible. What the .war approach gives you is the
ability to zip up your
project directory and drop it into your server's webapp directory and it
handles the rest.

And interestingly enough, I do see a trend in the Java world towards
using replaceable
frameworks or components as needed rather than buying into the
mega-appserver world of
j2ee. This is the approach of the Spring framework, of which I am a big
fan. So,
ironically, I'm on the "use the tools you need to get the job done" side
of things in Java.

However, I think the original concern I had was at a different level
than all that. I was
just looking for -- basically -- a webrick that I start running and into
which can add and
remove my apps and which also provides a virtual machine memory space
that I can use
without having to deal with running a differnt process.

You know, I'm actually not even talking about scalability concerns
here... I know that
servlet containers can be clustered and all that. Any decent web
platform will have some
way of doing that. I'm more concerned with what runtime environment is
available to me as
a developer. Now, since there's still plenty more for me to learn about
Ruby, I'll
probably find ways of doing everything I want.

Ok. Now I'll let this thread die a natural death!

b
Ezra Z. (Guest)
on 2006-02-18 00:37
(Received via mailing list)
On Feb 17, 2006, at 1:45 PM, Rick B. wrote:
> <snip great thoughts>
>
> Best,
> Rick

	I have to agree with Rick here. I don't really see what the benefit
of a servlet type environment would be for rails deployments of
various sizes. Rails has been built from the start with the Shared
Nothing mentality. Instead of a monolithic app server that does
everything, rails delegates to other processes the tasks they do
best. And it makes sense, I wouldn't want the complexity that would
come with trying to take a rails servlet container and scale it to
more and more machines as an app becomes more and more popular. As it
stands now it is relatively easy to add more hardware and spawn
external fcgi listeners on it from a lighttpd front end.

	Mongrel will be a great way to simplify things for rails deploys as
well. but i don't think the goals of mongrel are to become a servlet
container as much as just a different take on fcgi style processes.
So you wold still want to proxy from lighty or apache to multiple
back end mongrel listeners is how I see it working. Of course Zed is
a smart guy so we will see what he comes up with in this area. But
rails wasn't built for sharing like a servlet. Ruby doesn't have
native threads like java yet and so it becomes much more difficult
for rails to run multiple threads within one container then it is to
run multiple in memory fcgi processes.

	So the problem of having global shared assets is better solved with
drb or memcached or something similar and keeping as much as possible
sharing nothing. Only explicitely share something if you really need
it to be shared. I agree with Rick that shared nothing fits better
with the agile practices of many ruby developers and with ruby itself
as well. Rails is gaining popularity in leaps and bounds these days
and I think the deployment options are getting better and better.
Lighttpd is really a very solid piece of kit now. Its a paradigm
shift away from the java servlet container mentality, but its no less
ready for the enterprise. I think we will see more and more
widespread adoption of rails in the enterprise and with it will come
more widespread adoption of the shared nothing approach to scaling
web applications.

	Also the documentation on deploying rails apps for production is
severely fragmented and conflicted in its current state. This is the
reason I decided to write my book about it. Its one of the areas of
rails documentation that is sorely needed to be collected all in one
place. I hope to help solidify the communities understanding of how
rails apps work in production situations as much as i can with the
release of my book coming soon.

Cheers-
-Ezra Z.
Yakima Herald-Republic
WebMaster
http://yakimaherald.com
509-577-7732
removed_email_address@domain.invalid
ToddG (Guest)
on 2006-02-18 01:42
Ezra Z. wrote:
> 	Mongrel will be a great way to simplify things for rails deploys as
> well. but i don't think the goals of mongrel are to become a servlet
> container as much as just a different take on fcgi style processes.
> So you wold still want to proxy from lighty or apache to multiple
> back end mongrel listeners is how I see it working. Of course Zed is

(Pardon if this screws up the thread, I'm trying a post from ruby-forum
for the first time)

While the OP mentioned differences between shared-nothing and threaded
environments, I think the original point and that of the quote from TSS
(or at least the point I've thought many times) is more so the relative
immaturity of deploying Rails for production (i.e. stable) deployments,
rather than a debate on process vs. thread models. Yes FCGI is old, but
I think it's a safe bet the ruby-fcgi drivers and associated machinery
are not nearly as time-tested as FCGI itself and FCGI libs for Perl for
example.

My impression is it's the perceived flakiness that worries many people,
and as originally mentioned, the "Rails Bible" even mentioning the
flakiness wasn't especially encouraging. Lots of places can't switch to
lighty, or have to use Apache2 (the status of FCGI with at this point is
hard to ascertain - though it was noted mod_fcgi work was going to
resume(?)). And as Ezra mentions (and is working on improving), much
documentation for deployment is scattered and piecemeal. That's fine for
very roll-your-own types of shops, but that's not all shops. And it's
probably the main source of the "taped-together-ness" impressions and
comments.

Thankfully it's all getting better, and I think somewhat ironically it's
the contrast between how well and smooth all the Rails parts work
together and the still somewhat ad-hoc art (more than science) of
deploying a Rails app that makes this such a visible issue (for some at
least).

Another significant point is that most apps don't need to scale to
enterprise levels; for those having a single process with threads that
easily handles your load is a much easier proposition than setting up a
shared-nothing setup and managing multiple listeners and so on, esp when
they seem to be flaky and require kill -9's of zombie procs and so on.
This is one example of where Tomcat and friends are a much nicer option
than Rails, as Webrick doesn't cut it. Hopefully mongrel will be able to
fill that spot for Rails (though I'm not clear on how the threading but
locking around Rails part will affect speed).

Sorry this is long, I've been thinking about this stuff a lot myself and
just brain dumped.
This topic is locked and can not be replied to.