Ben,
Thanks for the time spent on your response!
I think in some ways we’re in complete agreement and in others we likely
have different priorities.
For example, it would be a decided retrograde maneuver to have to be
concerned with process management, request creation, and all the cruft
from the old CGI world. Servlets took care of that, mod_{php, perl,
ruby, etc. } took care of that, Rails takes care of that. It’s pretty
much a given these days. In that sense servlets solve a real problem
for the developer, but it’s also been solved in another half dozen ways
in other environments.
I also believe it would be nice if it were possible to pick up a little
bundle of model(s), view(s), and controller(s), with routes, drop it
into a directory and have it served off the URL root as a mini Rails
app. That’s actually not too far away – especially if you look at
plugins with the right perspective – given the non-trivial “if” of
having a properly configured web front-end in place.
Is making a plugin harder than making a .war file? Hard to tell.
Starting from novice in either paradigm (Java vs. Ruby/Rails) I would
estimate they are equally difficult – but that’s actually unlikely to
be improved for anyone over time, as there will always be a novice and
things will always be somewhat difficult for him. The more you try to
smooth away the wrinkles, the less powerful the tool he’s using becomes.
For those with some experience in the paradigm, making a .war is
probably less difficult than making a Rails plugin. That, however, is
unrelated to the validity of Rails deployment techniques: either the
overall non-servlet/non-container methodology is sound or it isn’t -
ease of making a drop-in package is more a question of optimizing to
developer audience.
But, on the question of whether we should be able to make drop in
“components” for pluggable deployment for Rails, I tend to side with DHH
on this one. Much of the software world has pre-occupied itself with
the “reusable components” holy grail for well over a decade now. The
Java economic ecosystem (not less the .Net market) seems to rely on this
reusability dream to keep its monetary Krebb’s Cycle flowing
perpetually. From where I sit, though, there still aren’t reusable
components. As David has said numerous times (I’m sure I’m
paraphrasing, so forgive me) once a component becomes big enough to be
of use, it’s unwieldy enough that writing it afresh is often easier
than fitting it to your system.
That as may be, I think where I see us having different priorities (and
if we’re lucky we’re representative of two broader groups with similarly
differing priorities) is in the value of “sharing” at the process level.
I think you’ve identified a trade-off that is of relevance. On one end
of the spectrum, to get the sort of persistence between calls or the
sharing between processes/threads that you highlight, certain constructs
must be available to the program. This means a certain amount of
infrastructure must be present to support that type of persistence
and/or sharing. The more important this sharing is, the more the
architecture is built around supporting sharing without burdening the
developer.
On the other end of the spectrum, to get the ability to pare down the
infrastructure, to be able to scale trivially and cheaply, to use small
and efficient specialized parts (“loosely joined”?), certain
architectural techniques are used which result in, essentially zero
sharing – sometimes known as the Share Nothing approach. Clearly, this
makes it more difficult for the programmer to share information between
requests or across the process/thread boundary (hence things like
storage in the session, DRb servers, memcache integration, even using
the database as a blackboard all become common techniques).
There’s a sharing continuum with tradeoffs. On the one end, while there
may be more overhead in setting up for easy sharing, it becomes easy for
the programmer to contact processes around the server (and cluster), and
easier to leave things in memory for later use. On the other end of the
spectrum, while it may be difficult to do certain types of
communication, the overhead in setting up a project is reduced.
The fundamental problem I think people are trying to solve in building
large-scale applications is building large-scale applications. Choosing
a point on the sharing spectrum is one part of that process of building.
All along that spectrum everyone is basically in agreement that
developer time is worth more than hardware cost. We can always throw
more hardware at the problem. The Share Nothing crowd says, “By sharing
nothing we can scale horizontally faster and cheaper than guys who don’t
Share Nothing.” The other crowd (Share Something?) says, “Yeah, but
you’re going to have to share eventually to get things done, so you
might as well plan for it. We can still throw hardware at the problem,
maybe it costs us 15% more, but that’s still cheap.”
Sitting mostly in the Share Nothing world, I perceive things a certain
way:
-
The sharing people talk about doing isn’t actually as common as it’s
made out to be. I.e., optimizing for it is premature optimization,
which is the famous Root Of All Evil. Corollary: the cost of sharing
architecture is often high, and the biggest proponents of installing
such architecture are usually the vendors who stand to make even more
money when the architecture becomes unwieldy and requires support.
-
While it appears that sharing and messaging in the way we’re talking
about will save developer effort, often the communication mechanisms
fail to work as intended and result, even when working properly, in
increased coupling and complexity. If that’s not the Root, it’s the
Trunk Of All Evil. Developer cost and support cost tends to go up
rather than the intended down.
-
Once one starts down the road away from Share Nothing the ability to
change storage strategies, to change how a system is accessed, to make
decisions on where to put which data all become compromised. It
becomes difficult to Refactor the application deployment layout.
I tend to see the Share Nothing philosophy as the Agile Deployment
philosophy. It stresses YAGNI. It stresses repeatability. It stresses
scalability both up and DOWN.
(…try running JBoss on a 486 laptop sometime, and try running Rails
in the same environment. And while deploying to tomcat may be as
simple as dropping in a .war, and setting up tomcat might require just
a couple of commands, the stack required is as light as a
self-assembling dump truck. I’m not saying we should consider the 486
laptop as a deployment target, merely that I feel the methodology
which gives up little if anything and foregoes no option for
deployment is more Agile than the one which closes off platforms as a
condition of adoption)
It stresses using what works for the problems at hand. Rails takes
things a step further and says, rather than building countless doomed
frameworks which try to predict what constructs will be useful (Big
Design Up Front), let’s extract the useful pieces from applications that
already work, and refactor those pieces for use where they make sense.
If what you do a lot of is sharing information between processes, then
you should optimize your design for that. If, on the other hand, that’s
something you do rarely, that’s something you shouldn’t be optimizing
for. Rails currently doesn’t optimize for sharing between
process/requests, because the real world applications from which it is
extracted never do that sort of thing. Will this kill Rails’ “adoption
into the enterprise”? Hard to tell now, but there’s no point in trying
to fabricate scenarios that have no applicability to real applications
since that’s the exact opposite technique to the one the Rails framework
developers took, and many believe is why Rails has been so successful.
Thinking otherwise is more BDUF than Agile – and, while there may be
advantages to both mindsets, it’s going to be hard to find kindred souls
in the Rails community who want to discard an Agile way of thinking.
Not sharing between entities in the lifecycle has led to a Share Nothing
approach which turns out to have huge benefits in Business Agility – in
a sense Share Nothing is very much like taking Agile to where the
Enterprise Unified Process was trying to take the Rational Unified
Process. Rather than focusing on just the hunk of code we’re crafting,
we must focus on the lifecycle before and afterwards as well. What
would Agile look like in that expanded environment? I believe it starts
with Share Nothing.
Since Rails has started down the Share Nothing road and accrued the
benefits, while realizing that it’s not seeing the need to share things
between requests and processes, it moves farther and farther away from
Share Something. Will there come a time when the Rails developers want
rails to storm the “enterprise” castle but Share Nothing is keeping them
from there? Hard to say (though many seem compelled to say so).
I tend to think not. I think the Era of The Big App Server is slowly
(to put on my Bruce Tate hat: probably as slow as the never-ending
demise of COBOL, of course) coming to an end. Rails is already storming
the “enterprise” castle, but it’s doing so not because a consortium of
vendors and standards bodies are pushing it in, but rather because the
people from inside the castle are pulling it in themselves. Deploying
in a Share Nothing fashion will get the job done, and I’m of the belief
that the lost sharing won’t be missed along the way.
Anyway, this is really all to say that I see a spectrum of different
approaches to sharing in resolving the problem of building heavy-use
systems. Many in the Java community appear to choose one end of that
spectrum, while many in the Rails community choose the other end of that
spectrum. From experience I happen to place a higher priority on the
factors that promote Share Nothing, and believe I’ve seen in the Java
“enterprise” community an evolutionary history that tells me that, with
the tools available today, for the applications we’re tasked to build,
Shared Something rarely ever works. So I’m compelled to go with what
does work, which today is Shared Nothing – and nothing enables Shared
Nothing like Rails.
But that’s because of how I see my priorities. In another world,
perhaps just a block away someone may say “that doesn’t work for me.”
If so, they should do what does work for them.
Best,
Rick
http://www.rickbradley.com MUPRN: 881
| communicate,
random email haiku | among other things, ideas,
| emotions, or thoughts.