Some architecture questions for my mongrelian friends

Server Mongrel
1

Hey,

I’m working on a project, and mongrel may be part of the stack, but
I’ve got some more general questions and ideas I’m hoping to run by
this list. The people on this list have a broader knowledgebase and
more experience than any place else I know - plus a general
friendliness and willingness to help!

I’m working with a company who has a really antique application stack.
Literally from 1998. IIS + ASP + MS SQL server. They want me to help
“modernize” things. In the abstract I’d say, “get a really good .NET
team and go that route.” But they want me to help. All I work in these
days is Ruby. And that’s all I want to work in. :slight_smile:

So my questions are like this:

  1. Can I in good conscience start migrating this company from IIS/ASP
    to Mongrel/Ruby/Merb/ORM (or something like that)? They have roughly
    2-3M page views per month.

1.a) No matter how good they think I am, wouldn’t it be smarter to move
forward with M$ since that’s what they’ve got already? I don’t want to
be the guy who screws them deeper into the hole by really confusing
their stack.

I hate it when new dudes come in with their “stack” and bias
development based on their preferences withou considering what’s
already there. I’d rather walk away from this if Microsoft is really
their odds-on smart choice (i.e. I don’t need the money - I have some
personal relations that led me here). All I want is the company to be
successful.

  1. Their MS SQL setup is relatively fine. Lots of wacky stored procs
    which bug me but mostly it’s fine. Am I crazy to try to run MS SQL
    against Ruby/ORM? Seems like there are some people doing it?

  2. If I do this, I’d plan to segment this site into two separate boxes
    and run the Ruby on a Linux box (and maybe outsource that management to
    a group like EngineYard). Then have the LB’s split traffic between the
    boxes based on url patterns. Again: crazy? unwise? Currently they’re at
    rackspace which knows poodle about Ruby/Mongrel afaict.

Context: The front-end site is not impossibly complex. But there is
“deep” integration with some backend admin processes which run a large
part of the business: some crm, PPC, finance/accounting, email and
billing: all partially implemented and built in hand coded ASP. It’s a
real tangle and it breaks all the time right now. I want to get most of
these processes out into third party systems with much narrower points
of contact between the production DB’s and the specific admin services.
This can only happen incrementally over time. This is in addition to
the front-end websystem migration.

Budgets for this work are not tiny but not enormous. Ditto timeframe.
Maybe $250k over 6-8 months.

Any tips or advice on taking on large migration projects such as this
would be appreciated. Advice such as “run!” is welcome also. I realize
there are no definite answers - I’m just looking for experience or
advice on how to reach conclusions here.

I realize this is horribly off-topic and impossibly vague. And I
wouldn’t ask for this input, except that I highly admire and regard the
capabilities and experience of many people who are on this list. I
can’t think of a smarter mail list who could help advise on this. Any
assistance at all will be greatly appreciated.

Thanks!

Steve

p.s. Anyone who has possible interest in this project professionally
can also contact me directly off-list.

2

On 16 May 2008, at 21:34, [email protected] wrote:

to help “modernize” things. In the abstract I’d say, “get a really
good .NET team and go that route.” But they want me to help. All I
work in these days is Ruby. And that’s all I want to work in. :slight_smile:

So my questions are like this:

  1. Can I in good conscience start migrating this company from IIS/
    ASP to Mongrel/Ruby/Merb/ORM (or something like that)? They have
    roughly 2-3M page views per month.

It can be done. Others can better tell you on the hosting costs, but
there are sites doing this and apparently making money still :slight_smile:

1.a) No matter how good they think I am, wouldn’t it be smarter to
move forward with M$ since that’s what they’ve got already? I don’t
want to be the guy who screws them deeper into the hole by really
confusing their stack.

The biggest problems you’ll hit with packing Ruby into a production
windows stack are:

  • No decent web proxy, at least, not like nginx and friends.
  • Apache I’ve benched up to 8000/s on a quad core on Windows with a
    hello world app and the one click installer. If I remember correctly
    the app was a camping one, so not too bad.
  • The mingw builds of ruby are about 30% faster than the OCI build.
  • You’ll have to wait for the release of a decent service manager
    for windows, or roll your own monitor for your services.
  • Logging integration is faster to do unintegrated - analogger has
    been working great for us, but you’ll need the trunk build from the
    mercurial repo that has my patches.
  • Windows ruby socket implementations (and ruby herself) have low
    limits for the number of file descriptors. JRuby can help I think, we
    proxy out our services where appropriate.

I hate it when new dudes come in with their “stack” and bias
development based on their preferences withou considering what’s
already there. I’d rather walk away from this if Microsoft is really
their odds-on smart choice (i.e. I don’t need the money - I have
some personal relations that led me here). All I want is the company
to be successful.

JRuby could be a good option for you due to the double-stack style of
it. You could switch to a Java stack underneath if you need speed /
less magic / java programmers / whatever.

  1. Their MS SQL setup is relatively fine. Lots of wacky stored procs
    which bug me but mostly it’s fine. Am I crazy to try to run MS SQL
    against Ruby/ORM? Seems like there are some people doing it?

Personally I’d go PostgreSQL, if they really like their stored
procedures, that’s not going to play nice with most ruby ORMs. We have
a product that seals the entire DB behind stored procedures for
security reasons (extreme case). The front end for that app is rails,
and we have a single point of control that basically replaces
activerecord (acts as our virtual-ORM) with about 200 lines of ruby,
and circa 10k lines of (largely generated) plpgsql.

  1. If I do this, I’d plan to segment this site into two separate
    boxes and run the Ruby on a Linux box (and maybe outsource that
    management to a group like EngineYard). Then have the LB’s split
    traffic between the boxes based on url patterns. Again: crazy?
    unwise? Currently they’re at rackspace which knows poodle about Ruby/
    Mongrel afaict.

If you want managed services, then yeah, go for a ruby company. There
are a lot of differences to deploying ruby fast than some places are
used to or can envisage. As far as splitting traffic goes, eugh, well
just think about it carefully.

Context: The front-end site is not impossibly complex. But there is
“deep” integration with some backend admin processes which run a
large part of the business: some crm, PPC, finance/accounting, email
and billing: all partially implemented and built in hand coded ASP.
It’s a real tangle and it breaks all the time right now. I want to
get most of these processes out into third party systems with much
narrower points of contact between the production DB’s and the
specific admin services. This can only happen incrementally over
time. This is in addition to the front-end websystem migration.

Do one piece at a time. It sounds like the kind of scenario where
you’ll be needing to refactor the ASP in order to keep things stable
whilst migrating pieces of a large system.

Budgets for this work are not tiny but not enormous. Ditto
timeframe. Maybe $250k over 6-8 months.

That’s not a lot of money to train up people on ruby. Sure ruby is
simple from the outset, but some ideas don’t map so well. Expect 3 to
6 months developer learning time before they become seriously
productive in the language, unless you’re in a house full of rock
stars or polygots. Bad ruby code can sometimes be a lot worse than bad
code in other languages, but it’s really going to depend on the team
and how they take to the language.

Any tips or advice on taking on large migration projects such as
this would be appreciated. Advice such as “run!” is welcome also. I
realize there are no definite answers - I’m just looking for
experience or advice on how to reach conclusions here.

Do small chunks at a time. Don’t be afraid to refactor the current
system in order to be sure of stability. Seriously write your tests,
and be careful around the interface boundaries particularly. The
problem with tests across interface boundaries is that they’re either
coupled (and slow), or functional (and slow), or mocked (and therefore
one-sided - it’ll only test one side of an interface).

3

I love Ruby, open source, and all that. However, I work with C# and
Java in a Microsoft-only shop. Well, MS-only until I wrote a
distributed cache in Java on Linux - the revolution has begun!

Anyway, we have discussed this scenario at work, many times. How do we
replace the old VB6 COM and ASP code that hides in the corners with
something … better? The long and the short of it is, if they insist
on using Windows and/or SQL Server, especially if they insist on
stored procedures, you’re screwed. Short of replacing the OS with
Linux, the database with PostgreSQL or MySQL, and running with all the
good Ruby/UNIX tools you’ve mentioned, you’ll never get the app where
you want it to be or where they want it to be.

If you must use Windows, you might go with .NET 3.5 and an MVC
framework like Monorail or the not-yet-final ASP.NET MVC (there are
others, too). There are just enough Ruby-esque features in .NET 3.5,
like lambda expressions and extension methods, to keep me from poking
my eyes out with a spork.

If you get them to agree to Linux servers and a more Ruby-friendly DB,
I think you could do them a favor in the long run with lower costs and
easier maintenance. The OS and DB are the keys, in my opinion.

Good luck!

On Fri, May 16, 2008 at 2:34 PM, [email protected] wrote:

and go that route." But they want me to help. All I work in these days is
the guy who screws them deeper into the hole by really confusing their
Ruby/ORM? Seems like there are some people doing it?
partially implemented and built in hand coded ASP. It’s a real tangle and it
be appreciated. Advice such as “run!” is welcome also. I realize there are

Steve

p.s. Anyone who has possible interest in this project professionally can
also contact me directly off-list.

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

–
Cheers,

Kevin W.
http://bantamtech.com/

http://kevwil.com/

4

The problem I see in your project is: if you use ruby + mysql, changing
things they are used to, your projects needs to be perfect (no delays,
no
bugs, no nothing).
Now, if you go and use M$ things, your project don’t need to be perfect,
cause they are used to it and it’s problems.
So it’s up to you and your trust in your team. I’ll tell you that ruby +
mongrel can handle your load - it handles my geo application running
with
mapserver + postgis + oracle, so just rails + mysql are piece of cake
for
it :slight_smile:

Cheers,

filipe

On Fri, 16 May 2008, [email protected] wrote:

go that route." But they want me to help. All I work in these days is Ruby.
guy who screws them deeper into the hole by really confusing their stack.

the time right now. I want to get most of these processes out into third
conclusions here.

p.s. Anyone who has possible interest in this project professionally can also
contact me directly off-list.

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

filipe {
@ icewall.org
GPG 1024D/A6BA423E
http://filipe.icewall.org/
}

5

There are some interesting issues beyond just the engineering here.
The main ones have to do with how happy the company is with their site
and the technology it uses. If they are happy then maybe they will be
completely satisfied with moving to a .NET type site.

But then you need to look at Microsoft and what their priorities are,
Let’s face it, Vista hasn’t been particularly successful. You almost
get the feeling that Windows is a cash cow that M$ will milk for as
long as they can, but they are going to move onto other things…

You also need to talk to the engineering staff and see how they feel.
Maybe give a presentation on Rails and see how it goes over. If the
team is intrigued, or better yet, excited, then A Rails application
may be in their future. If it is met with cold stares then it would be
best to move on.

On Fri, May 16, 2008 at 4:34 PM, [email protected] wrote:

and go that route." But they want me to help. All I work in these days is
Ruby. And that’s all I want to work in. :slight_smile:

So my questions are like this:

  1. Can I in good conscience start migrating this company from IIS/ASP to
    Mongrel/Ruby/Merb/ORM (or something like that)? They have roughly 2-3M page
    views per month.

I would consider a nginx/Mongrel/Rails app. The learning curve is a
bit lower and Rails will handle a lot of extras that Merb will not.
(Yet.)

Also consider using Rubber and hosting on Amazon EC2. Rubber makes
using EC2 a piece of cake and it is easy to add instances when your
traffic increases. Plus it pushes the responsibility of managing
server farms and the connection to the internet to Amazon.

here). All I want is the company to be successful.
See above…

  1. Their MS SQL setup is relatively fine. Lots of wacky stored procs which
    bug me but mostly it’s fine. Am I crazy to try to run MS SQL against
    Ruby/ORM? Seems like there are some people doing it?

They may need “wacky” stored procs because IIS and ASP was not up to
the task. You need to evaluate what they were trying to accomplish
with those “wacky” stored procs and see if they aren’t standard
features in Rails.

  1. If I do this, I’d plan to segment this site into two separate boxes and
    run the Ruby on a Linux box (and maybe outsource that management to a group
    like EngineYard). Then have the LB’s split traffic between the boxes based
    on url patterns. Again: crazy? unwise? Currently they’re at rackspace which
    knows poodle about Ruby/Mongrel afaict.

No idea.

Budgets for this work are not tiny but not enormous. Ditto timeframe. Maybe
$250k over 6-8 months.

Rails, especially if you drink the Koolaid and follow the patterns, is
a RAPID web development platform. Since we don’t know what you are
trying to accomplish we can’t judge whether you’ll be able to meet
this budget and time frame. But with Rails you are more likely to
succeed than with any other platform.

Any tips or advice on taking on large migration projects such as this would
be appreciated. Advice such as “run!” is welcome also. I realize there are
no definite answers - I’m just looking for experience or advice on how to
reach conclusions here.

Sorry. I’ve been lucky to start new projects so I have no idea about
migrating an old app to Rails.

6

On Fri, 16 May 2008 13:34:13 -0700
[email protected] wrote:

Hey,

I’m working on a project, and mongrel may be part of the stack, but
I’ve got some more general questions and ideas I’m hoping to run by
this list. The people on this list have a broader knowledgebase and
more experience than any place else I know - plus a general
friendliness and willingness to help!

I’m glad I don’t post here that often. I’d ruin the friendliness. :slight_smile:

So my questions are like this:

  1. Can I in good conscience start migrating this company from IIS/ASP
    to Mongrel/Ruby/Merb/ORM (or something like that)? They have roughly
    2-3M page views per month.

Ultimately it’d depend on what the application does and what they need
from a rewrite. If they need maintainability, then I’d say no, you
should work to move them on to .NET so they don’t have to retool
everyone. From my experience, you’ll run into insane amounts of
politics unless all the developers from inside the company are the ones
coming forward asking for a transition to Ruby.

If they just want to modernize and are willing to relearn how
deployment works, management, and coding practices, then I’d say it’s
worth a shot. Any language will be better than what they have, but
honesly they should evaluate all the potential options using a rubric
to make sure they cover everything.

Now, if your question is, “Can Ruby handle this kind of load?” then you
didn’t ask the question right. Let’s break down your 3M pv/month
metric into requests/second (the actual measurement you need). That
comes to effectively 34 requests/second on average. Now, I’m betting
you have a peak time like everyone else, so if your peak is about 4x
this mean then that’s 120 request/second, which a decent Rails
application can handle.

Still, you should go look at your web server logs and get a better
understanding of the traffic patterns.

1.a) No matter how good they think I am, wouldn’t it be smarter to move
forward with M$ since that’s what they’ve got already? I don’t want to
be the guy who screws them deeper into the hole by really confusing
their stack.

Yes, that’s what I recommend. Switching to Rails for them would be a
nightmare. They’d have to ditch all of their existing knowledge,
technology, and platforms to go with the new stuff. It’d be a whole
rewrite with no prior knowledge. And I know the existing devs will
revolt, system admins as well. Actually, the system administrators
will just have to be replaced. It’s rare that an MCSE windows admin
has the chops to change over to unix system admin.

I hate it when new dudes come in with their “stack” and bias
development based on their preferences withou considering what’s
already there. I’d rather walk away from this if Microsoft is really
their odds-on smart choice (i.e. I don’t need the money - I have some
personal relations that led me here). All I want is the company to be
successful.

That’s why I recommend the rubric with a shoot out. Get the devs
involved in coming up with what’s important and finding it. It’s a
pretty simple process and should only take a week. You might be
surprised to find that there’s a tech out there that makes the business
just melt.

  1. Their MS SQL setup is relatively fine. Lots of wacky stored procs
    which bug me but mostly it’s fine. Am I crazy to try to run MS SQL
    against Ruby/ORM? Seems like there are some people doing it?

Yeah, you’re screwed. The dual problem of logic in the DB (which Rails
HATES) combined with the fact that this means the DBAs control the show
and they’re always morons. Sorry to say it that way, but I have yet to
meet a DBA who is worth two shits and can’t be replaced by a good
script or two.

Hell, I’ve replaced entire departments with a good script or two.

  1. If I do this, I’d plan to segment this site into two separate boxes
    and run the Ruby on a Linux box (and maybe outsource that management to
    a group like EngineYard). Then have the LB’s split traffic between the
    boxes based on url patterns. Again: crazy? unwise? Currently they’re at
    rackspace which knows poodle about Ruby/Mongrel afaict.

Yep, that’s possible but probably not the smartest way to do it. In
fact, if you have to do this splitting of requests then it’s a bad
decision. Be honest with yourself and ask if this is the simplest
thing they could do. It’s not, so go find a simpler migration path for
them and help them deal with that.

Any tips or advice on taking on large migration projects such as this
would be appreciated. Advice such as “run!” is welcome also. I realize
there are no definite answers - I’m just looking for experience or
advice on how to reach conclusions here.

I’d say run but help them realize why you’re not the right guy and who
to get with for the next steps. Honestly if you can help them spend a
week or two picking the right technology that fits their business and
existing setup then you’d be saving them so much money and could at
least make a bit of extra.

As for actually doing migration projects, I’d say my experience is they
do NOT work unless you have the following:

  1. A project champion who fires half the staff as an example of what
    will happen if anyone fucks up his project.
  2. A project lead who focuses on the quality of the code and can have
    anyone above or below him fired by #1 for fucking with his project.
  3. Developers who have knowledge of the past, and another bunch who
    don’t and aren’t afraid to call bullshit on the first batch. If you
    don’t have people going around saying “WTF!” every two minutes for at
    least a month then you have the wrong team entirely. If you don’t have
    existing people going “WTF!” at the new guy’s dumbfuck ideas as well
    then you don’t have the right team.
  4. NOT ANYONE WITH AN ‘A’ IN THEIR ACRONYM ON THE PROJECT. No
    Database Admins, System Admins, Software Architects, Business Analysts,
    Or MBAs should be near the fucker until it is ready to deploy.
  5. Embed the champion from #1 or someone with just as much on the line
    into the team every day for the whole work day and make them
    participate.
  6. Don’t move until you’ve done a full UI design on paper and had a
    grahpic artist work it all out. If #1 doesn’t have this before even
    talking to you he’s fucked. If this requirement is satisfied by the
    existing site then that rocks.

Other than that, these projects always suck, are always full of
politics, are always budget sucks, and you don’t really make as much
money as you think you do.

–
Zed A. Shaw

7

Thanks James, Zed, Kevin and Filipe. This is really valuable input and
I hope it helps others besides just me from the archives someday!

I talked with the CEO yesterday more about this yesterday. I think we
were able to “level” his expectations about what’s possible vs. what’s
reasonable, affordable and not too risky.

Surprisingly in this company, the DBA/IT and programmer guys would love
to abandon their current systems: stored procs, IIS and the whole MS
house of cards. I guess that shows you how bad the rig is there. The
system is so old that no one at the company understands how it really
works and would all just as soon bury it if they could.

But from all your input (which jibes with my concerns), I think it
would prohibitively expensive to try to shut down the existing
infrastructure and write a new white box system, especially outside
their current MS systems.

They can probably milk their existing infrastructure for some good
profits for a few more years, if they can get people who will actually
work and learn on it. (Any ideas on how to find folks who are competent
AND willing to work on old systems? Outsource maybe?!)

Thanks again for all the insights.

Steve

p.s. Maybe I’ll see some of you at Railsconf? I’m giving a talk on
Search strategies in Rails on Friday 5/30 - afternoon sessions. Stop by
and say hi!