Need info on RoR for big e-commerce project

Rails
21

On 27/09/06, Daniel Smedegaard B. [email protected] wrote:

I tried looking in my history, and I think this might be the post I
was thinking about:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/211423

But this guy is also using ADO anyway. And I have no way near enough
knowledge about Ruby or this guy’s particular problem to know why his
code fails.

He’s opening a second statement against a connection while he already
has one open. That’s not supported by dbi drivers, and not used
within rails.

Okay, I gotta know now. Why exactly would one keep IIS instead of just
running the Webrick server? Does it scale badly, or…? Do I get any
kind of extra features by integrating with IIS that I’d otherwise miss
out on?

Webrick is really just a toy server. It’s great for development, but
no way near stable/secure enough for production.

In place of Webrick many people use Mongrel, which is extremely easy
to set up and use and excels at serving rails applications (though not
great at static content). It also comes with excellent documentation,
and a very helpful userbase. However, Ruby on Rails is not thread
safe, so when running rails, only a single concurrent request can be
handled. This is a rails issue - Mongrel itself can handle concurrent
requests.

To get around this, most deployments run many mongrel processes on
each server, each on a different port. To distribute requests between
each mongrel instance, some form of load-balancer or proxy is used
(there seem to be loads to choose from, though I don’t know what
exactly what your options are on windows). One common solution is to
use Apache; it may be possible to use IIS but doesn’t seem exactly
recommended :wink:

Tom

22

Tom W. wrote:

He’s opening a second statement against a connection while he already
has one open. That’s not supported by dbi drivers, and not used
within rails.

Okay, that floors the one post I’ve found with serious problems with
MSSQL integration. Perfect :slight_smile:

great at static content). It also comes with excellent documentation,
use Apache; it may be possible to use IIS but doesn’t seem exactly
recommended :wink:

Okay, wait a minute… This seems like quite an ugly hack? I assume
you’d keep the many instances running to avoid the delay from having to
start an instance for each request? Is there no plan to fix this soon?
If not threaded, then at least Rails should be able to fork itself? Or
am I completely off here…?
It seems that having severel Rails running concurrently would be hard
to administer, use excessive memory, and what if one instance crashes -
would some users then just hang in a queue waiting for it to come back
up while others get in one of the good queues? Okay, I admit, I’m more
confused than anything about this. Does a site like gbposters.com
really run a lot of Rails instances to handle their 5000+ hits per day?
Maybe I should just ask them :slight_smile:

Cheers,
Daniel

23

Tom W. wrote:

Thank you for that. That’s comforting! :slight_smile:

Tom

Cheers,
Daniel

24

confused than anything about this. Does a site like gbposters.com
really run a lot of Rails instances to handle their 5000+ hits per day?
Maybe I should just ask them :slight_smile:

Yes, basically.

Are you familiar with CGI? The web server takes the requests, starts
up an interpreter, and gives back the output every time. With
FastCGI and this mongrel setup, the web server (like IIS, Lighty, or
Apache) takes the request, talks to an already running instance of
mongrel or a Rails app on FastCGI through something like mod_proxy,
and then transmits the output. It’s much faster and more efficient
because the interpreter doesnt’ have to start every time because it’s
already running (I think I’m saying that right…I’m tired right now
so someone please correct me).

–Jeremy

P.S. - I can’t believe no one referred you to this:
http://www.agilewebdevelopment.com/rails-ecommerce

25

Okay, wait a minute… This seems like quite an ugly hack? I assume
you’d keep the many instances running to avoid the delay from having to
start an instance for each request? Is there no plan to fix this soon?
If not threaded, then at least Rails should be able to fork itself? Or
am I completely off here…?

It seems that having severel Rails running concurrently would be hard
to administer,

I’ve found using mongrel_cluster makes administration a breeze. Also,
as each instance is a fully-fledged http server it’s easy to monitor
their health as well as the health of the entire cluster, all with
standard monitoring tools.

use excessive memory,

Yup, this is certainly a downside. Rails/Mongrel isn’t really a
memory hog, but having several instances on a single server is
certaintly an inefficiency. That said, memory is cheap; my mongrel
instances seem to use around 20mb each, so even on an 80mb vps I can
easily run three.

and what if one instance crashes -
would some users then just hang in a queue waiting for it to come back
up while others get in one of the good queues?

If an cluster instance crashes, the load balancer will stop handing
requests to that instance until it returns, using the other instances
instead. This should mean that only a single request fails. If a
threaded application server crashed, all requests would fail until the
server restarts. So running a cluster of servers (even on a single
machine) can more stable than running just a single one.

There are other advantages to this deployment setup. It’s easy to add
more machines to the cluster as load increases, or to integrate other
web applications. Also, each part of the deployment is dedicated to a
single task (Apache to serve static content and balance, Mongrel to
serve dynamic content), and can be tested independently.

Okay, I admit, I’m more
confused than anything about this. Does a site like gbposters.com
really run a lot of Rails instances to handle their 5000+ hits per day?

Almost certainly.

Maybe I should just ask them :slight_smile:

Yeah :wink:

Tom

26

Hello,

This is - in other words - what I’m basically reading out of the posts
here. For my records - to use for argumentation about migrating the web
server if need be - could you elaborate about your experiences?

Running IIS with ROR with FastCGI mean :

  • Adding external util for rewriting request (ISAPIRewrite), luckully
    this tools is well writen, but add some complexity in the setup.
  • Patching some ROR Classes files with contributions
  • Using FASTCGI delivered as a standalone DLL with no support at all
  • Enjoy some registry editor
  • Tracking bugs in the request process, when sometime urls params are
    coming like controller/action*params=“foo” instead of ?, or
    strangeness like that.
  • And cherry, for handling more that one ROR site, you have to rename
    your fcgi files for second site into .fcgi2 and setup IIS and FastCGI
    for that.

After that, you still have 500 errors, try, stop, and so, and finally
it would work, but you dont know really what damned checkbox you’ve
activated in the setup screen of IIS. Or that’s because you give the
full control right to some internal account in the policies ?

AFIK, have this kind of setup running mean your Sysadmins are well
formed, such of knowledge you gain when you work daily and decently
with Apache (or else unix http server). So let me know somebody well
knowing Apache who wants use IIS.

Perhaps IIS is a nice httpd server, but it’s definitly not suited for
ROR because of lack of reverse_proxy, poor CGI support, fisherprice
logs, …

Holly Zed, we have Mongrel, but mongrel is nothing else than another
good HTTP server.

So using an http server behind IIS, assisted also with a balancer and
a contributed plugin for sending static content and an url rewriter…

In such case :

What do IIS exactly ?
Are your sysadmin able to track down performances issues or crash.
If the checkbox in the deep IIS MMC is checked again, will you babel
setup still work ?

In a ROI point of view Put they in some Apache consultant hands for 3
days.

Or also ask here : “Who run a decent public site with Rails and IIS,
even with FastCGI or Mongrel ?”

in FP Brooks, The Mythical Man-Month, Addison-Wesley, Reading, MA, 1975.

as I remember “Let the developers also design the architecture.” And
now architecture is also infrastructure…

Ciao’

27

On 9/27/06, Daniel Smedegaard B. [email protected] wrote:

Richard C. wrote:

Have a look at http://shopify.com, an e-commerce hosting provider, written in
Rails.

Thanks for the link, I just skimmed it, will check it out properly in a
little while. Is it just me, though, or are RoR apps generally more
nicely designed than “regular” web apps?

I am only touching base with Rails too, but I have managed to
accomplish some significant non-trivial results. Getting sophisticated
(in terms of complexity) views put together in Rails is shockingly
quick and simple, and its done right. In my experience, this
was done so quick that I had free time to make significant in-roads
into getting up to speed with CSS. Rails views are correctly
engineered practically from first authoring so they do not need
the levels of rework often seen in other frameworks.

Rails coders have far more free time left over for UI polishing
exercises. Rails also has one of the best AJAX integrations.

As a learning exercise, experimentation is very effective with
Rails, this conditions developers to be more confident about
trying new things, and will use more of the ‘pretty’ features.
There is also a significant after-market in Rails plugins.

My gut feeling and the herd opinion too is that Rails is perfectly scalable
beyond your needs. As long as you dont burn CPU frivolously, your individual
web servers will never be CPU pegged - you will max out your database
first in a properly architected system.

Thanks for saying that. I need as many opinions like this as humanly
possible to put my mind at rest :slight_smile:

But as a caveat, it doesn’t offer all of this ‘for free’ to scalability
novices. No more so than any other solution. You have to be familiar
with what is involved in achieving a scaling solution. But to be honest,
as a scalability novice myself, I would have more confidence engineering
a scalable rails solution than that of a different framework. Rails uses
a very simple, proven stack. .NET, EJB etc. seem to interfere with/
overcomplicate this too much making them much less simple to
architect into super-scaling solutions.

To get to Amazon levels, no web framework will suit your really, you will have
to write your own architecture from scratch.

I’ve heard this before. And I assume that when we talk about “Amazon
levels” it’s in regards to user load, and not functionality, right?

Correct. Any high-scaling solution implicitly has diminished user
functionality. There will be very little server-side CPU-intensive stuff
done on the behalf of the user, a user request will not be initiating
secondary/tertiary connections, and most of the work is involved
in taking database contents for a specific view and passing them
onto the user.

Big Websites actually have extra-ordinarily simple design architecture.
This simplicity is what allows them to scale. Its only when you
start introducing the scaling architecture that it starts to get wierd.
Even then the reasons for the scaling architecture are pretty
obvious.

28

Daniel:

One instance of mongrel can handle, on its own, anywhere from 5 requests
per
second up to 100 requests per second depending on the OS. Windows will
be
the low end, Linux will be the high end

5 requests per second = around 50,000 requests in 24 hours. You need
additional mongrel instances for your app if you start noticing
performance
problems. That’s “load balancing” and is common in web application
deployment. PHP uses FastCGI for this, and Rails can do that too…

but separate Mongrel instances using the same application means that you
can
actually deploy across physical servers as well to distribute the load
even
more.

I once heard that Alistapart.com uses only 4 dispatchers. They receive
lots
of traffic and I am sure they make good use of page and action caching.

One caution: This is Rails… new territory which has been discovered by
people who have been burned by previous experiences in other languages /
platforms. Things may not appear to make sense at first until you use
it.
If you look back at my posts from a year ago, you’ll see me asking
similar
questions :slight_smile:

29

Richard C. wrote:

into getting up to speed with CSS. Rails views are correctly
engineered practically from first authoring so they do not need
the levels of rework often seen in other frameworks.

Rails coders have far more free time left over for UI polishing
exercises. Rails also has one of the best AJAX integrations.

This is probably true, but I think it has more to do with Rails
attracting a more contemporary breed of programmer, like myself:
somebody who came from more of an interaction/graphic design background
and shifted over onto the server side and other more ‘serious’
programming concerns. People from these backgrounds tend not to have
learned staticly typed, compiled languages, so a powerful, intuitive
framework like Rails based on a beautiful, dynamic language like Ruby
is a natural fit. I think there is also something about the design of
Ruby in general that is more agreeable to the more creative types. See
why the lucky stiff as a shining example of the community in this
regard, though he is certainly not alone.

30

Okay, guys, this is gonna be a collective one with status for this
thing.

I’ve been kindly asked to leave research be until the control group has
reviewed the current document (which concludes that we should use TYPO3
with a note that RoR entered late in the process and may turn out to
beat TYPO3 as the system of choice), and either said that there’s no
way we can go with RoR, or that there’s still a possibility.

This means that I’ll now be reassigned to work on requirements, data
modelling, etc.

I just touched setting up Mongrel + IIS + ISAPIRewrite, and I read
about problems with zombie processes using FastCGI on Apache, and all
in all, Windows seems like such a crappy platform to do this on.

Good news is yes, we need to talk to an MS SQL Server, but it’s been
decided that this server will run on a dedicated machine, and thus
there’s a slight possibility that I can get permission to move the
currently running websites from the current IIS web server to a spare
machine, wipe Windows from the web server, install Debian, and set up
an Apache/Lighty/whatever/RoR combo to serve this thing. The machine is
a dual-core 3.2 GHz Xeon with 2GB mem and mirrored SCSI, and it’ll be
running ONLY this site. EXCLUSIVE database, which has its own machine.
I think it’d be safe to say that Ruby on Rails is now adequate beyond
any doubt to serve the maximum of maybe 100.000 page views on the
busiest day of the year. In my mind, there’s no way RoR should
represent any kind of bottleneck in this respect. It’ll be the MSSQL
machine, if any at all.

Okay, so while I’ve been told for the time being to use my day hours
here at work on forementioned work (and probably getting to know TYPO3
as well), I have received green light to use evenings to set up a Linux
environment on my personal laptop to do some testing with RoR, page
serving, clustering, whatever. If my tests are positive, then I can
probably sell this whole thing.

In this respect, if anyone can provide best-practices approaches
setting up the most stable, and preferably very scalable, Linux + RoR
combo, that would be wonderful.

Also, I’ve already received very convincing figures on the load
handling capabilities of gbposters.com, but if you know of other
figures that tell tales of RoR being a capable
“serve-many-serve-quickly” contender, I would love to see them.
Benchmarks comparing RoR to anything else in a web context (i.e. not
the “which language is quicker to render mandelbrots” type of deal)
would be useful. Basically anything you can use to sell the RoR
solution to the fat cats that make all the decisions.

For now, thank you so much for all your help with my many questions. I
hope this is not the end of my adventures with RoR. Well, I know it
won’t be cause I already have it planned for a different project (of my
own), I just hope that the next many months of developing this
particular website will be done in RoR :slight_smile:

Thanks again,
Daniel Buus

31

On 9/28/06, Daniel Smedegaard B. [email protected] wrote:

Okay, guys, this is gonna be a collective one with status for this
thing.

The machine is
a dual-core 3.2 GHz Xeon with 2GB mem and mirrored SCSI, and it’ll be
running ONLY this site. EXCLUSIVE database, which has its own machine.
I think it’d be safe to say that Ruby on Rails is now adequate beyond
any doubt to serve the maximum of maybe 100.000 page views on the
busiest day of the year. In my mind, there’s no way RoR should
represent any kind of bottleneck in this respect. It’ll be the MSSQL
machine, if any at all.

Hang on a bit here. There isn’t really a lot of hard facts on the
capability
of a properly engineered single-machine Rails install. Rails is designed
for scalability and that is achieved through the best practice of adding
more machines. It is more likely that the single Rails machine will be
the bottleneck than the SQL server (because Rails is designed to have
multiple Rails machines using a single database).

For single machine instances what you are looking for is performance
not scalability. Rails may still be enough for your needs, even without
heavy tweaking, and especially on *NIX boxes, but be under no illusions.

Okay, so while I’ve been told for the time being to use my day hours
here at work on forementioned work (and probably getting to know TYPO3
as well), I have received green light to use evenings to set up a Linux
environment on my personal laptop to do some testing with RoR, page
serving, clustering, whatever. If my tests are positive, then I can
probably sell this whole thing.

Your company gave you the green light to do the work for free? :confused:

With Rails the only way to prove it properly is to build the app that
was requested and have it ready by the end of the reqs gathering
stage. Don’t research, just build the app. Come back to your superiours
with a working app, instead of pro/con analysis.

Benchmarks comparing RoR to anything else in a web context (i.e. not
the “which language is quicker to render mandelbrots” type of deal)
would be useful. Basically anything you can use to sell the RoR
solution to the fat cats that make all the decisions.

The nice thing about Rails is how elegant it scales to best practices
solutions. You start small with a couple of Rails boxes and a database.
When your Rails boxes feel the heat, just add more. When your DB
starts to feel the heat, add another one and partition your custom.

32

2006/9/28, Daniel Smedegaard B. [email protected]:

In this respect, if anyone can provide best-practices approaches
setting up the most stable, and preferably very scalable, Linux + RoR
combo, that would be wonderful.

there is a series of blog entries by the developers of eins.de, a
german
community site server around 1M page views a day:
http://poocs.net/2006/03/13/the-adventures-of-scaling-stage-1

and, as a base of that, in order to set up an apache
2.2/mongrel/rails/capistrano
stack on linux, i find this blog article very useful:
http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/

again for the point of setting up such a stack: it sounds harder than it
is.
with some time to start and willing to read a few docs, such a wonderful
stack can be set up in no time. and if you’ve done it one, the next
times
(deploying the app on another server, or deploying another new project)
will
be even faster. it then just works.

Also, I’ve already received very convincing figures on the load

handling capabilities of gbposters.com, but if you know of other
figures that tell tales of RoR being a capable
“serve-many-serve-quickly” contender, I would love to see them.
Benchmarks comparing RoR to anything else in a web context (i.e. not
the “which language is quicker to render mandelbrots” type of deal)
would be useful. Basically anything you can use to sell the RoR
solution to the fat cats that make all the decisions.

IMHO, ruby/rails may be not the fastest of all language/framework
combinations, but as always said: hardware’s cheap, developers are not.
so
buying one ore two more servers will be less expensive as hiring one ore
two
more developers.and, what i consider the killer-argument: rails is fun,
rails makes (at least me) happy. and a happy programmer is a more
productive
programmer. (at least thats right for me!)

good luck, rock’em :slight_smile:

–
Michael S. [email protected]

www.siebert-wd.de - Gedanken lesen
www.stellar-legends.de - Weltraum-Browsergame im Alpha-Stadium

33

Okay everybody, here’s the final status for this project.

The bad news for me is that the control group down-thumbed Rails,
argumenting a too short history, too much risk, and too big a change to
migrate the server to Linux. We’re continuing with TYPO3, which I
really hope will turn out to be adequate and fast to develop with. The
learning curve is a bit scary, but at least I know what I’ll be doing
the next few weeks :slight_smile:

Thanks to every one who helped me. And even if this project will go
with an old-school CMF strategy, I’m still hanging in - I joined the
Copenhagen Ruby Brigade, and we’re gathering on october 21st it seems.
I’ll find a place for Rails yet.

Cheers everyone,
Daniel

Cheers,
Daniel