Has anyone attempted to deploy high transaction or heavy traffic ROR
sites/apps in a scalable virtual environment using Linux?
If so, would be great to hear some of the pros and cons.
Thanks in advance for any insights.
GN
I’m sure the Engine Y. guys will chime in on this issue, but from
our experience, especially with the grid system we have in
development, NFS and MySQL are the limitations of high traffic rails
apps. The NFS issue can be solved with a the use of a true SAN
instead of NFS, but that adds complexity and cost to the equation.
MySQL is a bit of a bigger problem, but moving to using a MySQL
cluster will handle quite a bit of load.
Using two dedicated boxes, or getting VPS shares on multiple boxes
allows you to get full redundancy in the event that hardware
crashes. Just make sure that if you’re going the multiple “vps”
route, that the vps units you are using are on different physical
boxes. Doing so also lets you manage posting updates to your cluster
without taking the whole site down.
How much load are you anticipating? Will it be sustained traffics,
or bursts from things like TV ads?
–
Jesse P.
Blue Box Group, LLC
p. +1.800.613.4305 x801
e. [email protected]
On Apr 28, 6:25 pm, Joe T. [email protected] wrote:
Has anyone attempted to deploy high transaction or heavy traffic ROR
sites/apps in a scalable virtual environment using Linux?
We have about 100 customers so deployed.
We’ve handled TechCrunchings, Diggs, and even a Fox News Live
interview with 30 minutes notice.
I highly recommend scalable virtual environments using Linux. 
Once you understand the ability to turn performance knobs using
virtualization, and migrate your environments to different hardware,
the though of running non-virtualized isn’t very appealing anymore.
–
– Tom M., CTO
– Engine Y., Ruby on Rails Hosting
– Support, Scalability, Reliability
– (866) 518-YARD (9273)
Out of curiosity, what kind of traffic have you seen from each of
those events?
–
Jesse P.
Blue Box Group, LLC
p. +1.800.613.4305 x801
e. [email protected]
[email protected] wrote:
On Apr 28, 6:25 pm, Joe T. [email protected] wrote:
Once you understand the ability to turn performance knobs using
virtualization, and migrate your environments to different hardware,
the though of running non-virtualized isn’t very appealing anymore.
I’ll confess I know very little about this at the moment… can you point
me in the right direction on where to start? Are we talking about
starting with Xen? Would really appreciate a few points in the right
direction…
On Apr 30, 12:18 pm, Jesse P. [email protected] wrote:
Out of curiosity, what kind of traffic have you seen from each of
those events?
Good question:
- Techcrunch: 50 concurrent connections
- Digg, category specific: 100 concurrent connections
- Digg Main: 200-300 concurrent connections
- Fox News: 2,000-3,000 concurrent
[email protected] wrote:
On Apr 28, 6:25 pm, Joe T. [email protected] wrote:
Once you understand the ability to turn performance knobs using
virtualization, and migrate your environments to different hardware,
the though of running non-virtualized isn’t very appealing anymore.
This pretty much hits the nail on the head. We’ve been doing some
development on an ROR environment using AppLogic and the flexibility of
virtual infrastructure opens up a lot of possibilities in terms of
deployment options, configuration, scaling and migration.
By breaking out each individual component of the hardware into a
virtualized instance and running each part on a separate physical node,
we’ve seen performance(so far) that rivals a physical solution with high
availability to boot.
Although there was definitely some “knob turning” involved.
On Apr 30, 8:27 pm, [email protected] wrote:
[email protected] wrote:
On Apr 28, 6:25 pm, Joe T. [email protected] wrote:
Once you understand the ability to turn performance knobs using
virtualization, and migrate your environments to different hardware,
the though of running non-virtualized isn’t very appealing anymore.This pretty much hits the nail on the head. We’ve been doing some
development on an ROR environment using AppLogic and the flexibility of
virtual infrastructure opens up a lot of possibilities in terms of
deployment options, configuration, scaling and migration.
I’d so love to discuss out solution with you!
Engine Y. sells complete clusters as well as hosting on our
clusters…
–
– Tom M., CTO
– Engine Y.
On Apr 30, 8:06 pm, “Vince W.” [email protected]
wrote:
[email protected] wrote:
On Apr 28, 6:25 pm, Joe T. [email protected] wrote:
Once you understand the ability to turn performance knobs using
virtualization, and migrate your environments to different hardware,
the though of running non-virtualized isn’t very appealing anymore.I’ll confess I know very little about this at the moment… can you point
me in the right direction on where to start? Are we talking about
starting with Xen? Would really appreciate a few points in the right
direction…
Start with Xen, add in SAN, LVM and the Red Hat Cluster Suite, LVS or
hardware load balancers, etc…
Or, pay someone else who has already spent a couple of man years’ of
development time to do it for you.
–
– Tom M., CTO
– Engine Y.
AppLogic is a very interesting platform and you’re smart to be
looking at it. We’ve been playing with it for quite some time, and
while I’m fond of the reliability it provides, the latest release we
were working with caused me some concerns in terms of what happens
when your management node dies (your whole cluster is rebooted). You
loose your whole concept of reliability if you still have a single
point of failure.
We’ve been developing our clustered scalable rails deployment system
and we’re about ready to bring it to market. The real “advantage” is
that it allows some one with a smaller budget to get into a system
that will grow with their site as their business grows. I’m fond of
using virtualized servers split into pieces and then providing nodes
on different physical machines for customers. These are great for
the customer who isn’t going to be getting 6-10 million hits a day
right away, but knows that’s where they’re headed.
From our experience, MySQL is going to be your first bottle neck you
run into, so that’s where I’d spend most of my time doing research,
but virtualization does provide a nice level of scalability in
regards to the individual app servers.
–
Jesse P.
Blue Box Group, LLC
p. +1.800.613.4305 x801
e. [email protected]
Jesse P. wrote:
AppLogic is a very interesting platform and you’re smart to be
looking at it. We’ve been playing with it for quite some time, and
while I’m fond of the reliability it provides, the latest release we
were working with caused me some concerns in terms of what happens
when your management node dies (your whole cluster is rebooted). You
loose your whole concept of reliability if you still have a single
point of failure.We’ve been developing our clustered scalable rails deployment system
and we’re about ready to bring it to market. The real “advantage” is
that it allows some one with a smaller budget to get into a system
that will grow with their site as their business grows. I’m fond of
using virtualized servers split into pieces and then providing nodes
on different physical machines for customers. These are great for
the customer who isn’t going to be getting 6-10 million hits a day
right away, but knows that’s where they’re headed.From our experience, MySQL is going to be your first bottle neck you
run into, so that’s where I’d spend most of my time doing research,
but virtualization does provide a nice level of scalability in
regards to the individual app servers.
From a user standpoint there is so much control over the environment
with something like AppLogic (including virtual load balancers,
firewalls, switches) and you can add infrastructure without additional
cost. Scaling and migration are extremely simple as well. It’s also nice
to not have to deal with colo and be able to handle a vast amount of
infrastructure through a web browser (big kudos to the 3tera guys on
that one).
The management node is certainly something to think about but that
doesn’t seem to take away reliability in any significant way (at least
that we’ve seen). We have had a large grid running now for about 6
months and it hasn’t gone down once. Nor has any appliance running on
it. From what we’ve heard, the next and future versions will mitigate or
remove these failure points.
MySQL…everyone’s fave. That is always a challenge. We’ve found that
since AppLogic allows granular control of dedicated resource
provisioning you can allocate and otpimize that specific appliance with
a great deal of control (on a per megabyte of RAM and 1% of a processor
to a full dedicated proc). We’re also looking into a heartbeat solution
or MySQL cluster implementation to check the viability. I see you guys
use that sort of setup (right??). Is that hard to get working right?
Another ineresting test will be a cluster within the grid for some
pretty extreme availability!
Out of curiosity, have you ran any internal tests to see the
performance you can get with the Pound software based load balancer
bundled with App Logic? We saw miserable performance numbers which
is why we’ve switched to hardware based solutions.
How much load are you running on this grid you’ve got running?
–
Jesse P.
Blue Box Group, LLC
p. +1.800.613.4305 x801
e. [email protected]
Was this using rails as the processing engine?
–
Jesse P.
Blue Box Group, LLC
p. +1.800.613.4305 x801
e. [email protected]
Jesse P. wrote:
Out of curiosity, have you ran any internal tests to see the
performance you can get with the Pound software based load balancer
bundled with App Logic? We saw miserable performance numbers which
is why we’ve switched to hardware based solutions.How much load are you running on this grid you’ve got running?
What kind of miserable performance numbers were you seeing? We haven’t
used that appliance extensively but the instance we created that
balanced 3 Apache servers didn’t show any unusual degradation. The
entire grid environment is load balanced and redundant by default so
additional load balancing is unnecessary for most applications.
If you were using one of the original evaluation grids that 3tera was
offering, that likely had a lot to do with any service impact you might
have seen (those were running on seriously old hardware). We are using
Dual Opteron 2200 with 4 GB of RAM minimum. That’s likely a big reason
why we’re not seeing poor performance (still not sure why 3tera let
people on those - we couldn’t use them and so we setup our own).
That’s interesting that you mention poor performance with the Pound
balancer, it’s been used with Ruby quite a bit. Since it runs through
the gigabit backend network on the grid and has such a small footprint
it’s quite suited for balancing Apache, Mongrel, Lighttpd. What were you
trying to run on it? Did you get a load balanced clustered configured in
AppLogic that worked?
We have run up to 4000 simultaneous connections on a single distributed
application (no Pound LB). That same app could have probably taken
another 3000 connections. That was with a single (heavily resourced)
MySQL instance as well.