# Ruby on the Grid - Deploying on Virtual Infrastructure **URL:** \ **Category:** Deployment\ **Created:** [April 29, 2007, 3:25am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197 "2007-04-29T03:25:22Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [April 29, 2007, 3:25am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/1 "2007-04-29T03:25:22Z")
Has anyone attempted to deploy high transaction or heavy traffic ROR sites/apps in a scalable virtual environment using Linux? If so, would be great to hear some of the pros and cons. Thanks in advance for any insights. GN ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [April 29, 2007, 4:16am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/2 "2007-04-29T04:16:10Z")
I’m sure the Engine Y. guys will chime in on this issue, but from our experience, especially with the grid system we have in development, NFS and MySQL are the limitations of high traffic rails apps. The NFS issue can be solved with a the use of a true SAN instead of NFS, but that adds complexity and cost to the equation. MySQL is a bit of a bigger problem, but moving to using a MySQL cluster will handle quite a bit of load. Using two dedicated boxes, or getting VPS shares on multiple boxes allows you to get full redundancy in the event that hardware crashes. Just make sure that if you’re going the multiple “vps” route, that the vps units you are using are on different physical boxes. Doing so also lets you manage posting updates to your cluster without taking the whole site down. How much load are you anticipating? Will it be sustained traffics, or bursts from things like TV ads? – Jesse P. Blue Box Group, LLC p. +1.800.613.4305 x801 e. removed\_email\_address@domain.invalid ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [April 30, 2007, 5:46pm UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/3 "2007-04-30T17:46:00Z")
On Apr 28, 6:25 pm, Joe T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Has anyone attempted to deploy high transaction or heavy traffic ROR > sites/apps in a scalable virtual environment using Linux? We have about 100 customers so deployed. We’ve handled TechCrunchings, Diggs, and even a Fox News Live interview with 30 minutes notice. I highly recommend scalable virtual environments using Linux. 🙂 Once you understand the ability to turn performance knobs using virtualization, and migrate your environments to different hardware, the though of running non-virtualized isn’t very appealing anymore. – – Tom M., CTO – Engine Y., Ruby on Rails Hosting – Support, Scalability, Reliability – (866) 518-YARD (9273) ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [April 30, 2007, 9:19pm UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/4 "2007-04-30T21:19:25Z")
Out of curiosity, what kind of traffic have you seen from each of those events? – Jesse P. Blue Box Group, LLC p. +1.800.613.4305 x801 e. removed\_email\_address@domain.invalid ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 5:06am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/5 "2007-05-01T05:06:56Z")
removed\_email\_address@domain.invalid wrote: > On Apr 28, 6:25 pm, Joe T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Once you understand the ability to turn performance knobs using > virtualization, and migrate your environments to different hardware, > the though of running non-virtualized isn’t very appealing anymore. I’ll confess I know very little about this at the moment… can you point me in the right direction on where to start? Are we talking about starting with Xen? Would really appreciate a few points in the right direction… ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 3:20am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/6 "2007-05-01T03:20:14Z")
On Apr 30, 12:18 pm, Jesse P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Out of curiosity, what kind of traffic have you seen from each of > those events? Good question: 1. Techcrunch: 50 concurrent connections 2. Digg, category specific: 100 concurrent connections 3. Digg Main: 200-300 concurrent connections 4. Fox News: 2,000-3,000 concurrent ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 5:27am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/7 "2007-05-01T05:27:52Z")
removed\_email\_address@domain.invalid wrote: > On Apr 28, 6:25 pm, Joe T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Once you understand the ability to turn performance knobs using > virtualization, and migrate your environments to different hardware, > the though of running non-virtualized isn’t very appealing anymore. This pretty much hits the nail on the head. We’ve been doing some development on an ROR environment using AppLogic and the flexibility of virtual infrastructure opens up a lot of possibilities in terms of deployment options, configuration, scaling and migration. By breaking out each individual component of the hardware into a virtualized instance and running each part on a separate physical node, we’ve seen performance(so far) that rivals a physical solution with high availability to boot. Although there was definitely some “knob turning” involved. ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 7:45am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/8 "2007-05-01T07:45:30Z")
On Apr 30, 8:27 pm, removed\_email\_address@domain.invalid wrote: > removed\_email\_address@domain.invalid wrote: > > > On Apr 28, 6:25 pm, Joe T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > Once you understand the ability to turn performance knobs using > > virtualization, and migrate your environments to different hardware, > > the though of running non-virtualized isn’t very appealing anymore. > > This pretty much hits the nail on the head. We’ve been doing some > development on an ROR environment using AppLogic and the flexibility of > virtual infrastructure opens up a lot of possibilities in terms of > deployment options, configuration, scaling and migration. I’d so love to discuss out solution with you! Engine Y. sells complete clusters as well as hosting on our clusters… – – Tom M., CTO – Engine Y. ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 7:44am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/9 "2007-05-01T07:44:29Z")
On Apr 30, 8:06 pm, “Vince W.” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > removed\_email\_address@domain.invalid wrote: > > > On Apr 28, 6:25 pm, Joe T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > Once you understand the ability to turn performance knobs using > > virtualization, and migrate your environments to different hardware, > > the though of running non-virtualized isn’t very appealing anymore. > > I’ll confess I know very little about this at the moment… can you point > me in the right direction on where to start? Are we talking about > starting with Xen? Would really appreciate a few points in the right > direction… Start with Xen, add in SAN, LVM and the Red Hat Cluster Suite, LVS or hardware load balancers, etc… Or, pay someone else who has already spent a couple of man years’ of development time to do it for you. 🙂 – – Tom M., CTO – Engine Y. ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 7:53am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/10 "2007-05-01T07:53:30Z")
AppLogic is a very interesting platform and you’re smart to be looking at it. We’ve been playing with it for quite some time, and while I’m fond of the reliability it provides, the latest release we were working with caused me some concerns in terms of what happens when your management node dies (your whole cluster is rebooted). You loose your whole concept of reliability if you still have a single point of failure. We’ve been developing our clustered scalable rails deployment system and we’re about ready to bring it to market. The real “advantage” is that it allows some one with a smaller budget to get into a system that will grow with their site as their business grows. I’m fond of using virtualized servers split into pieces and then providing nodes on different physical machines for customers. These are great for the customer who isn’t going to be getting 6-10 million hits a day right away, but knows that’s where they’re headed. From our experience, MySQL is going to be your first bottle neck you run into, so that’s where I’d spend most of my time doing research, but virtualization does provide a nice level of scalability in regards to the individual app servers. – Jesse P. Blue Box Group, LLC p. +1.800.613.4305 x801 e. removed\_email\_address@domain.invalid ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 8:50am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/11 "2007-05-01T08:50:31Z")
Jesse P. wrote: > AppLogic is a very interesting platform and you’re smart to be > looking at it. We’ve been playing with it for quite some time, and > while I’m fond of the reliability it provides, the latest release we > were working with caused me some concerns in terms of what happens > when your management node dies (your whole cluster is rebooted). You > loose your whole concept of reliability if you still have a single > point of failure. > > We’ve been developing our clustered scalable rails deployment system > and we’re about ready to bring it to market. The real “advantage” is > that it allows some one with a smaller budget to get into a system > that will grow with their site as their business grows. I’m fond of > using virtualized servers split into pieces and then providing nodes > on different physical machines for customers. These are great for > the customer who isn’t going to be getting 6-10 million hits a day > right away, but knows that’s where they’re headed. > > From our experience, MySQL is going to be your first bottle neck you > run into, so that’s where I’d spend most of my time doing research, > but virtualization does provide a nice level of scalability in > regards to the individual app servers. From a user standpoint there is so much control over the environment with something like AppLogic (including virtual load balancers, firewalls, switches) and you can add infrastructure without additional cost. Scaling and migration are extremely simple as well. It’s also nice to not have to deal with colo and be able to handle a vast amount of infrastructure through a web browser (big kudos to the 3tera guys on that one). The management node is certainly something to think about but that doesn’t seem to take away reliability in any significant way (at least that we’ve seen). We have had a large grid running now for about 6 months and it hasn’t gone down once. Nor has any appliance running on it. From what we’ve heard, the next and future versions will mitigate or remove these failure points. MySQL…everyone’s fave. That is always a challenge. We’ve found that since AppLogic allows granular control of dedicated resource provisioning you can allocate and otpimize that specific appliance with a great deal of control (on a per megabyte of RAM and 1% of a processor to a full dedicated proc). We’re also looking into a heartbeat solution or MySQL cluster implementation to check the viability. I see you guys use that sort of setup (right??). Is that hard to get working right? Another ineresting test will be a cluster within the grid for some pretty extreme availability! ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 8:54am UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/12 "2007-05-01T08:54:33Z")
Out of curiosity, have you ran any internal tests to see the performance you can get with the Pound software based load balancer bundled with App Logic? We saw miserable performance numbers which is why we’ve switched to hardware based solutions. How much load are you running on this grid you’ve got running? – Jesse P. Blue Box Group, LLC p. +1.800.613.4305 x801 e. removed\_email\_address@domain.invalid ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 2, 2007, 7:47pm UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/13 "2007-05-02T19:47:57Z")
Was this using rails as the processing engine? – Jesse P. Blue Box Group, LLC p. +1.800.613.4305 x801 e. removed\_email\_address@domain.invalid ---
**Author:** ![gridninja](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/f08c70/32.png) [@gridninja](https://www.ruby-forum.com/u/gridninja)\ **Post date:** [May 1, 2007, 3:50pm UTC](https://www.ruby-forum.com/t/ruby-on-the-grid-deploying-on-virtual-infrastructure/93197/14 "2007-05-01T15:50:12Z")
Jesse P. wrote: > Out of curiosity, have you ran any internal tests to see the > performance you can get with the Pound software based load balancer > bundled with App Logic? We saw miserable performance numbers which > is why we’ve switched to hardware based solutions. > > How much load are you running on this grid you’ve got running? What kind of miserable performance numbers were you seeing? We haven’t used that appliance extensively but the instance we created that balanced 3 Apache servers didn’t show any unusual degradation. The entire grid environment is load balanced and redundant by default so additional load balancing is unnecessary for most applications. If you were using one of the original evaluation grids that 3tera was offering, that likely had a lot to do with any service impact you might have seen (those were running on seriously old hardware). We are using Dual Opteron 2200 with 4 GB of RAM minimum. That’s likely a big reason why we’re not seeing poor performance (still not sure why 3tera let people on those - we couldn’t use them and so we setup our own). That’s interesting that you mention poor performance with the Pound balancer, it’s been used with Ruby quite a bit. Since it runs through the gigabit backend network on the grid and has such a small footprint it’s quite suited for balancing Apache, Mongrel, Lighttpd. What were you trying to run on it? Did you get a load balanced clustered configured in AppLogic that worked? We have run up to 4000 simultaneous connections on a single distributed application (no Pound LB). That same app could have probably taken another 3000 connections. That was with a single (heavily resourced) MySQL instance as well.