Cluster Design

Hello everyone! My apologies if this appears twice… I had trouble
getting it through earlier.

A collegue and I are beginning development on a site that will
eventually need to very scalable (assuming our business model is a
good idea). My problem is that I’m not a sysadmin guru, and I’m not
terribly comfortable with the design of the hosting platform. We’re
going to start with two servers initially (for cost reasons), and I’m
considering the following solution:

Run lighty on one machine, as well as a set of FCGI listeners. Use the
second machine as a DB server and more FCGI listeners on as we scale.
Keeping the DB server separate seems like a good idea, since
eventually I will want that machine to be ONLY a DB server.

As we scale, I’m thinking that we will add more machines to run FCGI
listeners, completely separate the DB server from app serving, and
front the whole thing with lighty on a machine that acts only as a web
server. If it gets any bigger than that, then I guess I’ll have to
hire a real sysadmin. But that’s putting the cart a couple miles in
front of the horse.

My concerns are:

  1. Redundancy. We’re going to be storing and serving large number of
    media files as part of the site, and I’m considering using MogileFS a
    la Robot Co-Op to handle storage of these files. I’m concerned about
    hardware failure and the implications of loss of data on these media
    files. Would I be best using hardware redundancy such as RAID, or are
    there better solutions?

  2. Off-site backups. (DB/Media Files, etc) Are there services that you
    can contract with? I see rsync is often used as a backup solution, so
    do you just a server with a different host and use it purely for
    backup, or what?

  3. Is the design above workable? We have to start small, but I want to
    make sure I don’t make bad decisions that will haunt me as we scale.

  4. So that I don’t waste everyone’s time with replying to this, are
    there any good places to find info on this topic? People that have
    done it before, etc? I’ve got Eric H.'s posts regarding their
    setup, so I’m wondering if there’s any other good sources of info.

Whew… Thanks for reading this far, and thanks for any info you can
offer!

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs