ive got a server with 6gb of memory which i plan on setting up with
apache (handling static - w/ mod_security) proxying to a mongrel cluster
with mysql databases setup on seperate xen instances with master/slave
replication. my question is 1. any opinions as far as whether this would
be the optimal config. and 2. ideas as to how to distribute memory… im
thinking 3gb for the apache/mongrel cluster and 1.5gb for each of the
xen instances running the databases. and 3. what kind of loads
-guesstimate- this could reasonably handle… ballpark guess
I’m wondering - what is the advantage of running xen over just running
all your processes on one instance of an OS. You’re still using all the
same hardware resources, plus overhead of running Xen. Maybe this is
obvious, but I don’t get it. Can anyone explain this to me?
Not speaking for Joe here but…
The reason to go with Xen is one of hardware mobility.
Once the configuration is in place, if you need to add in more hardware,
moving the instance is a no-brainer operation.
I, for one, will never install on bare hardware ever again. Properly
configured, the Xen overhead is amazingly low.
–
– Tom M., Engine Y.
On Sep 27, 2006, at 12:48 PM, Jason N. wrote:
cluster
Splitting up a server with 6 gigs of ram using Xen is a great way to
make it easy for yourself to scale later. It also separates concerns
so they run independently of each other. It is true that since it is
all on the same piece of hardware that you still have a single point
of failure if your hardware crashes. But it makes it much easier to
scale to more servers later. You can actually move a running Xen
instance to another physical server while it is still running.
Also Xen is a serious performance leader in the virt server field.
You really only have about 5% perf loss compared to raw hrdware and
Xen VM’s. We have built our entire business around Xen based rails
hosting http://engineyard.com . And even when we sell dedicated
servers to people we are still running Xen even if they only have one
Xen VM that takes up the entire server. This allows for much easier
migration to different hardware in the future.
Xen is awesome and I think it is a good idea to split up your server
like you are thinking about. You haven’t given enough details about
the hardware(processor type et al) and your app for me to give you
any performance or capacity estimates. But you will be able to serve
quite a lot of traffic off of a box like you have.
I would recommend if you haven't bought the hardware yet that you
consider getting two servers with maybe 2 or 4 gigs of ram each. And
then split up your app across two servers. Run identical VM’x on each
server so you have one DB VM on each server and a few App server VM’s
on each server and two Web static file VM’s on each server. This will
give you much more reliability then one server.
Cheers-
-Ezra
Ezra Z. wrote:
On Sep 27, 2006, at 12:48 PM, Jason N. wrote:
cluster
Splitting up a server with 6 gigs of ram using Xen is a great way to
make it easy for yourself to scale later. It also separates concerns
so they run independently of each other. It is true that since it is
all on the same piece of hardware that you still have a single point
of failure if your hardware crashes. But it makes it much easier to
scale to more servers later. You can actually move a running Xen
instance to another physical server while it is still running.Also Xen is a serious performance leader in the virt server field.
You really only have about 5% perf loss compared to raw hrdware and
Xen VM’s. We have built our entire business around Xen based rails
hosting http://engineyard.com . And even when we sell dedicated
servers to people we are still running Xen even if they only have one
Xen VM that takes up the entire server. This allows for much easier
migration to different hardware in the future.Xen is awesome and I think it is a good idea to split up your server
like you are thinking about. You haven’t given enough details about
the hardware(processor type et al) and your app for me to give you
any performance or capacity estimates. But you will be able to serve
quite a lot of traffic off of a box like you have.I would recommend if you haven’t bought the hardware yet that you
consider getting two servers with maybe 2 or 4 gigs of ram each. And
then split up your app across two servers. Run identical VM’x on each
server so you have one DB VM on each server and a few App server VM’s
on each server and two Web static file VM’s on each server. This will
give you much more reliability then one server.Cheers-
-Ezra
you guys hit the nail on the head as far as the reasoning for the
databases running on xen. scalability/portablility. while it is true
there is still a single point of hardware failure, this is a new ibm box
we’re not anticipating having hardware failures with initially.(also why
we’re going with 1 box instead of 2, as well as 1 is max horsepower for
the price e.i cheaper, and they’re paying by th u) the company we’re
building this for is hoping to need to scale/ have the $$ to scale
shortly which speaks to the use of xen… we can get two more boxes and
create 2 xen instances on each -migrate live- and have 4 node array w/ 2
points of hardware failure and then use heartbeat to monitor/redirect
between the two; maybe go cluster instead of master/slave… or
something like that, thats down the road 
and then we can leave the
proxy/apps on this box… add another proxy/app box after that w/ a
hardware load balancer etc. the point is scalability/portablility of the
databases. and no data -per se- will be on the bare hardware, we can run
normal backups, create copies of the xen database vm’s, etc as
necessary; and for the proxy/app fw/mongrel cluster, that can run bare
-and not have to take 1 more performance hit from xen- 'cause i can just
keep copies of the config, same w/ rails(capistrano) and mongrel.
monitor/restart processes via script etc. etc. and when a new box gets
introduced -bingo- live vm migrate. easy.
and as far as the performance you guys hit it to, from what i’ve read
xen handles resources quite well -much better than vmware- with the way
it interacts w/ said resources and with the horsepower we’re running the
hit will be negligible compared to the benefits. along w/ the 6gb ram,
procs are twin dual core 3.0gb xeons. can’t get into the app, sorry…
the company is launching the app we’re building in a few weeks… which
leads me to another, kinda amateur, question(sorry, haven’t lurked this
forum myself long:); when we get them/ they launch the beta whats the
policy regarding “hey, we launched this beta built on rails, check it
out see what ya think, give some input etc.” in this forum…
generally… and any suggestions on any good forums for that -rails,
web 2.0-; we’ve got a bunch but the more the merrier:) don’t wanna spam.
nobody likes that.
thanks all,
joe