Ruby game server (concurrency and speed)

Hello there,

I’m writing a game server for flash based game. Currently there is MySQL

  • activerecord + activesupport + eventmachine.

At timed intervals (lets say 1 minute) actions should be done:

  • buildings construction status should be updated.
  • resource entries should be updated.
  • technologies should be updated.
  • unit construction status should be updated.
  • unit movement status should be updated.

All those operations do heavy-duty actions with SQL and objects so they
take a while.

For example with 500 000 resource entries updating them all takes about
27 seconds.

500 000 resource entries are needed for 33 000 players.

I imagine that other status updates will add up to this time
considerably.

While eventmachine allows you to spawn green ruby threads, it seems that
mysql gem is blocking.

So far I had these ideas:

  1. run things in linear fashion, ensure that everything runs pretty
    speedy (under 1-2 seconds) because incoming requests would be blocked by
    eventmachine. Buy another server if it doesn’t.
  2. run things in concurrent fashion, ensure that everything fits into
    period (1 minute). Run updaters as separate ruby processes. Pretty
    complex to code.

Perhaps there would be any other ideas that I’ve missed?

Thank you.

2010/2/3 ArtÅ«ras Å lajus [email protected]:

While eventmachine allows you to spawn green ruby threads, it seems that
mysql gem is blocking.

There is an asynchronous mysql gem, but I haven’t done much with it.

Link:
http://www.espace.com.eg/neverblock/blog/2008/08/28/neverblock-mysql-support/

-Jonathan N.

While eventmachine allows you to spawn green ruby threads, it seems that
mysql gem is blocking.

Yeah check out mysqlplus, and possibly

-r

El Miércoles, 3 de Febrero de 2010, Jonathan N. escribió:

2010/2/3 ArtÅ«ras Å lajus [email protected]:

While eventmachine allows you to spawn green ruby threads, it seems that
mysql gem is blocking.

There is an asynchronous mysql gem, but I haven’t done much with it.

Link:
http://www.espace.com.eg/neverblock/blog/2008/08/28/neverblock-mysql-suppo
rt/

I use mysqlplus instead of mysql gem. It’s like a replacement, but it
allows
async operations (non-blocking) so a thread performing an async DB query
doesn’t block other threads.

On 3 Feb 2010, at 14:52, Artūras Šlajus wrote:

  • unit construction status should be updated.
    I imagine that other status updates will add up to this time
    period (1 minute). Run updaters as separate ruby processes. Pretty
    complex to code.

Perhaps there would be any other ideas that I’ve missed?

A few observations based on my own experiences of writing real-time
systems. YMMV.

To start with there are many architectural assumptions implicit in the
scenario you’ve outlined that you should reconsider to see if they’re
justified, not least of which is that you’re apparently trying to do all
of this through a single server application. Even if you were doing this
in C/C++ I’d view that as a huge disaster waiting to happen.

The rule of thumb with any “real-time” system is to break it down into
as many discrete parts as you can and to have these running as separate
services so that individual bottlenecks can be identified and scaled out
to separate hardware when load demands.

Is it really necessary that all of these updates happen concurrently?
And if not how would you prioritise the updates? Is it possible to limit
the updates to just those players who are logged on and active? Or
perhaps a slightly larger group of which they’re a subset? Perhaps you
can even partition the game world into zones where updates to an
individual zone can be scheduled in isolation.

Examine your query logs and figure out how much overhead you’re paying
for each type of query you run. It could be that for some updates you
should abandon ActiveRecord and use embedded SQL where you can tailor
the query to your specific requirements.

You should also look at message queues for handling your updates. This
will allow you to move the heavy data processing work into separate
processes and make scaling much easier to get your head around. With a
bit of thought you can probably turn what appears to be an atomic update
process into a staged pipeline which more effectively uses the
underlying computing resources at your disposal.

Depending on how much control you have over your database configuration
you could also look at sharding (the zone-splitting already mentioned is
a simple example of this), data denormalisation, and decoupling the
database behind a series of simple web services.

Finally don’t treat writes as immediate updates of the live data as
that’s a very difficult contract to fulfil. Instead consider each update
a discrete ‘turn’. Read the wikipedia article on Shannon-Nyquist and
you’ll see that with the right duty cycle you can make any digital
system appear continuous :slight_smile:

Also, based on the architecture you’ve described I’m not sure
EventMachine is a good approach to your problems. It’s really designed
for handling i/o traffic where latency is less of a consideration than
the need to handle many concurrent requests in a mostly timely fashion.

I hope there are some ideas in there that help. You might also find
various useful bits of code in the presentations linked from my
signature.

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

Another idea my friend suggested was to only calculate actual values
when requested.

So instead of recalculating all the resource entries each minute only
recalc those, which are requested (and that is not very often).

It sounds pretty reasonable for me.

The only thing being on_complete callbacks for units, technologies and
buildings.

On 3 Feb 2010, at 17:18, Artūras Šlajus wrote:

Another idea my friend suggested was to only calculate actual values
when requested.

So instead of recalculating all the resource entries each minute only
recalc those, which are requested (and that is not very often).

It sounds pretty reasonable for me.

The only thing being on_complete callbacks for units, technologies and
buildings.

You can calculate when actions start the ETA for completion and then
have a timed queue which check their completion at that point in time
and then reschedule if necessary. These kinds of scheduling systems have
all kinds of applicability elsewhere as well.

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net
http://www.linkedin.com/in/eleanormchugh