Forum: Ruby Ruby game server woes

Hi,

The following is a bit of a brain dump, as I'm thinking about things I
just don't have enough knowledge to think about clearly.  I'm hoping in
the mess someone might have some insights that will help me.  I've been
reading and thinking and researching for the last couple of days to work
out what I should do.  I had an idea to make a game (client/server
model, small number of players at a time - 4 usually, 20 at most), and I
thought I would use ruby for the game server.  My thinking was, I can
develop quickly with ruby, and where I need speed I can write a c
extension.  Best of both worlds, and I'll be able to make something in a
much shorter amount of time!

To make matters slower, I'm using javascript+html5's canvas for the
client.  It's a bit slow, but there's plenty of optimisations I have not
yet made that will ease the burden, so I'm not stressed there.

To communicate between server and client I'm using websockets.  To
implement that on the ruby side, I'm using eventmachine (em) with
em-websocket.  That has been working fine, until recently - now, I need
more control over how cpu time is allocated between server game loop and
network.  As I understand it, em loops itself repeatedly to check for
new messages, and then act on those messages.  To get a game loop
running, I put in a periodic timer so that it would call the main game
loop repeatedly.  However, this design is inefficient.  I am noticing
wasted cpu cycles that I want to make use of.  Network traffic is low,
so it would suit me much better if I were to have the main game loop
occasionally check for new network packets, rather than the other way.
So that's fine, I expect with more work I can ditch em and implement
something myself.

The problem is, I'm now having second thoughts about ruby.  I thought
I'd come here to get some second thoughts, and maybe to help me with
information I don't have.  Grouped, my thoughts are (in no particular
order):

* Javascript - Looking at this chart
(http://benchmarksgame.alioth.debian.org/u32/benchmark.php) javascript
v8 runs a lot faster.  That leaves me wondering if I should have gone
with nodejs in the end.  I am not at all eager on the idea of using
javascript where I don't need to.  However, one advantage is I could put
some server code into the client for predictive behaviour to reduce
network traffic.  That would be a nice advantage, but not critical for
my project.  I know benchmarks should be taken with a grain of salt, but
the speed different is quite large.  And then it leaves me wondering,
should I just bite the bullet and rewrite the server in c/c++?  And then
I think, why not just use c extensions in ruby?  And then I wonder if c
extensions in ruby will be much slower than a raw c app, and then I'm
left at the limits of my experience and knowledge!

* fibers - Doing some experimenting, I haven't found a way to put the EM
loop into a fiber.  I had thought perhaps I could have the game loop as
the main part of the program, and have it occasionally resume the EM
fiber, which would then yield back to the main game loop after checking
for new messages.  Trouble is, this didn't quite work - that is, I'd
create the fiber enclosing the em loop and assign it to a variable, but
the variable would be nil, leaving me to suspect some code in EM itself
was destroying the fiber.  Maybe I need to look back into this.

* processes - is there a way to create a new process of a separate ruby
app, and communicate via those processes in an efficient way (ie, not
just through writing to disk and reading from disk, or a shared sql
database)?  If so, I could run EM in one process, and have it
communicate back information to the main ruby app.  I'm not aware of a
way to do this, though.  Threads are an obvious choice, but I'm using
MRI ruby, and it seems that the game ends up with the same speed issues
using threads.  If there's a way to launch a unique process and have
them communicate, then I can also take advantage of multiple cores.

So essentially, my question is this - given all these streams of
thoughts, are there any insights you can give me on how to approach
this?  I lack enough knowledge to make an informed choice.

Thanks for your time!

Subject: Ruby game server woes
  Date: Mon 24 Dec 12 08:45:33AM +0900

Quoting Na Na (lists@ruby-forum.com):

> So essentially, my question is this - given all these streams of
> thoughts, are there any insights you can give me on how to approach
> this?  I lack enough knowledge to make an informed choice.

Did you actually try to add a C extension? It is something that has to
be learned. Not exactly easy, but once you've learned the mechanism,
and when it is properly used, results are quite important.

You do not describe your algorithm. In my experience, a C extension is
best justified when I have bursts of heavy maths. In that case I
generally create a C-defined class, which is instantiated at the
beginning of the run. At instantiation I pass those variables that
allow me to properly allocate all the memory that will be needed. I
also often precompute tables: let's say that you need very often the
value of the tangent of a specific range of angle values: allocate
say, one million floats and fill them with the appropriate values.

I have noticed that I gain the most with math, but also when I have to
deal with very large loops: the Ruby loops are wonderful and
marvellously flexible, but when you are in search of performance, it
is worth the effort to move your big loops to C.

An example? Take this simple Ruby script, that computes the tangent
table I mentioned above:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

FVAL=0.0
TVAL=90.0
NVALS=1024*1024

step=(TVAL-FVAL)/(NVALS-1).to_f
data=Array::new(NVALS)

ang=FVAL
NVALS.times do |i|
  data[i]=Math::tan(ang)
  ang+=step
end

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

I execute it here on my laptop (i5-2410M CPU @ 2.30GHz) and I get
these times:

real  0m0.297s
user  0m0.289s
sys  0m0.007s

The same computation in C:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

#include <math.h>
#include <malloc.h>

#define FVAL 0.0
#define TVAL 90.0
#define NVALS (1024*1024)

void main(void)
{
  float 
step=(TVAL-FVAL)/(float)(NVALS-1),ang,*data=malloc(sizeof(float)*NVALS);
  int i;

  for(ang=FVAL,i=0;i<NVALS;i++,ang+=step)
    data[i]=tanf(ang);

  free(data);
}

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

when compiled as follows:

gcc -ffast-math -o ts ts.c -lm

gives these times:

real  0m0.052s
user  0m0.050s
sys  0m0.001s

That is, roughly 6 times faster. But these sorts of time saving are
obtained at some expense.

First of all, the mechanism has to be learned, and there is not much
matterial available. I suggest you carefully study the README.EXT file
that is distributed with MRI, but prepare yourself for a steep
learning curve.

Then, you have to bring back into the picture those aspects that Ruby
had ridden you of - I just mention memory allocations/pointers and
typed variables. In general, it is hairier to program in C.

I mix C and Ruby in almost all my projects, and I can guarantee the
advantages are concrete. But the skillset has to be built, and the
first steps may require some dedication. A solid C experience is also
very welcome.

Carlo

Thanks for your reply, it has been more food for thought.  To answer
your question:

> Did you actually try to add a C extension?

No, I have not yet.  I have tried inline c, but only for something
*very* basic.  However, I have used c/c++ before, but it was years ago.
So I'm familiar with pointers and needing to do memory management, and
types, etc.  So hopefully it will just be a case of re-familiarising
myself with forgotten details.  I have had a look through someone else's
c extension to see what's involved, and I can see it's a lot of work to
do that.  But hopefully it's worth the extra hassle for the development
time I've saved overall by using ruby for other parts (and there are
indeed other things ruby has let me develop very quickly!).

Is this why you write in ruby?  Speed of development, and then c
extensions when you need quicker computations?

Are there issues with c extensions in terms of multiple platforms?  Or
is it a case that ruby c extensions typically will compile well for mac,
linux, and windows without platform-specific code?

Finally, if you know - is it possible to write a c extension that itself
uses multiple threads to speed up computations if it's a task that can
be done in parallel?

Thanks for taking the time to reply.

Ey guys, I'll enter my head just a little to ask a little simple 
question related to this topic. I'm relative new to programming, I've 
started with Ruby building desktop apps. I'm wondering if can be used 
eventmachine to comunicate two apps in different machines, differents 
IPs, can be in the local network or internet, to be exactly: a card 
game(I'll do everything with Ruby because there's not too much 
calculations to do between loops), one machine is the server, the other 
user connects to that server. Can the connection and translation of 
information be done just with eventmachine? Thank's for your time, sorry 
if my head bother here.

Subject: Re: Ruby game server woes
  Date: Mon 24 Dec 12 11:01:11PM +0900

Quoting Na Na (lists@ruby-forum.com):

> Is this why you write in ruby?  Speed of development, and then c
> extensions when you need quicker computations?

Well, this is what I *obtain* from writing in Ruby+C. The why is a bit
more difficult to explain: there is a strange feeling of
satisfaction/accomplishment that I get when I see my Ruby code
running. It is a bit the opposite of what I experienced with
Java. Java somehow cut the cake, but there always was that aftertaste
of unrealized promise, of vague awkwardness.

C is one with Unix: they were invented together and still nothing
better has been found (both Apple and Google had to bow their
heads...). IMHO it is only with Ruby that the promises of OO have been
delivered. And the delivery comes together with the acceptance that
more ease in development and maintenance is paid with a tad less
performance.

The highway towards better performance is open and available when you
need it - once you master the way.

(oops - I may have waxed a bit too lyrical ;-)

> Are there issues with c extensions in terms of multiple platforms?  Or
> is it a case that ruby c extensions typically will compile well for mac,
> linux, and windows without platform-specific code?

Ahha...

I use only Linux.

But: if you can compile MRI (including the C extensions that are found
under the ext subdirectory), your extension too will most probably
compile. After all, when you write C extensions to Ruby you write
similar code as the one that makes up the interpreter - which is
itself written in ANSI C. You interface with the running interpreter
in the same way. The Ruby-related stuff is all neatly included in .h
files, which happen to be themselves included in MRI.

If your Ruby extension is just there for performance reasons (and thus
makes no use of exotic libraries), there is no reason why it won't
compile. wherever Ruby itself compiles. And if the exotic libraries
are already ported to the various architectures, the dirty work is
done for you already.

Then, with macosx this is certainly more direct (after all, it is
still UNIX, and you use GCC there as default). With windows, to my
knowledge there are a range of options to give to that unlucky
platform an appearance of shape ;-), so there have to be ways. But
there, I cannot be of help.

> Finally, if you know - is it possible to write a c extension that itself
> uses multiple threads to speed up computations if it's a task that can
> be done in parallel?

Of course! I do it regularly. For example, my audio playback toys
always have a background thread that makes sure samples are fed to the
DSP in time.

(the DSP stuff is of course highly unportable and quirky, thanks to
the time-revered ALSA library, but this is another topic ;-)

Once you are in C land, you can do anything you do in any other C
program or library. And you can bring in any other library, too. There
is a function that the Ruby interpreter calls when your library file
is loaded, called

void Init_<whatever your library is called>(void)

(which is the only function that your library *needs* to export),
where you create the class objects and their accessible methods.

Then you need to write the code for those methods. Among them, you
generally have the initialization call (the equivalent of Class::new),
which is called every time you create a new instance of your
class. If you want a thread to be associated to each instantiated
object, you just call pthread_create from within the body of that
function.

There arev some gotchas you have to be aware of (basically, if you
want the thread function to make use of ruby objects, you must make
sure the garbage collector does not destroy them in the meantime - but
this is advanced stuff).

It is all about trying a bit. Create your directory, put into it your
extconf.rb (which is just a Ruby script making use of the mkmf
package, and may be refreshingly simple) plus one or more .c and/or .h
files. Run

ruby extconf.rb

which will create the makefile, then run

make

whick will create your shared library.

make install

will move the library wherever your Ruby installation is supposed to
find it. At that point your Ruby scripts can do

require <whatever your library is called>

and you will be able to instantiate your C-based objects.

> Thanks for taking the time to reply.

these winter days are sooo dark... ;-)

Carlo

Subject: Re: Ruby game server woes
  Date: Tue 25 Dec 12 01:35:50AM +0900

Quoting Dami??n M. Gonz??lez (lists@ruby-forum.com):

> I'm wondering if can be used
> eventmachine to comunicate two apps in different machines, differents
> IPs, can be in the local network or internet, to be exactly: a card
> game(I'll do everything with Ruby because there's not too much
> calculations to do between loops), one machine is the server, the other
> user connects to that server. Can the connection and translation of
> information be done just with eventmachine?

If you want both the server and client to be Ruby-based, the DRb
(Distributed Ruby) library is quite nifty.

About a month ago I wrote to this list a mail that you may find
interesting. You find it here:

http://www.ruby-forum.com/topic/4407985#1084982

Carlo

Thank you Carlo, I'm reading it.