Forum: Ruby CPU/Memory limiting

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Cc217306f0592921a59d9ec4344955d2?d=identicon&s=25 TongKe Xue (Guest)
on 2007-07-08 02:22
(Received via mailing list)
Hey Everyone,

  I'm working on a massive simulation project (like worldforge). I need
to
be able to allow users to script objects in the world. I am thinking of
ussing Ruby as my scripting language of choice. However:

  (1) I need to be able to limit the memory usage of the object.
  (2) I need to be able to limit the CPU cycle usage of the object.

  I.e. I don't want users to use my game to calculate digits of PI.

  Is there a way (or an existing implemntation of Ruby) to let me create
these light-weight tasklets/threadlets/processlets (there's too many
objects
to have a UNIX process for each object) such that I can limit the
memory/cpu
usage on each?


Thanks,
--TongKe
F48118fe74b0c7f6fd82a0ee422fa34e?d=identicon&s=25 snacktime (Guest)
on 2007-07-08 03:02
(Received via mailing list)
On 7/7/07, TongKe Xue <tongke@gmail.com> wrote:
>
>   Is there a way (or an existing implemntation of Ruby) to let me create
> these light-weight tasklets/threadlets/processlets (there's too many objects
> to have a UNIX process for each object) such that I can limit the memory/cpu
> usage on each?

Never used it before, but the Process class contains setrlimit which
might work.  Or you could send code fragments to be executed in a
different process with something like drb.

Chris
3bb23e7770680ea44a2d79e6d10daaed?d=identicon&s=25 M. Edward (Ed) Borasky (Guest)
on 2007-07-08 03:26
(Received via mailing list)
TongKe Xue wrote:
>
>
I don't think it's possible to use *any* Turing-complete scripting
language in such a manner, let alone Ruby. You'd be better off designing
your own language based on finite-state machines, for which you *can*
impose such control.
Cc217306f0592921a59d9ec4344955d2?d=identicon&s=25 TongKe Xue (Guest)
on 2007-07-08 03:56
(Received via mailing list)
I think there is a misunderstanding.
My goal is not to enforce these limits at compile time.

My goal is to enforece these limits at run time.

You exceed the memory limit? I don't allow you to allocate more memory.
You exceed your cpu slice? I swap you out and run another thread for a
while.
B9b5ff40232c1dfd61238c2a90467f84?d=identicon&s=25 Wayne E. Seguin (Guest)
on 2007-07-08 04:06
(Received via mailing list)
On Jul 07, 2007, at 21:55 , TongKe Xue wrote:

> I think there is a misunderstanding.
> My goal is not to enforce these limits at compile time.
>
> My goal is to enforece these limits at run time.
>
> You exceed the memory limit? I don't allow you to allocate more
> memory.
> You exceed your cpu slice? I swap you out and run another thread for a
> while.

You will have to do this external to the application via monitoring
utilities.

For Linux you might be able to use Monit (http://www.tildeslash.com/
monit/)

If you use Solaris, you could take advantage of its lovely SMF.

Good luck.
3bb23e7770680ea44a2d79e6d10daaed?d=identicon&s=25 M. Edward (Ed) Borasky (Guest)
on 2007-07-08 06:29
(Received via mailing list)
TongKe Xue wrote:
> I think there is a misunderstanding.
> My goal is not to enforce these limits at compile time.
>
> My goal is to enforece these limits at run time.
>
> You exceed the memory limit? I don't allow you to allocate more memory.
> You exceed your cpu slice? I swap you out and run another thread for a
> while.

As another poster pointed out, this is an operating-system-dependent
function, not one that belongs in a portable high-level language. So you
now need a system administrator to detect when a process is being a CPU
or memory hog and take some kind of action.

This is *exactly* what an OS does for a living! I don't know how to do
this on a Windows server, but on Linux and most other Unix-like systems,
there is a gizmo called "ulimit". This allows you to set a policy on how
big a process can get or how much total CPU time it can accumulate. The
bad news is that, as far as I know, when the process *does* violate its
ulimit, it is unceremoniously terminated and expunged from the system.
If that kind of behavior is compatible with your game, great. Otherwise,
you'll have to find a way to trap the ulimits and deal with them.

In any event, I think you're making this hard for yourself. There are
existing simulation game frameworks that work very well, and I think
there are even some with Ruby bindings. It sounds to me like you're
trying to reinvent some wheels rather than designing a user experience.
I'd recommend getting a good handle on the "whats" first before delving
much deeper into the "hows".
F05add2c1c1b613e8e318e9360eac5c2?d=identicon&s=25 Travis D Warlick Jr (Guest)
on 2007-07-08 12:32
(Received via mailing list)
TongKe Xue wrote:
>  I'm working on a massive simulation project (like worldforge). I need to
> be able to allow users to script objects in the world. I am thinking of
> ussing Ruby as my scripting language of choice. However:
>
>  (1) I need to be able to limit the memory usage of the object.
>  (2) I need to be able to limit the CPU cycle usage of the object.

Try using Process#setrlimit.  That being said, and aside from the fact
that I love Ruby -- I don't recommend it for this type of "massive"
endeavor.  Always try to choose the best language for your application,
not simply "the best language" because there isn't one.

As for what I would recommend: Not a scripting language.  For massive
projects, use a compiled language on the back-end and then you may
integrate it with Ruby on the front-end.

  *************************************
  *  Travis D Warlick, Jr
  *  Lead Developer
  *  Operis Systems, LLC
  *************************************
3bb23e7770680ea44a2d79e6d10daaed?d=identicon&s=25 M. Edward (Ed) Borasky (Guest)
on 2007-07-08 19:45
(Received via mailing list)
Travis D Warlick Jr wrote:
> endeavor.  Always try to choose the best language for your application,
>   *************************************
>
>
Yes, nearly all of the open-source MMORPG frameworks/games are
implemented in C/C++ and most of them provide some kind of
controlled/controllable scripting capability. I did find *one* that is
Ruby-based (MUES/FaerieMUD) but I was unable to find any evidence that
it is currently active). "daimonin" seemed to be the closest to "baked"
but it is masked in Gentoo because of some outstanding security issues,
so I didn't experiment with it at all.
Cc217306f0592921a59d9ec4344955d2?d=identicon&s=25 TongKe Xue (Guest)
on 2007-07-09 00:19
(Received via mailing list)
I can't use Unix processes because we have too many objects. And we
don't
wnat to pay the >= 8KB kernel memory for each process.

This is for simple stuff like:

dog:
  if dist(owner) > 5
   go_in_direction_of(owner)

but we have crap loads of these objects; and we want them to be user
scriptable (so C/C++ is a NO NO, we need an interpreted language), but
we
don't want people to do crap like:

while(1) {
  get some more memory;
}

or

calculate_ditits_of_pi(*)
7b4707f974af261f71943e1f2046c9ee?d=identicon&s=25 SonOfLilit (Guest)
on 2007-07-09 01:07
(Received via mailing list)
On 7/9/07, TongKe Xue <tongke@gmail.com> wrote:
> scriptable (so C/C++ is a NO NO, we need an interpreted language), but we
> On 7/8/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> > >
> > >   *  Travis D Warlick, Jr
> > but it is masked in Gentoo because of some outstanding security issues,
> > so I didn't experiment with it at all.
> >
> >
>

Smalltalk seems ideal.


Aur
F2d901ea9430646c6dd35a629bb3f119?d=identicon&s=25 Simen Edvardsen (Guest)
on 2007-07-09 02:45
(Received via mailing list)
On 7/9/07, TongKe Xue <tongke@gmail.com> wrote:
> scriptable (so C/C++ is a NO NO, we need an interpreted language), but we
> On 7/8/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> > >
> > >   *  Travis D Warlick, Jr
> > but it is masked in Gentoo because of some outstanding security issues,
> > so I didn't experiment with it at all.
> >
> >
>

Erlang, perhaps. It's got the ability to handle lots and lots of
lightweight threads, at least. Perhaps there's a way to limit the heap
of eachs process, since by design processes don't share data (well,
maybe they do under the hood.) I don't know about embedding it into an
application, though.
F48118fe74b0c7f6fd82a0ee422fa34e?d=identicon&s=25 snacktime (Guest)
on 2007-07-09 06:31
(Received via mailing list)
I wonder if any of the free ultima online implementations might have
something useful.  Been a while but I seem to remember they have a
rather extensive scripting system.

Chris
This topic is locked and can not be replied to.