# GSource-based scheduling considered harmful

**URL:** <https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886>\
**Category:** Ruby-Gnome 2\
**Created:** [November 21, 2007, 1:36pm UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886 "2007-11-21T13:36:43Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 21, 2007, 1:36pm UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/1 "2007-11-21T13:36:43Z")

</div>

(Originally a reply to the message  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid), but I decided to break  
the thread because it’s a new topic that deserves attention.)

To simplify part of a previous message about GSources for  
scheduling Ruby threads:

_Using a custom GSource to dispatch Ruby threads as in r2707 causes  
the Ruby process to wake up every millisecond. This is bad._

Demonstrating this is simple, using strace:

strace -r ruby -r gtk2 -e Gtk.main

This process should sleep, because nothing is happening. It should  
only wake up when something interesting happens.

- When using the GSource, the process wakes up every millisecond.
- When using the original poll function, the process wakes up every  
100 milliseconds to call the empty\_timeout\_func. I do not know  
why the empty\_timeout\_func exists, except for programs which are  
using Ruby/GTK+ wrongly.
- When using the poll function and eliminating the empty\_timeout\_func,  
the process does not wake up. This is what should happen.

—\> Drake W.

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 23, 2007, 4:37am UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/2 "2007-11-23T04:37:04Z")

</div>

Hi,

In [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
“[ruby-gnome2-devel-en] GSource-based scheduling considered harmful”  
on Wed, 21 Nov 2007 06:02:55 -0600,  
Drake W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> This process should sleep, because nothing is happening. It should  
> only wake up when something interesting happens.

I confirmed that understand sleeping is better. But is there  
a problem the current implementation? The current  
implementation doesn’t cause busy loop. The process doesn’t  
consume any CPU time on my machine. (I used ‘top -p PID’)

I know that the current implementation should be improved  
but it will be done step by step.

And I don’t want to add a new API like main\_thread\_do\_async.

## Thanks,

kou

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 23, 2007, 9:58pm UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/3 "2007-11-23T21:58:43Z")

</div>

Quoth Kouhei S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid), on 2007-11-23 12:36:50 +0900:

> consume any CPU time on my machine. (I used ‘top -p PID’)  
> It’s bad semantics. Sleep should be sleep. But this has some  
> practical effects also:

- Normally, power management systems can put the CPU into deeper  
sleep modes when it’s idle for long enough.

- Normally, if a process is completely idle, its pages can be  
swapped out, thus freeing RAM until the process wakes up.

- Normally, if everything else on the system is idle enough, a  
process doing heavy work might expect to have almost all the CPU  
and cache and such.

Not all of these are true on every system. You can probably think of  
other reasons.

But other software is not expecting processes to continually wake up  
for no reason at all. Broken semantics have a way of propagating, and  
that is why I think correct semantics is very important.

> I know that the current implementation should be improved  
> but it will be done step by step.

Using a GSource makes things worse. The custom poll function can  
avoid the continual waking entirely, if you’re careful; in fact, when  
you remove the empty\_timeout\_func, it does exactly that. The design  
of the custom GSource API makes this impossible.

> And I don’t want to add a new API like main\_thread\_do\_async.

I’ll respond to this in a second message.

> kou

—\> Drake W.

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 23, 2007, 11:10pm UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/4 "2007-11-23T23:10:08Z")

</div>

Quoth Kouhei S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid), on 2007-11-23 12:36:50 +0900:

> And I don’t want to add a new API like main\_thread\_do\_async.

If adding sources to the main loop from Ruby code is safe, we can make  
sure to wake up the main loop ourselves (preferably inside  
Ruby-GNOME2, but not necessarily in C) after adding each source.

If this is true, that is good. That is the most compatible with old  
Ruby-GNOME2 programs, and does not need a new function.

But adding sources to the main loop from Ruby code may _not_ be safe,  
as a result of GLib not knowing how to do appropriate locking.

If this is true, old Ruby-GNOME2 programs that use idle blocks that  
way are in trouble no matter what we do.

Then either Ruby-GNOME2 needs to add a new function, or each program  
needs to add that function, and it would be better to put it in  
Ruby-GNOME2 core because it is very useful to have.

In all of these cases, you have to have some way to wake up the main  
loop. That is why the pipe is there.

Now, when I tried making this in Ruby before (with an IOChannel), it  
didn’t seem to work; I don’t know why. It looks like it’s possible,  
though. Some Ruby code is attached showing multiple styles of this:

- main-idle-wakeup just adds idle blocks from other threads, then  
uses a wakeup function to make sure the main loop wakes.

- main-iochannel-queue uses a queue and has one function from the  
queue called for each byte read from the pipe.

In both of these, if you feed blocks too fast, the UI gets  
unresponsive, but that’s hard to avoid anyway. These would still need  
a few tweaks before being production-quality. (Checking for the main  
thread hanging on its own pipe, for instance.)

Both of these function as described under a version of Ruby-GNOME2  
equivalent to r2704, minus the empty\_timeout\_funcs (to make sure that  
they do the wakeup correctly). In that case, they both sleep and do  
not wake up until there is something to do.

They also function as described under the current version of  
Ruby-GNOME2 in Debian unstable. In that case, of course, they do not  
sleep, because they cannot get rid of the empty\_timeout\_func.

> kou

—\> Drake W.

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 24, 2007, 5:10am UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/5 "2007-11-24T05:10:57Z")

</div>

Quoth Kouhei S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid), on 2007-11-24 09:09:39 +0900:

> > Using a GSource makes things worse. The custom poll function can  
> > avoid the continual waking entirely, if you’re careful; in fact, when  
> > you remove the empty\_timeout\_func, it does exactly that. The design  
> > of the custom GSource API makes this impossible.
> 
> The custom poll function doesn’t work on Windows.  
> Did you see the following codes in old  
> src/rbglib\_mainloop.c?

I am aware of this. Let me try describing the effects with a table:

```
                       Windows Unixy
                 +----------------------------------+
  GSource | periodic poll periodic poll |

```

custom poll func | doesn’t work sleeps properly |  
weird idle thing | periodic poll? (don’t know) |  
±---------------------------------+

(I am using “weird idle thing” to describe the previous behavior  
on Windows.)

As far as I know, no one has proposed a way to make the main loop  
sleep properly on Windows. That means that on Windows, we  
periodically wake up. I have no objection to using a GSource on  
Windows, because it’s not worse than the original way. When someone  
invents a better main loop for Windows, we can remove the GSource and  
use that instead.

Using a GSource on Unixy platforms makes things worse. The custom  
poll function can sleep correctly. The GSource cannot. This means  
that a GSource should not be used on Unixy systems. We can  
distinguish between the types of systems using the GLib platform  
symbols, just like before. The custom poll function will simply not  
be used on Windows, just like before.

Do you consider it better to use a GSource rather than the custom poll  
function on Unixy platforms for some reason? If so, why?

(I suspect that proper GLib main loop integration on Windows is very  
tricky without changes to the Ruby interpreter code itself, by the  
way.)

> kou

—\> Drake W.

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 24, 2007, 1:10am UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/6 "2007-11-24T01:10:04Z")

</div>

Hi,

In [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
“Re: [ruby-gnome2-devel-en] GSource-based scheduling considered  
harmful” on Fri, 23 Nov 2007 14:57:57 -0600,  
Drake W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > I confirmed that understand sleeping is better. But is there  
> > a problem the current implementation? The current  
> > implementation doesn’t cause busy loop. The process doesn’t  
> > consume any CPU time on my machine. (I used ‘top -p PID’)

> But other software is not expecting processes to continually wake up  
> for no reason at all. Broken semantics have a way of propagating, and  
> that is why I think correct semantics is very important.

I know that. So I said ‘understand sleeping is better’ and

> > I know that the current implementation should be improved  
> > but it will be done step by step.

.

> Using a GSource makes things worse. The custom poll function can  
> avoid the continual waking entirely, if you’re careful; in fact, when  
> you remove the empty\_timeout\_func, it does exactly that. The design  
> of the custom GSource API makes this impossible.

The custom poll function doesn’t work on Windows.  
Did you see the following codes in old  
src/rbglib\_mainloop.c?

#ifdef G\_OS\_WIN32

# undef USE\_POLL\_FUNC

#endif

## Thanks,

kou

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 24, 2007, 5:22am UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/7 "2007-11-24T05:22:03Z")

</div>

In [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
“Re: [ruby-gnome2-devel-en] GSource-based scheduling considered  
harmful” on Fri, 23 Nov 2007 22:10:00 -0600,  
Drake W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Do you consider it better to use a GSource rather than the custom poll  
> function on Unixy platforms for some reason? If so, why?

It’s simple.

I’ll improve this. Could you remember that?  
For now, it’s enough that the current implementation  
functionally works. Performance tunings and so on are the  
next works.

I have an idea to avoid polling but don’t ask me the  
detail. I’ll commit that I implement that.

## Thanks,

kou

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 24, 2007, 10:59am UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/8 "2007-11-24T10:59:11Z")

</div>

In [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
“Re: [ruby-gnome2-devel-en] GSource-based scheduling considered  
harmful” on Fri, 23 Nov 2007 22:38:46 -0600,  
Drake W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > I’ll improve this. Could you remember that?  
> > I have an idea to avoid polling but don’t ask me the  
> > detail. I’ll commit that I implement that.
> 
> Okay. I’ll wait for that, and then we’ll see what happens. Can you  
> reply to this thread when you commit it?

If you are interested in the repository, you can subscribe  
ruby-gnome2-cvs ML.

–  
kou

---

<div class="post-metadata">

**Author:** ![Drake\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/b19c9b/32.png) [@Drake\_W](https://www.ruby-forum.com/u/Drake_W)\
**Post date:** [November 24, 2007, 5:39am UTC](https://www.ruby-forum.com/t/gsource-based-scheduling-considered-harmful/120886/9 "2007-11-24T05:39:35Z")

</div>

Quoth Kouhei S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid), on 2007-11-24 13:21:18 +0900:

> > Do you consider it better to use a GSource rather than the custom poll  
> > function on Unixy platforms for some reason? If so, why?
> 
> It’s simple.

Simple and wrong, yes.

(I happen to think the custom poll function is clearer anyway,  
actually, but I don’t want to argue about that.)

> I’ll improve this. Could you remember that?  
> I have an idea to avoid polling but don’t ask me the  
> detail. I’ll commit that I implement that.

Okay. I’ll wait for that, and then we’ll see what happens. Can you  
reply to this thread when you commit it?

> For now, it’s enough that the current implementation  
> functionally works. Performance tunings and so on are the  
> next works.

I don’t consider this to be only a performance tuning. I consider  
failure to put the process to sleep to be semantically broken. Not  
broken enough to make things unusable, but still broken.

(The other issue is waking the process up afterwards, but that’s  
covered in the other subthread.)

> kou

—\> Drake W.
