Stompserver & ActiveMessaging hacking

Rails
1

Hi,

== Context ==

I need a messaging system to have rails pushing tasks to processes in a
distributed environment. I need this to be small and hacker friendly, so
Stompserver and activemessaging looked good to me. There are some needs
that I can’t address as is with Stompserver and ActiveMessaging so I’m
currently hacking them for the following:

  • make Stompserver use a database for storing queues instead of relying
    on Madeleine. Madeleine is interesting but I’m not sure I understand the
    limitations of the journaling (I’ve only looked briefly at Madeleine’s
    code) + I need a way to inspect the queue backlogs to monitor the whole
    environment so a journal isn’t really what I’m after, a traditionnal
    database is far better for this.
  • as I want messages to stay in queue instead of being dispatched to
    subscribers if they are already busy, I’m delaying messages for
    subscribers with “ack: client” until they send an ack back (this helps
    with load distribution and can avoid stompserver being blocked for a
    while when a new subscriber comes).
  • this means that I have to modify ActiveMessaging to support
    acknowledgment (or support it more naturally).

== Current state ==

Unreadable :slight_smile: I’m a little hard on myself, but even if for example the
DB support in Stompserver works, I’ve lots of cleanups to do…

I began with the Madeleine -> ActiveRecord port which put pieces of
StompServer’s code all around on the floor :slight_smile: It runs now (though I’m
seeing warnings when I serialize the frames to a TEXT column :
Exception `TypeError’ at
/usr/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/active_record/base.rb:2068

  • can’t clone Fixnum).

I’ve implemented my modified ack/queuing behaviour in Stompserver and am
currently switching my attention to ActiveMessaging.

== Questions ==

Is anyone else interested in these modifications?
The more fair load distribution is completely independent of the DB
support so I could push a patch for Stompserver rather easily. Don’t
know yet for ActiveMessaging.
The DB support is more invasive that I though it would be, I began
thinking I could somehow allow users to choose a storage backend between
Madeleine and ActiveRecord at runtime but I let this behind for the
moment. If there’s interest in having a one-size fits all, I can look
into it.

Has someone a tip for the TypeError I get (described above)? It seems
harmless, but I don’t like having harmless warnings next to real error
messages :slight_smile:

== Thanks ==

Thanks for both StompServer and ActiveMessaging. It will probably save
me quite some development time (this is why I’m willing to contribute
back).

Best regards,

Lionel

2

On 7/8/07, Lionel B. [email protected] wrote:

Thanks for both StompServer and ActiveMessaging. It will probably save
me quite some development time (this is why I’m willing to contribute back).

I am pretty busy right now (with other things), but as long as you a
add tests and touch base with me before pushing a new gem welcome to
the project (I just added your rubyforge id). I am all for switching
to a database store on the persistent queues. I have encounters a
couple cases where Madeline did not work as well as I would have
hoped.

pth

3

You might take a look in branches/snacktime. It has some other
backing stores that work better, the file backing seemed to work the
best and have the best performance. I tried an sql store and it was
pathetic performance wise, can’t remember if I ever committed it.

Chris

4

Patrick H. wrote:

On 7/8/07, Lionel B. [email protected] wrote:

Thanks for both StompServer and ActiveMessaging. It will probably save
me quite some development time (this is why I’m willing to contribute
back).

I am pretty busy right now (with other things), but as long as you a
add tests and touch base with me before pushing a new gem welcome to
the project (I just added your rubyforge id).

Wow, ok. I’ll polish things until I’m not too ashamed of them. I’ll
probably have some details to work out with you before touching the svn
repository, I’ll contact you again when I’m ready.

I am all for switching
to a database store on the persistent queues. I have encounters a
couple cases where Madeline did not work as well as I would have
hoped.

Hum, I suspected it was not robust when I saw a Thread somewhere that
looked like it was linked to the journaling (not that having a separate
thread doing snapshots can’t be made to work correctly, just that it
seemed quite difficult to do so when I though about it).

Lionel.

5

snacktime wrote:

You might take a look in branches/snacktime.

Thanks for the tip.

It has some other
backing stores that work better, the file backing seemed to work the
best and have the best performance. I tried an sql store and it was
pathetic performance wise, can’t remember if I ever committed it.

Hum, I thought that the performance would suffer a lot (as I enforce
saving to DB as soon as the message comes in). In my case performance
isn’t a problem though (a message is triggering huge processings of
minutes or hours so limiting the throughput at say 10 messages/s is
completely ok for me) and a DB brings remote inspection of the queues
for free (which I need to monitor the whole system).
If we don’t enforce a realtime ‘message queued’ <-> ‘message dumped to
DB’ rule and allow writing messages to the database only if they stay in
queue/aren’t acknowledge for a configurable delay there is a huge
performance potential (think about all the messages that get through and
are processed in a matter of tens of milliseconds) to get at the expense
of reliability. So although it never will be the fastest backend,
hopefully it can meet some needs other than my own.

Hum, I’d better subscribe to the project’s devel list or I will bore
everyone to death here…

Best regards,

Lionel

6

Hi Lionel - glad to hear about such active adoption of stomp,
activemessaging, and stompserver - very cool.
I think I have some good news for you on ActiveMessaging, see below…

On 7/8/07, Lionel B. [email protected] wrote:

currently hacking them for the following:
I also recently checked in support for ReliableMessaging which supports
file
and database persistence of messages.
Are you aware of this gem?

http://trac.labnotes.org/cgi-bin/trac.cgi/wiki/Ruby/ReliableMessaging

It is now fully supported in ActiveMessaging as a simple, pure Ruby
broker
implementation, and pretty hackable as well.
Sounds like you are deep into StompServer, making headway and adding
good
stuff (which I am happy to see - glad StompServer is getting such
additional
attention) so don’t let me deter you, just wanted you to know there is
an
alternative.

  • make Stompserver use a database for storing queues instead of relying

on Madeleine. Madeleine is interesting but I’m not sure I understand the
limitations of the journaling (I’ve only looked briefly at Madeleine’s
code) + I need a way to inspect the queue backlogs to monitor the whole
environment so a journal isn’t really what I’m after, a traditionnal
database is far better for this.

I’ve looked at Madeleine, very interesting stuff, but I can understand
your
needs for better management/monitoring.
ReliableMessaging uses mysql or file based message storage, so it is
still
an alternative on this front.
One disad it has is that it is mysql specific, not using active record,
so
if you want to use a different db, you’re sol.

  • as I want messages to stay in queue instead of being dispatched to

subscribers if they are already busy, I’m delaying messages for
subscribers with “ack: client” until they send an ack back (this helps
with load distribution and can avoid stompserver being blocked for a
while when a new subscriber comes).

Totally agree, which is why I fixed the ActiveMessaging stomp adapter to
have ack mode support - checked it in last week :slight_smile:

  • this means that I have to modify ActiveMessaging to support
    acknowledgment (or support it more naturally).

Ummm not sure what you think is missing - please look at my recent
changes;
you should be happy.
Also you can check out the active messaging list and you’ll see my mail
about it.

ActiveMessaging now supports passing the :ack=>‘client’ mode in the
subscription, and has built in sending of acknowledgements after
sucessful
dispatch using the ‘ActiveMessaging::Adapters::Stomp.received’ method ,
which is standard now in ActiveMessgaing as the way to have any
connection
type acknowledge its messages (Amazon SQS uses this as well):

    def received message, headers={}
      #check to see if the ack mode for this subscription is auto or

client
# if the ack mode is client, send an ack
if (headers[:ack] === ‘client’)
ack_headers = message.headers.has_key?(:transaction) ?
message.headers[:transaction] : {}
ack message.headers[‘message-id’], ack_headers
end
end

All you need to do is add the :ack=>‘client’ header to the subscribes_to
call, and you’re all set. Let me know if there is something else you
think
is missing.

== Current state ==

  • can’t clone Fixnum).

I’ve implemented my modified ack/queuing behaviour in Stompserver and am
currently switching my attention to ActiveMessaging.

Again, please look at my recent changes, I really would be suprised if
there
is anything else regarding ack modes that needs to be added.

== Questions ==

Is anyone else interested in these modifications?
The more fair load distribution is completely independent of the DB
support so I could push a patch for Stompserver rather easily. Don’t
know yet for ActiveMessaging.

Always happy to get patches :slight_smile:

The DB support is more invasive that I though it would be, I began

Thanks for both StompServer and ActiveMessaging. It will probably save
me quite some development time (this is why I’m willing to contribute
back).

Glad you’re getting some use out of all this. Out of curiousity, can
you
talk about what you are using ActiveMessaging for?

In any case, let me know if the ack mode stuff, or anything else in
ActiveMessaging, isn’t cutting it for ya.

-Andrew K.
http://beginsinwonder.com

Best regards,

Lionel

–
Andrew K.