Message Queue Thread Safety

I have an application with some nasty race conditions I am trying to
pin-point.

The application starts with a 256 channel filterbank. Approx 45 outputs
are
connected to files, a handful to other blocks ending with a custom block
that puts messages in a GR message queue.

The application has a separate thread that is constantly de-queueing
items
produced by my custom blocks.

Occasionally, and randomly, one of the chains with my custom block
appears
to stall for an extended time and puts no messages into the queue. It
appears it may restart at a random later time, and in every case I have
observed restarting the application fixes the problem immediately. At
the
same time, another chain continues to send messages as I would expect.

Is having two or more producers and one consumer on one GR message queue
across multiple threads a bad idea? I’m not employing any additional
locking on the queue, due to wishful thinking and ignorance. I could
move
to a one consumer for each producer model and it would not significantly
change my application.

Ryan

On Sun, Feb 19, 2012 at 6:29 PM, Ryan P. [email protected] wrote:

Occasionally, and randomly, one of the chains with my custom block appears

Ryan

Ryan,
So the custom sinks are doing an insert_tail onto the msg_queue and the
other thread is doing a delete_head() to get the messages out?

Are they all going into the same queue, or does each sink have it’s own
queue to send messages to?

The insert_tail and delete_head both lock a mutex when called, so they
are
(or should be) thread safe. I’m trying to get a clearer picture of your
setup to see what else could be happening.

Tom

On Mon, Feb 20, 2012 at 10:28 AM, Tom R. [email protected] wrote:

Occasionally, and randomly, one of the chains with my custom block

The insert_tail and delete_head both lock a mutex when called, so they are
(or should be) thread safe. I’m trying to get a clearer picture of your
setup to see what else could be happening.

Tom, thank you for the reply.

You are correct with insert_tail from custom sink (2-5 in the
application,
1-10msgs/second each at peak) into ONE queue. A separate thread
continuously looping and delete_head()

On Tue, Feb 21, 2012 at 10:48 AM, Ryan P. [email protected] wrote:

are connected to files, a handful to other blocks ending with a custom
across multiple threads a bad idea? I’m not employing any additional

You are correct with insert_tail from custom sink (2-5 in the application,
1-10msgs/second each at peak) into ONE queue. A separate thread
continuously looping and delete_head()

Ryan,

There should be nothing inherently wrong with this approach.

Is there any other way to monitor the producer blocks to make sure they
are
actually getting data to send to the queue? Some output to stdout when
they
are active in any way?

Tom

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs