"Run to completion" not working with message passing blocks

musicdenotation · April 29, 2014, 3:35pm

Hi all,

I want to use message passing blocks for operations where strict
synchronization is not necessary. However in flowgraphs using message
passing with “Run to completion” option turned on the program doesn’t
exit on last processed sample. Is this a bug?

I want to use message passing in my programs (mainly in
https://github.com/Jakotako/gr-gsm) but completion after processing of
all input data is also very important for me.

–
Piotr K.

Perper · April 29, 2014, 4:56pm

On Tue, Apr 29, 2014 at 9:33 AM, Perper [email protected] wrote:

–
Piotr K.

Yes, this is a known issue and we are working on it.

Tom

Perper · February 16, 2015, 5:35pm

On Sat, Feb 14, 2015 at 3:34 PM, Piotr K. [email protected] wrote:

Yes, this is a known issue and we are working on it.
I’m working on a simple gsm network monitor application based on gr-gsm
Piotr K.

The fix for this was pushed a couple of weeks ago and will make it in
the
3.7.7 release. We’re looking to see any side effects and might be able
to
back-port it for the next 3.7.6 maint release. Commit here:

https://github.com/gnuradio/gnuradio/commit/035b9d016dffefec890323bd0b24dbaf23aa9186

Tom

Perper · March 25, 2015, 11:14am

W dniu 16.02.2015 o 17:29, Tom R. pisze:

>     synchronization is not necessary. However in flowgraphs
>     --
In GNU Radio 3.7.6 this is still an issue. It's quite serious because
The fix for this was pushed a couple of weeks ago and will make it in
the 3.7.7 release. We’re looking to see any side effects and might be
able to back-port it for the next 3.7.6 maint release. Commit here:

https://github.com/gnuradio/gnuradio/commit/035b9d016dffefec890323bd0b24dbaf23aa9186

Tom

I’ve tied gnuradio compiled with and without this commit. I didn’t
observe difference - run to completion doesn’t seem to work with message
passing. I’ve attached the gnuradio-companion flowgraph that I’ve used
for the test.

Question to the list: does “Run to completion” feature work for you in
the attached example or in any other case of a flowgraph with message
passing?

Perper · March 25, 2015, 4:50pm

On Wed, Mar 25, 2015 at 3:11 AM, Piotr K. [email protected] wrote:

>
processing of
>
I'm working on a simple gsm network monitor application based on
I've attached vary simple flowgraph that demonstrates the problem.

https://github.com/gnuradio/gnuradio/commit/035b9d016dffefec890323bd0b24dbaf23aa9186

the attached example or in any other case of a flowgraph with message
passing?

–
Piotr

Ah, well there’s the problem. In GRC open up “Help”->“Types”. The blocks
you are using are the old Message Queue based blocks, not the new
asynchronous message passing interface. The commit that I was talking
about
was specific to the async message passing, so yeah, it’d have no bearing
on
this.

This is definitely confusing since they are both “message” based even
though it’s a completely different system. The old style used here will
likely be removed at some point, though it’s use persists in a few
places.
For your stuff, I’d avoid using this (anything with that dark grey port)
and use the new async messages (the light grey ports).

Tom

Perper · February 14, 2015, 9:35pm

W dniu 29.04.2014 o 16:20, Tom R. pisze:

I want to use message passing in my programs (mainly in

Tom

Hi Tom,

In GNU Radio 3.7.6 this is still an issue. It’s quite serious because
whole class of applications that would use message passing and rely on
the processing in the flowgraph to end after processing of chunk of the
signal (i.e. to change the flowgraph in known conditions) is not
possible.

I’m working on a simple gsm network monitor application based on gr-gsm
(https://github.com/ptrkrysik/gr-gsm) and have quite hard time because
of this issue.

Is this some general problem where the source of it is that GNU Radio
was meant primarily to process stream of samples? Or is it some simpler
matter?

I’ve attached vary simple flowgraph that demonstrates the problem.

Best Regards,
Piotr K.

Perper · March 26, 2015, 8:41am

W dniu 25.03.2015 o 16:48, Tom R. pisze:

>     > <mailto:[email protected] <mailto:[email protected]>
>     >     exit on last processed sample. Is this a bug?
>     >
>     the processing in the flowgraph to end after processing of chunk
>     Is this some general problem where the source of it is that
>
> Tom
passing?
This is definitely confusing since they are both “message” based even
though it’s a completely different system. The old style used here
will likely be removed at some point, though it’s use persists in a
few places. For your stuff, I’d avoid using this (anything with that
dark grey port) and use the new async messages (the light grey ports).

Tom

Tom,

The attached flowgraph was an example. I wanted to prepare something
small that uses only GNU Radio’s message blocks, to avoid situation
where I messed something in C++. The blocks that I selected were not
appropriate for the situation.

However in
https://github.com/ptrkrysik/gr-gsm/blob/master/apps/airprobe_file.grc
I’m using only message passing interface, but can’t get “Run to
completion” to work as it should.

Can you show an example of GRC flowgraph that uses message passing and
exit when the input signal is over? I would greatly appreciate that as
I’m a little bit stuck on this problem.

–
Piotr

Perper · April 1, 2015, 12:12pm

Hi Piotr,

nice to hear you got a step ahead!
so,

I’d have a blind guess:
Thread 16 might be the “surviving” part of a python-spawned Timer()
thread, which caused a message _post at another thread, which might be
something that hangs if that block’s thread no longer exists.

Can you switch to that thread:
thread 16
and then try to get a python backtrace [1]
py-bt
and maybe a simple C-style backtrace
bt

that might give you some information what is actually waiting on a
condition (which is what I guess from “pthread_cond_wait”).

Greetings,
Marcus

[1] for this to work, you might need to follow these instructions from
http://gnuradio.org/redmine/projects/gnuradio/wiki/TutorialsGDB:

… make sure that the python development package is installed
(|python-devel| on Redhatoids, |python2.7-dev| on Debianoids); for some
systems, you should append the content of
|/usr/share/doc/{python-devel,python2.7-dev}/gdbinit[.gz]| to your
|~/.gdbinit|, and re-start |gdb|.

Perper · April 1, 2015, 8:35am

W dniu 26.03.2015 o 08:39, Piotr K. pisze:

<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>     program doesn't
>     >
and rely on
>
>     Piotr K.
>
the attached example or in any other case of a flowgraph with message

have no bearing on this.

Can you show an example of GRC flowgraph that uses message passing and

Hi,

Yesterday funkylab adviced me to debug the issue using GDB. The
description of the method in his own words:
1. rerun cmake with -DCMAKE_BUILD_TYPE=RelWithDebInfo
2. rebuild and install gr-gsm
3. load the right stuff into gdb:
gdb --args python apps/airprobe_rtlsdr.py
gdb will load up
type run
the flow graph will start to execute
now, we expect it to shut down
instead, parts of it continue to run, but we don’t know
which, right?
so at some point you just hit ctrl+z
in gdb
now you’re at the gdb prompt again
info threads will show you the threads that are still
active. If you built GNU Radio with debug info too (see the cmake thing
above), you will have threads with names that look like blocks

I did that and what I obtained was:

16 Thread 0x7fffbdffb700 (LWP 13462) “python”
[email protected]@GLIBC_2.3.2 () at
…/nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
3 Thread 0x7fffe9720700 (LWP 13449) “gsm_clock_offs1”
[email protected]@GLIBC_2.3.2 () at
…/nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238

1 Thread 0x7ffff7fc7740 (LWP 13444) “python” 0x00007ffff78e8da3 in
select () at …/sysdeps/unix/syscall-template.S:81

One of the threads indicates that clock_offset_control block didn’t end
as it should. So I disabled it for a moment and “Run to completion”
started to work as it should.

I thought that the block’s thread doesn’t end because I’ve used timer
inside of it. I’ve removed almost everything from this python message
only block almost everything and what was left was only this:

class clock_offset_control(gr.basic_block):
def init(self, fc):
gr.basic_block.init(self,
name=“gsm_clock_offset_control”,
in_sig=[],
out_sig=[])
self.message_port_register_in(pmt.intern(“measurements”))

… but still - if any message enters the “measurements” input, the
block’s thread doesn’t end.

Conclusions: thanks to you guys (Tom R. and funkylab) I’ve finally
found the source of the problem in my flowgraph. I still don’t know why
is it that my python message only block doesn’t end as it should. Even
if I won’t find the answer for that, I can work around the problem by
implementing the block in C++ (I have many C++ message only blocks that
don’t cause problems with “Run to completion”).

Best Regards,
Piotr K.

Perper · April 4, 2015, 12:32pm

W dniu 03.04.2015 o 13:25, Piotr K. pisze:

…/nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

Greetings,

Best Regards,
Piotr K.

Hi all,

Reimplementing the block that uses message passing from Python to C++
did help.

The question remains - in more precisely formulated form:

Why “Run to completion” doesn’t work with Python blocks that use
message passing?

For anyone who wants to debug this - I’ve attached Python code
demonstrating the problem to my previous message.

Best Regards,
Piotr K.

Perper · April 3, 2015, 1:27pm

W dniu 02.04.2015 o 15:30, Piotr K. pisze:

3 Thread 0x7fffe9720700 (LWP 13449) “gsm_clock_offs1”
Can you switch to that thread:
Marcus
Marcus,

Discuss-gnuradio mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Marcus,

I did what you recommended but my knowledge of GDB doesn’t let me
interpret what I get to find the probably cause of the problem.

Instead of sending output of GDB I’m sending simple python script so
every interested person can see what doesn’t work. Inside the script
there is one message source and one message sink.
The source sends one message to the sink. At the end the program should
exit but it doesn’t.

Best Regards,
Piotr K.

Perper · April 2, 2015, 3:32pm

W dniu 01.04.2015 o 12:10, Marcus M. pisze:

[email protected]@GLIBC_2.3.2 () at
Can you switch to that thread:
Marcus

[1] for this to work, you might need to follow these instructions from
http://gnuradio.org/redmine/projects/gnuradio/wiki/TutorialsGDB:

… make sure that the python development package is installed
(|python-devel| on Redhatoids, |python2.7-dev| on Debianoids); for
some systems, you should append the content of
|/usr/share/doc/{python-devel,python2.7-dev}/gdbinit[.gz]| to your
|~/.gdbinit|, and re-start |gdb|.

Marcus,

Regarding the timer - this is what I thought at first, so I removed it
from the block. It didn’t help, so I removed almost everything leaving
only message input of the block. But the problem persisted.

I will try you advice with GDB and let everybody know of the result.

Best Regards,
Piotr

"Run to completion" not working with message passing blocks

I did that and what I obtained was:

I thought that the block’s thread doesn’t end because I’ve used timer inside of it. I’ve removed almost everything from this python message only block almost everything and what was left was only this:

class clock_offset_control(gr.basic_block): def init(self, fc): gr.basic_block.init(self, name=“gsm_clock_offset_control”, in_sig=[], out_sig=[]) self.message_port_register_in(pmt.intern(“measurements”))

I thought that the block’s thread doesn’t end because I’ve used timer
inside of it. I’ve removed almost everything from this python message
only block almost everything and what was left was only this:

class clock_offset_control(gr.basic_block):
def init(self, fc):
gr.basic_block.init(self,
name=“gsm_clock_offset_control”,
in_sig=[],
out_sig=[])
self.message_port_register_in(pmt.intern(“measurements”))