GNURadio retrictions

GNU Radio
1

Hello GNURadioers!

I’ve been working with GNURadio since about one year ago. I just figured
out the following limitation of GNURadio’s framework that I want to
diacuss
them to clarify whether I’m wrong or not:

1- For general blocks with multiple output ports, there is a problem
with
the amount of producing items when the general work routine just gives
you
the “maximum” available number of output items for all output ports, and
also set_output_multiple or other functions like set_alignment dealing
with
all output ports rather than individual ones. So if one wants to produce
items on the output ports with large different numbers, such as 100
items
on one port and 2000 items on the other one, the scheduler will fail to
manage this high “diverse” ports.

2- The scheduler jumps through different blocks before finishing the
work
routine’s job which causes some problems. For example, consider the
below
code which is placed somewhere in the work (general_work ):

“”"""""""""""""""
memcpy (out, temp, n_out_produced * sizeof(gr_complex);

produce(0, n_out_produced);

add_item_tag(0, nitems_written(0), … , … );
“”"""""""""""""""

So if we change the order of the code like this:

“”"""""""""""""""
memcpy (out, temp, n_out_produced * sizeof(gr_complex);

add_item_tag(0, nitems_written(0), … , … );

produce(0, n_out_produced);
“”"""""""""""""""

In the second case there is no problem with tagging process, although in
the first case the scheduler may jump out of this block right after
calling
produce function, so the position of the tagging which is "
nitems_written(0)" will change to the new value. Suppose in the next
call
to the first code, after calling produce function, the add_item_tag is
called and the item is tagged on “nitems_written(0)” position which is
the
same as the previous call. So these two consecutive tags are placed on
the
same position in the stream. Why is this happening? This is just because
before finishing the job of this work routine, scheduler jumps somewhere
else.

Please help me find the rational reason!

Thanks in advance,
Alizadeh

2

Please help me find the rational reason!

Because calling “produce” means, “I’m done, go ahead and take those
sample”.
Don’t call it until you are reall done …

GR is a multi-threaded applications, each work() function is executed
in different threads and as soon as you call produce(), other threads
are signalled that the samples are ready.

Cheers,

Sylvain

3

Forgot to CC: the list:

Hi Mostafa,

On 11/23/2014 10:39 PM, Mostafa A. wrote:

on one port and 2000 items on the other one, the scheduler will fail to
manage this high “diverse” ports.
Ok, I think this needs some explanation. What does “fail to manage”
imply exactly?
Generally, I think if you’re having something like a block that produces
2000 samples on one, and 100 on another, but does not have a fixed rate
between in- and output, this sounds you like, from a logical point of
view as much as by performance considerations, be better of using
messages.

But I think if you illustrate more closely, it’s likely we can figure
out where the problem lies.

2- The scheduler jumps through different blocks before finishing the work
routine’s job which causes some problems.
No, it doesn’t. At least the GNU Radio scheduler won’t interrupt a work
function
Ok, so here’s the problem I have with your phrasing: you use
“scheduler”, and I assume you mean the GNU Radio scheduler.

It’s the thread-per-block scheduler, which executes multiple blocks
/simultaneously/.

Underneath, of course, lies a runtime environment (boost threads), which
uses underlying threading libraries (on unixoids usually pthreads),
which use the underlying operating system’s multithreading routines
(aka. the OS’s scheduler) , which will execute blocks interleaved
(depending on system architecture, number of CPUs etc).

So if we change the order of the code like this:

“”"""""""""""""""
memcpy (out, temp, n_out_produced * sizeof(gr_complex);

add_item_tag(0, nitems_written(0), … , … );

produce(0, n_out_produced);
“”"""""""""""""""
Obviously, calling produce() increments the value that nitems_written()
should return! (nitems_written is “number of produced samples”)
So these are two very different programs, semantically.

In the second case there is no problem with tagging process, although in
the first case the scheduler may jump out of this block right after calling
produce function,
No, he doesn’t :wink: you’ve just written buggy code.

Greetings,
Marcus

4

On Mon, Nov 24, 2014 at 4:11 PM, Sylvain M. [email protected]
wrote:

Cheers,

Sylvain

Hi Sylvain,

I infer from what you said that what I see is true and logical! In
another
word, the nitems_written can be updated permanently in the middle of the
work’s routine. OK

But I still think that this is not logical to update the number of items
written before returning from work call.

Thank you,
Mostafa

5

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

As Sylvain said, produce() actually means “hey, I’m finished with
producing these items, so do what you want in another block (ie. in
another thread”, so obviously, the things like nitems_written must
reflect this.

On 11/25/2014 07:42 AM, Mostafa A. wrote:

other threads are signalled that the samples are ready.
I infer from what you said that what I see is true and logical! In
_______________________________________________ Discuss-gnuradio
mailing list [email protected]
Discuss-gnuradio Info Page

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJUdDu+AAoJEAFxB7BbsDrL/ZoH/jhy/Mqgfe6EyU2gA1fId11o
fi9CMUYGd0n0vbqgqHap4+qYbDeKdFNk81jp2KgOyqPC8Al/JPlVQJNX3QwFW+w3
O4a1hMqHbGIf9ktP45kpuQH/PgH1yMJD805IueJYjxBqrU2VDXTuGr9P8dqfbfz9
xCrbkHMDLIV0eJEiZ67d3U3ldFUj1XSYBsPRdpAs+hnsmsfmyUB6fJHGxucil1DW
o6a0FxU4dZvAfdcjSUTkz2Kq8N9NUekPcMfBeTDDZqMctdDUtv0+Ggr8cqZ++uHd
SqscQLEbS8XXnTrSEXb3PLij/wz2f+4AZFkPlzRDc/0R4jHiCdW/E1lpkWznwtQ=
=N8Wx
-----END PGP SIGNATURE-----

6

Hi Marcus,

On Mon, Nov 24, 2014 at 1:33 AM, Marcus M. [email protected]
wrote:

you
2000 samples on one, and 100 on another, but does not have a fixed rate
between in- and output, this sounds you like, from a logical point of
view as much as by performance considerations, be better of using messages.

That’s a good idea ( using messages instead)!

But I think if you illustrate more closely, it’s likely we can figure
out where the problem lies.

2- The scheduler jumps through different blocks before finishing the work
routine’s job which causes some problems.
No, it doesn’t. At least the GNU Radio scheduler won’t interrupt a work
function

I believe in what I see in GNURadio! It forsakes the work’s routine
somewhere in amidst of it!!

(depending on system architecture, number of CPUs etc).

OK, these information doesn’t solve the problem at least for me who
don’t
know anything about boost.

should return! (nitems_written is “number of produced samples”)
So these are two very different programs, semantically.

In the second case there is no problem with tagging process, although in
the first case the scheduler may jump out of this block right after
calling
produce function,
No, he doesn’t :wink: you’ve just written buggy code.

Greetings,

Marcus

I swear that I wrote a clean code :), I see what I see.

This is also so complicated, I forgot to add, that sometime calling the
produce() function before add_item_tag() increments the nitems_written()
and sometime doesn’t!!! why??

Best,
Mostafa

7

Hi Mostafa

On 11/25/2014 07:37 AM, Mostafa A. wrote:

No, it doesn’t. At least the GNU Radio scheduler won’t interrupt a work
function

I believe in what I see in GNURadio! It forsakes the work’s routine
somewhere in amidst of it!!
This would really surprise me. What does “forsake” mean here? Does the
computer then execute another work function, just to come back to the
current one later?

OK, these information doesn’t solve the problem at least for me who don’t
know anything about boost.
Well, the point here is that it’s not about knowing boost, it’s about
understanding multithreading on current operating systems, which is
quite a bit more theoretical. However, you’re describing problems that
neither Sylvain nor me can reproduce and I’m afraid you have some
misconception about how things in GR work together, and how a thread
works.

I swear that I wrote a clean code :), I see what I see.
The code you shared did two different things, though you claimed it
should do the same. Sylvain and me both pointed out that it should
not do the same and the code works as designed. I swear your code does
what it should by definition. You just wrote the wrong code if you want
something else :slight_smile:

This is also so complicated, I forgot to add, that sometime calling the
produce() function before add_item_tag() increments the nitems_written()
and sometime doesn’t!!! why??
produce() will always instantly increase the value nitems_written()
returns, because that is the only job of produce().
block::nitems_written() just calls block_detail::nitems_written() which
returns its underlying buffer::nitems_written() which is defined in
buffer.h as
uint64_t nitems_written() { return d_abs_write_offset; }
The delegate chain for produce is block::produce returning
block_detail::produce which calls buffer::update_write_pointer which is
defined in buffer.cc as
d_abs_write_offset += nitems;
There’s no ambiguity anywhere here, no if()s, not a single point where
we don’t increase the return value of nitems_written() by the amount of
samples you produce(). I don’t see how that “sometimes” could increase
and sometimes not. Pretty sure you’re not really seeing GNU Radio
misbehave.

Best regards,
Marcus

8

On 11/25/2014 07:37 AM, Mostafa A. wrote:

No, it doesn't. At least the GNU Radio scheduler won't interrupt a work
function

I believe in what I see in GNURadio! It forsakes the work’s routine
somewhere in amidst of it!!

It really doesn’t. Other works() are also running, though.

Underneath, of course, lies a runtime environment (boost threads), which
uses underlying threading libraries (on unixoids usually pthreads),
which use the underlying operating system's multithreading routines
(aka. the OS's scheduler) , which will execute blocks interleaved
(depending on system architecture, number of CPUs etc).

OK, these information doesn’t solve the problem at least for me who
don’t know anything about boost.

Boost implements fairly standard multithreading paradigms, which you
should make yourself acquainted with if you want to do multithreading
stuff.

However, you don’t need to do this. Our stuff is designed to work
thread-safely. If you stick to all the rules and build blocks the way
they should be written, you will only run into issues if there’s
actually a bug, which it doesn’t sound like reading your emails.

This is also so complicated, I forgot to add, that sometime calling the
produce() function before add_item_tag() increments the nitems_written()
and sometime doesn’t!!! why??

It doesn’t. See here:

void
block_detail::produce(int which_output, int how_many_items)
{
if(how_many_items > 0) {
d_output[which_output]->update_write_pointer(how_many_items);
d_produce_or |= how_many_items;
}
}

and here:

void
buffer::update_write_pointer(int nitems)
{
gr::thread::scoped_lock guard(*mutex());
d_write_index = index_add(d_write_index, nitems);
d_abs_write_offset += nitems; // == nitems_written()
}

My guess is your debugging and inspection methods are inadequate.

M