A criticism to GNURadio's scheduler!

Hi,
I worked on GNURadio for many hours. After all, I prepared my blocks in
c++. However, the source by which I produce random bits (items with
sizeof(char) ) doesn’t work properly! By properly I mean, I wanted
GNURadio
to lead me control how it’s going to call the source. It’s crazily
calling the random bit generator so many times.

I think this is because of the GNURadio’s strategy for executing blocks
to
achieve as maximum throughput as possible! So GNURadio translates it* to
call the source as much as possible*.(no matter what is the source, here
is
the random bit generator)

Am I right? If I am, what is the solution?

Best,
Mostafa

The “Throttle” block is required if you are not using any external
hardware:

http://gnuradio.org/doc/doxygen/classgr_1_1blocks_1_1throttle.html

Mike


Mike J. M0MIK BSc MIET
Ettus R. Technical Support
Email: [email protected]
Web: http://ettus.com

On Sun, Jun 1, 2014 at 9:30 AM, Mostafa A.
[email protected]

Mostafa,

I know you’ve been working many hours on this :slight_smile: so don’t worry, you’re
being heard.

Nevertheless, GNU Radio is surely not calling the asking the source
“crazily” to produce items.

GNU Radio is a streaming-into-buffers architecture, which means that the
runtime will ask a source to produce output when there is space in the
output buffer. I fail to see the problem with this, since your source
can just take as much time as it wants to finish a work call, can
produce less than noutput_items, and generally should just be as fast as
it could. Not a single one of the items it produced is going to waste!

It is good practice and done by every externally rate-limited source
to just block in work until enough items can be produced. If you need to
wait to get more random seed, well, then wait in work(). I admit, that
gets a little tricky when you want your seed to come in using a message,
because messages are not going to disturb your work due to design for
thread-safety.

But then again, before I start proposing wait-notify/condition
multithreading methods, I’d like to hear a bit about your source and why
being called often is a problem; that’s usually not the case, so chances
are we might help you find a solution if we understood what’s wrong with
your source :wink:

Greetings,
Marcus

Hi Mike,

No, the throttle isn’t a source! It just controls the flow of items in
an
specific time interval. I don’t want this! I want cognitively tell the
source produces the random bits after some special procedures have done
(a
message can do this for the source). But the scheduler crazily wants the
source to produce items! :slight_smile:

How could I deal with this problem?
please.

Best,

Hi Marcus,
By mentioning that I spent many hours I wanted to say I’m hardly stuck
on
this problem yet! :slight_smile:

You said the there is no problem that we wait on the source to produce
as
long as we want! But this is not true when I have other blocks waiting
for
processing!! In other word, when the source produces one packet, I want
this packet go down through blocks and amid of these blocks a message
must
tells the source to produce again.
If the source doesn’t produce anything after generating some packets,
the
scheduler will be locked on the source and does nothing with other
blocks!!

As you said, the wait will work for this source, but it’s also has the
same problem I mentioned.

Here is the summary of random bit generator:

#include <gnuradio/io_signature.h>

#include <stdlib.h>

#include <time.h>

#include

#include <unistd.h>

namespace gr {

namespace my_lte {

lte_random_bit_gen::sptr

lte_random_bit_gen::make(int packet_len) // packet_len: the size of
generating packet bits

{

return gnuradio::get_initial_sptr

  (new lte_random_bit_gen_impl(packet_len));

}

/*

  • The private constructor

*/

lte_random_bit_gen_impl::lte_random_bit_gen_impl(int packet_len)

: gr::sync_block("lte_random_bit_gen",

        gr::io_signature::make(0, 0, 0),

        gr::io_signature::make(1, 1, sizeof(char))),

  packet_length(packet_len),

{

  srand (time(NULL)); // randomizing the seed


  set_max_noutput_items(packet_length);

}

lte_random_bit_gen_impl::~lte_random_bit_gen_impl()

{

}

int

lte_random_bit_gen_impl::work(int noutput_items,

        gr_vector_const_void_star &input_items,

        gr_vector_void_star &output_items)

{

  char *out = (char *) output_items[0];


  int random_val, nout_written = 0;



      for (int i=0; i<packet_length; i++)

      {

          random_val = rand();

          bitset<8> b(random_val);


          if (b[0] == 0)

              *out = 1;

          else

              *out = 0;


          out++;

          nout_written++;

      }


             // add tag

      add_item_tag(0,

                   nitems_written(0),

                   pmt::string_to_symbol("length"),

                   pmt::from_long(packet_length));


      cout << "bit generator " << endl ;


  return nout_written;

}

On Sun, Jun 1, 2014 at 1:43 PM, Marcus M. [email protected]

On Sun, Jun 1, 2014 at 4:56 PM, Mostafa A.
[email protected]
wrote:

Hi Mike,

No, the throttle isn’t a source! It just controls the flow of items in an
specific time interval. I don’t want this! I want cognitively tell the
source produces the random bits after some special procedures have done (a
message can do this for the source). But the scheduler crazily wants the
source to produce items! :slight_smile:

Mostafa,

Why not you tell us what you are trying to accomplish using this
flowgraph
(not this block).
Give us a big picture of what you try to accomplish, let us figure out
the
implementation details for you.

Chances are there is no problem with gnuradio but with your knowledge
about
gnuradio.

Hi Marcus,

On Sun, Jun 1, 2014 at 2:44 PM, Marcus M. [email protected]
wrote:

Hi Mostafa,

You said the there is no problem that we wait on the source to produce
as long as we want! But this is not true when I have other blocks
waiting for processing!!

Not true. If your block can’t produce something, it’s not GNU Radio’s
fault your downstream blocks don’t have anything to consume…

This is not true because when I produce 1-3 packet of bits, the
scheduler
doesn’t go to the other blocks while there is items for them! The
scheduler
calls the random bit generator for about 10 times!!

In other word, when the source produces one packet, I want this
packet go down through blocks and amid of these blocks a message must
tells the source to produce again.

This sounds like a bad design choice, why would a downstream block need
to tell the source to produce again?
I think we might have tried to convince you not to do this before, but I
just can’t find the thread where you describe your problem.

I think you’re right. I would control something somewhere else. :slight_smile:

Hi Mostafa,

this is no good. If you don’t tell us why and how you would want to do
that, we can’t discuss if and how GNU Radio is lacking.
Really, the GNU Radio architecture wasn’t meant for this kind of flow
control, but that’s less of a shortcoming than a design choice, keeping
flow graph designs a little saner.
Neither throttle nor message strobe are controlled by another block, and
the latter doesn’t even produce items.

Greetings,
Marcus

Hi Mostafa,

You said the there is no problem that we wait on the source to produce
as long as we want! But this is not true when I have other blocks
waiting for processing!!

Not true. If your block can’t produce something, it’s not GNU Radio’s
fault your downstream blocks don’t have anything to consume…

In other word, when the source produces one packet, I want this
packet go down through blocks and amid of these blocks a message must
tells the source to produce again.

This sounds like a bad design choice, why would a downstream block need
to tell the source to produce again?
I think we might have tried to convince you not to do this before, but I
just can’t find the thread where you describe your problem.

If the source doesn’t produce anything after generating some packets,
the scheduler will be locked on the source and does nothing with other
blocks!!

Exactly. If your source doesn’t produce anything, your downstream blocks
can’t consume anything – there’s nothing GNU Radio could do about this.
I’m fairly certain that GNU Radio will not stall a flow graph if there
is more than zero unprocessed items in the buffers. Haven’t tested that,
though. Maybe someone else might comment on this.
So you have a logical deadlock, not a GNU Radio problem, if I understand
you correctly.

As you said, the wait will work for this source, but it’s also has
the same problem I mentioned.

Sorry, still don’t undertand which problem you are writing about :frowning:

I’ve read your code. There’s absolutely no reason why the standard
behaviour of GNU Radio filling the output buffer of your source wouldn’t
work – these are pseudorandom numbers,
and there’s absolutely no difference between generating e.g. 8192 at
once and producing them one at a time.

Greetings,
Marcus

Activecat,

I think change my topology to get rid of this things. :slight_smile:
However, I still believe in that the GNURadio fails in this scenario.
As we have throttle or message_strobe blocks, I need to have a source
which
generates items controlled by another block.

I’ll try to see this problem in another way to escape from it!

Hi Mostafa,
On 01.06.2014 19:23, Mostafa A. wrote:

This is not true because when I produce 1-3 packet of bits, the
scheduler doesn’t go to the other blocks while there is items for
them! The scheduler calls the random bit generator for about 10 times!!
I have tried that, and can disprove by example. My python code is under
[1], feel free to test and modify.
It generates timestamped output:

21.341874 src: sleeping
22.342275 src: producing 3 items, 0 total so far
22.342600 src: sleeping
22.342774 sink: got 3 items, total 0 so far

23.343531 src: producing 3 items, 3 total so far
23.343843 src: sleeping
23.344011 sink: got 3 items, total 3 so far

As you can see, the items that got produced are being processed the
same millisecond
the sources’ work finishes.

My source sleeps a second, then produces three items.
Maybe there’s a block downstream that needs bigger multiples of inputs
than single bytes?

Greetings,
Marcus

[1] GitHub - marcusmueller/gr-trickling_items: Short Demo to show that the GNU Radio scheduler is not broken in ways described in http://lists.gnu.org/archive/html/discuss-gnuradio/2014-06/msg00008.html ; a GRC demo
flowgraph is under examples/

On Mon, Jun 2, 2014 at 1:27 AM, Mostafa A.
[email protected]
wrote:

Activecat,

I think change my topology to get rid of this things. :slight_smile:
However, I still believe in that the GNURadio fails in this scenario.
As we have throttle or message_strobe blocks, I need to have a source
which generates items controlled by another block.

It is very common to have blocks generating items while they are
controlled
by another block.
These blocks are not “source block” but they could be sync block,
decimation block, interpolation block, general block etc.
Apparently you have seriously wrong concept at the very basic level,
from
the very beginning.

I’ll try to see this problem in another way to escape from it!

As you refuse to tell us what you try to accomplish with this flowgraph,
I
have to leave you alone and wish all the best to you.
Please don’t seek help when you are not yet ready to be helped.
Don’t abuse the forum please.