What if noutput_itmes is too short?

I am trying to remove preamble and start frame delimiter from a
frame
and only the payload will be passed to the next block.

While in debugging, ninput_items is 7742, and noutput_items is 4096,
for instance. An offset where the frame starts is 790.

Simple calculation gives me that 7742 - 790 = 6952 samples should be
passed
to the next block. But output space given is only 4096. Thus, I can only
pass 4096 samples and 6952 - 4096 = 2856 samples will be discarded. But,
I
need them to be passed, too.

Even worse, a stream of samples are coming continuously from a previous
block.

Will it be helpful if I have some very long buffer in a block? I don’t
think it works since noutput_items is always smaller than
ninput_items
every time the block runs general_work(). In this case, I think the
buffer will be overflowed.

The link below is the log that shows

i/ninput_itmes and o/noutput_items,

where i and o are input and output indices. When ninput_itmes and
noutput_items change, it means a new general_work is called.

Log (it’s quite long…): https://bitbucket.org/snippets/gsongsong/By8x

Regards,
Jeon.

After posting the previous thread, I’ve searched APIs from GNU Radio
doxygen.

Several candidates have been found.

  1. produce()
  2. set_min_noutput_items()
  3. set_min_output_buffer()
  4. expand_minmax_buffer()

I think I can combine produce() and set_min_noutput_items() to
enlarge the noutput_items

If I get wrong, please let me know.

Regards,
Jeon.

2015년 5월 29일 (금) 오후 8:47, Jeon [email protected]님이 작성:

Thank you for your reply, Marcus.

Now I’m getting a little bit more.

As I understand, if I consume(0, n), where n is less than ninput_items,
then only n samples at the front of input_items[] are consumed and
unconsumed part remains. And then, at the next call of general_work(),
incoming samples are appended at the end of input_item[]. Is this right?

If it is right, I can get a single whole payload by not consuming input
samples after some time passed…

However, I am still doubtful that input_items[] will not be appended as
I
think since output buffer at the previous block isn’t cleared enough as
consume() is not called.

Well… Does forecast() which you’ve mentioned make the previous block
can send more streams and the current block can received them? So far, I
understand forecast() acts as just a blocker unless enough samples are
given.

In the developing stage, I am using fixed length payload. But, I want to
make it work with arbitrary length (but not too long).

Regards,
Jeon.

On Fri, May 29, 2015 at 9:52 PM Marcus Müller [email protected]

Hi Jeon,

On 05/29/2015 01:46 PM, Jeon wrote:

I am trying to remove preamble and start frame delimiter from a
frame and only the payload will be passed to the next block.

While in debugging, ninput_items is 7742, and noutput_items is
4096, for instance. An offset where the frame starts is 790.

Simple calculation gives me that 7742 - 790 = 6952 samples should be
passed to the next block. But output space given is only 4096. Thus, I
can only pass 4096 samples and 6952 - 4096 = 2856 samples will be
discarded. But, I need them to be passed, too.

There’s no reason not to pass through your payload in parts, if you
implement a state machine with a counter for remaining items to pass
through in your block.
However, there’s an advantage to waiting for the whole payload to be on
your input at once, which I’ll explain further down.
Does your payload always have the same length? In that case, your
forecast should always demand payload_length input items to produce any
amount of output items – that way, you can make sure that at one
point
, you get the whole payload into your input buffer.

No matter what you really do, you will need to use a state machine of
some kind.
In your case, that’s comparatively easy: Just check, at the beginning of
each general_work call, whether you’re in pass-through mode, or in
“search preamble mode”, etc, and let your work act based upon that. For
example, assume you’re in search preamble mode, you find a preamble, so
you consume all items up to the end of that preamble, and set your state
to “pass through”, and then return from general_work. Before GNU Radio
calls you next time, it will ask forecast() how many input items you
need. Forecast always says it needs payload_length items, so at the
point where work is actually called in “pass through” mode, you should
have at least payload_length input items; let your block consume and
produce these items, switch to “search preamble mode”, and return.

Even worse, a stream of samples are coming continuously from a
previous block.
Lucky you! You have GNU Radio, which makes sure there’s only one
instance of your general_work being run at once, and presenting you with
all the items you did not consume in the previous general_work calls.

Will it be helpful if I have some very long buffer in a block? I don’t
think it works since noutput_items is always smaller than
ninput_items every time the block runs general_work().
That’s not generally the case. It might be the case here, but that
simply means that the downstream block isn’t consuming the whole buffer,
or your upstream items are larger, or your upstream block is just much
faster at producing items than you and your downstream blocks are at
consuming them.

If you just try to rely on GNU Radio’s buffers, you would be safe from
something like buffer overflows happening – GNU Radio will only call
the upstream’s general_work function if there is enough space in its
output buffer (which is your input buffer).

Personally, I think what you’re doing is a complicated task, so I have a
lot of respect for the effort you’re putting into this.
I’d try to avoid the complexity by /not/ doing this in one block – use
a preamble correlator to mark the beginning of your frame with a stream
tag. Then, use a state-machine block to only copy the payload_length
items after such a tag (a whole payload at a time), adding a tag on the
first item containing the length of the payload as long:
I mentioned above there’s an advantage to passing the whole payload at
once; what you have when you have a length tag on the first item of
contigous packet items is a tagged stream. Tagged stream blocks are a
GNU Radio concept dedicated to processing packeted data, in which work()
methods are always called with the whole packet at once [1].

Best regards,
Marcus

[1] http://gnuradio.org/doc/doxygen/page_tagged_stream_blocks.html

Hi Jeon,

On 05/29/2015 05:25 PM, Jeon wrote:

As I understand, if I consume(0, n), where n is less than
ninput_items, then only n samples at the front of input_items[] are
consumed and unconsumed part remains. And then, at the next call of
general_work(), incoming samples are appended at the end of
input_item[]. Is this right?
yes :slight_smile:

If it is right, I can get a single whole payload by not consuming
input samples after some time passed…
you should always consume as many samples as possible to keep the flow
graph running; hence my recommendation to implement the forecast in a
manner that guarantees that your work will only be called with “enough”
input.
However, I am still doubtful that input_items[] will not be appended
as I think since output buffer at the previous block isn’t cleared
enough as consume() is not called.
the output buffer is completely independent from consume(); space in the
output buffer gets freed as soon as the downstream block calls
consume(), and gets filled by the amount you produce().

Well… Does forecast() which you’ve mentioned make the previous
block can send more streams and the current block can received them?
No, forecast is just the method that GNU radio calls to ask “hey, block!
If I want you to produce x items, how much input would you need?”.
In your case, the honest answer for any x>0 would be “I need a whole
payload length of items if I should produce anything”.
If you call set_output_multiple(payload_length), you will be guaranteed
enough space for at least one payload per general_work() call.

So far, I understand forecast() acts as just a blocker unless enough
samples are given.
It’s more of a way of telling GNU Radio what to expect from your block.

In the developing stage, I am using fixed length payload. But, I want
to make it work with arbitrary length (but not too long).
In that case, use a forecast() that adjusts the returned length as soon
as your block knows how long the payload will be – forecast() is really
just a hint for GNU Radio, so it doesn’t hurt if it’s not 100% accurate
(i.e. you produce less than what you’ve “promised” in forecast).

Best regards,
Marcus

Hi,

As I understand, if I consume(0, n), where n is less than ninput_items, then
only n samples at the front of input_items[] are consumed and unconsumed
part remains. And then, at the next call of general_work(), incoming samples
are appended at the end of input_item[]. Is this right?

If it is right, I can get a single whole payload by not consuming input
samples after some time passed…

Not really. Because the buffers are allocated only once at the start
and not resized and not really under your control.

So if you leave the sample in there, and the buffer fills up and you
still don’t have a full payload, the graph will stall forever.
It’s annoying and a pattern that comes up quite often but it’s the way
it works ATM.

Cheers,

Sylvain

Thanks to Sylvain and Marcus.

I think I’ve solved problem.

Regards,
Jeon.

2015-05-30 11:26 GMT+09:00 Sylvain M. [email protected]: