# What if noutput\_itmes is too short?

**URL:** <https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128>\
**Category:** GNU Radio\
**Created:** [May 29, 2015, 1:48pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128 "2015-05-29T13:48:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![musicdenotation](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ea666f/32.png) [@musicdenotation](https://www.ruby-forum.com/u/musicdenotation)\
**Post date:** [May 29, 2015, 1:48pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/1 "2015-05-29T13:48:40Z")

</div>

I am trying to remove `preamble` and `start frame delimiter` from a  
frame  
and only the payload will be passed to the next block.

While in debugging, `ninput_items` is 7742, and `noutput_items` is 4096,  
for instance. An offset where the frame starts is 790.

Simple calculation gives me that 7742 - 790 = 6952 samples should be  
passed  
to the next block. But output space given is only 4096. Thus, I can only  
pass 4096 samples and 6952 - 4096 = 2856 samples will be discarded. But,  
I  
need them to be passed, too.

Even worse, a stream of samples are coming continuously from a previous  
block.

Will it be helpful if I have some very long buffer in a block? I don’t  
think it works since `noutput_items` is always smaller than  
`ninput_items`  
every time the block runs `general_work()`. In this case, I think the  
buffer will be overflowed.

The link below is the log that shows

`i`/`ninput_itmes` and `o`/`noutput_items`,

where `i` and `o` are input and output indices. When `ninput_itmes` and  
`noutput_items` change, it means a new `general_work` is called.

Log (it’s quite long…): [https://bitbucket.org/snippets/gsongsong/By8x](https://bitbucket.org/snippets/gsongsong/By8x)

Regards,  
Jeon.

---

<div class="post-metadata">

**Author:** ![Jeon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/e8c25b/32.png) [@Jeon](https://www.ruby-forum.com/u/Jeon)\
**Post date:** [May 29, 2015, 2:50pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/2 "2015-05-29T14:50:37Z")

</div>

After posting the previous thread, I’ve searched APIs from GNU Radio  
doxygen.

Several candidates have been found.

1. [`produce()`](http://gnuradio.org/doc/doxygen/classgr_1_1block.html#aa5581727d057bdd8113f8b2a3fc5bd66)
2. [`set_min_noutput_items()`](http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a65cfc579150dc4d10c6180d3365aa9a8)
3. [`set_min_output_buffer()`](http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a9d10e3f6747f91b215abe81b60a003d5)
4. [`expand_minmax_buffer()`](http://gnuradio.org/doc/doxygen/classgr_1_1block.html#a9bbf96f6a81d5c289934a68ef44dd1b4)

I think I can combine `produce()` and `set_min_noutput_items()` to  
**enlarge** the `noutput_items`

If I get wrong, please let me know.

Regards,  
Jeon.

2015년 5월 29일 (금) 오후 8:47, Jeon [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)님이 작성:

---

<div class="post-metadata">

**Author:** ![Jeon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/e8c25b/32.png) [@Jeon](https://www.ruby-forum.com/u/Jeon)\
**Post date:** [May 29, 2015, 5:26pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/3 "2015-05-29T17:26:36Z")

</div>

Thank you for your reply, Marcus.

Now I’m getting a little bit more.

As I understand, if I consume(0, n), where n is less than ninput\_items,  
then only n samples at the front of input\_items[] are consumed and  
unconsumed part remains. And then, at the next call of general\_work(),  
incoming samples are appended at the end of input\_item[]. Is this right?

If it is right, I can get a single whole payload by not consuming input  
samples after some time passed…

However, I am still doubtful that input\_items[] will not be appended as  
I  
think since output buffer at the previous block isn’t cleared enough as  
`consume()` is not called.

Well… Does `forecast()` which you’ve mentioned make the previous block  
can send more streams and the current block can received them? So far, I  
understand `forecast()` acts as just a blocker unless enough samples are  
given.

In the developing stage, I am using fixed length payload. But, I want to  
make it work with arbitrary length (but not too long).

Regards,  
Jeon.

On Fri, May 29, 2015 at 9:52 PM Marcus Müller [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![Jeon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/e8c25b/32.png) [@Jeon](https://www.ruby-forum.com/u/Jeon)\
**Post date:** [May 29, 2015, 2:53pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/4 "2015-05-29T14:53:32Z")

</div>

Hi Jeon,

On 05/29/2015 01:46 PM, Jeon wrote:

> I am trying to remove `preamble` and `start frame delimiter` from a  
> frame and only the payload will be passed to the next block.
> 
> While in debugging, `ninput_items` is 7742, and `noutput_items` is  
> 4096, for instance. An offset where the frame starts is 790.
> 
> Simple calculation gives me that 7742 - 790 = 6952 samples should be  
> passed to the next block. But output space given is only 4096. Thus, I  
> can only pass 4096 samples and 6952 - 4096 = 2856 samples will be  
> discarded. But, I need them to be passed, too.

There’s no reason not to pass through your payload in parts, if you  
implement a state machine with a counter for remaining items to pass  
through in your block.  
However, there’s an advantage to waiting for the whole payload to be on  
your input at once, which I’ll explain further down.  
Does your payload always have the same length? In that case, your  
forecast should always demand payload\_length input items to produce any  
amount of output items – that way, you can make sure that at _one  
point_, you get the whole payload into your input buffer.

No matter what you really do, you will need to use a state machine of  
some kind.  
In your case, that’s comparatively easy: Just check, at the beginning of  
each general\_work call, whether you’re in pass-through mode, or in  
“search preamble mode”, etc, and let your work act based upon that. For  
example, assume you’re in search preamble mode, you find a preamble, so  
you consume all items up to the end of that preamble, and set your state  
to “pass through”, and then return from general\_work. Before GNU Radio  
calls you next time, it will ask forecast() how many input items you  
need. Forecast always says it needs payload\_length items, so at the  
point where work is actually called in “pass through” mode, you _should_  
have at least payload\_length input items; let your block consume and  
produce these items, switch to “search preamble mode”, and return.

> Even worse, a stream of samples are coming continuously from a  
> previous block.  
> Lucky you! You have GNU Radio, which makes sure there’s only one  
> instance of your general\_work being run at once, and presenting you with  
> all the items you did not consume in the previous general\_work calls.
> 
> Will it be helpful if I have some very long buffer in a block? I don’t  
> think it works since `noutput_items` is always smaller than  
> `ninput_items` every time the block runs `general_work()`.  
> That’s not generally the case. It might be the case here, but that  
> simply means that the downstream block isn’t consuming the whole buffer,  
> or your upstream items are larger, or your upstream block is just much  
> faster at producing items than you and your downstream blocks are at  
> consuming them.

If you just try to rely on GNU Radio’s buffers, you would be safe from  
something like buffer overflows happening – GNU Radio will only call  
the upstream’s general\_work function if there is enough space in its  
output buffer (which is your input buffer).

Personally, I think what you’re doing is a complicated task, so I have a  
lot of respect for the effort you’re putting into this.  
I’d try to avoid the complexity by /not/ doing this in one block – use  
a preamble correlator to mark the beginning of your frame with a stream  
tag. Then, use a state-machine block to only copy the payload\_length  
items after such a tag (a whole payload at a time), adding a tag on the  
first item containing the length of the payload as long:  
I mentioned above there’s an advantage to passing the whole payload at  
once; what you have when you have a length tag on the first item of  
contigous packet items is a tagged stream. Tagged stream blocks are a  
GNU Radio concept dedicated to processing packeted data, in which work()  
methods are always called with the whole packet at once [1].

Best regards,  
Marcus

[1] [http://gnuradio.org/doc/doxygen/page\_tagged\_stream\_blocks.html](http://gnuradio.org/doc/doxygen/page_tagged_stream_blocks.html)

---

<div class="post-metadata">

**Author:** ![Jeon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/e8c25b/32.png) [@Jeon](https://www.ruby-forum.com/u/Jeon)\
**Post date:** [May 29, 2015, 7:59pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/5 "2015-05-29T19:59:41Z")

</div>

Hi Jeon,

On 05/29/2015 05:25 PM, Jeon wrote:

> As I understand, if I consume(0, n), where n is less than  
> ninput\_items, then only n samples at the front of input\_items are  
> consumed and unconsumed part remains. And then, at the next call of  
> general\_work(), incoming samples are appended at the end of  
> input\_item. Is this right?  
> yes 🙂
> 
> If it is right, I can get a single whole payload by not consuming  
> input samples after some time passed…  
> you should always consume as many samples as possible to keep the flow  
> graph running; hence my recommendation to implement the forecast in a  
> manner that guarantees that your work will only be called with “enough”  
> input.  
> However, I am still doubtful that input\_items will not be appended  
> as I think since output buffer at the previous block isn’t cleared  
> enough as `consume()` is not called.  
> the output buffer is completely independent from consume(); space in the  
> output buffer gets freed as soon as the downstream block calls  
> consume(), and gets filled by the amount you produce().
> 
> Well… Does `forecast()` which you’ve mentioned make the previous  
> block can send more streams and the current block can received them?  
> No, forecast is just the method that GNU radio calls to ask “hey, block!  
> If I want you to produce x items, how much input would you need?”.  
> In your case, the honest answer for any x\>0 would be “I need a whole  
> payload length of items if I should produce anything”.  
> If you call set\_output\_multiple(payload\_length), you will be guaranteed  
> enough space for at least one payload per general\_work() call.

> So far, I understand `forecast()` acts as just a blocker unless enough  
> samples are given.  
> It’s more of a way of telling GNU Radio what to expect from your block.
> 
> In the developing stage, I am using fixed length payload. But, I want  
> to make it work with arbitrary length (but not too long).  
> In that case, use a forecast() that adjusts the returned length as soon  
> as your block knows how long the payload will be – forecast() is really  
> just a hint for GNU Radio, so it doesn’t hurt if it’s not 100% accurate  
> (i.e. you produce less than what you’ve “promised” in forecast).

Best regards,  
Marcus

---

<div class="post-metadata">

**Author:** ![Jeon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/e8c25b/32.png) [@Jeon](https://www.ruby-forum.com/u/Jeon)\
**Post date:** [May 30, 2015, 4:27am UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/6 "2015-05-30T04:27:55Z")

</div>

Hi,

> As I understand, if I consume(0, n), where n is less than ninput\_items, then  
> only n samples at the front of input\_items are consumed and unconsumed  
> part remains. And then, at the next call of general\_work(), incoming samples  
> are appended at the end of input\_item. Is this right?
> 
> If it is right, I can get a single whole payload by not consuming input  
> samples after some time passed…

Not really. Because the buffers are allocated only once at the start  
and not resized and not really under your control.

So if you leave the sample in there, and the buffer fills up and you  
still don’t have a full payload, the graph will stall forever.  
It’s annoying and a pattern that comes up quite often but it’s the way  
it works ATM.

Cheers,

```
Sylvain

```

---

<div class="post-metadata">

**Author:** ![Jeon](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/e8c25b/32.png) [@Jeon](https://www.ruby-forum.com/u/Jeon)\
**Post date:** [June 1, 2015, 4:01pm UTC](https://www.ruby-forum.com/t/what-if-noutput-itmes-is-too-short/242128/7 "2015-06-01T16:01:11Z")

</div>

Thanks to Sylvain and Marcus.

I think I’ve solved problem.

Regards,  
Jeon.

2015-05-30 11:26 GMT+09:00 Sylvain M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):