# Flowgraph latency with UHD and continuous stream

**URL:** <https://www.ruby-forum.com/t/flowgraph-latency-with-uhd-and-continuous-stream/228459>\
**Category:** GNU Radio\
**Created:** [April 23, 2013, 8:00am UTC](https://www.ruby-forum.com/t/flowgraph-latency-with-uhd-and-continuous-stream/228459 "2013-04-23T08:00:02Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Phelps\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/b9bd4f/32.png) [@Phelps\_W](https://www.ruby-forum.com/u/Phelps_W)\
**Post date:** [April 23, 2013, 8:00am UTC](https://www.ruby-forum.com/t/flowgraph-latency-with-uhd-and-continuous-stream/228459/1 "2013-04-23T08:00:02Z")

</div>

I created a block which in the work(), sits in select on a udp socket  
with  
a timeout of 10msec. In the event it times out rather than getting a  
read  
signal, a pattern of noutput\_items is written. In the event it receives  
data from the udp socket, the udp dgram is copied to the output items  
vector. This flowgraph ends with a UHD USRP sink.

I am consistently observing latency through my flow graph of around 2 -  
2.5sec. The total latency doesn’t appear to be dependent on the sample  
rate (I’ve tested 1Msps and 2Msps). When intentionally underruning the  
flow graph by setting set\_max\_noutput\_items to less than 10msec of  
samples,  
the latency through the graph is between 80msec and 200msec.  
Intentionally underrunning UHD isn’t an option for my application as an  
intermittent modulated signal from the USRP causes issues.

I am attempting to use the global (via gr.top\_block.start()) and block  
specific max\_noutput\_items settings to limit the total amount of latency  
in  
my flow graph but this doesn’t appear to be the primary source of  
latency.

My suspicion is that UHD or the USRP has a transmit buffer which is the  
source of my problem. I tried messing with the send\_buff\_size and  
recv\_buff\_size specified here:  
[http://files.ettus.com/uhd\_docs/manual/html/transport.html](http://files.ettus.com/uhd_docs/manual/html/transport.html) but these  
didn’t  
seem to make any impact.

I played with setting set\_max\_noutput\_items on the source block to the  
exact value expected during the 10msec select timeout in my custom  
source  
block. This works for a while but occasionally underruns. Also as udp  
traffic moves through the block, the total amount of latency slowly  
increases. It also seems wrong to throttle the flowgraph at both the  
source and the sink. Much like it is problematic to use a throttle with  
a  
UHD source or sink block. set\_max\_noutput\_items doesn’t seem like the  
right solution for this problem.

Decreasing GR\_FIXED\_BUFFER\_SIZE in gr\_flat\_flowgraph makes a significant  
improvement but it isn’t a viable option for my application because it  
is a  
universal setting for all gnuradio based radios running on my system.  
Some  
of which require the default buffer size to operate efficiently.

There has been a little discussion of this topic on the mailing list in  
the  
past but I haven’t found examples that include UHD / USRP with a  
flowgraph  
that isn’t intentionally underrunning (ie with a typical UDP source  
block).

I have messed with the maximum socket buffer sizes allowed by linux.  
The  
uhd recommended config and much smaller variations don’t appear to make  
an  
impact to this latency.  
$ sudo sysctl net.core.rmem\_max  
net.core.rmem\_max = 50000000  
$ sudo sysctl net.core.wmem\_max  
net.core.wmem\_max = 1048576

For reference I am running:  
UHD 003.005.002  
Gnuradio 3.6.4.1

_tl;dr_  
My expectation is that there is a buffer UHD is attempting to maintain,  
when this buffer gets low, it triggers upstream work operations in the  
flowgraph to refill this buffer. Is this not the case? How can I  
manipulate the size of this buffer? Is there any way to confirm the  
size  
of this buffer?

Thanks!

-Phelps

---

<div class="post-metadata">

**Author:** ![Phelps\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/b9bd4f/32.png) [@Phelps\_W](https://www.ruby-forum.com/u/Phelps_W)\
**Post date:** [April 23, 2013, 11:16pm UTC](https://www.ruby-forum.com/t/flowgraph-latency-with-uhd-and-continuous-stream/228459/2 "2013-04-23T23:16:54Z")

</div>

Hi Phelps,

I’ve been dealing with latency issues of my own, and this seems to be  
the best solution so far (big thanks to Josh!):

[http://lists.gnu.org/archive/html/discuss-gnuradio/2013-04/msg00211.html](http://lists.gnu.org/archive/html/discuss-gnuradio/2013-04/msg00211.html)

The nice thing about this method is that it allows you to manage  
buffering at a system level as opposed to per-block. I’ve been able to  
limit my latency to about 50 ms, but you can probably do better  
depending on how much processing you’re doing. Hope this works for you!

Jordan

---

<div class="post-metadata">

**Author:** ![Phelps\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/b9bd4f/32.png) [@Phelps\_W](https://www.ruby-forum.com/u/Phelps_W)\
**Post date:** [April 24, 2013, 1:39am UTC](https://www.ruby-forum.com/t/flowgraph-latency-with-uhd-and-continuous-stream/228459/3 "2013-04-24T01:39:15Z")

</div>

Thanks Jordan, this is super helpful.

I made the uhd/gr-uhd changes that josh suggested and using the  
following  
code snippet. I now can see EVENT\_CODE\_BURST\_ACK (event code == 0)  
messages coming back from UHD at approximately 30msec intervals.

self.async\_msgq = gr.msg\_queue(0)  
self.uhd\_amsg\_source\_0 = uhd.amsg\_source(“addr=192.168.102.1”,  
self.async\_msgq)  
self.async\_rcv = gru.msgq\_runner(self.async\_msgq, self.async\_callback)

def async\_callback(self, msg):  
md = self.uhd\_amsg\_source\_0.msg\_to\_async\_metadata\_t(msg)  
print “Channel: %i Time: %f Event: %i” % (md.channel,  
md.time\_spec.get\_real\_secs(), md.event\_code)

In josh’s example he sets the user\_payload field with presumably a  
buffer  
occupancy value. I’m getting lost in the swiggyness here and can’t  
figure  
out how to print a value of type \<Swig Object of type ‘uint\_fast16\_t \*’  
at  
0x9894188\> any suggestions there?

I’m guessing what you are doing is writing a block which handles  
messages  
and in the event the occupancy drops below a certain amount, you write  
more  
source data out.

---

<div class="post-metadata">

**Author:** ![Phelps\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/b9bd4f/32.png) [@Phelps\_W](https://www.ruby-forum.com/u/Phelps_W)\
**Post date:** [April 24, 2013, 4:33am UTC](https://www.ruby-forum.com/t/flowgraph-latency-with-uhd-and-continuous-stream/228459/4 "2013-04-24T04:33:55Z")

</div>

The payload field contains the sequence numbers of packets consumed by  
the USRP’s DUC. As a note, you may need to reorder the bytes  
(uhd::ntohx). I’m handling the messages in C++, so I’m not too sure  
about the swig. My source reads the messages and calculates the number  
of samples consumed by the USRP (this is possible because the USRP  
packet size is fixed). The number of produced output samples are  
limited such that there are only N samples (50 ms) between my source and  
the USRP’s DUC.
