Gr-ctrlport-monitor timeout exception

GNU Radio
1

I’m using ControlPort to monitor transmissions through a USRP. I have a
flowgraph responsible for generating burst traffic and streaming to a
uhd_sink. Then I have a uhd_source tuned to the same frequency as the
uhd_sink, and I connect it to a ctrlport_probe2_c block with length=128.
I have ControlPort support compiled-in and enabled from a config file.
I’m able to connect to a remotely running flowgraph using
gr-ctrlport-monitor and plot the PSD of the “samples” vector pulled from
the probe2 block every 100 milliseconds. The problem is that after (what
seems to be) a nondeterministic time, the ICE port stops responding and
gr-ctrlport-monitor reports an error:

ctrlport-monitor: radio.get threw exception (exception
::Ice::ConnectTimeoutException
{
}).

When I close and restart, gr-ctrlport-monitor times out and segfaults:

2013-11-11 16:02:47.329422 /usr/local/bin/gr-ctrlport-monitor: error:
Traceback (most recent call last):
File “/usr/lib/pymodules/python2.7/Ice.py”, line 984, in main
status = self.doMain(args, initData)
File “/usr/lib/pymodules/python2.7/Ice.py”, line 1031, in doMain
return self.run(args)
File
“/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py”,
line 97, in run
radio = self.getRadio(host, port)
File
“/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py”,
line 36, in getRadio
radio = GNURadio.ControlPortPrx.checkedCast(base)
File “/usr/local/lib/python2.7/dist-packages/gnuradio_ice.py”, line
1257, in checkedCast
return
_M_gnuradio.ctrlport.GNURadio.ControlPortPrx.ice_checkedCast(proxy,
‘::GNURadio::ControlPort’, facetOrCtx, _ctx)
ConnectTimeoutException: exception ::Ice::ConnectTimeoutException
{
}

Segmentation fault (core dumped)

So there are two issues to note here:

  •      Something in the ICE instance is breaking on the GNU Radio

flowgraph side. The port is still open; it just times out. Trying to
instantiate gr-ctrlport-monitor to an incorrect port just says
“connection refused,” as expected.

  •      gr-ctrlport-monitor is not robust to connection-related

errors like timeouts or refused connections.

Is there any advice of what I can turn on or enable in GNU Radio or my
flowgraph to debug the first problem? I can live with the second problem
as long as I can make sure ICE doesn’t break on me.

Thanks,
Sean

2

}).
File
“/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py”, line
36, in getRadio

  • Something in the ICE instance is breaking on the GNU Radio flowgraph
    side. The port is still open; it just times out. Trying to instantiate
    gr-ctrlport-monitor to an incorrect port just says “connection refused,” as
    expected.
  • gr-ctrlport-monitor is not robust to connection-related errors like
    timeouts or refused connections.

Is there any advice of what I can turn on or enable in GNU Radio or my flowgraph
to debug the first problem? I can live with the second problem as long as I can
make sure ICE doesn’t break on me.

Thanks,
Sean

Sorry for getting antsy about this, but I’m really not sure how to go
about debugging ICE server stuff. It seems like it’s buried pretty
deeply in gnuradio-runtime. Where’s the best place to start looking to
find out why the ICE server stops responding?

Sean

3

I’ve seen ICE do this before, but usually its because my flowgraph dies,
via segfault or something. When the flowgraph dies, the ice endpoints
aren’t available anymore so the controlport monitor timesout when
attempting to query them. Are you sure your flowgraph isn’t crapping
out
for some reason or another?

Tim

On Wed, Nov 13, 2013 at 11:10 AM, Nowlan, Sean

4

Sorry it took me a bit of time to respond. Yes, I’m sure my flowgraph is
still running and producing data as expected. Can anybody think of why
the ice endpoint would die unexpectedly?

Sean

From: Tim N. [mailto:[email protected]]
Sent: Wednesday, November 13, 2013 11:42 AM
To: Nowlan, Sean
Cc: [email protected]
Subject: Re: [Discuss-gnuradio] gr-ctrlport-monitor timeout exception

I’ve seen ICE do this before, but usually its because my flowgraph dies,
via segfault or something. When the flowgraph dies, the ice endpoints
aren’t available anymore so the controlport monitor timesout when
attempting to query them. Are you sure your flowgraph isn’t crapping
out for some reason or another?

Tim

On Wed, Nov 13, 2013 at 11:10 AM, Nowlan, Sean
<[email protected]mailto:[email protected]> wrote:

}).
File
“/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py”, line
36, in getRadio

  • Something in the ICE instance is breaking on the GNU Radio flowgraph
    side. The port is still open; it just times out. Trying to instantiate
    gr-ctrlport-monitor to an incorrect port just says “connection refused,” as
    expected.
  • gr-ctrlport-monitor is not robust to connection-related errors like
    timeouts or refused connections.

Is there any advice of what I can turn on or enable in GNU Radio or my flowgraph
to debug the first problem? I can live with the second problem as long as I can
make sure ICE doesn’t break on me.

Thanks,
Sean

Sorry for getting antsy about this, but I’m really not sure how to go
about debugging ICE server stuff. It seems like it’s buried pretty
deeply in gnuradio-runtime. Where’s the best place to start looking to
find out why the ICE server stops responding?

Sean

5

Perhaps this is happening due to the ICE endpoint’s TCP timeout setting.
I have it configured to 300 ms based on the example ctrlport.conf file
distributed with GNU Radio. I’ll try increasing this. A few questions:

  1.  What disadvantages would there be to setting this to something

really high?

  1.  I need to monitor things over a potentially unreliable

connection. It looks straightforward to configure the ICE endpoint to
use UDP. What would it take to make gr-ctrlport-monitor work with a UDP
endpoint?

Thanks,
Sean

From: discuss-gnuradio-bounces+sean.nowlan=removed_email_address@domain.invalid
[mailto:discuss-gnuradio-bounces+sean.nowlan=removed_email_address@domain.invalid] On
Behalf Of Nowlan, Sean
Sent: Tuesday, December 03, 2013 3:13 PM
To: Tim N.
Cc: [email protected]
Subject: Re: [Discuss-gnuradio] gr-ctrlport-monitor timeout exception

Sorry it took me a bit of time to respond. Yes, I’m sure my flowgraph is
still running and producing data as expected. Can anybody think of why
the ice endpoint would die unexpectedly?

Sean

From: Tim N. [mailto:[email protected]]
Sent: Wednesday, November 13, 2013 11:42 AM
To: Nowlan, Sean
Cc: [email protected]mailto:[email protected]
Subject: Re: [Discuss-gnuradio] gr-ctrlport-monitor timeout exception

I’ve seen ICE do this before, but usually its because my flowgraph dies,
via segfault or something. When the flowgraph dies, the ice endpoints
aren’t available anymore so the controlport monitor timesout when
attempting to query them. Are you sure your flowgraph isn’t crapping
out for some reason or another?

Tim

On Wed, Nov 13, 2013 at 11:10 AM, Nowlan, Sean
<[email protected]mailto:[email protected]> wrote:

}).
File
“/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py”, line
36, in getRadio

  • Something in the ICE instance is breaking on the GNU Radio flowgraph
    side. The port is still open; it just times out. Trying to instantiate
    gr-ctrlport-monitor to an incorrect port just says “connection refused,” as
    expected.
  • gr-ctrlport-monitor is not robust to connection-related errors like
    timeouts or refused connections.

Is there any advice of what I can turn on or enable in GNU Radio or my flowgraph
to debug the first problem? I can live with the second problem as long as I can
make sure ICE doesn’t break on me.

Thanks,
Sean

Sorry for getting antsy about this, but I’m really not sure how to go
about debugging ICE server stuff. It seems like it’s buried pretty
deeply in gnuradio-runtime. Where’s the best place to start looking to
find out why the ICE server stops responding?

Sean