Forum: GNU Radio gr-ctrlport-monitor timeout exception

2474e637cd95190b38f78b5655f795e2?d=identicon&s=25 Nowlan, Sean (Guest)
on 2013-11-11 22:10
(Received via mailing list)
I'm using ControlPort to monitor transmissions through a USRP. I have a
flowgraph responsible for generating burst traffic and streaming to a
uhd_sink. Then I have a uhd_source tuned to the same frequency as the
uhd_sink, and I connect it to a ctrlport_probe2_c block with length=128.
I have ControlPort support compiled-in and enabled from a config file.
I'm able to connect to a remotely running flowgraph using
gr-ctrlport-monitor and plot the PSD of the "samples" vector pulled from
the probe2 block every 100 milliseconds. The problem is that after (what
seems to be) a nondeterministic time, the ICE port stops responding and
gr-ctrlport-monitor reports an error:

ctrlport-monitor: radio.get threw exception (exception
::Ice::ConnectTimeoutException
{
}).

When I close and restart, gr-ctrlport-monitor times out and segfaults:

2013-11-11 16:02:47.329422 /usr/local/bin/gr-ctrlport-monitor: error:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/Ice.py", line 984, in main
    status = self.doMain(args, initData)
  File "/usr/lib/pymodules/python2.7/Ice.py", line 1031, in doMain
    return self.run(args)
  File
"/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py",
line 97, in run
    radio = self.getRadio(host, port)
  File
"/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py",
line 36, in getRadio
    radio = GNURadio.ControlPortPrx.checkedCast(base)
  File "/usr/local/lib/python2.7/dist-packages/gnuradio_ice.py", line
1257, in checkedCast
    return
_M_gnuradio.ctrlport.GNURadio.ControlPortPrx.ice_checkedCast(proxy,
'::GNURadio::ControlPort', facetOrCtx, _ctx)
ConnectTimeoutException: exception ::Ice::ConnectTimeoutException
{
}

Segmentation fault (core dumped)

So there are two issues to note here:

-          Something in the ICE instance is breaking on the GNU Radio
flowgraph side. The port is still open; it just times out. Trying to
instantiate gr-ctrlport-monitor to an incorrect port just says
"connection refused," as expected.

-          gr-ctrlport-monitor is not robust to connection-related
errors like timeouts or refused connections.

Is there any advice of what I can turn on or enable in GNU Radio or my
flowgraph to debug the first problem? I can live with the second problem
as long as I can make sure ICE doesn't break on me.

Thanks,
Sean
2474e637cd95190b38f78b5655f795e2?d=identicon&s=25 Nowlan, Sean (Guest)
on 2013-11-13 17:12
(Received via mailing list)
>}).
>  File
"/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py", 
line
36, in getRadio
>    -    Something in the ICE instance is breaking on the GNU Radio flowgraph
side. The port is still open; it just times out. Trying to instantiate
gr-ctrlport-monitor to an incorrect port just says "connection refused," as
expected.
>    -    gr-ctrlport-monitor is not robust to connection-related errors like
timeouts or refused connections.
>
>Is there any advice of what I can turn on or enable in GNU Radio or my flowgraph
to debug the first problem? I can live with the second problem as long as I can
make sure ICE doesn't break on me.
>
>Thanks,
>Sean

Sorry for getting antsy about this, but I'm really not sure how to go
about debugging ICE server stuff. It seems like it's buried pretty
deeply in gnuradio-runtime. Where's the best place to start looking to
find out why the ICE server stops responding?

Sean
A85d6f5d2b64dc5177ad2583c3fdc2fa?d=identicon&s=25 Tim Newman (Guest)
on 2013-11-13 17:43
(Received via mailing list)
I've seen ICE do this before, but usually its because my flowgraph dies,
via segfault or something.  When the flowgraph dies, the ice endpoints
aren't available anymore so the controlport monitor timesout when
attempting to query them.  Are you sure your flowgraph isn't crapping
out
for some reason or another?

Tim


On Wed, Nov 13, 2013 at 11:10 AM, Nowlan, Sean
2474e637cd95190b38f78b5655f795e2?d=identicon&s=25 Nowlan, Sean (Guest)
on 2013-12-03 21:14
(Received via mailing list)
Sorry it took me a bit of time to respond. Yes, I'm sure my flowgraph is
still running and producing data as expected. Can anybody think of why
the ice endpoint would die unexpectedly?

Sean

From: Tim Newman [mailto:tim.newman@gmail.com]
Sent: Wednesday, November 13, 2013 11:42 AM
To: Nowlan, Sean
Cc: discuss-gnuradio@gnu.org
Subject: Re: [Discuss-gnuradio] gr-ctrlport-monitor timeout exception

I've seen ICE do this before, but usually its because my flowgraph dies,
via segfault or something.  When the flowgraph dies, the ice endpoints
aren't available anymore so the controlport monitor timesout when
attempting to query them.  Are you sure your flowgraph isn't crapping
out for some reason or another?

Tim

On Wed, Nov 13, 2013 at 11:10 AM, Nowlan, Sean
<Sean.Nowlan@gtri.gatech.edu<mailto:Sean.Nowlan@gtri.gatech.edu>> wrote:
>}).
>  File
"/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py", 
line
36, in getRadio
>    -    Something in the ICE instance is breaking on the GNU Radio flowgraph
side. The port is still open; it just times out. Trying to instantiate
gr-ctrlport-monitor to an incorrect port just says "connection refused," as
expected.
>    -    gr-ctrlport-monitor is not robust to connection-related errors like
timeouts or refused connections.
>
>Is there any advice of what I can turn on or enable in GNU Radio or my flowgraph
to debug the first problem? I can live with the second problem as long as I can
make sure ICE doesn't break on me.
>
>Thanks,
>Sean

Sorry for getting antsy about this, but I'm really not sure how to go
about debugging ICE server stuff. It seems like it's buried pretty
deeply in gnuradio-runtime. Where's the best place to start looking to
find out why the ICE server stops responding?

Sean
2474e637cd95190b38f78b5655f795e2?d=identicon&s=25 Nowlan, Sean (Guest)
on 2013-12-03 21:25
(Received via mailing list)
Perhaps this is happening due to the ICE endpoint's TCP timeout setting.
I have it configured to 300 ms based on the example ctrlport.conf file
distributed with GNU Radio. I'll try increasing this. A few questions:


1)      What disadvantages would there be to setting this to something
really high?

2)      I need to monitor things over a potentially unreliable
connection. It looks straightforward to configure the ICE endpoint to
use UDP. What would it take to make gr-ctrlport-monitor work with a UDP
endpoint?

Thanks,
Sean

From: discuss-gnuradio-bounces+sean.nowlan=gtri.gatech.edu@gnu.org
[mailto:discuss-gnuradio-bounces+sean.nowlan=gtri.gatech.edu@gnu.org] On
Behalf Of Nowlan, Sean
Sent: Tuesday, December 03, 2013 3:13 PM
To: Tim Newman
Cc: discuss-gnuradio@gnu.org
Subject: Re: [Discuss-gnuradio] gr-ctrlport-monitor timeout exception

Sorry it took me a bit of time to respond. Yes, I'm sure my flowgraph is
still running and producing data as expected. Can anybody think of why
the ice endpoint would die unexpectedly?

Sean

From: Tim Newman [mailto:tim.newman@gmail.com]
Sent: Wednesday, November 13, 2013 11:42 AM
To: Nowlan, Sean
Cc: discuss-gnuradio@gnu.org<mailto:discuss-gnuradio@gnu.org>
Subject: Re: [Discuss-gnuradio] gr-ctrlport-monitor timeout exception

I've seen ICE do this before, but usually its because my flowgraph dies,
via segfault or something.  When the flowgraph dies, the ice endpoints
aren't available anymore so the controlport monitor timesout when
attempting to query them.  Are you sure your flowgraph isn't crapping
out for some reason or another?

Tim

On Wed, Nov 13, 2013 at 11:10 AM, Nowlan, Sean
<Sean.Nowlan@gtri.gatech.edu<mailto:Sean.Nowlan@gtri.gatech.edu>> wrote:
>}).
>  File
"/usr/local/lib/python2.7/dist-packages/gnuradio/ctrlport/IceRadioClient.py", 
line
36, in getRadio
>    -    Something in the ICE instance is breaking on the GNU Radio flowgraph
side. The port is still open; it just times out. Trying to instantiate
gr-ctrlport-monitor to an incorrect port just says "connection refused," as
expected.
>    -    gr-ctrlport-monitor is not robust to connection-related errors like
timeouts or refused connections.
>
>Is there any advice of what I can turn on or enable in GNU Radio or my flowgraph
to debug the first problem? I can live with the second problem as long as I can
make sure ICE doesn't break on me.
>
>Thanks,
>Sean

Sorry for getting antsy about this, but I'm really not sure how to go
about debugging ICE server stuff. It seems like it's buried pretty
deeply in gnuradio-runtime. Where's the best place to start looking to
find out why the ICE server stops responding?

Sean
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.