USRP2 stops after several-days-continuous-operation

GNU Radio
1

Hi, folks,

I’m testing my application with USRP2.
After I verify the function works fine logically, I started aging test
by
letting the program run over a week.
Although it varies, the program hangs eventually after running several
days.
(from 2~3 days to 8 days)

The followings are part of my python code

In the top block,

~~~
self.u = usrp2.source_32fc(self.interface, self.MAC_addr)
s2v = gr.stream_to_vector(gr.sizeof_gr_complex, self.fft_size)
fft = gr.fft_vcc(self.fft_size, True, mywindow)
c2mag = gr.complex_to_mag_squared(self.fft_size)
self.msgq = gr.msg_queue(1)
stats = gr.message_sink(gr.sizeof_float * self.fft_size, self.msgq,

True)

self.connect(self.u, s2v, fft, c2mag, stats)
~~~

In main_loop thread

~~~
while(1):
    rcvd_param = rx_msg.check_udp()

    if (rcvd_param[0] == 1):        # Valid UDP packet received
        tb.u.set_decim(cp.decim)    # Set(Update) USRP2 decim
        cp.init_ctrl_param(rcvd_param, tb.fft_size)

    radio.tune(cp.ctr_freq)                                # This is

to
control other independent receiver
cp.spectrum_inv = radio.spectrum_status()

    # Force buffer flushing
    samples = msg.payload(tb.msgq.delete_head(), tb.fft_size)
    samples = msg.payload(tb.msgq.delete_head(), tb.fft_size)
    samples = msg.payload(tb.msgq.delete_head(), tb.fft_size)
    samples.make_rawdata()
    tx_msg.send_packet_over_UDP(cp, samples.raw_data)
~~~
    cp.update()                                                # End

of
while loop
~~~

In the program, I generated a message queue to get the sample from
USRP2, do
some process it a bit, and send it to other server over UDP.

In the while loop,

  1. Check the user defined control parameter (concerning tuning frequency
    and
    USRP2 decim. rate)
    If any control message is delivered, it update the control
    parameters.
    Otherwise, skips.
  2. Tune the independent receiver module. This is a frontend of USRP2.
  3. Flushing sample queue
  4. Get the sample out of queue and send it over UDP

The program hangs between first and second buffer flushing after several
days of continuous operation.
When it hangs, I figured out that the USRP2 does not send sample data to
PC.
(I confirmed it with wireshark.)

I attached the program to gdb and found out that the program is stuck
waiting for the queue to be filled, as you see in the following gdb
back-trace message.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
(gdb) info program
Using the running image of attached LWP 28651.
Program stopped at 0xa08a32e9.
(gdb) bt
#0 0x00007f26a08a32e9 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib/libpthread.so.0
#1 0x00007f269f3ec9c0 in omni_condition::wait (this=)
at posix.cc:152
#2 0x00007f269f794c98 in gr_msg_queue::delete_head (this=0x2aea520) at
gr_msg_queue.cc:81
#3 0x00007f269fa3b125 in gr_py_msg_queue__delete_head (q= {px =
0x7fffb561f150, pn = {pi_ = 0xffffffffffffffff}}) at
gnuradio_swig_py_runtime.cc:4840
#4 0x00007f269fa43649 in _wrap_gr_py_msg_queue__delete_head
(args=, kwargs=) at
gnuradio_swig_py_runtime.cc:13477
#5 0x000000000041d3bd in PyObject_Call ()
#6 0x00000000004a1460 in PyEval_EvalFrameEx ()
#7 0x00000000004a4649 in PyEval_EvalCodeEx ()
#8 0x00000000004a2cb0 in PyEval_EvalFrameEx ()
#9 0x00000000004a3dae in PyEval_EvalFrameEx ()
#10 0x00000000004a4649 in PyEval_EvalCodeEx ()
#11 0x00000000004a4762 in PyEval_EvalCode ()
#12 0x00000000004c4c3c in PyRun_FileExFlags ()
#13 0x00000000004c4f6b in PyRun_SimpleFileExFlags ()
#14 0x00000000004189ce in Py_Main ()
#15 0x00007f269fca05a6 in __libc_start_main () from /lib/libc.so.6
#16 0x0000000000417ae9 in _start ()
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

So, I am suspicious if this is a problem caused simply by the USRP2 not
sending data.
Is there anybody experienced similar case or having any idea?

Regards,

ILKYOUNG.