Packet mod/demod race condition?

stevenc · September 17, 2007, 1:23pm

Hi all-

I am testing out the blks2.mod_pkts & blks2.demod_pkts with something
similar to what is in the benchmark_loopback example, only I do not use
the
USRP at all. Basically, I am transfering the contents of one audio file
to
another by way of a mod/demod packet chain, and hoping to see the same
complete, intact audio file come out the other side. Relevant code
follows:

class gmsk_tester(gr.top_block):
def init(self, rx_callback):
gr.top_block.init(self, “gmsk_tester”)

    self.mod = gmsk_mod()
    self.pkt_tx = mod_pkts(self.mod)

    self.demod = gmsk_demod()
    self.pkt_rx = demod_pkts(self.demod, callback=rx_callback)

    self.connect(self.pkt_tx, self.pkt_rx)

def main():
src_fn = ‘data.ogg’
dst_fn = ‘data2.ogg’

src_file = file(src_fn, 'r')
src_data = src_file.read()
src_file.close()

global dst_file
dst_file = file(dst_fn, 'w')

def rx_callback(ok, payload):
    global dst_file
    dst_file.write(payload)

gt = gmsk_tester(rx_callback)

gt.start()

try:
    i = 0
    pkt_size = 800
    pkt_data = src_data[i:i+pkt_size]
    while len(pkt_data) > 0:
        print len(pkt_data)
        gt.pkt_tx.send_pkt(pkt_data)
        i = i+pkt_size
        pkt_data = src_data[i:i+pkt_size]
except:
    pass

gt.pkt_tx.send_pkt('')

gt.pkt_tx.send_pkt(eof=True)

gt.wait()

#time.sleep(1)

dst_file.flush()

if name == ‘main’:
main()

Sometimes this code runs to completion without complaint, sometimes at
the
end I get:
Exception in thread Thread-1 (most likely raised during interpreter
shutdown):
Traceback (most recent call last):
File “threading.py”, line 460, in __bootstrap
File
“/usr/local/lib/python2.5/site-packages/gnuradio/blksimpl2/pkt.py”,
line 153, in run
<type ‘exceptions.AttributeError’>: ‘NoneType’ object has no attribute
‘unmake_packet’
Unhandled exception in thread started by
Error in sys.excepthook:

Original exception was:

If I uncomment the sleep, I never see this message. So:
Q1) Any idea what this error is all about? Is this a race condition that
needs to be addressed, or am I doing something wrong?

Q2) The original audio file is 350.0KB. Sometimes the resulting audio
file
is complete, other times it never gets the last few KB (ends up 341.0KB,
for
example). How do I ensure that all the bytes make it across
successfully?

Q3) What is the effect of packet size? Is there an optimum size? A max
size?
For USRP, packets need to be padded to a multiple of XXX?

Q4) Some examples use gr.enable_realtime_scheduling(). What is the
effect of
this, and do I need it? I note that it requires a sudo.

Thanks for your time!
-Steven

stevenc · September 17, 2007, 1:24pm

Pardon the bump…anyone have any ideas?

stevenc · September 17, 2007, 8:43pm

Hi Steven,

If I uncomment the sleep, I never see this message. So:
Q1) Any idea what this error is all about? Is this a race condition that
needs to be addressed, or am I doing something wrong?

This part I’m not sure about.

Q2) The original audio file is 350.0KB. Sometimes the resulting audio file
is complete, other times it never gets the last few KB (ends up 341.0KB, for
example). How do I ensure that all the bytes make it across successfully?

The benchmark loopback code makes no guarantee of delivery. In order
to do so, you need some method or higher level protocol to detect and
retransmit any dropped frames. Though in your case, I wouldn’t expect
the received file to vary per run. Again, I’m not sure here.

Q3) What is the effect of packet size? Is there an optimum size? A max size?
For USRP, packets need to be padded to a multiple of XXX?

The optimal size depends on the given application. Latency and
overhead might be two considerations. The max packet size is limited
by the length of the whitener PN sequence. I don’t believe there are
any restrictions beyond that. The USRP will view the data in a stream
form; it won’t have any notion of the data packet length at that
point.

Q4) Some examples use gr.enable_realtime_scheduling(). What is the effect of
this, and do I need it? I note that it requires a sudo.

Realtime scheduling changes the scheduling policy of selected threads
to a priority over normal threads. In Linux, root access is required
in order to do so. It should result in more lower latencies and more
deterministic behavior.

Thanks for your time!
-Steven

Anytime.

-TT

stevenc · September 25, 2007, 11:03pm

Tom & John-

Thanks for your comments. I checked out rev 6473 today, it installed
fine
under Ubuntu. I’m still seeing mostly the same behavior. With that 1
second
sleep present, the file is always completely transferred with no errors.
With the sleep commented out, 90-95% of the file is transferred, then I
usually get one of a variety of errors. Sometimes I get:

Unhandled exception in thread started by <bound method
_queue_watcher_thread.__bootstrap of <_queue_watcher_thread(Thread-1,
stopped daemon)>>
Error in sys.excepthook:
Traceback (most recent call last):
File “/var/lib/python-support/python2.5/apport_python_hook.py”, line
30,
in apport_excepthook
import apport.report, apport.fileutils
ImportError: No module named apport.report

Original exception was:
Traceback (most recent call last):
File “threading.py”, line 473, in __bootstrap
File “traceback.py”, line 236, in format_exc
File “traceback.py”, line 142, in format_exception
File “traceback.py”, line 76, in format_tb
File “traceback.py”, line 101, in extract_tb
File “linecache.py”, line 14, in getline
File “linecache.py”, line 40, in getlines
File “linecache.py”, line 109, in updatecache
TypeError: ‘NoneType’ object is not iterable

Sometimes, I get:

Exception in thread Thread-1 (most likely raised during interpreter
shutdown):
Traceback (most recent call last):
File “threading.py”, line 460, in __bootstrap
File
“/usr/local/lib/python2.5/site-packages/gnuradio/blks2impl/pkt.py”,
line 162, in run
<type ‘exceptions.AttributeError’>: ‘NoneType’ object has no attribute
‘unmake_packet’
Unhandled exception in thread started by
Error in sys.excepthook:

Original exception was:

And sometimes I get no error at all, python exits gracefully, but the
file
is still incomplete.

I have no problem with leaving this sleep in, I just thought this might
be
indicative of something that needs to be looked at.

Tom- good to hear from you again. Hope school is going well.

-Steven

stevenc · July 14, 2009, 12:48am

Look into /usr/local/lib/python2.x/site-packages/gnuradio directory. You
will find answers to all your questions.

Good luck

stevenc · July 15, 2009, 1:14am

On Mon, Jul 13, 2009 at 02:32:57PM -0700, udadidd wrote:

I was trying to understand benchmark_tx and benchmark_rx files. I see
blks2.mod_pkts being used. and blks2 being imported from gnuradio

if i call “from gnuradio import blks2”, where exactly are these files
called from?

i do not see any mod_pkts() function in the blks2 directory. instead i
find those in blks2impl/pkt.py and packet_utils.py. I wanted to see the
packet form and tried “Print pkt” in packet_utils.py but it doesnt reflect
in the output at all. How do i know which files are being used in this?

Because of how we glue together the gr python namespace,
the stuff that appears in the blks2 namespace actually lives in a
directory called blks2impl.

Eric

stevenc · July 13, 2009, 11:40pm

I was trying to understand benchmark_tx and benchmark_rx files. I see
blks2.mod_pkts being used. and blks2 being imported from gnuradio

if i call “from gnuradio import blks2”, where exactly are these files
called from?
i do not see any mod_pkts() function in the blks2 directory. instead
i
find those in blks2impl/pkt.py and packet_utils.py. I wanted to see the
packet form and tried “Print pkt” in packet_utils.py but it doesnt
reflect
in the output at all. How do i know which files are being used in this?

–
View this message in context:
http://www.nabble.com/Packet-mod-demod-race-condition--tp12660275p24469786.html
Sent from the GnuRadio mailing list archive at Nabble.com.

stevenc · July 18, 2009, 8:59pm

Am trying to make data(read from keyboard) into packets, save this into
a
file, and in another program, read this data, deframe it and display.

I followed similar steps as in benchmark_rx and benchmark_tx. instead i
removed modulator and demodulator. I alos avoided the use of usrp in
between, instead of which i use a file to write to and read from.
http://www.nabble.com/file/p24550834/check.py check.py

check.py asks you to the data u wanna send, converts it into packets
and
saves them into packetizer.data
http://www.nabble.com/file/p24550834/functions.py functions.py
it uses the file functions.py
check2.py is does teh deframing job
http://www.nabble.com/file/p24550834/check2.py check2.py
http://www.nabble.com/file/p24550834/chk_rxpath.py chk_rxpath.py
http://www.nabble.com/file/p24550834/deframe_pkts.py deframe_pkts.py

Running check2.py gives me the following output :

I am inside ‘if name = main’ loop
Main() function is called now
Defining tb with rx_callback
inside my_top_block
in chk_rxpath now
printing rx_callback : <function rx_callback at 0x2a71170>
is this self.u being called?
inside deframe_packets now
inside queue watcher now
rcvd packet in queue watcher :
<gnuradio.gr.gnuradio_swig_py_runtime.gr_msg_queue_sptr; proxy of <Swig
Object of type ‘gr_msg_queue_sptr *’ at 0x2a6b4c0> >
callback in queue watcher : <function rx_callback at 0x2a71170>
inside queue watcher : next is self.start()
inside run(self) of deframe packets
Connecting source file to packet_receiver
connected
<chk_rxpath.chk_rxpath object at 0x2a6b2b0>
<gr_block file_source (6)>
next is tb.start()
terminate called after throwing an instance of ‘std::invalid_argument’
what(): port number 0 exceeds max of (none)
Abort trap
~/gnuradio-examples/python/audio/>

I did not follow where exactly i am going wrong!and what is happening
when
tb.start() is executed!!

–
View this message in context:
http://www.nabble.com/Packet-mod-demod-race-condition--tp12660275p24550834.html
Sent from the GnuRadio mailing list archive at Nabble.com.

stevenc · July 18, 2009, 8:52pm

I have used some print commands in these programs to follow the flow of
the
code. I used print commandsin a) benchmark_rx, b) receive_path c)
blks2impl/pkt.py d) packet_utils.py.
Running the program is get the output as follows :

In main now
in def main() now
calling tb in main()
in receive_path now
calling __setup_usrp_source
inside _setup_usrp_source: getting self.u now
usb_control_msg failed: usb_control_msg(DeviceRequestTO): pipe is
stalled
in def set_gain
in def set_freq
calculated channel coeff’s (-0.0, 2.611279946859197e-18,
-9.7454292139741081e-18, 1.9490858427948216e-17,
-2.9236288469102937e-17,
3.6370440010964533e-17, 1.0, 3.6370440010964533e-17,
-2.9236288469102937e-17, 1.9490858427948216e-17,
-9.7454292139741081e-18,
2.611279946859197e-18, -0.0)
making packet_receiver module…i will call pkt.py here

gr_fir_fff: using SSE # Here the pkt.py must be called
(demod_pkts() ), but instead it does not show me any output from pkt.py
made packet_receiver module…
options.log_rx_power = false
executing self.connect of self.u and packet_receiver
tb.start() in a while # I did not follow what exactly happens when
tb.start() is called? and what is the flow here?
tb.wait() now
in def rx_callback now
1
ok = True pktno = 0 n_rcvd = 1 n_right = 1
in def rx_callback now
2
ok = True pktno = 1 n_rcvd = 2 n_right = 2
.
.
.

I made changes in the
/usr/local/lib/python2.x/site-packages/gnuradio/blks2impl/pkt.py
and /usr/local/lib/python2.x/site-packages/gnuradio/packet_utils.py
files

Eric B. wrote:

reflect

Discuss-gnuradio mailing list
[email protected]
Discuss-gnuradio Info Page

–
View this message in context:
http://www.nabble.com/Packet-mod-demod-race-condition--tp12660275p24550733.html
Sent from the GnuRadio mailing list archive at Nabble.com.

stevenc · July 29, 2009, 10:27pm

Hi

i used the same code as benchmark_tx.py to make packets and saved
this in
a file
Now i try to read this file back and unmake packets. But the code
doesnt
do any thing at msg.delete_head().

My code is as follows:

class my_top_block(gr.top_block):
def init(self,rx_callback):
gr.top_block.init(self)

access_code=functions.conv_packed_binary_string_to_1_0_string(‘\xAC\xDD\xA4\xE2\xF2\x8C\x20\xFC’)
threshold = 1

    u = gr.file_source(gr.sizeof_char,"packetizer.data")
    correlator = gr.correlate_access_code_bb(access_code, threshold)
    rcvd_pktq = gr.msg_queue()
    framer_sink = gr.framer_sink_1(rcvd_pktq)
    self.connect(u,correlator,framer_sink)

   watcher = _queue_watcher_thread(rcvd_pktq,rx_callback)

class _queue_watcher_thread(_threading.Thread):
def init(self, rcvd_pktq, rx_callback):
_threading.Thread.init(self)
self.setDaemon(1)
self.rcvd_pktq = rcvd_pktq
self.rx_callback = rx_callback
self.keep_running = True
self.start()

def run(self):
    while self.keep_running:
     msg = self.rcvd_pktq.delete_head()

        ok,payload = packet_utils.unmake_packet(self.rcvd_pktq, 0)
        print "payload",payload
        if self.callback:
             self.rx_callback(ok, payload)
             break

def main():
print “in ‘def main()’ now”
def rx_callback(ok, payload):
print "in the def of rx_callback "
print payload
(pktno,) = struct.unpack(‘!H’, payload[0:2])
print ok

r = gr.enable_realtime_scheduling()
if r != gr.RT_OK:
    print "Warning: Failed to enable realtime scheduling."
tb = my_top_block(rx_callback)

tb.start()        # start flow graph

tb.wait()         # wait for it to finish

if name == ‘main’:

try:
    main()
except KeyboardInterrupt:
    pass

–
View this message in context:
http://www.nabble.com/Packet-mod-demod-race-condition--tp12660275p24726750.html
Sent from the GnuRadio mailing list archive at Nabble.com.

stevenc · July 29, 2009, 10:25pm

Hi

i used the same code as benchmark_tx.py to make packets and saved
this in
a file
Now i try to read this file back and unmake packets. But the code
doesnt
do any thing at msg.delete_head().

My code is as follows:

class my_top_block(gr.top_block):
def init(self,rx_callback):
gr.top_block.init(self)

access_code=functions.conv_packed_binary_string_to_1_0_string(‘\xAC\xDD\xA4\xE2\xF2\x8C\x20\xFC’)
threshold = 1

    u = gr.file_source(gr.sizeof_char,"packetizer.data")
    correlator = gr.correlate_access_code_bb(access_code, threshold)
    rcvd_pktq = gr.msg_queue()
    framer_sink = gr.framer_sink_1(rcvd_pktq)
    self.connect(u,correlator,framer_sink)

   watcher = _queue_watcher_thread(rcvd_pktq,rx_callback)

class _queue_watcher_thread(_threading.Thread):
def init(self, rcvd_pktq, rx_callback):
_threading.Thread.init(self)
self.setDaemon(1)
self.rcvd_pktq = rcvd_pktq
self.rx_callback = rx_callback
self.keep_running = True
self.start()

def run(self):
    while self.keep_running:
     msg = self.rcvd_pktq.delete_head()

        ok,payload = packet_utils.unmake_packet(self.rcvd_pktq, 0)
        print "payload",payload
        if self.callback:
             self.rx_callback(ok, payload)
             break

def main():
print “in ‘def main()’ now”
def rx_callback(ok, payload):
print "in the def of rx_callback "
print payload
(pktno,) = struct.unpack(‘!H’, payload[0:2])
print ok

r = gr.enable_realtime_scheduling()
if r != gr.RT_OK:
    print "Warning: Failed to enable realtime scheduling."
tb = my_top_block(rx_callback)

tb.start()        # start flow graph

tb.wait()         # wait for it to finish

if name == ‘main’:

try:
    main()
except KeyboardInterrupt:
    pass

Kindly Help…
Thanks

–
View this message in context:
http://www.nabble.com/Packet-mod-demod-race-condition--tp12660275p24726759.html
Sent from the GnuRadio mailing list archive at Nabble.com.

stevenc · July 30, 2009, 10:23pm

Thanks a ton. That helped a lot!

The same program now i use it with delete_head_nowait() which returns
0
when there’s nothing in the queue instead of blocking.
I also wanted to see if the queue is empty and used : empty_p()
function.
It gives a TRUE!!

i connected : file_source → correlator → framer_sink

So if my queue is empty, where is the data that i am reading, going?

udadidd wrote:

Now i try to read this file back and unmake packets. But the code
doesnt
do any thing at msg.delete_head().

delete_head() is a blocking function, which blocks until there is a
message in the queue. You might want to try using the non- blocking
version instead?

You may compile the documentation (if it isn’t done per default) which
per default locates itself in
/usr/local/share/doc/gnuradio-X.Xsvn/html/index.html

Cheers!
//Mattias

–
View this message in context:
http://www.nabble.com/Packet-mod-demod-race-condition--tp12660275p24746639.html
Sent from the GnuRadio mailing list archive at Nabble.com.

stevenc · July 30, 2009, 11:43pm

On Thu, Jul 30, 2009 at 01:20:25PM -0700, udadidd wrote:

It’s probably not seeing any valid packets, and thus there’s no
messages being sent from the framer_sink.

Eric

stevenc · July 30, 2009, 11:24am

udadidd wrote:

Now i try to read this file back and unmake packets. But the code doesnt
do any thing at msg.delete_head().

delete_head() is a blocking function, which blocks until there is a
message in the queue. You might want to try using the non- blocking
version instead?

You may compile the documentation (if it isn’t done per default) which
per default locates itself in
/usr/local/share/doc/gnuradio-X.Xsvn/html/index.html

Cheers!
//Mattias