Tunnel.py exception

GNU Radio
1

I am running tunnel.py on gnuradio 3.3.0 . It run successfully for a
while but
after a period of time (around an hour) the following exception prints
out:

Rx: ok = True len(payload) = 82
Tx: len(payload) = 82
Exception in thread Thread-1:
Traceback (most recent call last):
File “/usr/lib64/python2.6/threading.py”, line 525, in
__bootstrap_inner
self.run()
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/blks2impl/pkt.py”,
line 162, in run
ok, payload = packet_utils.unmake_packet(msg.to_string(),
int(msg.arg1()))
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line
183, in unmake_packet
payload_with_crc = dewhiten(whitened_payload_with_crc,
whitener_offset)
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line
95, in dewhiten
return whiten(s, o) # self inverse
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line
91, in whiten
z = sa ^ random_mask_vec8[o:len(sa)+o]
ValueError: shape mismatch: objects cannot be broadcast to a single
shape

After this exception the receive chain seems to stop working. I am still
able to
transmit but no receive packets are recorded.

Anyone have any clue what could be causing this issue?

Thanks,
Dave

2

On Wed, Apr 20, 2011 at 9:25 AM, David B.
[email protected]wrote:

File
File “/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
Thanks,
Dave

I think that the problem is when the receiver thinks it has a
zero-length
packet (that is, something gets screwed up with the header and it sees
0’s
there). I’m not positive that this is the real problem, but I’d say it
has
something to do with a packet getting corrupted in a particular way
that’s
causing this to happen.

We would need to put some protections into the unmake_packet to handle
this
as a dropped packet and then continue, once we find the exact problem.

Tom

3

I am working with two USRPS wired connection with 25 dB attenuator
between them
so I didnt expect to much corruption of the packets.

The error seems to occur quicker at lower bit rates for some reason ,
like
around after 10s of minutes for below 250 kbps and more like after an
hour or
more for 1 Mbps. Unfortunatly this makes it unusable for longer
durationlower
bandwidth tests until I find a way to fix the problem.

Has anyone else had this problem with tunnel.py?

Thanks,
Dave

From: Tom R. [email protected]
To: David B. [email protected]
Cc: [email protected]
Sent: Thu, April 21, 2011 10:22:39 AM
Subject: Re: [Discuss-gnuradio] Tunnel.py exception

On Wed, Apr 20, 2011 at 9:25 AM, David B. [email protected]
wrote:

I am running tunnel.py on gnuradio 3.3.0 . It run successfully for a
while but
after a period of time (around an hour) the following exception prints
out:

File “/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”, line

After this exception the receive chain seems to stop working. I am still able to
transmit but no receive packets are recorded.

Anyone have any clue what could be causing this issue?

Thanks,
Dave

I think that the problem is when the receiver thinks it has a
zero-length packet
(that is, something gets screwed up with the header and it sees 0’s
there). I’m
not positive that this is the real problem, but I’d say it has something
to do
with a packet getting corrupted in a particular way that’s causing this
to
happen.

We would need to put some protections into the unmake_packet to handle
this as a
dropped packet and then continue, once we find the exact problem.

Tom

4

Dave,

To bypass this problem, change the pkt.py file. In the end, after

msg = self.rcvd_pktq.delete_head()

add

if (string_len > 18) & (string_len < 4096) :
ok, payload = packet_utils.unmake_packet(msg.to_string(),
int(msg.arg1()))

Andrew

5

I’ve recently been using tunnel.py for 1/2 hr tests with no problems.
I’m
using the basic_tx/rx boards with a low frequency (5 MHz) and 1 Mbit
datarate.

6

Sorry that I skipped one line when I made the copy

msg = self.rcvd_pktq.delete_head()
string_len = len(msg.to_string())

7

On Thu, Apr 21, 2011 at 4:26 PM, Feng Andrew Ge
[email protected]wrote:

ok, payload = packet_utils.unmake_packet(msg.to_string(),

int(msg.arg1()))

Andrew

Thanks, Andrew, that looks about right. I’ll keep that in mind to apply
a
patch soon.

Tom

8

Hello William, Hello David,

i had this problem too.

I changed my tunnel.py to understand mac better into a simple program
which
sends a packet and when the receiver gets the packet the receiver sends
an
acknowledge back.

After a short time period the exception was thrown.

I found out that the gnu which is the receiver cannot handle the speed
of
the sender - gnu so the exception was raised. I inserted some
time.sleep()'s
in the sender and the exception was gone.

Kind Regards

Von: [email protected]
[mailto:[email protected]] Im Auftrag
von
William C.
Gesendet: Donnerstag, 21. April 2011 16:40
An: David B.
Cc: [email protected]
Betreff: Re: [Discuss-gnuradio] Tunnel.py exception

I’ve recently been using tunnel.py for 1/2 hr tests with no problems.
I’m
using the basic_tx/rx boards with a low frequency (5 MHz) and 1 Mbit
datarate.

On Thu, Apr 21, 2011 at 10:31 AM, David B.
[email protected]
wrote:

I am working with two USRPS wired connection with 25 dB attenuator
between
them so I didnt expect to much corruption of the packets.

The error seems to occur quicker at lower bit rates for some reason ,
like
around after 10s of minutes for below 250 kbps and more like after an
hour
or more for 1 Mbps. Unfortunatly this makes it unusable for longer
duration
lower bandwidth tests until I find a way to fix the problem.

Has anyone else had this problem with tunnel.py?

Thanks,

Dave

From: Tom R. [email protected]
To: David B. [email protected]
Cc: [email protected]
Sent: Thu, April 21, 2011 10:22:39 AM
Subject: Re: [Discuss-gnuradio] Tunnel.py exception

On Wed, Apr 20, 2011 at 9:25 AM, David B. [email protected]
wrote:

I am running tunnel.py on gnuradio 3.3.0 . It run successfully for a
while
but after a period of time (around an hour) the following exception
prints
out:

Rx: ok = True len(payload) = 82
Tx: len(payload) = 82
Exception in thread Thread-1:
Traceback (most recent call last):
File “/usr/lib64/python2.6/threading.py”, line 525, in
__bootstrap_inner
self.run()
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/blks2impl/pkt.py”,
line 162, in run
ok, payload = packet_utils.unmake_packet(msg.to_string(),
int(msg.arg1()))
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line 183, in unmake_packet
payload_with_crc = dewhiten(whitened_payload_with_crc,
whitener_offset)
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line 95, in dewhiten
return whiten(s, o) # self inverse
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line 91, in whiten
z = sa ^ random_mask_vec8[o:len(sa)+o]
ValueError: shape mismatch: objects cannot be broadcast to a single
shape

After this exception the receive chain seems to stop working. I am still
able to transmit but no receive packets are recorded.

Anyone have any clue what could be causing this issue?

Thanks,
Dave

I think that the problem is when the receiver thinks it has a
zero-length
packet (that is, something gets screwed up with the header and it sees
0’s
there). I’m not positive that this is the real problem, but I’d say it
has
something to do with a packet getting corrupted in a particular way
that’s
causing this to happen.

We would need to put some protections into the unmake_packet to handle
this
as a dropped packet and then continue, once we find the exact problem.

Tom

9

Dave,

In the patch I told you, please change 4096 to 4095, that is,

if (string_len> 18)& (string_len< 4095) :

Here is how this happened:

When you send a packet in GNU Radio, there is a header right ahead the
payload (plus CRC bits).
The header has 4 bytes which consist of two same 2-byte information. In
each 2-byte, the 12 least significant bits represent the length
of your payload plus CRC; the 4 most significant bits represent
whitening offset information.

When you receive the packet, the two lengths contained in each of the
2-byte in the header are compared. If they are the same, say, both x,
the receiver
will get the next x bytes information; otherwise, the receiver assumes
that the packet is corrupted.

Currently GNU Radio doesn’t have any error correction code. Therefore,
the header can be corrupted under low SNR or interference.
Even with corruption or interference, in probability, sometimes you will
still see the two length information in the header are the same.

In your case (of course it happened to me before, otherwise I won’t
know), you see 4095 of string_len, it means that the 12 least
significant bits
in each of the 2 bytes (in the header) are all 1’s, that is " 1111 1111
1111 " (=4095). However, the contained payload—not matter what it
is—is actually wrong.

Therefore, the cause is corruption under low SNR or interference. The
missing part is error correction code.

By applying the above patch, you can bypass this python code problem.

Andrew

Posted by David B. (Guest)
on 2011-04-29 15:26

I tried the previously suggested patch that checked the str_len of the
message
before unmaking the packet but errors still occurred. I noticed that
when
printing the size fo packet out the errors occur when the str_len is
exactly
equal to 4095. There is no reason it should be that length as I am only
periodically sending short ping packets. I have yet to figure out why it
is
mistaking the message length. Anyone have any ideas?

Thomas,

What do you mean that the receiver couldnt handle the sender speed?
Where were
the sleeps put in?

Thanks,
Dave

10

Thank you for the explanation. I will try this and let you know.

I had one question though. It seems odd to me that the interference will
always
cause the header to be corrupted to all ones for both sets of 2 bytes .
Wouldnt
it be more likely to have sent 80 bytes payload and have lets say 1 bit
corrupted ineach (like both change to 81 instead of 80) which would
cause a
error I would think. Since I always see 4095 length as error it seems
weird that
so many bits are corrupted and all corrputed to be all 1’s. I just want
to make
sure I understand the root cause of the issue.

Thanks,
Dave

From: Feng Andrew Ge [email protected]
To: [email protected]; David B. [email protected]
Sent: Fri, April 29, 2011 10:35:52 AM
Subject: Re: [Discuss-gnuradio] Tunnel.py exception

Dave,

In the patch I told you, please change 4096 to 4095, that is,

if (string_len> 18)& (string_len< 4095) :

Here is how this happened:

When you send a packet in GNU Radio, there is a header right ahead the
payload
(plus CRC bits).
The header has 4 bytes which consist of two same 2-byte information. In
each
2-byte, the 12 least significant bits represent the length
of your payload plus CRC; the 4 most significant bits represent
whitening offset
information.

When you receive the packet, the two lengths contained in each of the
2-byte in
the header are compared. If they are the same, say, both x, the receiver
will get the next x bytes information; otherwise, the receiver assumes
that the
packet is corrupted.

Currently GNU Radio doesn’t have any error correction code. Therefore,
the
header can be corrupted under low SNR or interference.
Even with corruption or interference, in probability, sometimes you will
still
see the two length information in the header are the same.

In your case (of course it happened to me before, otherwise I won’t
know), you
see 4095 of string_len, it means that the 12 least significant bits
in each of the 2 bytes (in the header) are all 1’s, that is " 1111 1111
1111 "
(=4095). However, the contained payload—not matter what it is—is
actually
wrong.

Therefore, the cause is corruption under low SNR or interference. The
missing
part is error correction code.

By applying the above patch, you can bypass this python code problem.

Andrew

Posted by David B. (Guest)
on 2011-04-29 15:26

I tried the previously suggested patch that checked the str_len of the
message
before unmaking the packet but errors still occurred. I noticed that
when
printing the size fo packet out the errors occur when the str_len is
exactly
equal to 4095. There is no reason it should be that length as I am only
periodically sending short ping packets. I have yet to figure out why it
is
mistaking the message length. Anyone have any ideas?

Thomas,

What do you mean that the receiver couldnt handle the sender speed?
Where were
the sleeps put in?

Thanks,
Dave

11

I tried the previously suggested patch that checked the str_len of the
message
before unmaking the packet but errors still occurred. I noticed that
when
printing the size fo packet out the errors occur when the str_len is
exactly
equal to 4095. There is no reason it should be that length as I am only
periodically sending short ping packets. I have yet to figure out why it
is
mistaking the message length. Anyone have any ideas?

Thomas,

What do you mean that the receiver couldnt handle the sender speed?
Where were
the sleeps put in?

Thanks,
Dave

From: Thomas H. [email protected]
To: William C. [email protected]; David B.
[email protected]
Cc: [email protected]
Sent: Wed, April 27, 2011 8:39:26 AM
Subject: AW: [Discuss-gnuradio] Tunnel.py exception

Hello William, Hello David,

i had this problem too.

I changed my tunnel.py to understand mac better into a simple program
which
sends a packet and when the receiver gets the packet the receiver sends
an
acknowledge back.

After a short time period the exception was thrown.

I found out that the gnu which is the receiver cannot handle the speed
of the
sender – gnu so the exception was raised. I inserted some time.sleep()’s
in the
sender and the exception was gone.

Kind Regards

Von:[email protected]
[mailto:[email protected]] Im Auftrag
von
William C.
Gesendet: Donnerstag, 21. April 2011 16:40
An: David B.
Cc: [email protected]
Betreff: Re: [Discuss-gnuradio] Tunnel.py exception

I’ve recently been using tunnel.py for 1/2 hr tests with no problems.
I’m using
the basic_tx/rx boards with a low frequency (5 MHz) and 1 Mbit datarate.
On Thu, Apr 21, 2011 at 10:31 AM, David B.
[email protected] wrote:
I am working with two USRPS wired connection with 25 dB attenuator
between them
so I didnt expect to much corruption of the packets.

The error seems to occur quicker at lower bit rates for some reason ,
like
around after 10s of minutes for below 250 kbps and more like after an
hour or
more for 1 Mbps. Unfortunatly this makes it unusable for longer
duration lower
bandwidth tests until I find a way to fix the problem.

Has anyone else had this problem with tunnel.py?
Thanks,
Dave

From:Tom R. [email protected]
To: David B. [email protected]
Cc: [email protected]
Sent: Thu, April 21, 2011 10:22:39 AM
Subject: Re: [Discuss-gnuradio] Tunnel.py exception

On Wed, Apr 20, 2011 at 9:25 AM, David B. [email protected]
wrote:
I am running tunnel.py on gnuradio 3.3.0 . It run successfully for a
while but
after a period of time (around an hour) the following exception prints
out:

Rx: ok = True len(payload) = 82
Tx: len(payload) = 82
Exception in thread Thread-1:
Traceback (most recent call last):
File “/usr/lib64/python2.6/threading.py”, line 525, in
__bootstrap_inner
self.run()
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/blks2impl/pkt.py”,
line 162, in run
ok, payload = packet_utils.unmake_packet(msg.to_string(),
int(msg.arg1()))
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line
183, in unmake_packet
payload_with_crc = dewhiten(whitened_payload_with_crc,
whitener_offset)
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line
95, in dewhiten
return whiten(s, o) # self inverse
File
“/usr/local/lib64/python2.6/site-packages/gnuradio/packet_utils.py”,
line
91, in whiten
z = sa ^ random_mask_vec8[o:len(sa)+o]
ValueError: shape mismatch: objects cannot be broadcast to a single
shape

After this exception the receive chain seems to stop working. I am still
able to
transmit but no receive packets are recorded.

Anyone have any clue what could be causing this issue?

Thanks,
Dave

I think that the problem is when the receiver thinks it has a
zero-length packet
(that is, something gets screwed up with the header and it sees 0’s
there). I’m
not positive that this is the real problem, but I’d say it has something
to do
with a packet getting corrupted in a particular way that’s causing this
to
happen.

We would need to put some protections into the unmake_packet to handle
this as a
dropped packet and then continue, once we find the exact problem.

Tom

12

Dave, yes, what you described is more likely to happen. That will
corrupt your received data.

What I described is a special case (with less probability) explaining
why you got a payload with a length of 4095 bytes.

Andrew

13

Ok. Would the case I described also cause the same shape exception
killing the
receive chain. Because anytime it fails on me it always is with 4095
which I
cant figure out why.

Thanks,
Dave

From: Feng Andrew Ge [email protected]
To: David B. [email protected]
Cc: [email protected]
Sent: Fri, April 29, 2011 2:57:28 PM
Subject: Re: [Discuss-gnuradio] Tunnel.py exception

Dave, yes, what you described is more likely to happen. That will
corrupt your
received data.

What I described is a special case (with less probability) explaining
why you
got a payload with a length of 4095 bytes.

Andrew

On 04/29/2011 01:52 PM, David B. wrote:
Thank you for the explanation. I will try this and let you know.

Dave

From: Feng Andrew Ge [email protected]

14

From my experience, the patch does the trigger.

if (string_len> 18)& (string_len< 4095) :

Andrew

15

I have been running it for a few days and have no issues now. The tunnel
seems
to continue working for days.

Thanks for your help!
Dave

From: Feng Andrew Ge [email protected]
To: David B. [email protected]
Cc: [email protected]
Sent: Fri, April 29, 2011 4:02:06 PM
Subject: Re: [Discuss-gnuradio] Tunnel.py exception

From my experience, the patch does the trigger.

if (string_len> 18)& (string_len< 4095) :

Andrew

On 04/29/2011 04:00 PM, David B. wrote:
Ok. Would the case I described also cause the same shape exception
killing the
receive chain. Because anytime it fails on me it always is with 4095
which I
cant figure out why.

Thanks,
Dave

From: Feng Andrew Ge [email protected]

got a payload with a length of 4095 bytes.

corrupted ineach (like both change to 81 instead of 80) which would cause a
error I would think. Since I always see 4095 length as error it seems weird that
so many bits are corrupted and all corrputed to be all 1’s. I just want to make
sure I understand the root cause of the issue.

Thanks,
Dave

From: Feng Andrew Ge [email protected]