Strange output of "0" at the terminal

GNU Radio
1

Hi all,

I’m now using message_sink and msg_queue to receive data from USRP. I do
some calculation for all the data in the msg_queue one by one and write
some
of them into a file. Everything seems to be working smoothly. But once
in a
while, a “0” is printed in the terminal. (There is no code to print “0”
in
my program.) I checked the data, and found that every time a “0” is
printed,
some data are lost, and the length of lost data seems to be of hundreds
of
messages.

There is no other error information, so I’m really confused by this
problem.
Does anyone has a clue of what happened and how should I deal with it?

Thanks,

Wu

2

The output is O (Oh) not 0 (zero).

I made more tests and feel the problem may be from use of write() to
write
data into files. Anyone had similar problem?

Wu

From:
discuss-gnuradio-bounces+wu.ting=removed_email_address@domain.invalid
[mailto:discuss-gnuradio-bounces+wu.ting=removed_email_address@domain.invalid
g] On Behalf Of Wu Ting
Sent: 2012222 14:48
To: [email protected]
Subject: [Discuss-gnuradio] Strange output of “0” at the terminal

Hi all,

Im now using message_sink and msg_queue to receive data from USRP. I do
some calculation for all the data in the msg_queue one by one and write
some
of them into a file. Everything seems to be working smoothly. But once
in a
while, a 0 is printed in the terminal. (There is no code to print 0
in my program.) I checked the data, and found that every time a 0 is
printed, some data are lost, and the length of lost data seems to be of
hundreds of messages.

There is no other error information, so Im really confused by this
problem. Does anyone has a clue of what happened and how should I deal
with
it?

Thanks,

Wu

3

“O” means there has been an overflow, some part of your system is not
fast enough to keep up with the incoming data, probably your hard
drive, or you may not have a fast enough CPU to process as the sample
rate you have chosen.

2012/2/22 Wu Ting [email protected]:

4

Hi! Thank you for your response. I’ve kept working on this problem for
two
days, but still cannot find a way to solve it.

I simplified the program and have determined that the ‘O’ is produced is
this while loop:

while msgCount<10000:
msg = tb.queue.delete_head()
payload = msg.to_string()
f.write(payload)
msgCount += 1

I also tried to make it sleep for a short time after each operation:

while msgCount<10000:
msg = tb.queue.delete_head()
sleep(0.00001)
payload = msg.to_string()
sleep(0.00001)
f.write(payload)
sleep(0.00001)
msgCount += 1

It still has the problem. I’m using a USRP source with 4M smapling rate,
and
connect it with message_sink.

self.source = uhd.usrp_source(device_addr="",
stream_args=uhd.stream_args(‘sc16’, ‘sc16’, args=“scalar=1024”))
self.source.set_samp_rate(4e6)
self.queue = gr.msg_queue()
self.sink = gr.message_sink(gr.sizeof_short*2, self.queue, False)
self.connect(self.source, self.sink)

This is really a serious problem for our application because we want to
continuously record some data. Does anyone has any idea how to deal with
this problem, or at least catch this error when it happens? Any
suggestions
will be greatly appreciated.

Wu

5

As I said, your system is not processing the samples fast enough and a
buffer overflows and samples are lost. You say it happends durring that
loop, that might use enough CPU time to cause these problems. Your
options
are to get a faster system, sample at a lower rate or find a way to make
your code faster.

On Thursday, February 23, 2012, Wu Ting <
[email protected]> wrote:

msgCount += 1
msgCount += 1

It still has the problem. I’m using a USRP source with 4M smapling rate,
and
continuously record some data. Does anyone has any idea how to deal with
this problem, or at least catch this error when it happens? Any
suggestions

6

Hi! Thank you for your suggestions!

I realized it is a problem related with computer speed. Today I used a
laptop to run the same code, and ‘O’ is printed much more frequently.

However, the computer I’m using is a quite good one. It has intel i7
3.4GHz 8-core, and when the code is running, CPU is only used by about
10%, and memory is also used by no more than 15%. On the other hand, I
also checked number of messages in message queue, and I’ve found that
when
‘O’ is printed, sometimes there are only several messages in the queue.
So
I don’t understand why it has this outflow problem.

And is it possible to catch this error when it happens, so that I can
deal
with it in the problem?

Thanks!

Wu

7

2012/2/23 Wu Ting [email protected]

msgCount += 1
msgCount += 1

Two quick things. First, the tb.queue.delete_head() is a blocking call,
so
it will wait until there is a message to process. You don’t need to
sleep.
Adding a sleep call here is probably only making things worse since
you’re
already not keeping up with the samples. Second, the sleep() call is
generally only accurate to about ~10 ms, but you’re asking it to sleep
for
10 us. I’m not sure if it rounds up or down. Worse case, you’re making
the
loop sleep for about 30 ms total; best case is you actually aren’t
pausing
at all.

Tom

8

You are writing to a file in a blocking thread, its not the CPU it’s
the hard drive. You should try the program without the write. Then if
it is the problem you should write out to a file on a ram disk then
save it to a real disk later.

2012/2/24 Wu Ting [email protected]:

9

Modern disk subsystems can easily sustain 80MB or more per second
even on relativley wimpy hardware

What is the sample rate involved
here? Seriously, I’ve been able to stream multi-MHz of bandwidth to disk
for long periods without any ‘O’ happening.

-Marcus

On Fri, 24 Feb
2012 11:23:05 -0500, Andrew D. wrote:

You are writing to a file
in a blocking thread, its not the CPU it’s
the hard drive. You should
try the program without the write. Then if
it is the problem you
should write out to a file on a ram disk then
save it to a real disk
later.

2012/2/24 Wu Ting :

Hi! Thank you for your suggestions! I
realized it is a problem related with computer speed. Today I used a
laptop to run the same code, and ‘O’ is printed much more frequently.
However, the computer I’m using is a quite good one. It has intel i7
3.4GHz 8-core, and when the code is running, CPU is only used by about
10%, and memory is also used by no more than 15%. On the other hand, I
also checked number of messages in message queue, and I’ve found that
when ‘O’ is printed, sometimes there are only several messages in the
queue. So I don’t understand why it has this outflow problem. And is it
possible to catch this error when it happens, so that I can deal with it
in the problem? Thanks! Wu

2012/2/23 Wu Ting

Hi!
Thank you for your response. I’ve kept working on this problem for two
days, but still cannot find a way to solve it. I simplified the program
and have determined that the ‘O’ is produced is this while loop: while
msgCount
Two quick things. First, the tb.queue.delete_head() is a
blocking call, so it will wait until there is a message to process. You
don’t need to sleep. Adding a sleep call here is probably only making
things worse since you’re already not keeping up with the samples.
Second, the sleep() call is generally only accurate to about ~10 ms, but
you’re asking it to sleep for 10 us. I’m not sure if it rounds up or
down. Worse case, you’re making the loop sleep for about 30 ms total;
best case is you actually aren’t pausing at all. Tom

_______________________________________________ Discuss-gnuradio mailing
list [email protected] [3]
Discuss-gnuradio Info Page [4]

Links:

[1] mailto:[email protected]
[2]
mailto:[email protected]
[3]
mailto:[email protected]
[4]
Discuss-gnuradio Info Page

10

Hi! Marcus,

The sample rate is 4MHz. I also tried 1MHz and 10MHz. On my laptop, the
frequency of ‘O’ is related with sampling rate and is quite common. On
the
desktop, I always have to wait quite a while before an ‘O’ is printed.
But
it is printed even when the sampling rate is 200kHz.

Following is my code. Any suggestions?

#!/usr/bin/env python

from gnuradio import gr
from gnuradio import uhd
import time
from struct import unpack
from time import sleep

class probe_this(gr.top_block):
def init(self):
gr.top_block.init(self)
self.source = uhd.usrp_source(device_addr="",
stream_args=uhd.stream_args(‘sc16’, ‘sc16’, args=“scalar=1024”))

    self.source.set_samp_rate(4e6)
    self.source.set_gain(1)
    self.source.set_center_freq(0)

    self.queue = gr.msg_queue()
    self.sink = gr.message_sink(gr.sizeof_short*2, self.queue,

False)
self.connect(self.source, self.sink)

if name==“main”:
tb = probe_this()
tb.start()

while True:
    msg = tb.queue.delete_head()
    payload = msg.to_string()
    loadLen = int(msg.arg2())
    format = str(loadLen*2)+'h'
    data = unpack(format, payload)
    datach1 = data[0::2]
    datach2 = data[1::2]

    maxVal = max(datach1)
    minVal = min(datach1)
    if maxVal>6 or minVal<-6:
        print 'K'
        fileName = time.time()
        f = open(str(fileName), 'wb')
        f.write(payload)

        msgCount = 1
        print 'T'
        while msgCount<10000:
            msg = tb.queue.delete_head()
            payload = msg.to_string()
            f.write(payload)
            msgCount += 1
        f.close()
        msgNum = tb.queue.count()
        print msgNum
11

I tried the code without write(), and it still has the problem. But it
seems to be less frequent. I will try your method. Thank you!