RAM consumption

GNU Radio
1

I am running some Zigbee code from UCLA with some modifications of my
own…
When I run the (modified) code, RAM is slowly (but surely) consumed with
high CPU usage. After about half an hour, the CPU/RAM monitor shows
100%
RAM usage and then operation stalls and CPU usage drops down once again
to
quiescent levels, with RAM remaining tied up until the program is ended
(ctrl-C). Until the stall, program operation appears to be correct.

I have reviewed the code by hand for memory leaks and also used some of
the
tools available for detecting memory leaks without any results. Has
anyone
else had similar problems and, if so, how did they debug them? Is this
kind
of inexorable creep of system memory usage symptomatic of anything else?
Within GnuRadio, is there any method for quickly determining how program
memory (heap space) or data memory is being consumed?

/ David K.

View this message in context:
http://old.nabble.com/RAM-consumption-tp30041914p30041914.html
Sent from the GnuRadio mailing list archive at Nabble.com.

2

On Sun, Oct 24, 2010 at 09:19:25AM -0700, David K. wrote:

else had similar problems and, if so, how did they debug them? Is this kind
of inexorable creep of system memory usage symptomatic of anything else?
Within GnuRadio, is there any method for quickly determining how program
memory (heap space) or data memory is being consumed?

99% of the base code distributed with GNU Radio (runtime + blocks)
allocates NO MEMORY once the flow graph has started (see below for
exceptions).

DO NOT use a vector_sink for anything other than QA code that
produces a small amount of output. The vector_sink will allocate an
unbounded amount of memory if you keep writing to it.

It is also possible to consume memory by sending an unbounded number
of messages into a message queue (gr.msg_queue) that doesn’t have a
queue size limit and has a slow (or compute bound) reader.

If there’s a GUI of any kind involved, or python code that’s being
executed beyond initialization time, look there for problems.

Eric

3

Eric B. wrote:

of messages into a message queue (gr.msg_queue) that doesn’t have a
queue size limit and has a slow (or compute bound) reader.

If there’s a GUI of any kind involved, or python code that’s being
executed beyond initialization time, look there for problems.

Eric

Thanks for such a quick response. I am modifying C++ code, rather than
Python code. There is no GUI (just an XTERM accessed via std::cout
statements). I am using std::queue data structures and the push and pop
to
these queues occurs in the same routine (see below).

I think that your vector sink comments refer to the Python construct,
don’t
they? What is the C++ equivalent of your ‘DO NOT DO THIS’ statement?
Does
this mean I have to access the queue in a critical section or using
explicit
thread-safe methods? My output seems to be sequenced just fine and
there
are no duplicates or the like.

My C++ implementation is:
…
if ( (int)queue.size() >= MAX) { queue.pop() }
queue.push(value)
…
It is possible that the CPU is not able to keep up with the emptying
task,
but then I’d expect the queue size to be growing beyond MAX and it
isn’t…
based on std::cout output anyway. I suppose that this output could be
also
lagging further and further behind, but the output all appears to
complete
properly as each of my relatively rare test events occur (~1 second
apart).

Is there something specific that I must do inside the C++ code to avoid
GnuRadio run-time memory allocation issues? I have not specifically
added
any new malloc/dalloc/calloc statements to the existing code myself.
Could
declaring data structures (e.g. scalars, arrays, std::queue) inside or
external to subroutines have this kind of side effect? Would I be
better
off implementing the queue using an array/circular buffer or std::vector
of
my own, rather than using the one from the std:: library?

My overall objective is that I want to be able to capture and print off
samples from the ADC on a specific trigger characteristic of the sample
values (e.g. magnitude). So, I need to remember (either print out to
the
XTERM or store to a file) small subsets of contiguous ADC samples that
occurred just prior to my trigger condition. Once they are printed, I
want
to ‘forget’ them entirely and start looking for my trigger again. In my
testing, this amounts to printing about 50 lines every second or so, but
I
am consuming RAM at a steady rate of about 3 MB/s.

/ David K.

View this message in context:
http://old.nabble.com/RAM-consumption-tp30041914p30043424.html
Sent from the GnuRadio mailing list archive at Nabble.com.

4

On Sun, Oct 24, 2010 at 02:28:04PM -0700, David K. wrote:

unbounded amount of memory if you keep writing to it.

Thanks for such a quick response. I am modifying C++ code, rather than
Python code. There is no GUI (just an XTERM accessed via std::cout
statements). I am using std::queue data structures and the push and pop to
these queues occurs in the same routine (see below).

Is there only a single thread that manipulates the queue, or is one
for example inside of a GNU Radio block, and one is outside of the
block?

I think that your vector sink comments refer to the Python construct, don’t
they? What is the C++ equivalent of your ‘DO NOT DO THIS’
statement?

They map 1:1 from Python to C++:

gr.msg_queue -> gr_msg_queue
gr.vector_sink_* -> gr_vector_sink_*

Does this mean I have to access the queue in a critical section or using
explicit
thread-safe methods?

As in any multithreaded programming, if there is more than one thread
involved, and you are accessing a shared data structure from more than
one thread, and somebody else isn’t already handling the critical
section for you, you WILL have to implement a critical section.

STL containers are not inherently thread-safe.

My output seems to be sequenced just fine and there
are no duplicates or the like.

Is there something specific that I must do inside the C++ code to avoid
GnuRadio run-time memory allocation issues?

No.

I have not specifically added
any new malloc/dalloc/calloc statements to the existing code myself. Could
declaring data structures (e.g. scalars, arrays, std::queue) inside or
external to subroutines have this kind of side effect?

Of course they could. If you’re using some kind of container, you
need to know what it’s worst case running time and memory usage is.

Would I be better
off implementing the queue using an array/circular buffer or std::vector of
my own, rather than using the one from the std:: library?

It all depends.

Have you written a new C++ block for GNU Radio?
If so, have you written any QA code for it?

If you’ve written a block, and you replace the guts of your work or
general_work method with:

return noutput_items; // turn block into a NOP

does the memory increase stop?

If you haven’t written a new block, how are you getting access to the
samples?

My overall objective is that I want to be able to capture and print off
samples from the ADC on a specific trigger characteristic of the sample
values (e.g. magnitude). So, I need to remember (either print out to the
XTERM or store to a file) small subsets of contiguous ADC samples that
occurred just prior to my trigger condition. Once they are printed, I want
to ‘forget’ them entirely and start looking for my trigger again. In my
testing, this amounts to printing about 50 lines every second or so, but I
am consuming RAM at a steady rate of about 3 MB/s.

Seems simple enough. Instead of us trying to guess what you’re actually
doing, can you post a link to the code including an example that
exercises it?

/ David K.

Eric