Crashes, memory errors and valgrind

GNU Radio
#1

Hello list,

I’m playing around with gqrx, an excellent example which great
standalone software can be built with gnuradio.

I’m running phirsch’s rtlsdr fork, do not have a Funcube Dongle or USRP
at hand, but rather a Terratex NOXON DAB stick - 25 part does the trick
fine enough for me :wink:

I’m running a self-compiled gnuradio rev3.6.0 on my Debian testing amd64
machine.

Sometime it crashes with messages like “glibc detected *** gqrx:
corrupted double-linked list”. Firing up valgrind on it I find -besides
several uninitialized variables- messages of the kind:

==31294== Invalid read of size 8
==31294== at 0x4F782AC: float_dotprod_sse (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
[deleted some lines…]
==31294== Address 0x1a211ba8 is 440 bytes inside a block of size 447
alloc’d
==31294== at 0x402894D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31294== by 0x4FB3C5E: malloc16Align (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4FB3CA9: calloc16Align (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F75E27: gr_fir_fff_simd::set_taps(std::vector<float,
std::allocator > const&) (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
[deleted some lines…]

Full trace of that error below, I can provide a full trace of a run if
needed.

Usually the program runs stable, but occasionally it crashes. I played
around with the internal bandwidth to get better FM reception - 75kHz is
quite narrow for FM reception. This change seems to trigger the problems
much faster. I hardly can change any parameters while running without a
crash.

I’m no expert, but it seems to me there is not enough memory allocated.
Maybe an off-by-one error? Did anyone see things like that or used
valgrind recently on gnuradio programs? What could I do to find out more
about this problem?

Regards

Patrick

==31294== Invalid read of size 8
==31294== at 0x4F782AC: float_dotprod_sse (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F8A27F:
gr_rational_resampler_base_fff::general_work(int, std::vector<int,
std::allocator >&, std::vector<void const*, std::allocator<void
const*> >&, std::vector<void*, std::allocator<void*> >&) (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F45CBD: gr_block_executor::run_one_iteration() (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F65AA8:
gr_tpb_thread_body::gr_tpb_thread_body(boost::shared_ptr<gr_block>, int)
(in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F60645:
boost::detail::function::void_function_obj_invoker0<gruel::thread_body_wrapper<tpb_container>,
void>::invoke(boost::detail::function::function_buffer&) (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x52BBCCD:
boost::detail::thread_data<boost::function0 >::run() (in
/usr/local/lib/libgruel-3.6.0.so.0.0.0)
==31294== by 0x81EA928: ??? (in /usr/lib/libboost_thread.so.1.49.0)
==31294== by 0x6C08B4F: start_thread (pthread_create.c:304)
==31294== by 0x76959DC: clone (clone.S:112)
==31294== Address 0x1a211ba8 is 440 bytes inside a block of size 447
alloc’d
==31294== at 0x402894D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31294== by 0x4FB3C5E: malloc16Align (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4FB3CA9: calloc16Align (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F75E27: gr_fir_fff_simd::set_taps(std::vector<float,
std::allocator > const&) (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F8A144:
gr_rational_resampler_base_fff::install_taps(std::vector<float,
std::allocator > const&) (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F8A5B6:
gr_rational_resampler_base_fff::gr_rational_resampler_base_fff(unsigned
int, unsigned int, std::vector<float, std::allocator > const&)
(in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F8A826: gr_make_rational_resampler_base_fff(unsigned
int, unsigned int, std::vector<float, std::allocator > const&)
(in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x44A41D: resampler_ff::resampler_ff(unsigned int,
unsigned int) (resampler_ff.cpp:71)
==31294== by 0x44A025: make_resampler_ff(unsigned int, unsigned int)
(resampler_ff.cpp:33)
==31294== by 0x4141C9: receiver::receiver(std::string, std::string)
(receiver.cpp:80)
==31294== by 0x41DBD9: MainWindow::MainWindow(QString, QWidget*)
(mainwindow.cpp:64)
==31294== by 0x41D4D7: main (main.cpp:52)

–
Engineers motto: cheap, good, fast: choose any two
Patrick S.
Student of Telematics, Graz Univ. of Technology, Austria

#2

On Sat, Jun 2, 2012 at 5:50 AM, Patrick S.
[email protected] wrote:

machine.
alloc’d

Maybe an off-by-one error? Did anyone see things like that or used
==31294== at 0x4F782AC: float_dotprod_sse (in
/usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0)
==31294== by 0x4F8A27F:
gr_rational_resampler_base_fff::general_work(int, std::vector<int,

Patrick,

It looks like you’re problem is in the rational_resampler code. I
wonder if there’s something about the resampling rate being used
that’s causing something to go out of bounds here. Can you dig into
the code and figure out what interpolation and decimation rates are
being used?

Tom

#3

Hi Tom,

Tom R. wrote on 2012-06-03 17:12:

On Sat, Jun 2, 2012 at 5:50 AM, Patrick S.
Patrick,

It looks like you’re problem is in the rational_resampler code. I
wonder if there’s something about the resampling rate being used
that’s causing something to go out of bounds here. Can you dig into
the code and figure out what interpolation and decimation rates are
being used?

Interpolation is 1, decimation is 2.

I compiled GNU Radio branch v3.6.0 with
$ cmake -DCMAKE_BUILD_TYPE=Debug …
which resulted in a compile with flags -g -O2 .
I tried to track what’s happening in gdb, in gr_fir_fff_simd::set_taps,
but -O2 was not helping, a lot optimized out, loops unrolled.

Full valgrind log in
http://pastebin.com/7GCs3bWy

regards

Patrick

Engineers motto: cheap, good, fast: choose any two
Patrick S.
Student of Telematics, Graz Univ. of Technology, Austria

#4

Patrick S. wrote on 2012-06-03 20:22:

Hi Tom,

Tom R. wrote on 2012-06-03 17:12:

On Sat, Jun 2, 2012 at 5:50 AM, Patrick S.
Patrick,

It looks like you’re problem is in the rational_resampler code. I
wonder if there’s something about the resampling rate being used
that’s causing something to go out of bounds here.

I’m looking for some test program, that uses C++ and filters/decimators.
The only examples in C++ I found are either dialtone, which does no
processing, or programs that require some hardware that I do not have
(fcd_nfm_rx.cc,

Does anyone know of an small GNU Radio C++ application that does some
filtering/resampling?

Regards

Patrick

Engineers motto: cheap, good, fast: choose any two
Patrick S.
Student of Telematics, Graz Univ. of Technology, Austria

#5

On Mon, Jun 4, 2012 at 2:18 PM, Tom R. [email protected] wrote:

that’s causing something to go out of bounds here. Can you dig into

(or fft_filter_fff) just for testing purposes? That’ll help us see if
it’s the rational resampler itself or something more general.

Patrick: You can now try my own osmosdr branch from
https://github.com/csete/gqrx
I have made many changes since I told you to stay away few days ago
and I think it is slightly usable now.

This code adds proper multirate support by using the arbitrary PFB
resamplers on both the quadrature and the audio end. It also takes
advantage of the device discovery API in gr-osmosdr so you really just
plug in your devices and go :slight_smile:

It still works very well with the Funcube Dongle and is supposed to
work with UHD devices but I really haven’t had time to debug that part
yet. Gr-osmosdr source works though with UHD devices in GRC, so it’s a
gqrx issue if it doesn’t work.

Concerning the original topic of this thread: I can’t dismiss that
it’s a rarely seen bug in the GR code, but keep in mind that the fork
was made when much of my code assumed 96ksps input rate and 48ksps
audio rate.

Alex

#6

Tom R. wrote on 2012-06-04 14:18:

that’s causing something to go out of bounds here. Can you dig into
the code and figure out what interpolation and decimation rates are
being used?

Interpolation is 1, decimation is 2.

Full valgrind log in
http://pastebin.com/7GCs3bWy

I was hoping you’d say the interpolation and/or decimation were some
ridiculously large numbers. Since the block is only actually
decimating, could you replace it in the code with an fir_filter_fff
(or fft_filter_fff) just for testing purposes? That’ll help us see if
it’s the rational resampler itself or something more general.

What seems strange to me is the allocation of a buffer of a non by 8
dividable size, that is accessed in blocks of 8 bytes. So the last read
always either does not touch the end of the buffer, or it reads beyond
the end.

I’m a little sparse at time until Sunday evening. I tried to write some
mini program starting from dial_tone.cc which uses a filter, have to
convince cmake to compile it… learning :wink:

Regards

Patrick

Engineers motto: cheap, good, fast: choose any two
Patrick S.
Student of Telematics, Graz Univ. of Technology, Austria

#7

On Tue, Jun 5, 2012 at 10:06 AM, Patrick S.
[email protected] wrote:

wonder if there’s something about the resampling rate being used
ridiculously large numbers. Since the block is only actually
decimating, could you replace it in the code with an fir_filter_fff
(or fft_filter_fff) just for testing purposes? That’ll help us see if
it’s the rational resampler itself or something more general.

What seems strange to me is the allocation of a buffer of a non by 8
dividable size, that is accessed in blocks of 8 bytes. So the last read
always either does not touch the end of the buffer, or it reads beyond
the end.

Sorry, Patrick, I think there was a typo or something in your first
sentence, and I’m not sure I understood. But from the gist of it, yes,
if the items used in a block have size 8, then the buffer allocation
should be sized enough to handle an integer number of items. This is
true because the scheduler sends the block noutput_items, which are
the number of items to process (so the buffer size would be
8*noutput_items).

I’m a little sparse at time until Sunday evening. I tried to write some
mini program starting from dial_tone.cc which uses a filter, have to
convince cmake to compile it… learning :wink:

Regards

Patrick

Thanks. Let us know how it goes.

Tom

#8

Tom R. wrote on 2012-06-06 01:12:

On Tue, Jun 5, 2012 at 10:06 AM, Patrick S.
[email protected] wrote:

Tom R. wrote on 2012-06-04 14:18:

On Sun, Jun 3, 2012 at 2:22 PM, Patrick S.
[email protected] wrote:

Full valgrind log in
http://pastebin.com/7GCs3bWy

the number of items to process (so the buffer size would be
8*noutput_items).

To make it more clear:
From the valgrind log:

==19688== Invalid read of size 8
[…]
==19688== Address 0x1a200600 is 1 bytes after a block of size 447
alloc’d
[…]
==19688== by 0x4F7AF67: gr_fir_fff_simd::set_taps(std::vector<float,
std::allocator > const&) (gr_fir_fff_simd.cc:88)
[…]
==19688== by 0x4F8DF06: gr_make_rational_resampler_base_fff(unsigned
int, unsigned int, std::vector<float, std::allocator > const&)
(gr_rational_resampler_base_fff.cc:45)

I do not completely understand where this number is calculated, but a
buffer of size 447 is allocated, and later accessed in steps of 8. Btw
even if the buffer was a multiple of 8, the read would be out of bounds.
Anyway, reading out of bounds may trigger a segfault if you are lucky,
if you are not, it goes unnoticed and you get wrong/weird results. I
wonder if this stems from an error in the application. Is it possible to
mis-use the API to trigger something like this that would not happen if
everything was used rigth?

I’m sorry if my mails are not always completely clear and
understandable, still trying to improve my English on technical topics
that are new to me :wink:

Regards

Patrick

–
Engineers motto: cheap, good, fast: choose any two
Patrick S.
Student of Telematics, Graz Univ. of Technology, Austria

#9

On Sun, Jun 3, 2012 at 2:22 PM, Patrick S.
[email protected] wrote:

being used?
http://pastebin.com/7GCs3bWy

regards

Patrick

I was hoping you’d say the interpolation and/or decimation were some
ridiculously large numbers. Since the block is only actually
decimating, could you replace it in the code with an fir_filter_fff
(or fft_filter_fff) just for testing purposes? That’ll help us see if
it’s the rational resampler itself or something more general.

Tom