Some problems with library path when running test and proposed fix

GNU Radio
#1

In brief: Depending on the settings make test does not always test what
should test.

In detail:
The following is based on gnuradio v3.7.2.1-159-g3fa56a46 though the
problems already exist with older versions.

If I run make test on a fresh build ( but not installed! ) version of
gnuradio I get a lot of errors.

Mainly symbol not found.

This depends on my special ( but I think not really unusual ) setup.

I work with multiple version of gnuradio all installed in /usr/local,
for example:

drwxr-xr-x 9 root root 88 21. Jan 17:27 /usr/local/gnuradio-3.7.1
drwxr-xr-x 9 root root 88 13. Sep 12:13 /usr/local/gnuradio-3.7-debug

The installation to use is selected by an symbolic link:

lrwxrwxrwx 1 root root 14 26. Jan 12:30 /usr/local/gnuradio ->
gnuradio-3.7.1

The gnuradio-libs a referenced by:
LD_LIBRARY_PATH=/usr/local/gnuradio/lib

and

PYTHONPATH=/usr/local/gnuradio/lib64/python2.7/site-packages/

Switching between versions is easy, I’ve just change the symbolic link.
This works very well.

Installing a new(er) version of gnuradio is as follow.

Clone the repository.
Run cmake in the build- directory defining a new installation directory
and run

make test.

Only if the test are O.K. I’d really like to install the new version.

No the problems arise:
As LD_LIBRARY_PATH is set, the volk test uses the wrong library:

ldd lib/test_all
linux-vdso.so.1 (0x00007fff45bff000)
libvolk.so.0.0.0 => /usr/local/gnuradio/lib/libvolk.so.0.0.0
(0x00007eff7c501000)
libboost_filesystem.so.1.55.0 =>
/usr/lib64/libboost_filesystem.so.1.55.0 (0x00007eff7c2e9000)
libboost_system.so.1.55.0 => /usr/lib64/libboost_system.so.1.55.0
(0x00007eff7c0e5000)
libboost_unit_test_framework.so.1.55.0 =>
/usr/lib64/libboost_unit_test_framework.so.1.55.0 (0x00007eff7be39000)
libboost_program_options.so.1.55.0 =>
/usr/lib64/libboost_program_options.so.1.55.0 (0x00007eff7bbc6000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007eff7b9c2000)
liborc-0.4.so.0 => /usr/lib64/liborc-0.4.so.0 (0x00007eff7b73b000)
libstdc++.so.6 =>
/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/libstdc++.so.6
(0x00007eff7b433000)
libm.so.6 => /lib64/libm.so.6 (0x00007eff7b133000)
libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/libgcc_s.so.1
(0x00007eff7af1c000)
libc.so.6 => /lib64/libc.so.6 (0x00007eff7ab6c000)
librt.so.1 => /lib64/librt.so.1 (0x00007eff7a964000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007eff7a747000)
/lib64/ld-linux-x86-64.so.2 (0x00007eff7c81b000)

So the test results in:

lib/test_all: symbol lookup error: lib/test_all: undefined symbol:
volk_32f_invsqrt_32f_get_func_desc
This can be avoided be adding . to the LD_LIBRARY_PATH in the test
script.

Similar problems arise in many tests that depend on the volk library.
For example the analog/qa_agc.py test results in
undefined symbol: volk_malloc

If you have a look into the test script you’ll find

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH
export
PYTHONPATH=/home/schroer/gnuradiocomponents/gnuradio/build-master/gnuradio-runtime/python:$PYTHONPATH

The setting of the LD_LIBRARY_PATH makes no real sense. It should be
prefixed with the path to the newly compiled volk library in the build
directory, similarly like the PYTHONPATH is set.

For example:

export
LD_LIBRARY_PATH=/home/schroer/gnuradiocomponents/gnuradio/build-master/volk/lib:$LD_LIBRARY_PATH

There are some other tests affected.

The attached patch adresses these problems and the patch for
FindQwt.cmake solves a problem on gentoo that the wrong qwt library is
used. The original FindQwt.cmake finds the include file of qwt6 but the
lib of qwt5.

– Volker

#2

On Jan 26, 2014, at 11:23 AM, Volker S. [email protected] wrote:

In brief: Depending on the settings make test does not always test what should
test.

Hi Volker - If GNU Radio is already installed into the PREFIX provided
to cmake during configuration, then during “make test” some of the
already-installed libraries will be found and used instead of those just
build. This condition can lead to some interesting “make test” errors!

I can say with confidence that the -built- libraries are all OK (I’ve
checked both the CMakeLists.txt files as well as used ldd or otool to
see what linkage is actually there; it’s all good), it’s just those
found for -testing- which can get mixed up.

This is a known problem for a while now (to me), and in my queue to work
on sooner or later. I believe that the issue is more extensive than
just adding VOLK to the library dependency list for each component. I
encounter this in MacPorts because, generally, the GNU Radio port is
already installed when an update comes along. The nice part is that
it’s trivial to replicate in MacPorts, and thus easy to see what needs
to be fixed; and, the “how” part isn’t too difficult either, just takes
perseverance and time.

Thanks for the information and patch! - MLD

#3

Hi Michael,

just for clarification:

My CMAKE_INSTALL_PREFIX points to a non existing directory at build and
test time. The libraries used depend on the LD_LIBRARY_PATH settings
that point to an existing installation.

In my case, I can avoid the problems by unsetting the LD_LIBRARY_PATH.
It seems that in such a case the libs in the build directory are found.
But I suppose that this could not be a general solution as in some
cases other important libraries like boost may be found be means of the
LD_LIBRARY_PATH.

I agree that my patch is not a general solution for this problem, but
only a starting point.
A general solution could be to prepend the required test libraries to
the LD_LIBRARY_PATH as I did it in my patch for the volk lib.

– Volker

Am 26.01.2014 23:15, schrieb Michael D.:

#4

Tom,
you are right. I’ll use github.
I looked at the issue tracker but may be I’m blind. How to log in ?

– Volker

Am 27.01.2014 14:59, schrieb Tom R.:

#5

On Mon, Jan 27, 2014 at 2:08 AM, Volker S. [email protected] wrote:

suppose that this could not be a general solution as in some cases other
important libraries like boost may be found be means of the LD_LIBRARY_PATH.

I agree that my patch is not a general solution for this problem, but only a
starting point.
A general solution could be to prepend the required test libraries to the
LD_LIBRARY_PATH as I did it in my patch for the volk lib.

– Volker

Hey Volker,

When submitting patches, it’s best to:

a) separate the patches to identify the specific purpose. In your
case, there’s a patch to fix your QWT problem and another for the VOLK
library tests.

b) It’s also best to submit this either on our issue tracker or as a
pull request on github; otherwise, it’s likely that this will be
forgotten before the issue is fully resolved or missed entirely.

Thanks,
Tom