Libtool/linker/dynamic loader stuff

GNU Radio
1

It seems like it’s time to fully revist the interlibrary dependency
issue, and the related issue of reliably and robustly testing with
build tree libs vs install tree libs.

My priority order for this working is:

Free systems are first priority:

  • GNU/Linux must work as expected using unmolested libtool.
    Ubuntu may be broken, and if it is, we need to fully understand
    why and how it’s broken. If it makes sense, we should file a bug
    upstream with good explanation for the behavior that we expect.
    I.e., testing with uninstalled libs.

  • NetBSD must work as expected

  • OS/X is next (at least their kernel is free)

  • MinGW/Cygwin/Windows: lowest priority. If we have to break this
    to fix the others, it’s not the end of the world. I’d prefer a
    solution that works reliably for all, but if it comes down to
    triage,
    this one dies first.

What “working” means:

We are able to test our apps and Python code in the build tree in a
way that is completely independent of whether or not there is an
earlier installation of our code or not. I’ve seen some flaky
behavior on some platforms where it appears that we get different
behavior depending on whether there’s already something in the install
path. This usually manifests itself as “undefined foo…” where foo
is the mangled name of a new function or method that is in fact
contained in the build tree libs, but isn’t contained in the installed
libs.

When make check is run, the installed libs, if any, MUST NOT BE
REFERENCED to resolve any symbols.

Likewise, when installed, our libraries should only resolve symbols
using the installed libraries.

I believe that libtool will do this out of the box. It normally
causes stuff to be linked twice: once at compile time, and once again
at install time. However, for this to work, we need to do things “the
libtool way”. I’m not sure we’re fully compliant.

I’m looking for someone who’s willing to sort this out and to test on
all platforms. To succeed in this venture the person ought to have
(or be willing to develop) a solid understanding of how dynamic
linking works in general, how libtool works, and what ld flags such as
-rpath, -L, and -l do. Understanding how python finds and loads
native modules is essential too. OS-specific knowledge of dynamic
linking and shared libs on all the platforms would probably come in
handy :wink:

Under Linux, see man ld, man ld-linux.so, man ldd, man ldconfig
Under OS/X, don’t confuse glibtool with libtool :wink:

http://www.gnu.org/software/libtool/
http://www.gnu.org/software/libtool/manual.html
(You can also build the pdf doc if you download the libtool src)

You’ll also want to look at the automake docs and see how it interacts
with libtool:

http://www.gnu.org/software/automake/
http://sources.redhat.com/automake/automake.pdf

http://www.gnu.org/software/libtool/manual.html#Using-Automake

Relevant environment variables include:

LD_LIBRARY_PATH, PYTHONPATH, DYLD_LIBRARY_PATH (os/x)

In all of this discussion, please assume libtool 1.5.22. It’s the
latest stable version (18 December 2005).

Any volunteers? Fame and glory awaits!

(You don’t need to be an existing committer. If you’re willing to take
this on, I’ll set you up with the access you need to do the work.)

I’m sure we can find a way to provide access to all the relevant
platforms.

Thanks!
Eric

2

That sounds good to me, but if we’re talking a major change, I’d like
to throw out something somewhere between desire and requirement:

If a component A is enabled, but a GNU Radio component B that is a
dependency for A is not enabled (by configure), then build and make
check both use the installed version of B.

This property is needed in order to be able to build separate packages
for separate parts of GNU Radio, or at least to do so reasonably.
Berndt: can you comment on whether this works ok now? If so, can you
explain how/why?

I’ll also comment that some systems, including the BSDs, use -R (or
-rpath) to encode the search path for dependencies in executables and
libraries. This is different from the Linux use of a global path, and
the resulting apparently not infrequent use of LD_LIBRARY_PATH. It is
also different from the Mac OS X way of encoding full paths for
dependencies. I think libtool deals with this correctly.

It is likely that if all works ok on NetBSD that it will also be ok on
OpenBSD and FreeBSD.

I think GNU Radio largely has the desired properties already, but not
quite. I believe that having a linker search path that looks in the
build tree and then the installed system at build time, and that then
avoids the build tree when final installation happens is sufficient.
It may be that there is a -lfoo instead of libfoo.la in a Makefile
now.

Greg T. <[email protected]>
3

On Sun, Feb 25, 2007 at 08:58:25AM -0500, Greg T. wrote:

That sounds good to me, but if we’re talking a major change,

Hopefully, this isn’t a major change. I’m thinking that it’s a bug
fix and the creation of developer documentation that says exactly how
we’re linking everything and why.

I believe that the recent breakage is a result of some “fixes” for
win32.

I’d like to throw out something somewhere between desire and requirement:

If a component A is enabled, but a GNU Radio component B that is a
dependency for A is not enabled (by configure), then build and make
check both use the installed version of B.

This property is needed in order to be able to build separate packages
for separate parts of GNU Radio, or at least to do so reasonably.

Seems like a reasonable request.

dependencies. I think libtool deals with this correctly.

It is likely that if all works ok on NetBSD that it will also be ok on
OpenBSD and FreeBSD.

Good.

I think GNU Radio largely has the desired properties already, but not
quite. I believe that having a linker search path that looks in the
build tree and then the installed system at build time, and that then
avoids the build tree when final installation happens is sufficient.
It may be that there is a -lfoo instead of libfoo.la in a Makefile
now.

I believe that’s the current problem. We used to always use
libfoo.la. There was a recent change that added -lfoo in addition to
libfoo.la.

Eric

4

Eric B. wrote:

It may be that there is a -lfoo instead of libfoo.la in a Makefile
now.

I believe that’s the current problem. We used to always use
libfoo.la. There was a recent change that added -lfoo in addition to
libfoo.la.

This was done to fix ticket 138 on Win32 platforms, on the trunk in
r4484. There is one part of the changeset that results in -lfoo being
added in front of libfoo.la in a few places. I’m going to back out just
that portion (later today.) This will result in partial breakage of
Win32, and we’ll have to figure out a different way of solving it.

–
Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com