Ticket #13 was reporting failure with parallel makes.
Invoking (after bootstrap & configure) make with the -j option causes it
to run as many instances in parallel as possible, dependencies allowing,
in order to take advantage of multiple processors or cores, or to have a
second instance run while the first was waiting for I/O, etc. Normally
you put -j2 or -j3 to limit the maximum, as beyond that there is little
benefit.
This was failing in gnuradio-core, but was fixed back in r3222.
Those of you tracking the new svn trunk–could you test this and
confirm?
Incidentally, make -j doesn’t break anything, and at one point had 1100
processes running in parallel to build various parts of the whole tree.
So
when we get to 1024 core processors, we know we’ll be able to take
advantage of them. 
-Johnathan
On a Dual G5 OSX 10.4.7, “make -j” results in lots of:
/opt/local/bin/glibtool: fork: Resource temporarily unavailable
and
powerpc-apple-darwin8-g+±4.0.1: : No such file or directory
and
/bin/sh: fork: Resource temporarily unavailable
Running it time after time slowly gets everything made.
“make -j2” and “make -j3” work just fine.
On Thu, Aug 10, 2006 at 04:22:30PM -0400, Michael D. wrote:
/bin/sh: fork: Resource temporarily unavailable
Running it time after time slowly gets everything made.
“make -j2” and “make -j3” work just fine.
You are most likely running into some configured limit relevant to
fork(2). Could be number of processes per user, or memory, etc. Take
a look at “$ man 2 fork” for OS/X particulars.
On Thu, 2006-08-10 at 11:56 -0700, Johnathan C. wrote:
Ticket #13 was reporting failure with parallel makes.
This was failing in gnuradio-core, but was fixed back in r3222.
Those of you tracking the new svn trunk–could you test this and confirm?
I tried the svn source with “make -j 6” yesterday and it built
everything (except portaudio,osx,windows etc) in 4:50 A-OK. That’s
on a dual opteron x86_64 running Fedora 4.
That’s ok, there’s, um, some issue with make check. Did “make install”
and simple graphs do run alright, but for “make check” :
make[4]: Entering directory
`/mnt/sata/src/gnuradio/gnuradio-core/src/tests’
*** glibc detected
*** /mnt/sata/src/gnuradio/gnuradio-core/src/tests/.libs/lt-test_all:
free(): invalid pointer: 0x000000000058a8d0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x32db16a94e]
/lib64/libc.so.6(__libc_free+0x6e)[0x32db16ae7e]
/usr/lib64/libcppunit-1.11.so.0(_ZN7CppUnit8TestPathC1EPNS_4TestERKSs
+0x49f)[0x3787d36e65]
/usr/lib64/libcppunit-1.11.so.0(_ZNK7CppUnit4Test15resolveTestPathERKSs
+0x9)[0x3787d2eac9]
/usr/lib64/libcppunit-1.11.so.0(_ZN7CppUnit10TestRunner3runERNS_10TestResultERKSs+0x16)[0x3787d415bc]
/usr/lib64/libcppunit-1.11.so.0(_ZN7CppUnit14TextTestRunner3runESsbbb
+0x47)[0x3787d45c23]
/mnt/sata/src/gnuradio/gnuradio-core/src/tests/.libs/lt-test_all(__gxx_personality_v0+0x149)[0x400c71]
/lib64/libc.so.6(__libc_start_main+0xdc)[0x32db11c4cc]
/mnt/sata/src/gnuradio/gnuradio-core/src/tests/.libs/lt-test_all(__gxx_personality_v0+0x41)[0x400b69]
–Chuck
Charles S. wrote:
That’s ok, there’s, um, some issue with make check. Did “make install”
and simple graphs do run alright, but for “make check” :
make[4]: Entering directory
`/mnt/sata/src/gnuradio/gnuradio-core/src/tests’
*** glibc detected
*** /mnt/sata/src/gnuradio/gnuradio-core/src/tests/.libs/lt-test_all:
free(): invalid pointer: 0x000000000058a8d0 ***
Could you please file a new bug in Trac for this latter issue? You
should be able to do so using your developer account.
Thanks,
-Johnathan
On Fri, August 11, 2006 10:03, Jan S. wrote:
I know I might be starting a holy war here, but I believe that a better
way to do this reliably is for the top-level Makefile to include the
lower-level ones, which just state the dependencies for the particular
module.
This is a valid point and one that deserves a lengthier reply than I
have
time to write up at the moment. This weekend…
-Johnathan
I am still seeing occasional, semi-random build failures, although it
mostly appears to work. This is on a dual-core Intel Mac with -j 3. I
haven’t run enough of them to spot a pattern, but sometimes the make
fails and sometimes the make check.
With all due respect, I don’t think the current approach of recursive
makefiles is going to get us to a point where you can get a consistent
and repeatable build with arbitrary numbers of processors or build
machines (in the case of a distributed build), with multiple developers,
over long periods of time. It is really really hard to do this without
the top-level make being aware of all the dependencies throughout the
whole code base. It is so easy to subtly break a build by introducing a
dependency in a lower-level Makefile that the top-level make isn’t aware
of. Then multi-process builds will sometimes fail and sometimes succeed,
depending on the execution order, which is completely arbitrary.
Spotting and debugging the resulting inconsistencies is usually pretty
painful, as most of you will know.
I know I might be starting a holy war here, but I believe that a better
way to do this reliably is for the top-level Makefile to include the
lower-level ones, which just state the dependencies for the particular
module. All the actual work is controlled by a single instance of make,
which has the whole picture of the dependencies and can schedule
accordingly. That doesn’t preclude building individual modules, as they
can be stated as sub-targets and built individually.
A reference to the landmark paper “Recursive make considered harmful”
may be in order
(http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html). This is
almost 10 years old, but still well worth reading.
I just wondered whether this was on anybody else’s radar.
Cheers,
Jan
On Fri, Aug 11, 2006 at 10:03:49AM -0700, Jan S. wrote:
may be in order
(http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html). This is
almost 10 years old, but still well worth reading.
I just wondered whether this was on anybody else’s radar.
No holy war here. Johnathan and I were talking about this a couple of
weeks ago. I agree that a non-recursive-make-solution is probably the
right path. If you’re game, I’d be delighted to accept a patch which
“does the right thing”. Automake is supposed to have (some) support for
doing this. We thought that doing the conversion in the midst of the
migration was a bit more than we wanted to bite off.
Cheers,
Jan
Eric
On Fri, 2006-08-11 at 07:02 -0700, Johnathan C. wrote:
Could you please file a new bug in Trac for this latter issue? You
should be able to do so using your developer account.
Gaack, that turned out to be from using a development version
of cppunit, 1.11. Uninstalling the rpm’s and building 1.10 and…
all tests passed. No ticket submitted - problem was all on this end.
–Chuck
