Distcheck error on latest trunk

+++++++++++++++
Making all in gr-usrp
Making all in src
/opt/local/bin/swig -c++ -fvirtual -python -modern -keyword -w511 -
DOMNITHREAD_POSIX=1
[snip]
-I/GNURadio/trunk/build/gnuradio-3.2svn/_build/usrp/host/lib/legacy -
module usrp_swig -o usrp_swig.cc …/…/…/gr-usrp/src/usrp.i
Unable to open file …/…/…/gr-usrp/src/usrp_swig.py: Permission denied
+++++++++++++++

I think this is a SWIG issue. I’m using version 1.3.37, which for
some reason requires the addition of “-outdir $(builddir)” in order to
work correctly. Without this addition, one file is written to $
(srcdir) and the other to $(builddir) [which, when doing distcheck,
will error out because $(srcdir) is not writable].

I think the correct solution is to add this change to the
Makefile.common variable “SWIGPYTHONFLAGS” … but even with that some
of the gr-foo components need to be updated to handle this correctly
(e.g., gr-audio-osx, gr-atsc, gr-audio-windows, gr-comedi, gr-video-
sdl, … probably others).

I’m happy to make these changes into a test branch to verify
compatibility with older SWIG version(s) … that is, if this is the
correct way to go. Thoughts? Comments? Suggestions? - MLD

On Wed, 2009-01-21 at 21:41 -0500, Michael D. wrote:

I’m happy to make these changes into a test branch to verify
compatibility with older SWIG version(s) … that is, if this is the
correct way to go. Thoughts? Comments? Suggestions? - MLD

Thanks for the offer…go for it. Can you identify what SWIG version
introduced that particular command-line option? We need to survey the
distributions to find out how common it is.

Johnathan

On Wed, Jan 21, 2009 at 09:41:38PM -0500, Michael D. wrote:

I think this is a SWIG issue. I’m using version 1.3.37, which for some
reason requires the addition of “-outdir $(builddir)” in order to work
correctly. Without this addition, one file is written to $(srcdir) and
the other to $(builddir) [which, when doing distcheck, will error out
because $(srcdir) is not writable].

Strange. By the two files, do you mean the .cc and the .py?

Sounds like a swig error. AFAIK $(builddir) is always ‘.’

I think the correct solution is to add this change to the
Makefile.common variable “SWIGPYTHONFLAGS” … but even with that some
of the gr-foo components need to be updated to handle this correctly
(e.g., gr-audio-osx, gr-atsc, gr-audio-windows, gr-comedi, gr-video-sdl,
… probably others).

I’m happy to make these changes into a test branch to verify
compatibility with older SWIG version(s) … that is, if this is the
correct way to go. Thoughts? Comments? Suggestions? - MLD

I don’t think we should kludge our code to work around this without
further investigation. Is there a SWIG bug filed on this and/or any
discussion about it? What version of swig introduced the -outdir
option?

Eric

After further investigation, this is a SWIG 1.3.37 bug. According to
the documentation [1], “the default output directory for these
language specific files is the same directory as the generated C/C++
file” … which is clearly being violated since the .py file is being
written to the directory where the .i file is located while the .cc
file is being written to “.”. I will submit a bug fix to the SWIG
folks, since there doesn’t seem to be one yet; I think I’ve figured
out how to patch their code to correct this issue.

[1] SWIG-1.3 Documentation

That said: The state of use of SWIGPYTHONFLAGS in individual
components varies significantly (some do not even us it, instead just
recreating those flags in the local Makefile.am) and should probably
be cleaned up.

On Jan 22, 2009, at 11:43 AM, Johnathan C. wrote:

On Thu, Jan 22, 2009 at 11:39:25AM -0500, Michael D. wrote:

After further investigation, this is a SWIG 1.3.37 bug.

I retract this statement. I’m no longer sure if it’s a bug. Issuing
“swig -help” in the different versions results in:

1.3.36:

  -outdir <dir>   - Set language specific files output directory

1.3.37:

  -outcurrentdir  - Set default output dir to current dir instead

of input file’s path
-outdir - Set language specific files output directory
to

This seems to imply a “split” in where output files are written:

  • By default in 1.3.36, all files are written to the output directory
    (the one from which the ‘swig’ command was called).

  • By default in 1.3.37, C++ files are written to the output directory,
    while non-C++ files are written to the input directory (the one
    containing the input .i file). To get the output directory correct,
    one now needs to specify “-outcurrentdir” (or “-outdir .”, which
    should be backwards compatible back to about version 1.3.20). It is
    clear from SVN [1] that this change is intentional, hence I’m inclined
    to believe that the SWIG docs are just out of date and this is a new
    feature in 1.3.37.

[1]

Thanks for looking into this. What platform configuration are you
using that is shipping 1.3.37?

MacPorts keeps pretty up to date on most releases, which can be a good
or bad thing.

I will submit a bug fix to the SWIG folks, since there
doesn’t seem to be one yet

Great.

I will submit a complaint to them, to resolve this issue. If they
want to go with the “split” system that’s fine, but it broke backwards
compatibility and should be highlighted somewhere … and the
documentation needs to be updated to reflect this -important- change.

That said: The state of use of SWIGPYTHONFLAGS in individual
components
varies significantly (some do not even us it, instead just recreating
those flags in the local Makefile.am) and should probably be
cleaned up.

That would be a useful patch. Thanks for volunteering :slight_smile:

I’ll get there. I’m inclined to include “-outdir .” in
SWIGPYTHONFLAGS just to provide both backward and forward compatibility.

BTW> What’s the estimated timeline for 3.2 release? I’m going to
release a variety of OSX-specific GNU Radio items at that time,
including Portfiles for the various components.

On Thu, Jan 22, 2009 at 11:39:25AM -0500, Michael D. wrote:

After further investigation, this is a SWIG 1.3.37 bug.

Thanks for looking into this. What platform configuration are you using
that is shipping 1.3.37?

I will submit a bug fix to the SWIG folks, since there
doesn’t seem to be one yet

Great.

That said: The state of use of SWIGPYTHONFLAGS in individual components
varies significantly (some do not even us it, instead just recreating
those flags in the local Makefile.am) and should probably be cleaned up.

That would be a useful patch. Thanks for volunteering :slight_smile:

Johnathan

On Thu, Jan 22, 2009 at 12:02:20PM -0500, Michael D. wrote:

To get the output directory correct, one
now needs to specify “-outcurrentdir” (or “-outdir .”, which should be
backwards compatible back to about version 1.3.20). It is clear from SVN
[1] that this change is intentional, hence I’m inclined to believe that
the SWIG docs are just out of date and this is a new feature in 1.3.37.

I’ll get there. I’m inclined to include “-outdir .” in SWIGPYTHONFLAGS
just to provide both backward and forward compatibility.

Looks like we’re going to need to change the SWIG detection mechanism in
the build to conditionally define SWIGPYTHONARGS based on the detected
version. Do you want to take stab at that?

BTW> What’s the estimated timeline for 3.2 release? I’m going to
release a variety of OSX-specific GNU Radio items at that time,
including Portfiles for the various components.

I just forked the 3.2 release branch from the trunk, and am prepping it
for a 3.2rc0 release. I expect to go through a series of release
candidates for the official 3.2, so you’ve got an opportunity to get
these in. Just update the trunk as needed; I’ll merge as appropriate
from the trunk to the release branch. The release branch will have a
(small) portion of the trunk code removed, so it might be subtly
different, but probably not in any way that affects what you’re doing.

Or will the stuff you’re describing live entirely outside the source
code tree?

Johnathan

On Jan 22, 2009, at 12:29 PM, Johnathan C. wrote:

Looks like we’re going to need to change the SWIG detection
mechanism in the build to conditionally define SWIGPYTHONARGS based
on the detected version. Do you want to take stab at that?

GNU Radio right now requires SWIG 1.3.31 or newer, which should all
work correctly using “-outdir .” (it being redundant with the behavior
of those versions of SWIG). 1.3.37 also works correctly with this
flag (I’ve checked it). Hence I would -much- prefer to just add this
flag to SWIGPYTHONARGS in Makefile.common, then change those
Makefile.am’s which require SWIG to make use of SWIGPYTHONARGS. It’s
a global (to GNU Radio), backwards and forwards compatible (to SWIG),
solution, and in the whole scheme of things a really minimal change to
GNU Radio. Unless you (or Eric or someone else) really objects, I’ll
create a branch with those changes ASAP to get them into the 3.2
release (and thus have them be compatible with MacPorts’ installed
SWIG).

I just forked the 3.2 release branch from the trunk, and am prepping
it for a 3.2rc0 release. I expect to go through a series of release
candidates for the official 3.2 [snip] Or will the stuff you’re
describing live entirely outside the source code tree?

The MacPorts folks like to control Portfiles once submitted, hence I
don’t think placing those files into the trunk makes any sense. I
haven’t written the other files yet for creating / using GNU Radio on
OSX, so I’m not even thinking about where to place them at this
time … once that chunk of code is working, I’ll be thinking about it.

I was more wondering if there is a goal as to the release time, e.g.,
1 month, or March 1, or “whenever we get there”? - MLD

On Thu, Jan 22, 2009 at 12:45:24PM -0500, Michael D. wrote:

require SWIG to make use of SWIGPYTHONARGS. It’s a global (to GNU
Radio), backwards and forwards compatible (to SWIG), solution, and in the
whole scheme of things a really minimal change to GNU Radio. Unless you
(or Eric or someone else) really objects, I’ll create a branch with those
changes ASAP to get them into the 3.2 release (and thus have them be
compatible with MacPorts’ installed SWIG).

This sounds good. Thanks for sorting this out.

Eric