Removing '.py' from system path installed Python scripts

A while back we did some clean up of the USRP examples, removing some
bit-rotted cruft, and moving the commonly used programs into gr-utils.

These included things like usrp_fft.py, usrp_rx_cfile.py, and those
scripts that over time have become more utilities than examples.

In the gr-utils component, we are now installing these into the
$prefix/bin directory, so they’ll end up on the user’s PATH and be
callable from anywhere without prefixing them with the examples path.

However, a common convention on Linux, at least on Debian, Ubuntu, and
derived systems (probably Redhat too), is to strip the language specific
extension off scripts in the path.

Would anyone have any heartache if we did this for the gr-utils scripts
as well as the relatively few other scripts we install on the path?

usrp_fft.py → usrp_fft
usrp_rx_cfile.py → usrp_rx_cfile

etc.

It’s not a critical item, but if we do this, it will need to be before
the 3.1 stable branch is officially released.


Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com

However, a common convention on Linux, at least on Debian, Ubuntu, and
derived systems (probably Redhat too), is to strip the language
specific
extension off scripts in the path.

Would anyone have any heartache if we did this for the gr-utils
scripts
as well as the relatively few other scripts we install on the path?

usrp_fft.py -> usrp_fft
usrp_rx_cfile.py -> usrp_rx_cfile

From the NetBSD viewpoint, I think that’s perfectly fine.

It would be good if all the scripts started with a small set of prefixes
like usrp_foo and gr_foo, to avoid collisions with at least most of the
other 5000 packages. Sounds like you are headed this way.

Removing the ‘.py’ works fine on OSX 10.4.10, so long as the file is
executable (a+x) and has “#!/usr/bin/env python” as the first line to
get the runtime environment correct. - MLD

Removing the ‘.py’ works fine on OSX 10.4.10, so long as the file is
executable (a+x) and has “#!/usr/bin/env python” as the first line to
get the runtime environment correct. - MLD

Agreed.

But that will only really work if the version of python found as
‘python’ is the same one as the version gnuradio found when it was
compiled, and the packaging system installs an interpreter as python.
In a world with multiple python versions and site-libs this leads to
incorrect behavior.

See http://www.gnuradio.org/trac/ticket/151

This is an orthogonal issue to the renaming, so I don’t mean to object,
and I haven’t written an install script to change

#!/usr/bin/env python

to

#!@PYTHON@

when doing the install, so it’s fair to say ENOPATCH to my mail.

Greg T. wrote:

From the NetBSD viewpoint, I think that’s perfectly fine.

It would be good if all the scripts started with a small set of prefixes
like usrp_foo and gr_foo, to avoid collisions with at least most of the
other 5000 packages. Sounds like you are headed this way.

I agree that the $PATH installed files are in a global name space and
need usrp_ (if they are USRP related) or some other prefix (like gr_ as
you suggest) if they are not.

At least in gr-utils, we specifically made sure everything that gets
installed into $PATH starts with usrp_. The ‘print-db’ script became
‘usrp_print_db.py’, for example.

There are other things which probably don’t conform to this, and I want
to clean that up at the same time I remove .py suffix, if we go forward
with that.


Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com

Michael D. wrote:

Removing the ‘.py’ works fine on OSX 10.4.10, so long as the file is
executable (a+x) and has “#!/usr/bin/env python” as the first line to
get the runtime environment correct.

Anything we put on the path already conforms to the above. This may
actually cause some grief on systems that have multiple Python versions
installed, as it will invoke the Python interpreter on the path instead
of a different version if that’s what the user wants.


Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com

**
Can this be handled with symbolic links rather then renaming the
scripts?

Greg T. wrote:

But that will only really work if the version of python found as
‘python’ is the same one as the version gnuradio found when it was
compiled, and the packaging system installs an interpreter as python.
In a world with multiple python versions and site-libs this leads to
incorrect behavior.

True. When someone downloads GNU Radio as a tarball, runs configure,
make, etc., and installs it, everything gets put into the right place to
be used with the version of Python discovered on the path. This is a
good thing, I think, as it covers the most common case.

See http://www.gnuradio.org/trac/ticket/151

Understood. There really isn’t a clean solution to this, however. I’d
be happy if could figure out one that doesn’t make the common case more
difficult.

This is an orthogonal issue to the renaming

Right.


Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com

Johnathan C. [email protected] writes:

be used with the version of Python discovered on the path. This is a
good thing, I think, as it covers the most common case.

The case where there is no interpreter called ‘python’, but one called
‘python2.4’ is not handled. This is how pkgsrc works, specifically
avoiding providing ‘python’, which would change if a new version is
instaled. I realize that whether this is reasonable is a tough call.

See http://www.gnuradio.org/trac/ticket/151

Understood. There really isn’t a clean solution to this, however. I’d
be happy if could figure out one that doesn’t make the common case more
difficult.

Given that new versions of python can be installed and made default
(meaning invoked as ‘python’), it’s necessary to bind the scripts to the
same version of python used to build .so modules and install .py files
in site-packages. I think it can be done quite cleanly by either of the
two methods in the ticket, in the software engineering aesthetics sense,
and that the modify-on-install avoids a lot of non-local hair.

Specifically, add a new script install-py that is generated from
install-py.in via configure that has

#!/bin/sh

really this might need to parse install args better

INSTALLTMP=/tmp/$1.install.$$
sed -e “s,/usr/bin/env python,@PYTHON@,” < $1 > $INSTALLTMP
install -m 555 $INSTALLTMP $2

and then have the install-script rule use this instead.

I looked in the tree, and it seems it should go at top level alongside
py-compile.

Greg T. wrote:


Given that new versions of python can be installed and made default
(meaning invoked as ‘python’), it’s necessary to bind the scripts to the
same version of python used to build .so modules and install .py files
in site-packages…

I’m curious – really just curious – why not create a chroot
environment for gnuradio? Or, where the kernel and the CPU allow,
a fully virtualized stable OS install simply for gnuradio?

Frank

Tim M. wrote:

Can this be handled with symbolic links rather then renaming the scripts?

It could, if need be, but that’s an extra complication that I’d want to
avoid. Symlink behavior is easy to break across all the systems that
run GNU Radio, and would probably invite future grief.


Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com

Frank B. schrieb:

I’m curious – really just curious – why not create a chroot
environment for gnuradio? Or, where the kernel and the CPU allow,
a fully virtualized stable OS install simply for gnuradio?

If you have the possibility to do so, no problem at all. It’s quite
expansive in respect to resources and administration time. But there may
be reasons not create a special area for gnuradio, and then you have to
think about robust integration.

You would not want to create a whole new chroot or virtulization for
every package you install.

Patrick

Engineers motto: cheap, good, fast: choose any two
Patrick S.
Student of Telematik, Techn. University Graz, Austria

Frank B. [email protected] writes:

Given that new versions of python can be installed and made default
(meaning invoked as ‘python’), it’s necessary to bind the scripts to the
same version of python used to build .so modules and install .py files
in site-packages…

I’m curious – really just curious – why not create a chroot
environment for gnuradio? Or, where the kernel and the CPU allow,
a fully virtualized stable OS install simply for gnuradio?

You could, but a chroot per package would lead to a vast number of
chroots, and it wouldn’t solve the underlying problem of maintaining
dependencies.

If one views the computer’s sole purpose as providing a way to run Gnu
Radio, that might be reasonable. If one views GNU Radio as one of many
components in an overall system - for instance providing the low-level
network connectivity, then it isn’t reasonable.

The lack of python binding really is just plain suboptimal (or wrong) -
but becaues most people rarely change python bindings, and because GNU
Radio hackers constantly rebuild/reinstall GNU Radio, it hardly ever
causes trouble.

I don’t expect other people to fix this if they don’t want to spend the
effort. pkgsrc currently works around this by having a list of files in
which the interpreter is changed as they are installed.

Given the potential impact of making this change, I’ve decided it won’t
happen for the 3.1 stable branch, but on the trunk at some point in the
future.


Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com