Q's regarding C++ <-> SWIG <-> Python

[mdeering]

It seems like the “usual” C constructs port easily between C++ /
SWIG / Python: char, short, int, long, float, double. I’m fine in C+
+, but SWIG and Python are new to me, and combining them I generally
take to be an SEP (“someone else’s problem”, thanks Douglas Adams).
Given my time constraints over the summer, I’m hoping that someone
else knows about these issues, and can provide some definitive
answers. Thanks! - MLD

1. What I need are std::vector’s of the usual C types, as well as
   references to gr-blocks. In looking through gnuradio-core/src/lib/
   general and runtime, it looks like to get a std::vector argument
   in the constructor, what I need to do is something like:

   const std::vector foo;

but “&foo” won’t work? … at least it didn’t for me.

OTOH, I can also so a std::vector<gr_block_sptr>'s via something like:

const std::vector<gr_block_sptr> &foo;

… or at least these seem to be the way things are done in the
code. So no “&” for the usual types, but an “&” for sptr types?
Does this sound correct, or can anyone offer corrections? Can
someone -briefly- explain these further?

2. Is it possible to do a 2d-matrix-like type with something built-in
   to Python / SWIG / C++, such as the std::vector<std::vector>?
   Clearly I can do a 1d vector (as per (1) above) then mess around with
   it internally (which is what I currently do) … but I would certainly
   prefer a better solution.

3. Is it possible to assign default values to only -some- arguments
   to the “friend” method? I can see some codes which assign default
   values to -all- arguments, while most do none.

4. Can I have multiple constructors, and thus multiple “friend”
   methods … in the SWIG context (clearly I can in just C++, but can
   SWIG handle it correctly)? For example, the constructor for a
   “feedback” system typically requires 1 more argument than the
   “feedforward” system (the feedback portion; both require the
   feedforward portion) … so I can either do 1 block with 2
   constructors or create 2 blocks each with 1 constructor, to handle
   the different cases.

[mdeering]

  gr_fir_filter_fff (int decimation, const std::vector<float>  

&taps);

 public:
  ~gr_fir_filter_fff ();

  void set_taps (const std::vector<float> &taps);
};

OK. So what’s the difference between:

const std::vector<float> &taps

and
const std::vector taps

in terms of the way Python -> SWIG -> C++ works? In C++, the “&” is
a reference to - no copying, and “const” means that you can’t change
it at the destination (or, rather that the compiler will complain if
you try to ;). Does a reference (not copying) really hold true for
moving data from Python to C++? That would imply that the underlying
data structure is the same between them … which I can understand
for the “usual” types (char, short, int, long, float, double,
etc…) … so how would that work with a C++ class such as
std::vector? Does Python actually implement a “list” (e.g. “[5,6]”)
as a std::vector, and hence allow for reference passing?

OK. That’s a bit off-topic; just curious. Sounds like I can use
wither “&foo” or “foo” so long as it’s “const”. Good. I’ll try that
tomorrow. Hopefully my problem was elsewhere.

than the “it’s safe to make a copy of the data” strategy currently
used. You’ll need to instantiate a real STL vector in Python, not use
a Python list or tuple.

What I’m interested in is:

const std::vector<std::vector<int>>  &foo;

The receiving block will do error checking, but otherwise there will
be no changes to the data. Any SWIG users out there who could offer
advice? I’ll try the SWIG list & docs too, tomorrow.

3. Is it possible to assign default values to only -some- arguments
   to the “friend” method? I can see some codes which assign default
   values to -all- arguments, while most do none.

It’s a C++ thing. Everything to the right of the first argument
with a default value must also have a default value.

Yes yes, standard C++ In the context of how SWIG interprets the
declarations for the friend method(s), that’s my question. I tried
having the last 3 arguments of the friend method have defaults (of 7,
so 4 required, 3 optional), but then the instantiation from Python
doesn’t work. Could be that I messed up something else along the
way, but this was the last change I made (removing the 3 default
values) and -poof- things worked again.

Yes, you can have multiple constructors, but they need to have
different names. (If they have the same name, C++ can sort them out
at compile time based on argument type at the calling site, but Python
can’t do that because of its dynamic typing.)

Ah, so just name the friend method differently than that of the
“default” friend method? Hmmm, I’ll try that tomorrow when I have a
chance. Clearly for a given C++ class, the constructor names will be
the same but with different arguments … so the difference has to
come in the friend naming. I would guess it would require it’s own
GR_SWIG_BLOCK_MAGIC stuff too, with the different friend name?

[mdeering]

On Fri, Jul 07, 2006 at 07:24:26PM -0400, Michael D. wrote:

general and runtime, it looks like to get a std::vector argument
in the constructor, what I need to do is something like:

  const std::vector<int> foo;

but “&foo” won’t work? … at least it didn’t for me.

Passing a non-const reference would require that python and C++ share
the same
underlying data structure. If the ref is const, or if the vector is
passed by value, then SWIG can safely make a copy of the Python data
for C++ to play with.

[You may need to instantiate a non-anonymous STL template in the .i
file.
What we’re currently (mostly) using are anonymous (unnamed) templates.]

Non-anonymous template instantiation:

// support vectors of these...
namespace std {
  %template(x_vector_gr_block_sptr) vector<gr_block_sptr>;
};

Anonymous template instantiation (lower cost, doesn’t wrap all the STL
vector cruft):

// instantiate the required template specializations

namespace std {
  %template()         vector<unsigned char>;
  %template()         vector<char>;
  %template()         vector<short>;
  %template()         vector<int>;
  %template()         vector<float>;
  %template()         vector<double>;
  %template()         vector<std::complex<float> >;
};

N.B., there are some differences between SWIG versions as to how
(or if) the STL stuff works. The idioms in use in the code are known
to work OK in SWIG 1.3.23, 1.3.24, 1.3.25, 1.3.27, 1.3.28, and 1.3.29
(1.3.26 had problems and was immediately replaced with 1.3.27.)

OTOH, I can also so a std::vector<gr_block_sptr>'s via something like:

  const std::vector<gr_block_sptr> &foo;

… or at least these seem to be the way things are done in the
code. So no “&” for the usual types, but an “&” for sptr types?
Does this sound correct, or can anyone offer corrections? Can
someone -briefly- explain these further?

Not true. Take a look at this:

GR_SWIG_BLOCK_MAGIC(gr,fir_filter_fff);

gr_fir_filter_fff_sptr gr_make_fir_filter_fff (int decimation, const 

std::vector &taps);

class gr_fir_filter_fff : public gr_sync_decimator
{
 private:
  gr_fir_filter_fff (int decimation, const std::vector<float> 

&taps);

 public:
  ~gr_fir_filter_fff ();

  void set_taps (const std::vector<float> &taps);
};

2. Is it possible to do a 2d-matrix-like type with something built-in
   to Python / SWIG / C++, such as the std::vector<std::vector>?
   Clearly I can do a 1d vector (as per (1) above) then mess around with
   it internally (which is what I currently do) … but I would certainly
   prefer a better solution.

Not sure, ask on swig-user.

If you plan on messing with the contents of a vector in C++ and expect
Python to see the result, you’re going to need a different approach
than the “it’s safe to make a copy of the data” strategy currently
used. You’ll need to instantiate a real STL vector in Python, not use
a Python list or tuple.

3. Is it possible to assign default values to only -some- arguments
   to the “friend” method? I can see some codes which assign default
   values to -all- arguments, while most do none.

It’s a C++ thing. Everything to the right of the first argument
with a default value must also have a default value.

4. Can I have multiple constructors, and thus multiple “friend”
   methods … in the SWIG context (clearly I can in just C++, but can
   SWIG handle it correctly)? For example, the constructor for a
   “feedback” system typically requires 1 more argument than the
   “feedforward” system (the feedback portion; both require the
   feedforward portion) … so I can either do 1 block with 2
   constructors or create 2 blocks each with 1 constructor, to handle
   the different cases.

Yes, you can have multiple constructors, but they need to have
different names. (If they have the same name, C++ can sort them out
at compile time based on argument type at the calling site, but Python
can’t do that because of its dynamic typing.)

If you’ve got more SWIG questions, they’re probably better addressed
on the swig-user mailing list. They’re nice people too

swig-user List Signup and Options

Have you tried the swig docs?
I’d start with these sections:

SWIG Basics
SWIG and C++
SWIG and Python

Eric

[mdeering]

Yes yes, standard C++ In the context of how SWIG interprets the
declarations for the friend method(s), that’s my question. I tried
having the last 3 arguments of the friend method have defaults (of 7,
so 4 required, 3 optional), but then the instantiation from Python
doesn’t work. Could be that I messed up something else along the
way, but this was the last change I made (removing the 3 default
values) and -poof- things worked again.

Should work OK. Be sure that the C++ .h and the .i files agree.

Yup. This works OK now … it was the probably the multiple
constructors which were the primary problem … could have had other
issues as well.

come in the friend naming. I would guess it would require it’s own
GR_SWIG_BLOCK_MAGIC stuff too, with the different friend name?

The second constructor will require a subset of what’s in
GR_SWIG_BLOCK_MAGIC. Probably just the renaming part.

OK. I’ll look into it. - MLD

[mdeering]

On Fri, Jul 07, 2006 at 10:31:43PM -0400, Michael D. wrote:

OK. So what’s the difference between:

const std::vector &taps
and
const std::vector taps

in terms of the way Python -> SWIG -> C++ works?

Using the default typemaps that convert Python lists/tuples to
STL vectors, there is no difference. The default typemaps will
convert the Python sequence into a STL vector by copying the
elements.

In C++, the “&” is a reference to - no copying, and “const” means
that you can’t change it at the destination (or, rather that the
compiler will complain if you try to ;). Does a reference (not
copying) really hold true for moving data from Python to C++?

You’re going to want to read up on SWIG typemaps… Have fun

That would imply that the underlying
data structure is the same between them … which I can understand
for the “usual” types (char, short, int, long, float, double,
etc…) … so how would that work with a C++ class such as
std::vector? Does Python actually implement a “list” (e.g. “[5,6]”)
as a std::vector, and hence allow for reference passing?

Nope. Two reasons:

[5, “second”, [3, 4, lambda a, b: a + b], 28+3j]

Python’s coded in C

OK. That’s a bit off-topic; just curious. Sounds like I can use
wither “&foo” or “foo” so long as it’s “const”. Good. I’ll try that
tomorrow. Hopefully my problem was elsewhere.

OK.

so 4 required, 3 optional), but then the instantiation from Python
doesn’t work. Could be that I messed up something else along the
way, but this was the last change I made (removing the 3 default
values) and -poof- things worked again.

Should work OK. Be sure that the C++ .h and the .i files agree.

GR_SWIG_BLOCK_MAGIC stuff too, with the different friend name?
The second constructor will require a subset of what’s in
GR_SWIG_BLOCK_MAGIC. Probably just the renaming part.

Eric