Control plane/Data plan communication using file descriptors

GNU Radio
1

‘waterfallsink.py’ uses pipes as an IPC mechanism to get data from one
thread (running gnuradio dispatch loop) to another thread (running
wxPython’s dispatch loop), by creating a pipe and using
file_descriptor_sink/source to move the data.

This seems to be a good way to do things, and I’d like to see how well
it fits in a more general case.

One could implement the ‘data plane’ of one’s application as a set of
gnuradio blocks that implement all the signal processing, and have a set
of pipes that output various display updates. The user interface code
could select() on these pipes and handle updating the various GUI
elements after reading the available data.

For control, having the GUI code invoke public methods on the gnuradio
blocks seems ok, bearing in mind any inter-thread data access locking
required.

Aside from select not working with pipes on Win32, does this seem like a
good application structure?

How have people dealt with the separation of user interface and data
flow in a scalable way?

-Johnathan (wanting to think about something besides autotools!)

2

You may lose many of your Windows/MinGW customers if you use pipes.
Pipes on Windows will only work if gnuradio and Python can agree on
which version of the C runtime library to use. Because the binary
distribution
of Python 2.4 for Windows is compiled with Microsoft’s VC 7 and uses the
msvcr71.dll, pipes in gnuradio will only work if it is also compiled
with
the
same Microsoft compiler. The choices are (1) get a Microsoft compiler,
(2) go back to Python 2.3, (3) try to build Python from source using
gcc,
or (4) make MinGW use a different C library (e.g., msvcr71.dll).
I’m not doing (1). Does anyone know if gnuradio has any problem with
Python 2.3? (3) may be difficult, since the Python build process for
Windows is likely geared to using the Microsoft tools. I’ve seen hints
that
(4) may be possible, but I don’t think the process is documented or
supported.

An alternative for Windows users is to use Cygwin. I am currently
working
on this and all seems to be going well, but I haven’t yet tackled to
problem
of
building wxPython.

– Don W.

----- Original Message -----
From: “Johnathan C.” [email protected]
To: [email protected]
Sent: Monday, August 21, 2006 11:39 AM
Subject: [Discuss-gnuradio] Control plane/Data plan communication using
filedescriptors

3

Don W. wrote:

You may lose many of your Windows/MinGW customers if you use pipes.
Pipes on Windows will only work if gnuradio and Python can agree on
which version of the C runtime library to use.

Here there be dragons. I don’t even know where to begin thinking about
this.

An alternative for Windows users is to use Cygwin. I am currently working
on this and all seems to be going well, but I haven’t yet tackled to
problem of
building wxPython.

There is another person on the list here who is trying to get this done
(Owen with a .tw email address).

-Johnathan

4

Eric B. wrote:

waterfallsink.py should be updated to work like fftsink.
fftsink does not use pipes.

I see now that fftsink uses message queues (gr.msg_queue).

For a single producer and single consumer, this looks straightforward
and the consumer can do a blocking read to wait for a message
(delete_head).

I see in the gr_msg_queue implementation that locks are used on insert
and removal, so is there anything more to it than having all the
producer gnuradio blocks calling insert_tail and having the consumer
thread do a blocking delete_head()? (Of course the message source needs
to be encoded in the message so they can be dispatched on the other side
to the right place.)

-Johnathan

5

On Mon, Aug 21, 2006 at 12:16:17PM -0400, Don W. wrote:

You may lose many of your Windows/MinGW customers if you use pipes.

waterfallsink.py should be updated to work like fftsink.
fftsink does not use pipes.

Patch welcome :wink:

Eric

6

On Mon, Aug 21, 2006 at 08:39:05AM -0700, Johnathan C. wrote:

‘waterfallsink.py’ uses pipes as an IPC mechanism to get data from one
thread (running gnuradio dispatch loop) to another thread (running
wxPython’s dispatch loop), by creating a pipe and using
file_descriptor_sink/source to move the data.

This seems to be a good way to do things, and I’d like to see how well
it fits in a more general case.

The pipe part of it sucks, but the idea seems to hold up pretty well.

Other code (fftsink, scopesink) has been re-written to use
gr_messages and gr_msg_queue, which work independent of the underlying
OS.

One could implement the ‘data plane’ of one’s application as a set of
gnuradio blocks that implement all the signal processing, and have a set
of pipes that output various display updates. The user interface code
could select() on these pipes and handle updating the various GUI
elements after reading the available data.

For control, having the GUI code invoke public methods on the gnuradio
blocks seems ok, bearing in mind any inter-thread data access locking
required.

Use messages and message queues. They are thread safe and are
designed specifically to solve this problem.

You can look at fftsink.py or the
gnuradio-core/src/python/gnuradio/blksimpl/pkt.py for examples.

Aside from select not working with pipes on Win32, does this seem like a
good application structure?

See above :wink:

How have people dealt with the separation of user interface and data
flow in a scalable way?

See above :wink:

The mblock stuff under developement will provide a similar approach
using messages to communicate between interested parties. In
addition, since all communication is message based, it eliminates
the inter-thread data access problem (actually it puts all “behind the
scenes” where it can be taken care of in one place by the underlying
runtime system).

-Johnathan (wanting to think about something besides autotools!)

I can imagine :wink:

Eric