To implement WiMAX with GnuRadio or not?

alcost · May 28, 2011, 5:32pm

On Sat, May 28, 2011 at 18:23, Marcus D. Leech [email protected]
wrote:

I also wonder - have you quantified overhead of those methods - in
latency, jitter, memory, etc?

Well, in terms of the FIFO I/O, they’re pretty-darned efficient in
Linux, at least as efficient as disk I/O,
although since FIFOs (named pipes) are implemented in kernel memory,
they’re typically a lot
faster.

Problem here is that FIFO’s are not very well suited for real-time
operation, IIRC. Have you tried a shared memory and shared signals
across applications?

I’ve never needed the XMLRPC stuff to “go real fast”, since I typically
use it to modify flow-graph
parameters that change infrequently. One of the things I use it for
in one of my applications that I
did for a customer is for active RFI excision for a science
application. The flow-graph produces
a spectral estimate which is sent over a FIFO to an external program
that analyses the spectrum, and
then, if necessary, sends back parameters for a multi-notch filter
into the flow-graph.

Good idea. With only one problem - XML is a bit of overhead for
real-time application messaging

I wonder - have anyone considered using Google’s Protocol Buffers or a
similar messaging scheme with fast serialization/deserialization?

–
Regards,
Alexander C…

alcost · May 28, 2011, 5:39pm

On May 28, 2011, at 11:26 AM, Alexander C. wrote:

How well is GnuRadio suited for real-time operation?

In a general sense, yes, GNU Radio is well suited for real-time signal
processing of data streams. That said: “Real time” is only meaningful
knowing the performance criteria. What sort of performance do you
require processing for?

5.1) Specifically - is memory allocated from heap with malloc() or it
supports non-blocking memory pools?

IIRC (Tom? Eric?): GNU Radio uses the standard C++ “new” command for
most memory allocations – and thus it is memory from the heap. One can
always overload this method globally or locally if required, to provide
memory from pools or other means.

5.2) Can the whole flowgraph be executed from a single thread to
minimize cross-thread communication overhead (and jitter)?

Yes. Your choices are a single thread for all or 1 thread per block. -
MLD

alcost · May 28, 2011, 5:49pm

On Sat, May 28, 2011 at 19:38, Michael D. [email protected] wrote:

On May 28, 2011, at 11:26 AM, Alexander C. wrote:

How well is GnuRadio suited for real-time operation?

In a general sense, yes, GNU Radio is well suited for real-time signal
processing of data streams. That said: “Real time” is only meaningful knowing the
performance criteria. What sort of performance do you require processing for?

Real-time is not about performance, but about predictability
I have to be sure that my flowgraph always executes before the
deadline is hit. So everything that introduces jitter is a no-no.

5.1) Specifically - is memory allocated from heap with malloc() or it
supports non-blocking memory pools?

IIRC (Tom? Eric?): GNU Radio uses the standard C++ “new” command for most memory
allocations – and thus it is memory from the heap. One can always overload this
method globally or locally if required, to provide memory from pools or other
means.

Is this “new” called during the flowgraph execution loop? It’s too
bad, if it is. Standard memory allocation schemes introduce random
delays in a thread execution. And implementation of a lock-free memory
allocator is a complex task. We used memory pools instead, but this
requires special care - you can’t replace a “new” with memory pool
get() with a regexp.

5.2) Can the whole flowgraph be executed from a single thread to
minimize cross-thread communication overhead (and jitter)?

Yes. Your choices are a single thread for all or 1 thread per block. - MLD

Great.

–
Regards,
Alexander C…

alcost · May 28, 2011, 5:53pm

Problem here is that FIFO’s are not very well suited for real-time
operation, IIRC. Have you tried a shared memory and shared signals
across applications?

It depends on what you mean by “real time”. Certainly FIFO I/O will be
slower than
intra-flowgraph ring buffers, but not so horribly sluggish and
latency prone that they
can’t be used for a large class of real-time applications.

I use them extensively in a radio astronomy application, and they don’t
seem to add any
noticable latency above the already-not-spectacular latency within
Gnu Radio.

Good idea. With only one problem - XML is a bit of overhead for
real-time application messaging

Are you concerned about parsing overhead? And what do you mean by “real
time”??
Are you concerned about reacting to stimuli on microsecond
timescales? In which case,
deep thought would certainly be required about the entire
architecture, and not just
the protocol “syntax”.

I wonder - have anyone considered using Google’s Protocol Buffers or a
similar messaging scheme with fast serialization/deserialization?

Don’t know anything about it. Might be worth looking at for someone.

–
Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

alcost · May 28, 2011, 6:00pm

Real-time is not about performance, but about predictability I have
to be sure that my flowgraph always executes before the deadline is
hit. So everything that introduces jitter is a no-no.
In general, Gnu Radio executes on general-purpose OSes, which means that
there will always be microsecond-to-milllisecond scale
jitter in executing code. If you want 100s-of-nanosecond to
microsecond scale execution timing predicability, Gnu Radio is going to
be the least of your worries, in my experience. The kernel can
prempt you at any time to deal with interrupts, etc.

–
Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

alcost · May 28, 2011, 5:29pm

Hi community,

On Mon, May 23, 2011 at 23:50, Alexander C.
[email protected] wrote:

may be an option. So, I’m looking for the community help with the the

Related to (3) - how can we make sure our protocol stack can be
embedded into a closed-source application/system?

I want to add some more questions:

How well is GnuRadio suited for real-time operation?

5.1) Specifically - is memory allocated from heap with malloc() or it
supports non-blocking memory pools?

5.2) Can the whole flowgraph be executed from a single thread to
minimize cross-thread communication overhead (and jitter)?

–
Regards,
Alexander C…

alcost · May 28, 2011, 6:05pm

On Sat, May 28, 2011 at 19:52, Marcus D. Leech [email protected]
wrote:

Problem here is that FIFO’s are not very well suited for real-time
operation, IIRC. Have you tried a shared memory and shared signals
across applications?

It depends on what you mean by “real time”. Certainly FIFO I/O will be
slower than
intra-flowgraph ring buffers, but not so horribly sluggish and latency
prone that they
can’t be used for a large class of real-time applications.

I mean “Soft Real-Time”:

So, I actually do care about two parameters:

“Real-timeness” - i.e. whether using of this primitive can
introduce unexpected delays into execution.
Latency and throughput - if “real-timeness” requirement is met,
then I want to know how well performance of the primitive is.

I use them extensively in a radio astronomy application, and they don’t seem
to add any
noticable latency above the already-not-spectacular latency within Gnu
Radio.

Is there information about what is the biggest latency-injector in
GnuRadio?

Good idea. With only one problem - XML is a bit of overhead for
real-time application messaging

Are you concerned about parsing overhead? And what do you mean by “real
time”??
Are you concerned about reacting to stimuli on microsecond timescales? In
which case,
deep thought would certainly be required about the entire architecture, and
not just
the protocol “syntax”.

That’s what I try to do right now - evaluate whether GnuRadio can
perform well enough in general

I probably spent too much time developing VoIP media processing where
you can hear every “non-realtimness” with your ears. But RF processing
should be no less real-time, imho.

–
Regards,
Alexander C…

alcost · May 28, 2011, 6:07pm

On Sat, May 28, 2011 at 19:59, Marcus D. Leech [email protected]
wrote:

Real-time is not about performance, but about predictability I have to
be sure that my flowgraph always executes before the deadline is hit. So
everything that introduces jitter is a no-no.

In general, Gnu Radio executes on general-purpose OSes, which means that
there will always be microsecond-to-milllisecond scale
jitter in executing code. If you want 100s-of-nanosecond to microsecond
scale execution timing predicability, Gnu Radio is going to
be the least of your worries, in my experience. The kernel can prempt you
at any time to deal with interrupts, etc.

You can do with sub-ms jitter on a decent Linux kernel. We performed a
tons of experiments with our VoIP media processing code (sipXmediaLib)
to make sure our processing is real-time.

–
Regards,
Alexander C…

alcost · May 29, 2011, 6:43am

I thought the 1 thread execution scheduler was deprecated in gnuradio?

al fayez

alcost · May 29, 2011, 7:13am

I thought the 1 thread execution scheduler was deprecated in gnuradio?

al fayez

You may still turn it off, but the TPB scheduling policy is now the
default.

alcost · May 28, 2011, 6:25pm

Is there information about what is the biggest latency-injector in GnuRadio?

Nearly all of the basic computational blocks are as blazing-fast as they
can be on a general-purpose
CPU. The biggest latency injector is the scheduler in general, and
the buffer management part of
that scheduler in particular. The Gnu Radio architecture is
optimized for throughput, not latency, which
means BIG ring-buffers between blocks, and a complicated mechanism
for calculating the “shape” of
those ring buffers.

Scheduling across multiple threads is great for throughput, to be
sure, but it complicates optimization in
other directions. This is not an easy problem to solve.

the protocol “syntax”.

That’s what I try to do right now - evaluate whether GnuRadio can
perform well enough in general

I probably spent too much time developing VoIP media processing where
you can hear every “non-realtimness” with your ears. But RF processing
should be no less real-time, imho.

It depends very much on the type of RF processing you’re doing. Gnu
Radio is used for a large number of different applications–some of
those applications are a more-awkward fit, and some fit really well.
RF signal processing doesn’t have a single set of universal
constraints that
everyone can agree on are “vital”. Even among different types of
telecom applications, there isn’t general agreement–some have
quite-sloppy requirements for latency, while others have very tight
requirements for latency.

For my use of Gnu Radio, I care very much about overall throughput-type
performance, and reliability of the data. I dont’ have a “stimulus
response” type of model that needs to respond on timescales of a few
sample times. But others have much-tighter requirements. I don’t
need to know in real-time, for example, that the ionosphere has
changed reflectivity by 1%–I’m perfectly happy to find out about that
up to several seconds after it’s happened In my science
application that does active RFI-excision, it needs to respond on very
lazy
timescales, since one spends perhaps several days building up an “RFI
map” (and the corresponding notch filter coefficients) before
commencing “real science”.

But there have certainly been plenty of other folks on this list trying
to get a good feel for the “shape” of the real-timeness of Gnu Radio,
and whether it would be suitable “out of the box” for their
application. All I can suggest is to try some experiments on various
host
hardware, with the latest code, and see where it takes you.

–
Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium