New to GNUradio: AFSK and codec2 gmsk modem

Hi,

I’m still very new to GNUradio, so please excuse my (perhaps dump)
question. I did look around at different places at the web but did not
really find an answer:

  • My first step with GNUradio was to learn a bit more about using it; so
    … do some test.
    I wanted to use GNUradio to determine the width of an AFSK signal; so I
    started looking for a AFSK modulator / demodulator; to come to the
    conclussion I didn’t find one.

I looked in the GR source, on CGRAN, using several generic search
engines without look.

Am I missing something here? I would be surprised there is no AFSK
modulator / demodulator in GR as it would mean one cannot do packetradio
or APRS with gnuradio.

  • The main reason I am interested in gnuradio is because one of the
    projects I work on is “c2gsmk”, a GMSK-based modem for VHF/UHF using the
    codec2 vocoder. Gnuradio looks to me like a very interesting platform to
    do simulation and work on the modem.
    The modem is now rewriten as an API which can operate purely on
    bitstreams (i.e. all GMSK modulation / demodulation taken out of the
    code): 56 bits in (40 ms @ 1400 bps codec2), “n” times 96 or 192 Bits
    out (40 ms @ 2400 or 4800 bps c2gmsk).

I have been browsing the different documents on how to write a block and
also looking in some code to find simular examples on how it is done. I
have three things I kind-of do not understand.

  • How does GNU radio deal with “sessions”? A c2gmsk session is not just
    a stream of bits; it has the notion of sessions. It has a beginning and
    a stream also ends; and it can end in a number of different ways.

I’m trying to understand how to fit the idea of “sessions” into
GNUradio.

Or am I seeing it all wrong?

  • How does GNUradio deal with information about events in the API.

The c2gmsk API does not only return a bitstream, but also information
about events; like a change of state in the statemachine of the API, or
an event like “a GMSK sync-pattern has been detected”, or “the stream
has terminated due to to many out-of-sync frames”, or the versionid of
the stream received, or statistics of the golay decoding process, etc.

  • To get some idea of how a block is written. I have been looking at the
    codec2 encoder/decoder block. It is interesting as it is also “x bits
    in, y bits out”.
    However, c2gmsk is a bit different: there always is a fixed predefined
    number of bits in, but the number of bits “out” can vary. Sometimes just
    send in bits, but do not get anything back (e.g. in the decoder when the
    stream is not syncronised), sometimes one frame inbound results in one
    frame outbound, but sometimes one single frame in can result in (say) 5
    frames out. (e.g. when encodig the start of a stream).

Is this an issue for gnuradio?

Any information welcome.

Chéééério!
Kr. Bonne.

Hallo Marcus,

On 29-07-13 01:44, Marcus D. Leech wrote:

packetradio or APRS with gnuradio.
(…)

Just found this, it might help:
GitHub - dl1ksv/gr-ax25: Some hamradio ax.25 routines for gnuradio
Hmmm. Looks interesting. A good start. :slight_smile:

However, it seams I have another problem: it does not build on my system
(ubuntu 12.04 LTS).
It needs gnuradio-runtime and as I currently run gnuradio 3.6.4.1 +
Boost 1.49 build from source, it doesn’t have that.

I tried to build 3.7.1 -which seams to include gnuradio-runtime in its
source- but that fails on a number of tests (qa_fir_filter,
qa_freq_xlating_fir_filter, qa_constellation_receiver and
qa_codec2_vocoder).

Grrr! :frowning:

Also, keep in mind that AFSK is a kind of “double modulation scheme”.
To demodulate it, first demodulate the FM-voice using a narrowband
FM demodulator, then run that into a suitaby-configured FSK demod.
AFSK was always a “hack” to allow you to send data over narrowband
FM-voice radios without having to do anything to the radio. So, they
chose
Bell-202 synchronous modem tones, because they’d “fit” in most
narrowband FM-voice radios.
Well, that is exactly the reason I would like to test it out. :slight_smile:

We currently do packet/ax.25/aprs at 1200 baud AFSK, which is not that
much less then the 1400 bps needed for codec2. Add a little bit frame
sync-patterns and -perhaps- some FEC, we might end up with 1600.

It would be kind-of-nice to be able to try out DV with standard FM
handhelds without any need to have a radio that does 9k6.
But, this is for me just as much a practicle test and just a way to
getting to know GNUradio. :slight_smile:

Cheerio!
Kr. Bonne - ON1ARF

On 07/28/2013 07:35 PM, Kristoff B. wrote:

I started looking for a AFSK modulator / demodulator; to come to the

and also looking in some code to find simular examples on how it is

results in one frame outbound, but sometimes one single frame in can
Kr. Bonne.


Discuss-gnuradio mailing list
[email protected]
Discuss-gnuradio Info Page
Just found this, it might help:

Also, keep in mind that AFSK is a kind of “double modulation scheme”.
To demodulate it, first demodulate the FM-voice using a narrowband
FM demodulator, then run that into a suitaby-configured FSK demod.

AFSK was always a “hack” to allow you to send data over narrowband
FM-voice radios without having to do anything to the radio. So, they
chose
Bell-202 synchronous modem tones, because they’d “fit” in most
narrowband FM-voice radios.

Tom,

(inline comments)

On 29-07-13 17:01, Tom R. wrote:

1.49 build from source, it doesn’t have that.
take a while), try rerunning ‘make test’.
Good news and bad news.

First test: to run volk_profile without doing the “make install” (I did
not want to install something where the “make tests” indicated is wasn’t
100 % OK). Same result as before. “make test” still gives 4 errors.

2nd test: ran “sudo make install” anyway, then volk_profile and then
“make test”:
Good news and bad news.
3 tests are now OK, only the codec2 vocoder still fails.

— cut here — cut here — cut here — cut here — cut here —
The following tests FAILED:
168 - qa_codec2_vocoder (Failed)
Errors while running CTest
— cut here — cut here — cut here — cut here — cut here —

If it still fails, run
‘ctest -V -R volk’ and send us the output.
Is now OK:
— cut here — cut here — cut here — cut here — cut here —
1: *** No errors detected
1/1 Test #1: qa_volk_test_all … Passed 0.92 sec

The following tests passed:
qa_volk_test_all

100% tests passed, 0 tests failed out of 1
— cut here — cut here — cut here — cut here — cut here —

What is strange is that I do get an error when I do this test on the
codec2 vocoder block:
— cut here — cut here — cut here — cut here — cut here —

168: Test command: /bin/sh
“/home/kristoff/ham/gnuradio/gnuradio-3.7.0/build/gr-vocoder/python/vocoder/qa_codec2_vocoder_test.sh”
168: Test timeout computed to be: 9.99988e+06
168: F
168:

168: FAIL: test001_module_load (main.test_codec2_vocoder)
168:

168: Traceback (most recent call last):
168: File
“/home/kristoff/ham/gnuradio/gnuradio-3.7.0/gr-vocoder/python/vocoder/qa_codec2_vocoder.py”,
line 54, in test001_module_load
168: self.assertEqual(expected_data, actual_result)
168: AssertionError: Tuples differ: (0, 0, 0, 3, 2, 0, 1, 5, 6, 7,… !=
(0, 0, 0, 3, 2, 0, 1, 5, 6, 7,…
168:
168: First differing element 112:
168: 24
168: 25
— cut here — cut here — cut here — cut here — cut here —

Am I correct to assume that the errors means there is a difference
between what it actually gets back from the module based on a
test-pattern and what is expects.
Is this related to the volt-libraries that produce a different result on
my particular test.

But I must say that there does is quite a difference between the test in
gnuradio 3.6.4.1. and 3.7.0

All the stuff about “data” and “expected data” are simply not present in
the test-script in 3.6.4.1.

Perhaps the thing here is that the script for 3.6.4.1. simply does not
do this “compair with what we expect” test and it might be that this
codec2 library is just as wrong on 3.6.4.1 as in 3.7.0

Both versions I have are build with boost 1.49; as described in
README.building-boost

I can try with newer versions of boost; but I must say that when using
Boost 1.54, GNUradio doesn’t even compile on my box.

What is the recommended version of boost to be used with gnuradio 3.7.0
?

What’s your processor?
The machine is a dell laptop. The CPU is a quad-core “Intel® Core™
i5 CPU M 520 @ 2.40GHz”
(which is a bit strange as the wikipedia says the 520m is supposed to be
dual-core).

Tom

Cheerio!
Kr. Bonne

On Mon, Jul 29, 2013 at 7:26 PM, Kristoff B. [email protected]
wrote:

wrote:

It needs gnuradio-runtime and as I currently run gnuradio 3.6.4.1 + Boost
after you run ‘make install’. If a given kernel fails for an
test":
Good news and bad news.
3 tests are now OK, only the codec2 vocoder still fails.

— cut here — cut here — cut here — cut here — cut here —
The following tests FAILED:
168 - qa_codec2_vocoder (Failed)
Errors while running CTest
— cut here — cut here — cut here — cut here — cut here —

Yes, you need to have GNU Radio installed for volk_profile to work
properly.

qa_volk_test_all

“/home/kristoff/ham/gnuradio/gnuradio-3.7.0/build/gr-vocoder/python/vocoder/qa_codec2_vocoder_test.sh”

168: AssertionError: Tuples differ: (0, 0, 0, 3, 2, 0, 1, 5, 6, 7,… != (0,
Is this related to the volt-libraries that produce a different result on my
particular test.

The codec2 needs a bit of attention. I’ve not known it to fail like
this (and that’s a very close result you’re seeing), but we need to
update the version of codec2 that we’re using. So unless you really
wanted to use codec2 with GNU Radio, just ignore this error since it’s
very isolated.

But I must say that there does is quite a difference between the test in
gnuradio 3.6.4.1. and 3.7.0

All the stuff about “data” and “expected data” are simply not present in the
test-script in 3.6.4.1.

Perhaps the thing here is that the script for 3.6.4.1. simply does not do
this “compair with what we expect” test and it might be that this codec2
library is just as wrong on 3.6.4.1 as in 3.7.0

Apparently, in 3.6, we weren’t actually properly testing that code.

Both versions I have are build with boost 1.49; as described in
README.building-boost

I can try with newer versions of boost; but I must say that when using Boost
1.54, GNUradio doesn’t even compile on my box.

What is the recommended version of boost to be used with gnuradio 3.7.0 ?

No recommended version. We just recommend not to use 1.46, 1.47, and
1.52. I haven’t yet tried 1.54 and no one’s reported any problems with
it, yet. I just use the default Ubuntu 13.04 version which is 1.53.

What’s your processor?

The machine is a dell laptop. The CPU is a quad-core “Intel(R) Core™ i5
CPU M 520 @ 2.40GHz”
(which is a bit strange as the wikipedia says the 520m is supposed to be
dual-core).

Ok, well the processor makes a bigger difference with Volk, but since
that actually seems to be working fine for you, the codec2 issue is
something else. Probably related to another version of a library.

Tom

On Mon, Jul 29, 2013 at 5:48 AM, Kristoff B. [email protected]
wrote:

I wanted to use GNUradio to determine the width of an AFSK signal; so

1.49 build from source, it doesn’t have that.

I tried to build 3.7.1 -which seams to include gnuradio-runtime in its
source- but that fails on a number of tests (qa_fir_filter,
qa_freq_xlating_fir_filter, qa_constellation_receiver and
qa_codec2_vocoder).

Grrr! :frowning:

Those QA failures are probable a Volk issue. Try running volk_profile
after you run ‘make install’. If a given kernel fails for an
architecture, it will be disabled. Once volk_profile completes (it can
take a while), try rerunning ‘make test’. If it still fails, run
‘ctest -V -R volk’ and send us the output.

What’s your processor?

Tom

Tom,

(inline comments)

On 30-07-13 15:53, Tom R. wrote:

Perhaps the thing here is that the script for 3.6.4.1. simply does not do
this “compair with what we expect” test and it might be that this codec2
library is just as wrong on 3.6.4.1 as in 3.7.0
Apparently, in 3.6, we weren’t actually properly testing that code.

Some time ago, there was a discussion in the mailing-list about exactly
this, about the effects of code-optimalisation.

It turns out that when you start really optimizing code for specific
processors or specific features (like Single-instruction-multiple-data)
it is very difficult to do so without effecting the outcome of the
codec2 decoding process.

I’m not an expert on ARM code optimalisation but it seams to be that the
way floats are processed will under certain conditions result in very
small differences in the result when you decode a codec2 frame to a PCM
frame.
The difference are very small (after all, you do not hear a difference
of 1 or 2 over a 65536 scale of 16 PCM audio) but they are there!
This was also seen when running the codec2 code on (say) a PPC processor
instead of a intel or a ARM.

Now, I’m not a expert in the low-level code of the codec2 libraries or
on volk, but I think it may be we are seeing the same thing here: that
the code-optimalisation operations in volk result in these small
differences -depending on what processor the code runs on- which is what
causes this particular test to fail in the GNUradio 3.7.0 “make test”
script.

So it might be that it would be better to remove these bit-comparison
tests again as it will probably confuse people. People do expect a “make
test” to result in a 100 % success-rate.

Tom
Chrio!
Kr. Bonne.

Marcus,

(repost due to technical problems. My appolologies for a possible double
post).

On 30-07-13 21:41, Marcus D. Leech wrote:

Are the SIMD subsystems not IEEE compliant, for performance reasons?
As said, I’m not an expert in code-optimisation.

I did look up the discussions. It was more then “a few weeks ago”, they
date back to January this year. :slight_smile:

There are two discussions that might be interesting:
http://sourceforge.net/mailarchive/forum.php?forum_name=freetel-codec2&viewmonth=201301

“c2dec produces different wav-files on i386 and on ARM” and “adding
compiler flags”

Cheerio! Kr. Bonne.

Marcus,

On 30-07-13 21:41, Marcus D. Leech wrote:

Are the SIMD subsystems not IEEE compliant, for performance reasons?
As said, I’m not an expert in code-optimisation.

I did look up the discussions. It was more then “a few weeks ago”, they
date back to January this year. :slight_smile:

There are two discussions that might be interesting:
http://sourceforge.net/mailarchive/forum.php?forum_name=freetel-codec2&viewmonth=201301

“c2dec produces different wav-files on i386 and on ARM
http://sourceforge.net/mailarchive/forum.php?thread_name=50F137B8.8000403%40coppice.org&forum_name=freetel-codec2
and “adding compiler flags
http://sourceforge.net/mailarchive/forum.php?thread_name=CABfYdSp2EkG9AHbsoXT7n_hLb1gSur2HH-noznGEJ7Eak6eiqw%40mail.gmail.com&forum_name=freetel-codec2

Cheerio! Kr. Bonne.

On Tue, Jul 30, 2013 at 4:47 PM, Kristoff B. [email protected]
wrote:

Tom,

(inline comments)

My understanding was that all IEEE floating-point implementations followed
the same rounding rules, etc? Is that not the case?

Are the SIMD subsystems not IEEE compliant, for performance reasons?

Yeah, actually, not all SIMD subsytems are IEEE compliant, though off
the top of my head, I can’t tell you any more than that. So it is
possible that precision issues can come into play here. There are a
number of Issues on our project page that I believe are related to
this exact problem (basically on different ARM processors with and
without NEON).

As said, I’m not an expert in code-optimisation.

I did look up the discussions. It was more then “a few weeks ago”, they date
back to January this year. :slight_smile:

There are two discussions that might be interesting:

http://sourceforge.net/mailarchive/forum.php?forum_name=freetel-codec2&viewmonth=201301

“c2dec produces different wav-files on i386 and on ARM” and “adding compiler
flags”

So codec2 shouldn’t be using Volk, but if it’s using some other SIMD
code itself, that could be a problem.

The problem with just turning off a QA test because it fails is that
we’re now just hiding a potential problem. I’d rather understand the
problem and try and patch it as opposed to shutting our eyes and
pretending everything’s ok :slight_smile:


Tom
Visit us at GRCon13 Oct. 1 - 4
http://www.trondeau.com/grcon13

On 07/30/2013 03:35 PM, Kristoff B. wrote:

Tom,

(inline comments)

My understanding was that all IEEE floating-point implementations
followed the same rounding rules, etc? Is that not the case?

Are the SIMD subsystems not IEEE compliant, for performance reasons?


Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

On 07/30/2013 06:03 PM, Douglas G. wrote:

can’t speak to whether this is actually an issue for any given

Doug


Doug G.
[email protected] mailto:[email protected]
Ah, that makes perfect sense.

In the “olden days”, various CPUs had various FP implementations, with
subtle differences in things like rounding rules, etc. Then IEEE-754
was published,
and most CPUs began to use IEEE-754 formats and semantics. But
there, I guess exceptions, and this is one case where doing a QA test
that is
looking for bit-of-bit identical patterns would be just wrong.

On Tue, Jul 30, 2013 at 3:41 PM, Marcus D. Leech [email protected]
wrote:

Are the SIMD subsystems not IEEE compliant, for performance reasons?

Marcus,
(Forgive my ignoring the context of the parent discussion to answer
your
question):

I can tell you that in the case of ARM at least NEON instructions are
is
not IEEE-754 compliant. One optimization in particular that I can
recall
off the top of my head is that denormalized floats are flushed to zero
(i.e. extremely small values get interpreted as zeros). I can’t speak
to
whether this is actually an issue for any given application, but ARM
specifically notes that NEON instructions are not IEEE compliant, and
that
therefore users should aware of potential issues. They also point out
other
non-compliant bits in their TRM’s (e.g. for the Cortex-A9:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0409e/CIHEJAGA.html).
I’ll
note that most versions of GCC do not emit NEON instructions for
floating
point math unless you pass set -funsafe-math-optimizations (see
ARM Options (Using the GNU Compiler Collection (GCC))). Of
course that statement doesn’t apply to hand assembly (or intrinsics).

Doug

On Tue, Jul 30, 2013 at 6:43 PM, Marcus D. Leech [email protected]
wrote:

I agree. For floating-point-derived “stuff”, it seems that tests should
not be doing bit-wise-exact comparisons (and indeed I think most of
our QA tests understand that there’s a reasonable “window” for answers
coming out of the QA tests).

Yes, exactly. The main trouble is finding the balance between handling
these idiosyncrasies and just being sloppy so that anything passes.
But almost never should we require an exact match, especially if
there’s any floating point math at all.


Tom
Visit us at GRCon13 Oct. 1 - 4
http://www.trondeau.com/grcon13

But there, I guess exceptions, and this is one case where doing a QA test that
is
looking for bit-of-bit identical patterns would be just wrong.

How about enclosing the bit-for-bit tests in
#ifdef IEEE754_Compliant

#endif

And adding appropriate definitions for the various platforms (or check
in autoconf)?

-Greg

On 07/30/2013 05:55 PM, Tom R. wrote:

So codec2 shouldn’t be using Volk, but if it’s using some other SIMD
code itself, that could be a problem.

The problem with just turning off a QA test because it fails is that
we’re now just hiding a potential problem. I’d rather understand the
problem and try and patch it as opposed to shutting our eyes and
pretending everything’s ok :slight_smile:

I agree. For floating-point-derived “stuff”, it seems that tests
should not be doing bit-wise-exact comparisons (and indeed I think most
of
our QA tests understand that there’s a reasonable “window” for answers
coming out of the QA tests).

On 07/30/2013 08:51 PM, Gregory W. wrote:

How about enclosing the bit-for-bit tests in
#ifdef IEEE754_Compliant

#endif

Merely knowing that a platform conforms to IEEE 754 is not sufficient to
guarantee bit-for-bit compatibility. The order of operations also
matters, and compliers can, and do, re-arrange, eliminate, and combine
operations as an optimization. Even if the rounding rules are the same,
the cost of various operations or the primitive operations that are
available are not the same between platforms, so the optimizations also
differ, and also depend on compiler version and build environment.

Search Optimize Options (Using the GNU Compiler Collection (GCC)) for
“float” for some idea of the myriad complier options which influence
this.

Getting bit-for-bit IEEE 754 behavior on some platforms comes at the
expense of both speed and precision. For example, on the x87, the FPU
registers are 80 bits, and all register instructions operate on the full
80 bits, while IEEE 754 defines no such format (64 bits being the
closest). To avoid this requires -ffloat-store, which forces all
variables to be stored in RAM, which is a significant performance hit
for most programs and makes rounding error worse. I can’t see this as
being good for gnuradio in any way.

O

based on this: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

Nathan
I learned, back in the Pleistocene Era, never to test floating-point
values for exact equality. Anyone who is doing so is living in a state
of sin.

But my surprise was in learning that two modern-CPU floating-point
implementations, starting from the same starting conditions, would
produce
slightly different answers, and the reason, is that not everybody
supports IEEE-754, or not, perhaps, all the rounding rules of IEEE-754,
while
perhaps supporting the format.

It’s not like there’s non-determinism going on. The same CPU will
always produce the same floating-point result, given the same starting
conditions, and the same chain of floating-point operations.


Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium