Dealing with framed data

GNU Radio
1

Hi,

I’m currently trying to take the output of a fft block and pipe it in a
tcp sink for (near) real-time processing outside of gnuradio.

My question is:

Assuming that I have vectors of n floats (the fft block outputs vectors
of floats or complexes, so I may need to pipe it to a complex-to-mag2
block to convert it to an array of floats, but that’s not the main point
of my question here), if I send them directly to the tcp sink, it will
become a stream of bytes, and my code outside of gnuradio won’t be able
to detect the boundaries between vectors of n bytes. Actually it won’t
even be able to detect the boundaries between floats.

So I need to “frame” the data in a kind of basic protocol. For example:
each vector of n floats, insert a m bytes header with a magic
identifier, and perhaps a length and/or sequence number.

I’m confused about what technique to use to achieve this, between tagged
streams, messages, pdus, packets:

  • I tried using tagged stream (with “stream to tagged stream”), and
    piping it to “Packet Header Generator”, but the output of the “packet
    header generator” seems to be only the packet header, and I don’t know
    how to recombine it with the payload.

  • I tried using “tagged stream to pdu”, but then I don’t know what to do
    of the pdu output, the only block I found which may input this data type
    is “Socket PDU” but I don’t understand its purpose.

  • Or should I use “simple framer”?

I’d appreciate some advice on the best / most simple way to achieve
this. I’m new to gnuradio, so feel free to correct me on any errors (and
accept my apologizes if my question seems trivial).

best,

–
Matthieu I. [email protected]
http://graal.ens-lyon.fr/~mimbert/
INRIA / LIP ENS-Lyon research engineer
http://www.inria.fr Home
+33(0)472728149
Monod campus, room GN1 Nord 3.52
LIP ENS-Lyon, 46 alle d’Italie
69364 Lyon cedex 07, FRANCE

2

Use the packet encoder block

Regards,
Vanush

On Thu, Apr 24, 2014 at 1:05 AM, Matthieu I.

3

On Apr 23, 2014, at 8:05, Matthieu I. [email protected]
wrote:

I’m currently trying to take the output of a fft block and pipe it in a tcp sink
for (near) real-time processing outside of gnuradio.

My question is:

Assuming that I have vectors of n floats (the fft block outputs vectors of
floats or complexes, so I may need to pipe it to a complex-to-mag2 block to
convert it to an array of floats, but that’s not the main point of my question
here), if I send them directly to the tcp sink, it will become a stream of bytes,
and my code outside of gnuradio won’t be able to detect the boundaries between
vectors of n bytes. Actually it won’t even be able to detect the boundaries
between floats.

So I need to “frame” the data in a kind of basic protocol. For example: each
vector of n floats, insert a m bytes header with a magic identifier, and perhaps a
length and/or sequence number.

Actually, you don’t need any framing, and the boundaries can be
well-defined.

Unlike a radio transmission, TCP is a reliable protocol – it delivers
the bytes that are sent without any omissions. Furthermore, there is a
defined start-of-data point (when the connection was created).
Therefore, you can simply count bytes.

The first 4 bytes the receiving program gets make up a float. The next 4
bytes are another float, and so on. Collect groups of 4 bytes to make
floats, then collect groups of N floats to make arrays of floats.

The one tricky bit is that (at least in POSIX) there is no guarantee the
socket read operations on the receiving side will give you multiples of
4 bytes, so you will need to buffer at least up to 3 bytes (or an entire
vector length), or use high-level facilities in your language/framework
to read guaranteed lengths.

The only reason to use a more elaborate protocol (with headers or
whatever) on top of TCP is if your data is not homogeneous – if it
consists of variable-length messages. That is not the case here (unless,
for example, you want to add the capability to change your FFT-length
mid-stream).

–
Kevin R. http://switchb.org/kpreid/