# Dealing with framed data **URL:** \ **Category:** GNU Radio\ **Created:** [April 23, 2014, 5:07pm UTC](https://www.ruby-forum.com/t/dealing-with-framed-data/240625 "2014-04-23T17:07:08Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Detlef\_R](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/58f4c7/32.png) [@Detlef\_R](https://www.ruby-forum.com/u/Detlef_R)\ **Post date:** [April 23, 2014, 5:07pm UTC](https://www.ruby-forum.com/t/dealing-with-framed-data/240625/1 "2014-04-23T17:07:08Z")
Hi, I’m currently trying to take the output of a fft block and pipe it in a tcp sink for (near) real-time processing outside of gnuradio. My question is: Assuming that I have vectors of n floats (the fft block outputs vectors of floats or complexes, so I may need to pipe it to a complex-to-mag2 block to convert it to an array of floats, but that’s not the main point of my question here), if I send them directly to the tcp sink, it will become a stream of bytes, and my code outside of gnuradio won’t be able to detect the boundaries between vectors of n bytes. Actually it won’t even be able to detect the boundaries between floats. So I need to “frame” the data in a kind of basic protocol. For example: each vector of n floats, insert a m bytes header with a magic identifier, and perhaps a length and/or sequence number. I’m confused about what technique to use to achieve this, between tagged streams, messages, pdus, packets: - I tried using tagged stream (with “stream to tagged stream”), and piping it to “Packet Header Generator”, but the output of the “packet header generator” seems to be only the packet header, and I don’t know how to recombine it with the payload. - I tried using “tagged stream to pdu”, but then I don’t know what to do of the pdu output, the only block I found which may input this data type is “Socket PDU” but I don’t understand its purpose. - Or should I use “simple framer”? I’d appreciate some advice on the best / most simple way to achieve this. I’m new to gnuradio, so feel free to correct me on any errors (and accept my apologizes if my question seems trivial). best, – Matthieu I. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) [http://graal.ens-lyon.fr/~mimbert/](http://graal.ens-lyon.fr/~mimbert/) INRIA / LIP ENS-Lyon research engineer [http://www.inria.fr](http://www.inria.fr) [Home](http://www.ens-lyon.fr/LIP) +33(0)472728149 Monod campus, room GN1 Nord 3.52 LIP ENS-Lyon, 46 alle d’Italie 69364 Lyon cedex 07, FRANCE ---
**Author:** ![Matthieu\_I](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9de053/32.png) [@Matthieu\_I](https://www.ruby-forum.com/u/Matthieu_I)\ **Post date:** [April 23, 2014, 9:37pm UTC](https://www.ruby-forum.com/t/dealing-with-framed-data/240625/2 "2014-04-23T21:37:21Z")
Use the packet encoder block Regards, Vanush On Thu, Apr 24, 2014 at 1:05 AM, Matthieu I. ---
**Author:** ![Matthieu\_I](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9de053/32.png) [@Matthieu\_I](https://www.ruby-forum.com/u/Matthieu_I)\ **Post date:** [April 24, 2014, 1:13am UTC](https://www.ruby-forum.com/t/dealing-with-framed-data/240625/3 "2014-04-24T01:13:49Z")
On Apr 23, 2014, at 8:05, Matthieu I. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > I’m currently trying to take the output of a fft block and pipe it in a tcp sink > for (near) real-time processing outside of gnuradio. > > My question is: > > Assuming that I have vectors of n floats (the fft block outputs vectors of > floats or complexes, so I may need to pipe it to a complex-to-mag2 block to > convert it to an array of floats, but that’s not the main point of my question > here), if I send them directly to the tcp sink, it will become a stream of bytes, > and my code outside of gnuradio won’t be able to detect the boundaries between > vectors of n bytes. Actually it won’t even be able to detect the boundaries > between floats. > > So I need to “frame” the data in a kind of basic protocol. For example: each > vector of n floats, insert a m bytes header with a magic identifier, and perhaps a > length and/or sequence number. Actually, you don’t need any framing, and the boundaries can be well-defined. Unlike a radio transmission, TCP is a reliable protocol – it delivers the bytes that are sent without any omissions. Furthermore, there is a defined start-of-data point (when the connection was created). Therefore, you can simply _count bytes_. The first 4 bytes the receiving program gets make up a float. The next 4 bytes are another float, and so on. Collect groups of 4 bytes to make floats, then collect groups of N floats to make arrays of floats. The one tricky bit is that (at least in POSIX) there is no guarantee the socket read operations on the receiving side will give you multiples of 4 bytes, so you will need to buffer at least up to 3 bytes (or an entire vector length), or use high-level facilities in your language/framework to read guaranteed lengths. The only reason to use a more elaborate protocol (with headers or whatever) on top of TCP is if your data is not homogeneous – if it consists of variable-length messages. That is not the case here (unless, for example, you want to add the capability to change your FFT-length mid-stream). – Kevin R. [http://switchb.org/kpreid/](http://switchb.org/kpreid/)