Forum: GNU Radio hdtv hacking

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Dc48f9c00e3e6de9640898a531c26d89?d=identicon&s=25 Charles Swiger (Guest)
on 2006-04-07 21:30
(Received via mailing list)
Well, (this is a boy trying a grown-ups job ;) I managed to get
two new modules to compile in gr-atsc,   atsc_interleaver and
atsc_deinterleaver, just by copying atsc_rs_encoder and atsc_rd_decoder
(and accounting for atsci_reed_solomon having one class
and two methods(?) encode and decode, whereas atsci_data_interleaver
has two classes with one method each - IIUC).

So, basically all I did was change the names and put
in for atsc_interleaver.cc:

  for (int i = 0; i < noutput_items; i++){
    d_interleaver.interleave (out[i], in[i]);

and in atsc_deinterleaver.cc

  for (int i = 0; i < noutput_items; i++){
    d_deinterleaver.deinterleave (out[i], in[i]);


Anyway, after putting in another python test_loopback_002(self):
with the two stages interleaver/deinterleaver it will fail
make check , printing out a bunch of numbers.

--Chuck
learning a lot
Dc48f9c00e3e6de9640898a531c26d89?d=identicon&s=25 Charles Swiger (Guest)
on 2006-04-07 23:07
(Received via mailing list)
On Fri, 2006-04-07 at 15:28 -0400, Charles Swiger wrote:
> Well, (this is a boy trying a grown-ups job ;) I managed to get
> two new modules to compile in gr-atsc,   atsc_interleaver and
> atsc_deinterleaver, just by copying atsc_rs_encoder and atsc_rd_decoder

> Anyway, after putting in another python test_loopback_002(self):
> with the two stages interleaver/deinterleaver it will fail
> make check , printing out a bunch of numbers.
>

Ah, I need a clue how to do this:

>When you loop the deinterleaver back, you'll need to account for the
>52 segment delay through it.  Using gr.skiphead is probably the
>easiest solution (thanks Martin!).


As this sure didn't help:

        src = vector_source_ts(self.fg, src_data)
        rand = atsc.randomizer()
        rs_enc = atsc.rs_encoder()
        inter = atsc.interleaver()
=?=>    skip = gr.skiphead(atsc.sizeof_atsc_mpeg_packet_rs_encoded,52)
        deinter = atsc.deinterleaver()
        rs_dec = atsc.rs_decoder()
        derand = atsc.derandomizer()
        dst = vector_sink_ts(self.fg)
        self.fg.connect(src, rand, rs_enc, inter, skip, deinter, rs_dec,
derand, dst)


apologies if the above is laughably inept ;)

--Chuck
745d8202ef5a58c1058d0e5395a78f9c?d=identicon&s=25 Eric Blossom (Guest)
on 2006-04-07 23:41
(Received via mailing list)
On Fri, Apr 07, 2006 at 03:28:17PM -0400, Charles Swiger wrote:
>   for (int i = 0; i < noutput_items; i++){
>     d_interleaver.interleave (out[i], in[i]);
>
> and in atsc_deinterleaver.cc
>
>   for (int i = 0; i < noutput_items; i++){
>     d_deinterleaver.deinterleave (out[i], in[i]);


Looks good.

> Anyway, after putting in another python test_loopback_002(self):
> with the two stages interleaver/deinterleaver it will fail
> make check , printing out a bunch of numbers.

OK.  The most likely reason it doesn't pass, is that there is a 52
segment delay between what you write into the interleaver, and what you
get out of the deinterleaver.  To work around the behavior, drop the
first 52 segments of the output and then compare the remainder with
the input.

That is:

   Drop first 52 segments of output
   Ignore last 52 segments of input
   The resulting pieces should be the same length and value.

   Raw input and output:

     Input:  A----------------------------------------ZXXXXXX
     Output: 000000A----------------------------------------Z

   Where XXXXXX and 0000000 represent 52 segments of data.

   After adjustment:

     Input:  A----------------------------------------Z
     Output: A----------------------------------------Z


> --Chuck
> learning a lot

I did too!
Eric
745d8202ef5a58c1058d0e5395a78f9c?d=identicon&s=25 Eric Blossom (Guest)
on 2006-04-07 23:44
(Received via mailing list)
On Fri, Apr 07, 2006 at 05:05:24PM -0400, Charles Swiger wrote:
> Ah, I need a clue how to do this:
>         rs_enc = atsc.rs_encoder()
> apologies if the above is laughably inept ;)
I think you're close.

> derand, dst)
Then you'll still need to drop the last 52 segments of the source
before comparing.

Eric
Dc48f9c00e3e6de9640898a531c26d89?d=identicon&s=25 Charles Swiger (Guest)
on 2006-04-08 00:51
(Received via mailing list)
> The most likely reason it doesn't pass, is that there is a 52
> segment delay between what you write into the interleaver, and what you
> get out of the deinterleaver.  To work around the behavior, drop the
> first 52 segments of the output and then compare the remainder with
> the input.
>

Well - writing expected_results and result_data out to files
and manually comparing show they do match, so interleaver/deinterleaver
work - cool! (putting skip AFTER  inter, deinter). I'll finish the
test code and move on to Trellis Encoder / Viterbi Decoder.

The latest benchmark I have for the 0.9 atsc is 10 minutes of data
takes about 14 hours to decode on an Athlon 3200+ (2200 MHz real),
and over HALF that time is spent resampling and up-mixing the data
to emulate an mc4020 to atsc_rx.
821c1adf3f4be2f2aa6fbc89cce4c69a?d=identicon&s=25 Michael Dickens (Guest)
on 2006-04-08 01:12
(Received via mailing list)
On Apr 7, 2006, at 6:49 PM, Charles Swiger wrote:
> work - cool! (putting skip AFTER  inter, deinter). I'll finish the
> test code and move on to Trellis Encoder / Viterbi Decoder.

I have a trellis encoder and the V decoder is in debugging (by me) -
hope to have version 1 finished by the end of this weekend.  These
are generic encoder / decoder blocks, taking arbitrary # of input
streams and element sizes and reproducing that on the output.
Throughput of the encoder using a (2 output, 2 input, 2 memory /
input) code (meaning 16 states) is around 10 Mb/s without optimizing
the programming.  As the decoder isn't operational, I don't know how
fast it will be ... but it's mush more complex than the encoder (no
surprise there) and thus will definitely be slower than the encoder.

I don't know if these would be of interest for use by ATSC, but I'm
happy to provide them once I know they work.  Would also make for
good beta testing ;-)  I will provide them to the overall GR project
once I know they work and are reasonable optimized. - MLD
745d8202ef5a58c1058d0e5395a78f9c?d=identicon&s=25 Eric Blossom (Guest)
on 2006-04-08 03:22
(Received via mailing list)
On Fri, Apr 07, 2006 at 06:49:07PM -0400, Charles Swiger wrote:
> work - cool! (putting skip AFTER  inter, deinter). I'll finish the
> test code and move on to Trellis Encoder / Viterbi Decoder.

That's great.
Did you figure out how to compare them without writing to files?

> The latest benchmark I have for the 0.9 atsc is 10 minutes of data
> takes about 14 hours to decode on an Athlon 3200+ (2200 MHz real),
> and over HALF that time is spent resampling and up-mixing the data
> to emulate an mc4020 to atsc_rx.

Did you do the 5:2 resampling using the rational_resampler, or did you
interpolate then decimate in two steps?

Great work!

Eric
745d8202ef5a58c1058d0e5395a78f9c?d=identicon&s=25 Eric Blossom (Guest)
on 2006-04-08 04:11
(Received via mailing list)
On Fri, Apr 07, 2006 at 07:11:21PM -0400, Michael Dickens wrote:
> the programming.  As the decoder isn't operational, I don't know how
> fast it will be ... but it's mush more complex than the encoder (no
> surprise there) and thus will definitely be slower than the encoder.
>
> I don't know if these would be of interest for use by ATSC, but I'm
> happy to provide them once I know they work.  Would also make for
> good beta testing ;-)  I will provide them to the overall GR project
> once I know they work and are reasonable optimized. - MLD

Thanks!

Eric
3d1bbca20dabfeca5cce1687c1e70660?d=identicon&s=25 Jens Elsner (Guest)
on 2006-04-08 10:05
(Received via mailing list)
On Fri, Apr 07, 2006 at 07:11:21PM -0400, Michael Dickens wrote:
> I don't know if these would be of interest for use by ATSC, but I'm
> happy to provide them once I know they work.  Would also make for
> good beta testing ;-)  I will provide them to the overall GR project
> once I know they work and are reasonable optimized. - MLD

Maximum likelyhood decoding with the Viterbi algorithm is needed
almost everywhere - a generic one for gnuradio would be great.

Jens
Dc48f9c00e3e6de9640898a531c26d89?d=identicon&s=25 Charles Swiger (Guest)
on 2006-04-08 16:15
(Received via mailing list)
On Fri, 2006-04-07 at 18:20 -0700, Eric Blossom wrote:
> On Fri, Apr 07, 2006 at 06:49:07PM -0400, Charles Swiger wrote:

> > The latest benchmark I have for the 0.9 atsc is 10 minutes of data
> > takes about 14 hours to decode on an Athlon 3200+ (2200 MHz real),
> > and over HALF that time is spent resampling and up-mixing the data
> > to emulate an mc4020 to atsc_rx.
>
> Did you do the 5:2 resampling using the rational_resampler, or did you
> interpolate then decimate in two steps?
>

currently it's using the rational_resampler on one machine, then pipes
and sockets to atsc_rx on another to parallelize things a bit. Has been
running all night ;)  Resampler on an Athlon 3200+ at 99% cpu and
atsc_rx on a 1.4Ghz Pentium-M running about 70%.
3719f4fea703e38bcbf8de6fe6bcdf55?d=identicon&s=25 Martin Dvh (Guest)
on 2006-04-09 13:28
(Received via mailing list)
Michael Dickens wrote:
> On Apr 7, 2006, at 6:49 PM, Charles Swiger wrote:
>
>> work - cool! (putting skip AFTER  inter, deinter). I'll finish the
>> test code and move on to Trellis Encoder / Viterbi Decoder.
>
There is a very fast (SSE,SSE2) viterbi decoder by Phil Karns.
(He also wrote a reed-solomon decoder)
They are available here:
http://www.ka9q.net/code/fec/

Greetings,
Martin
745d8202ef5a58c1058d0e5395a78f9c?d=identicon&s=25 Eric Blossom (Guest)
on 2006-04-09 20:17
(Received via mailing list)
On Sun, Apr 09, 2006 at 01:26:50PM +0200, Martin Dvh wrote:
> Michael Dickens wrote:
> > On Apr 7, 2006, at 6:49 PM, Charles Swiger wrote:
> >
> >> work - cool! (putting skip AFTER  inter, deinter). I'll finish the
> >> test code and move on to Trellis Encoder / Viterbi Decoder.
> >
> There is a very fast (SSE,SSE2) viterbi decoder by Phil Karns.
> (He also wrote a reed-solomon decoder)
> They are available here:
> http://www.ka9q.net/code/fec/

FYI, we're using an early version of his RS code.

Eric
821c1adf3f4be2f2aa6fbc89cce4c69a?d=identicon&s=25 Michael Dickens (Guest)
on 2006-04-10 19:50
(Received via mailing list)
Phil's work is quite famous, and his programs are of very good quality
for C programs.  Not the best commented, but they do work for the
platforms for which they are designed (generic or SSE/2 optimized).
Fortunately or not, I use a PPC PowerMac, and hence the SSE/2 stuff is
of no use to me.  Maybe I'll get an IntelMac once the hardware has
stabilized and I have the resources ;-)

My encoder and decoder blocks will augment his programs somewhat in
various ways including adding many useful comments, user options, and
computational advances in decoding, all without compromising
throughput. - MLD
Dc48f9c00e3e6de9640898a531c26d89?d=identicon&s=25 Charles Swiger (Guest)
on 2006-04-10 20:09
(Received via mailing list)
On Fri, 2006-04-07 at 18:20 -0700, Eric Blossom wrote:
> On Fri, Apr 07, 2006 at 06:49:07PM -0400, Charles Swiger wrote:
> >
> > > The most likely reason it doesn't pass, is that there is a 52
> > > segment delay between what you write into the interleaver, and what you
> > > get out of the deinterleaver.  To work around the behavior, drop the
> > > first 52 segments of the output and then compare the remainder with
> > > the input.
> > >
> >

> That's great.
> Did you figure out how to compare them without writing to files?
>

Yes, using gr.skiphead(atsc.sizeof_atsc_mpeg_packet_rs_encoded,52)
right after inter/deinterleaving, and also trim 52 * 188 off the
end of expected_result with

expected_result = src_data[0:len(src_data)-(52*188)]


I'm a little cornfused with the data sizes, in light of this:

"An MPEG transport stream packet is 188 bytes long.  Internally we use a
packet that is 256 bytes long to help with buffer alignment.  This
function adds the appropriate trailing padding to convert each packet
from 188 to 256 bytes."

and these:

>>> atsc.sizeof_atsc_data_segment
1024
>>> atsc.sizeof_atsc_data_segment_pad
188
>>> atsc.sizeof_atsc_mpeg_packet
256
>>> atsc.sizeof_atsc_mpeg_packet_no_sync
256
>>> atsc.sizeof_atsc_mpeg_packet_no_sync_pad
65
>>> atsc.sizeof_atsc_mpeg_packet_pad
68
>>> atsc.sizeof_atsc_mpeg_packet_rs_encoded
256
>>> atsc.sizeof_atsc_mpeg_packet_rs_encoded_pad
45
>>> atsc.sizeof_atsc_soft_data_segment
4096


I would hav expected "atsc_mpeg_packet" to be 188 and
"atsc_mpeg_packet_pad" to be 256 ??

Anyway, seems we should use one of those instead of 188 in
the expected_result trim statement above.

Other than that, make check runs and passes, ready to check
into cvs.

--Chuck
This topic is locked and can not be replied to.