Work function - how it fits in the big picture


#1

I’m stuck on how to loop over the data in calling a primitive
that “does 12 segments at a time”. What is it that calls the
work function and what sets the value of noutput_items?

If I completely ignore noutput_items and shovel in the code
from 0.9 :

for (int i = 0; i < atsci_trellis_encoder::NCODERS; i +=
atsci_trellis_encoder::NCODERS){
d_trellis_encoder.encode(&out[i], &in[i]);

it compiles and runs but every NCODERS blocks there a bunch
of missing or corrupted data.

Anything else, like

for (int i = 0; i < noutput_items; i +=
atsci_trellis_encoder::NCODERS){
d_trellis_encoder.encode(&out[i], &in[i]);

just segfaults :wink: I think it needs to relate noutput_items
to NCODERS somehow that I don’t understand.

Or, what code in gnuradio-core might shed some light on the
plumbing?

–Chuck


#2

Chuck - A variety of comments; someone please correct me if my
understanding is not correct. - MLD

  1. The “work()” method is used by the “gr_sync_block” class as a
    subcall from “general_work()”. The latter (“general_work()”) is
    called by the scheduler (quoted from a previous email from Eric; see
    “gr_single_threaded_scheduler::main_loop()” for the actual
    “scheduler” code which does all the ‘work’ … ha ha):
    +++++++++++++
    Part of the setup is in Python, the runtime part is in C++.

gnuradio-core/src/python/gnuradio/gr:

basic_flow_graph.py
flow_graph.py
scheduler.py

gnuradio-core/src/lib/runtime:

gr_single_threaded_scheduler.{h,cc}		# the runtime scheduler

   you'll also want to look at

gr_block.{h,cc} and gr_block_detail.{h,cc}

+++++++++++++
2) The scheduler calls each block’s “forecast()” method repeatedly
starting with a “large” buffer size (e.g. 2^16) and dropping down by
powers of 2 nearest the block’s “output_multiple”. The end result is
that each block’s “general_work()” method is provided with a correct
“output_multiple” multiple of number of output items to create (i.e.
if the “o_m” is 10, then “noutput_items” = 10*X, where X is any non-
negative integer). The number is clearly dependent on the number of
“input_items” available for processing, as well as the “forecast()”
to create a reasonable number of output items given those input items
(really vice versa, but it’s one way of looking at things).

  1. The “forecast()” method is important (as per 2 above); make sure
    it returns an accurate number of input items required for the
    provided number of output items.

  2. The return value for “work()” or “general_work()” is the number of
    output items actually created, but no more than the input
    “noutput_items”. Remember, these are -items- as defined by the
    “output_io_signature”, not necessarily Bytes or whatever.

  3. Instead of looping over “atsci_trellis_encoder::NCODERS” or the
    number of output items, you should use the size of the input_items or
    output_items (depending on how your computation works;e.g.
    “output_items.size()”). These may or not be the same depending on
    how your "io_signature"s are set. Looking at the code from 0.9,
    maybe something like:

    for (int i = 0; i < output_items.size(); i++) {
    d_trellis_encoder.encode (&out[i], &in[i]);

But what you do really depends on how the rest of the block is
configured.


#3

On Wed, Apr 19, 2006 at 05:50:38PM -0400, Charles S. wrote:

Ah [lightbulbs flash] - I’m blindly returning noutput_items
but only processing NCODERS items, explains the missing data.

This’ll take a while.

–Chuck

Chuck,

To make the new code match the behavior of GrAtscTrellisEncoder
(which is what I think you’re up to) you’ll need to derive from
gr_sync_block and add this to your constructor:

set_output_multiple(atsci_trellis_encoder::NCODERS);

If you’re explicitly calling set_history, remove that.

I’d try it with the standard forecast method first. If that doesn’t
work you may have to modify it such that it returns NCODERS + the
“regular” answer. Normally we’d just call
set_history(atsci_trellis_encoder::NCODERS), but that would have the
side-effect of the first 12-segments we saw being binary zero, which
means it wouldn’t have the pipeline info fields initalized properly.

Hope this helps. The code is a bit odd because it must maintain the
12-segment alignment, even in the face of errors upstream (which I
don’t think can happen).

I also suggest adding the “paranoid check” that’s at the bottom of
GrAtscTrellisEncoder if you haven’t already.

Eric


#4

On Wed, 2006-04-19 at 17:41 -0700, Eric B. wrote:

To make the new code match the behavior of GrAtscTrellisEncoder
(which is what I think you’re up to) you’ll need to derive from
gr_sync_block and add this to your constructor:

set_output_multiple(atsci_trellis_encoder::NCODERS);

Great - that seems to work.

I also suggest adding the “paranoid check” that’s at the bottom of
GrAtscTrellisEncoder if you haven’t already.

That check is in the atsci_trellis_encoder.cc.

Ok - I’m going to spend time trying to understand all that. Putting
the same thing in atsc_viterbi_decoder.cc compiles and runs and
gets this further down the loopback:

sched: <gr_block atsc_viterbi_decoder (21)> is requesting more input
data
than we can provide.
ninput_items_required = 12
max_possible_items_available = 7
If this is a filter, consider reducing the number of taps.

It wants 12 4096-byte soft_data_segment, or 49152 bytes. Max
available is 28672.

Well, one solution is to go into
gnuradio-core/src/python/gnuradio/gr/flow_graph.py and change
fixed_buffer_size to:

self.fixed_buffer_size = 64*1024

from 32*1024

then make, make install. Don’t know if that can by dynamically
changed from the processing block.

Runs, few issues in the qa script then will update cvs.

–Chuck


#5

On Thu, Apr 20, 2006 at 11:53:03AM -0400, Charles S. wrote:

Runs, few issues in the qa script then will update cvs.

–Chuck

Don’t commit the change to flow_graph.py, I believe the right answer
is elsewhere. Please do commit the atsc changes even though it fails
make check, and I’ll use it to test and fix. I think this is a symptom
of the same problem Marcus is seeing when trying to use large numbers
of taps in the FFT filters.

Eric


#6

On Thu, 2006-04-20 at 09:12 -0700, Eric B. wrote:

atsc_trellis_encoder, atsc_viterbi_decoder and atsc_ds_to_softds to
glue them together have been checked in. The python qa_atsc.py script
has another loopback test 002 for

src -> randomizer -> rs_encoder -> trellis_encoder -> ds_to_softds ->
viterbi_decoder -> rs_decoder -> derandomizer -> dst
assert src = dst

Notice that it does NOT have the interleaver/deinterleaver as I
don’t know how to handle the 52 segment delay which causes
trellis to choke on regular segment flag check.

–Chuck


#7

On Thu, Apr 20, 2006 at 03:31:05PM -0400, Charles S. wrote:

of taps in the FFT filters.

atsc_trellis_encoder, atsc_viterbi_decoder and atsc_ds_to_softds to
glue them together have been checked in. The python qa_atsc.py script
has another loopback test 002 for

src -> randomizer -> rs_encoder -> trellis_encoder -> ds_to_softds ->
viterbi_decoder -> rs_decoder -> derandomizer -> dst
assert src = dst

OK.

Notice that it does NOT have the interleaver/deinterleaver as I
don’t know how to handle the 52 segment delay which causes
trellis to choke on regular segment flag check.

I think that if you move the skiphead to the very end (or just do it
by slicing the 52 out of the python expected vs actual vals), it will
work with the interleaver in.

Eric


#8

On Wed, 2006-04-19 at 16:54 -0400, Michael D. wrote:

Chuck - A variety of comments; someone please correct me if my
understanding is not correct. - MLD

  1. The return value for “work()” or “general_work()” is the number of
    output items actually created, but no more than the input
    “noutput_items”. Remember, these are -items- as defined by the
    “output_io_signature”, not necessarily Bytes or whatever.

Ah [lightbulbs flash] - I’m blindly returning noutput_items
but only processing NCODERS items, explains the missing data.

This’ll take a while.

–Chuck


#9

Has anyone begun/completed work on a Digital Radio Mundial block for
gnu-radio/USRP?

I’m interested in receiving these signals, but no such module seems to
exist currently.

If not, what would it take to reference the libraries from
drm.sourceforge.net from within python in order to decode these signals?


#10

On Thu, 2006-04-20 at 13:54 -0700, Eric B. wrote:

Notice that it does NOT have the interleaver/deinterleaver as I
don’t know how to handle the 52 segment delay which causes
trellis to choke on regular segment flag check.

I think that if you move the skiphead to the very end (or just do it
by slicing the 52 out of the python expected vs actual vals), it will
work with the interleaver in.

Done - but I had to drop skiphead alltogether for some reason it
emits lots of zeros, maybe just on mpeg_packet size items. I don’t
know, but just trimming expected_result and result_data in python
works and all is ok now. Updated and checked in.

Oh, and you have to include another 12 packet delay from the viterbi
decoder. 52+12.

All that was just tying a ribbon on nearly finished code - going forward
the field sync and bit timing stuff looks pretty formidable and sketchy.

It’d be nice to find the Wayne Bretl text.

–Chuck


#11

Take Swiger’s HF modules, tune to DRM frequency and run the audio into
the DRM (Dream) code. That is a hardware/analog (sound card needed)
solution. A soft connection will require more work.

Bob

Brett L Trotter wrote:


Discuss-gnuradio mailing list
removed_email_address@domain.invalid
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio


AMSAT VP Engineering. Member: ARRL, AMSAT-DL, TAPR, Packrats,
NJQRP/AMQRP, QRP ARCI, QCWA, FRC. ARRL SDR Wrk Grp Chairman
Laziness is the number one inspiration for ingenuity. Guilty as
charged!


#12

On Thu, Apr 20, 2006 at 06:16:06PM -0400, Charles S. wrote:

Done - but I had to drop skiphead alltogether for some reason it
emits lots of zeros, maybe just on mpeg_packet size items. I don’t
know, but just trimming expected_result and result_data in python
works and all is ok now. Updated and checked in.

Good.

Oh, and you have to include another 12 packet delay from the viterbi
decoder. 52+12.

Oh, that :wink:

All that was just tying a ribbon on nearly finished code - going forward
the field sync and bit timing stuff looks pretty formidable and sketchy.

The segment sync stuff isn’t too bad. It should be pretty straight
forward. Basically you’re looking for the “pulse” at the beginning of
each segment. The bit timing might take a bit more to understand, but
again, it’s not too bad. You’re going to want to base the port on
GrAtscBitTimingLoop3. The other two were failed experiments :wink:

It’d be nice to find the Wayne Bretl text.

A bit of googling, and then remembering that I found it on the Zenith
site gives:

http://www.zenith.com/sub_hdtv/iscemain.zip

which contains a doc file. See also:

http://www.zenith.com/sub_hdtv/hdtv_papers.html

Eric


#13

On Fri, Apr 21, 2006 at 12:31:23AM -0400, Robert McGwier wrote:

Take Swiger’s HF modules, tune to DRM frequency and run the audio into
the DRM (Dream) code. That is a hardware/analog (sound card needed)
solution. A soft connection will require more work.

Where’s the source for the DRM libraries? This stuff came off of
sourceforge, right?

Eric