Randomness Introduced in Gnuradio WAV Source Block?

Dear Gnuradio,

I wrote a program that uses a WAV source block. I found that my
program ran as expected on my CENTOS and UBUNTU machines (producing the
same output for a given input), but I noticed something unusual on my
Gentoo machine. If I run the decoder against the same input WAV file, I
get different output every time! My program is an APT decoder, so the
resulting output is an image. The images look similar, but they have
different intensities. (some are darker, some lighter)

I eventually determined, that the randomness was added somewhere in
the WAV Source block. Has anyone seen this?

This little program shows where the problem may be. The input is a WAV
file. The float stream is converted to chars, and then dumped to a file.
This should be completely reproducible, but it doesn’t seem to be in the
case of my Gentoo machine.


class top_block(gr.top_block):

    def __init__(self):
            gr.top_block.__init__(self, "WAV TEST")

            inputFileName = str(sys.argv[1])
            outputFileName = str(sys.argv[2])

            self.gr_wavfile_source_0 = 

gr.wavfile_source(inputFileName, False)
self.gr_float_to_uchar_0 = gr.float_to_uchar()
self.gr_file_sink_0 = gr.file_sink(gr.sizeof_char*1,
outputFileName)
self.gr_file_sink_0.set_unbuffered(False)

            self.connect((self.gr_wavfile_source_0,0), 

(self.gr_float_to_uchar_0, 0))
self.connect((self.gr_float_to_uchar_0, 0),
(self.gr_file_sink_0, 0))

if name == ‘main’:

    tb = top_block()
    tb.run(True)

I used the same WAV file input 4 times, and got 4 different outputs.

This is the same problem-computer that gives me the make test errors, so
that may be related.

               Sincerely,
      Tommy James Tracy II
        PhD Student

High Performance Low Power Lab
University of Virginia

On Tue, Jan 15, 2013 at 01:48:47AM -0500, Tommy T. II wrote:

I eventually determined, that the randomness was added somewhere in the WAV
Source block. Has anyone seen this?

Tommy,

can you please post more details:

  • Which GR version?
  • What format are your WAV files? (8/16 Bit, Mono/Stereo)
    • Does that matter?
  • How large is the WAV file?
  • To be clear, this works on Ubuntu, but not on Gentoo? (works on my
    Ubuntu …)
  • Do you know how the output is corrupted?

This little program shows where the problem may be. The input is a WAV file.
The float stream is converted to chars, and then dumped to a file. This should
be completely reproducible, but it doesn’t seem to be in the case of my Gentoo
machine.

Are the Ubuntu & Gentoo machine different hardware?

[code]

if name == ‘main’:
tb = top_block()
tb.run(True)

Hm, I’m not even sure what the ‘True’ does here… normally you limit the
FG to a number of items with the run() argument.

Also, can you please run this test: Add a throttle after the file
source, and set it to a low value (50k or something). Then run again.
(I’m serious :).

MB


Karlsruhe Institute of Technology (KIT)
Communications Engineering Lab (CEL)

Dipl.-Ing. Martin B.
Research Associate

Kaiserstraße 12
Building 05.01
76131 Karlsruhe

Phone: +49 721 608-43790
Fax: +49 721 608-46071
www.cel.kit.edu

KIT – University of the State of Baden-Württemberg and
National Laboratory of the Helmholtz Association

On Tue, Jan 15, 2013 at 01:48:47AM -0500, Tommy T. II wrote:

I eventually determined, that the randomness was added somewhere in the WAV
Source block. Has anyone seen this?

Tommy,

Martin,

can you please post more details:

  • Which GR version?
    $gnuradio-config-info -v
    Ubuntu: v3.6.2-265-gb15b38df
    Gentoo: v3.6.x-xxx-xunknown

The version of gnu radio installed on the Gentoo box was installed from
source from the latest stable release of gnuradio-3.6.3.
As per some prior posts, I get failed ‘make test’ results, but the
install completes.

  • What format are your WAV files? (8/16 Bit, Mono/Stereo)
    This is an 8 Bit/Sample, 11025Hz, Mono WAV file.
  • Does that matter?
    I don’t think it should. There is an 8-bit to float (32-bit)
    conversion somewhere in that block, but it should be deterministic. It
    appears to be deterministic on the Ubuntu machine.
  • How large is the WAV file?
    The WAV file is 10.9 MBytes
  • To be clear, this works on Ubuntu, but not on Gentoo? (works on my
    Ubuntu )
    Yes. This is how I determined it ‘worked’.
for i in 1 2 3 4 5; do ./WAV_TEST.py ../sources/<source.wav> $i; 

done
for i in 1 2 3 4 5; do md5sum $i; done

  Ubuntu: They are the same
  143a5b3be71e8476ff2f5edd1086e79f 1
  143a5b3be71e8476ff2f5edd1086e79f 2
  143a5b3be71e8476ff2f5edd1086e79f 3
  143a5b3be71e8476ff2f5edd1086e79f 4
  143a5b3be71e8476ff2f5edd1086e79f 5

  Gentoo: They are not
  6f328ef003b3a60cf4b9675a5453d142 1
  c544a257aef0a2f95c9cda82f92acbd6 2
  c544a257aef0a2f95c9cda82f92acbd6 3
  6f328ef003b3a60cf4b9675a5453d142 4
  cf9dc9f7d0c80a3c205c47c60c6c7ac9 5

Interesting note though; all 10 files have the same size.
  • Do you know how the output is corrupted?

This is where it gets weird.

for i in 1 2 3 4 5; do hexdump $i > $i.hex; done

To compare, I looked at a few lines of 1.hex, 2.hex, and 5.hex (they are
all different)

1.hex
0000000 a27e a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
0000010 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
*
0000170 a1a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
0000180 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
*

0019710 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a3
0019720 a1a1 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
0019730 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
*
001a1d0 a2a3 a1a1 a3a3 a1a2 a2a2 a2a2 a2a2 a2a2
001a1e0 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

04f09a0 a1a1 a3a3 a1a2 a3a2 a1a2 a2a1 a2a3 a2a1
04f09b0 a3a3 a1a1 a3a2 a1a2 a3a1 a2a3 a2a1 a2a3
04f09c0 a1a1 a3a2 a1a2 a3a2 a1a3 a2a1 a2a3 a1a1
04f09d0 a3a3 a1a2 a3a2 a2a2 a2a1 a2a3 a2a1 a3a3

2.hex
0000000 421e 4242 4242 4242 4242 4242 4242 4242
0000010 4242 4242 4242 4242 4242 4242 4242 4242
*
0000170 4142 4242 4242 4242 4242 4242 4242 4242
0000180 4242 4242 4242 4242 4242 4242 4242 4242
*

0019710 4242 4242 4242 4242 4242 4242 4242 4243
0019720 4141 4242 4242 4242 4242 4242 4242 4242
0019730 4242 4242 4242 4242 4242 4242 4242 4242
*
001a1d0 4243 4141 4343 4142 4242 4242 4242 4242
001a1e0 4242 4242 4242 4242 4242 4242 4242 4242
*

04f09a0 4141 4343 4142 4342 4142 4241 4243 4241
04f09b0 4343 4141 4342 4142 4341 4243 4241 4243
04f09c0 4141 4342 4142 4342 4143 4241 4243 4141

5.hex
0000000 c29e c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
0000010 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
*
0000170 c1c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
0000180 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
*

0019710 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c3
0019720 c1c1 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
0019730 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
*
001a1d0 c2c3 c1c1 c3c3 c1c2 c2c2 c2c2 c2c2 c2c2
001a1e0 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2
*

04f09a0 c1c1 c3c3 c1c2 c3c2 c1c2 c2c1 c2c3 c2c1
04f09b0 c3c3 c1c1 c3c2 c1c2 c3c1 c2c3 c2c1 c2c3
04f09c0 c1c1 c3c2 c1c2 c3c2 c1c3 c2c1 c2c3 c1c1

Other than the first 2-byte entry (which seems to take the form of
2e, all other bytes seem to be identical with a simple
substitution.

This little program shows where the problem may be. The input is a WAV file.
The float stream is converted to chars, and then dumped to a file. This should
be completely reproducible, but it doesn’t seem to be in the case of my Gentoo
machine.

Are the Ubuntu & Gentoo machine different hardware?

Yes. They are very different.
The Ubuntu machine is a core i5, 2GB desktop.
The gentoo machine is a storage-less single-board computer, with NFS
filesystem.

[code]

if name == ‘main’:
tb = top_block()
tb.run(True)

Hm, I’m not even sure what the ‘True’ does here… normally you limit the
FG to a number of items with the run() argument.

I’m not sure either The ‘True’ does slow it down considerably. I removed
it and still get the same behavior.

Also, can you please run this test: Add a throttle after the file
source, and set it to a low value (50k or something). Then run again.
(I’m serious :).

Updated Throttle Code:

class top_block(gr.top_block):

    def __init__(self):
            gr.top_block.__init__(self, "WAV TEST")

            ##################################################
            # Blocks
            ##################################################

            inputFileName = str(sys.argv[1])
            outputFileName = str(sys.argv[2])

            self.gr_wavfile_source_0 = 

gr.wavfile_source(inputFileName, False)

            self.throttle = gr.throttle(gr.sizeof_float, 50000)
            self.gr_float_to_uchar_0 = gr.float_to_uchar()
            self.gr_file_sink_0 = gr.file_sink(gr.sizeof_char*1, 

outputFileName)
#self.gr_file_sink_0.set_unbuffered(False)

            ##################################################
            # Connections
            ##################################################
            #self.connect((self.gr_wavfile_source_0,0), 

(self.gr_float_to_uchar_0, 0))
self.connect((self.gr_wavfile_source_0, 0),
(self.throttle, 0))
self.connect((self.throttle, 0),
(self.gr_float_to_uchar_0, 0))
self.connect((self.gr_float_to_uchar_0, 0),
(self.gr_file_sink_0, 0))

if name == ‘main’:

    tb = top_block()
    start = time()
    tb.run()
    end = time()
    print str(end-start)

As expected, the throttle slowed down the execution considerably.
But after hex dumping the new files, I get the following:
(1-5 are done without the throttle; 6-10 are done with the throttle)

$ ls -sh *.hex
3.6M 1.hex 3.6M 2.hex 3.6M 4.hex 4.0K 6.hex 4.0K 8.hex
4.0K 10.hex 3.6M 3.hex 3.6M 5.hex 3.6M 7.hex 3.6M 9.hex

You’ll notice that some of the resulting hex files for 6-10 are much
smaller

Ignoring the hex-dump failures

7.hex
0000000 bebe bebe bebe bebe bebe bebe bebe bebe
*
0000170 bdbe bebe bebe bebe bebe bebe bebe bebe
0000180 bebe bebe bebe bebe bebe bebe bebe bebe
*

0019710 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e3
0019720 e1e1 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2
0019730 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2
*
001a1d0 e2e3 e1e1 e3e3 e1e2 e2e2 e2e2 e2e2 e2e2
001a1e0 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2
*

04f09a0 e1e1 e3e3 e1e2 e3e2 e1e2 e2e1 e2e3 e2e1
04f09b0 e3e3 e1e1 e3e2 e1e2 e3e1 e2e3 e2e1 e2e3
04f09c0 e1e1 e3e2 e1e2 e3e2 e1e3 e2e1 e2e3 e1e1

The difference in 7.hex is that lines 0000000-0001c70 are padded with
bebe; from there on, I get the same e<1,2,3>e<1,2,3> bytes.

9.hex
0000000 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e
*
0000170 7d7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e
0000180 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e
*

0019710 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a3
0019720 a1a1 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
0019730 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
*
001a1d0 a2a3 a1a1 a3a3 a1a2 a2a2 a2a2 a2a2 a2a2
001a1e0 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2
*

04f09a0 a1a1 a3a3 a1a2 a3a2 a1a2 a2a1 a2a3 a2a1
04f09b0 a3a3 a1a1 a3a2 a1a2 a3a1 a2a3 a2a1 a2a3
04f09c0 a1a1 a3a2 a1a2 a3a2 a1a3 a2a1 a2a3 a1a1

The difference in 9.hex is that lines 0000000-0001c70 are padded with
7e7e; from there on, I get the same a<1,2,3>a<1,2,3> bytes as shown in
1.hex

Kaiserstrae 12
Discuss-gnuradio mailing list
[email protected]
Discuss-gnuradio Info Page

               Sincerely,
      Tommy James Tracy II
        PhD Student

High Performance Low Power Lab
University of Virginia