Gr.head, and flushing of file sinks

GNU Radio
#1

  1. Has anybody ever noticed a case where gr.head doesn’t seem to limit
    the
    amount of data that flows through it? I have (some data source) ->
    (head) ->
    (file sink), and sometimes the writing to the file shuts off
    appropriately,
    other times it seems to skip over the limit and would happily consume my
    entire hard drive (I watch the file grow to 50MB, 100MB, …) if I
    didn’t
    Ctrl-C. I’m having trouble reproducing this now, but thought I would
    ask.

b = MyTopBlockWhichWritesToAFileSink()
b.start()
b.wait()
raw_input() #wait indefinitely for user to hit enter
del b

b.wait() doesn’t seem to do a final flush of the file sink…ie if b is
supposed to write 10000 bytes to a file, after b.wait() the file will
have
8.0KB (~8192 bytes) in it. Not until del b is the file complete (9.8KB,
~10000 bytes). What exactly does b.wait() do? Is there any way to force
a
file flush without deleting b?

  1. Is there anything wrong with having multiple sequential top blocks?
    Something like:

a = MyTopBlock1()
a.start()
a.wait()
del a

for i in range(10):
b = MyTopBlock2()
b.start()
b.wait()
del b

#2

Steven C. wrote:

  1. Has anybody ever noticed a case where gr.head doesn’t seem to limit
    the amount of data that flows through it? I have (some data source) ->
    (head) -> (file sink), and sometimes the writing to the file shuts off
    appropriately, other times it seems to skip over the limit and would
    happily consume my entire hard drive (I watch the file grow to 50MB,
    100MB, …) if I didn’t Ctrl-C. I’m having trouble reproducing this now,
    but thought I would ask.

Please let us know when you have a way to reliably reproduce this, and
document a bug on our Trac website.

(9.8KB, ~10000 bytes). What exactly does b.wait() do? Is there any way
to force a file flush without deleting b?

‘wait()’ will cause the calling thread to block until all the top block
scheduler threads exit. This may happen on its own or be caused by a
SIGINT.

The gr.file_sink block doesn’t close its file handle and flush to disk
until its destructor is called, which doesn’t happen until it entirely
goes out of scope (that’s what your ‘del b’ is forcing.)

There isn’t at present a way to cause a flush to occur at random.
Partly this is because the file sink fwrite calls are happening in a
separate thread, so to add a flush method to the block would require
adding locking, etc.

Another method might be to add a constructor passed flag that would
optionally cause a fflush after every fwrite. This would really slow it
down, but would give you the results you want.

  1. Is there anything wrong with having multiple sequential top blocks?
    Something like:

No, as long as you are deleting each before creating the next one. At
present, there is a somewhat artificial limitation of one top block per
application. This will go away in release 3.2.

–
Johnathan C.
Corgan Enterprises LLC
http://corganenterprises.com

#3

On Tue, Oct 23, 2007 at 12:22:31PM -0400, Steven C. wrote:

b.start()
b.wait()
raw_input() #wait indefinitely for user to hit enter
del b

b.wait() doesn’t seem to do a final flush of the file sink…ie if b is
supposed to write 10000 bytes to a file, after b.wait() the file will have
8.0KB (~8192 bytes) in it. Not until del b is the file complete (9.8KB,
~10000 bytes). What exactly does b.wait() do? Is there any way to force a
file flush without deleting b?

You could call my_file_sink_instance.close(), which will flush and
close the underlying file. This may not be exactly what you want.

b = MyTopBlock2()
b.start()
b.wait()
del b

It should work. Doesn’t mean that there isn’t a bug somewhere,
particularly if you are using the pkt based stuff. There was a report
of a hang related to something (the queue reader I think) hanging on a
message queue that was never seeing any further messages.

FWIW, the existing QA code uses serial top blocks without a problem.

If you’re using the gr.udp_source, it’s broken, and will result in
gr.head() never stopping. It’ll just hang.

In the random suggestion category: stylistically, instead of
b.start(), b.wait(), you may want to consider b.run(). It does the
start(), wait() for you.

If this didn’t help, can you cook a simple example that has the
problem. E.g., gr.file_source() -> gr.head() -> gr.file_sink()

Eric