Forum: GNU Radio Slow message queues... How to speed up?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
E. Ornelas (Guest)
on 2009-03-18 18:45
(Received via mailing list)
Hello!

I'm trying to write a program that read data from the usrp, then
preforms a
fft and plots
the values of the maximum frequency and correspondent amplitude in real
time.

I've come across this problem:
- At first I was using vector_sink but it gave me problems with memory
management, always
crashing after a few moments.
- Now I've implemented a message queue system. With a queue for the
frequency values and
one for the amplitude.
- I've noticed that it became much slower (about 10 fft's per second, 2
per
message)
- I need to have a much higher number to do what I want in the final
version

Is there any way to speed up the process? (Have more values per message
or
more messages per second maybe..)

Following is my code:

class top(gr.top_block):
  def __init__(self):
    gr.top_block.__init__(self)

    #Settings
    rx_subdev_spec = (0, 0)
    decim = 256
    freq = None
    gain = 10

    self.u = usrp.source_c(decim_rate=decim)
    self.u.set_mux(usrp.determine_rx_mux_value(self.u, rx_subdev_spec))

    # determine the daughterboard subdevice we're using
    self.subdev = usrp.selected_subdev(self.u, rx_subdev_spec)

    # FFT Parameters
    fftsize = 4096
    mywin = window.blackmanharris(fftsize)
    fft = gr.fft_vcc(fftsize, True, mywin)

    # Get Maximum Value
    c2m = gr.complex_to_mag(fftsize)
    imax = gr.argmax_fs(fftsize)    # index
    amax = gr.max_ff(fftsize)      # amplitude

    s2f1 = gr.short_to_float()
    s2f2 = gr.short_to_float()
    p0 = gr.probe_signal_f()
    p1 = gr.probe_signal_f()

    # Vector Sink/Source
    ss2v = gr.stream_to_vector(gr.sizeof_gr_complex, fftsize)
    v2ss = gr.vector_to_stream(gr.sizeof_float, fftsize)

    # Message Settings
    self.qsize = 10

    self.msgq0 = gr.msg_queue(self.qsize)  # queue frequency
    self.msgq1 = gr.msg_queue(self.qsize)  # queue amplitude

    # Message sink for frequency(0) and amplitude(1)
    m_sink0 = gr.message_sink(gr.sizeof_float, self.msgq0, True)
    m_sink1 = gr.message_sink(gr.sizeof_float, self.msgq1, True)

    self.connect(self.u, ss2v, fft, c2m, (imax,0), s2f1, m_sink0)
    self.connect((imax,1), s2f2, p1)

    self.connect(c2m, amax, m_sink1)

        # set initial values

    if gain is None:
    # if no gain was specified, use the mid-point in dB
      g = self.subdev.gain_range()
      gain = float(g[0]+g[1])/2

      if freq is None:
        freq = 10.7e6

        self.set_gain(gain)

        if not(self.set_freq(freq)):
          pass
    else:
      if freq is None:
        freq = 10.7e6

        self.set_gain(gain)

        if not(self.set_freq(freq)):
          pass

  def set_freq(self, target_freq):

    r = usrp.tune(self.u, 0, self.subdev, target_freq)

    return False

  def set_gain(self, gain):
    self.subdev.set_gain(gain)

def shift(x,size=4096):
  """
    Shifts indexes so that the first half corresponds to FS-FS/2
  and the second half to FS+FS/2.
    @ x: index number
  """
  if x <= (size/2):
    return x+(size/2)-1
  elif x > (size/2):
    return x-(size/2)-1


def fill_buffer(buffer, data, buffer_size=1000):
  """
    Fill a buffer in a circular form. When buffer_size is reached
  the oldest values are discarded. The number of values discarded
  depend on the size of new ones to add to the buffer.
    @ buffer: vector to add the values
    @ data: list of values to add
    @ buffer_size: 1000 by default
    @ out: final buffer with data values added
  """

  if len(buffer) < buffer_size:
    out = concatenate((buffer,data))
  elif len(buffer) >= buffer_size:
    out = concatenate((buffer[len(data):],data))
  return out

def main ():

  # Start Top Block
  tb = top()

  #Start Flowgraph
  tb.start()

  # Set Buffers
  yf=[]
  ya=[]
  fftsize=4096

  # Read first message

  fmsg = tb.msgq0.delete_head()  # get first frequency message
  amsg = tb.msgq1.delete_head()  # get first amplitude message

  raw_f = fmsg.to_string()  # raw frequency data
  raw_a = amsg.to_string()  # raw amplitude data

  f_data = numpy.fromstring(raw_f, numpy.float32, count =
int(fmsg.arg2()))  #
converted frequency data
  a_data = numpy.fromstring(raw_a, numpy.float32, count =
int(amsg.arg2()))  #
converted amplitude data

  # fill frequency buffer
  yf = fill_buffer(yf,f_data)

  # fill frequency buffer
  ya = fill_buffer(ya,a_data)

  # Set Initial Plot Values

  x = [i for i in range(len(yf))]

  a_data = 10.0*scipy.log10(a_data)
  yf = [shift(i,fftsize) for i in yf]  # index shifting
  yf = [10.7e6+i*250e3/fftsize-125e3 for i in yf]  # convertion from
index to
frequency

  pylab.ion()

  (....)

  pylab.draw()

  while(True):

    fmsg = tb.msgq0.delete_head()  # get first frequency message
    amsg = tb.msgq1.delete_head()  # get first amplitude message

    raw_f = fmsg.to_string()  # raw frequency data
    raw_a = amsg.to_string()  # raw amplitude data

    f_data = numpy.fromstring(raw_f, numpy.float32, count =
int(fmsg.arg2()))
# converted frequency data
    a_data = numpy.fromstring(raw_a, numpy.float32, count =
int(amsg.arg2()))
# converted amplitude data

    a_data = 10.0*scipy.log10(a_data)
    f_data = [shift(i,fftsize) for i in f_data]      # index shifting
    f_data = [10.7e6+i*250e3/fftsize-125e3 for i in f_data]  #
convertion from
index to frequency

    # fill frequency buffer
    yf = fill_buffer(yf, f_data)

    # fill frequency buffer
    ya = fill_buffer(ya, a_data)

    (....)

    pylab.draw()

if __name__ == '__main__':
    main ()

--
View this message in context:
http://www.nabble.com/Slow-message-queues...-How-t...
Sent from the GnuRadio mailing list archive at Nabble.com.
Eric B. (Guest)
on 2009-03-18 19:00
(Received via mailing list)
On Wed, Mar 18, 2009 at 09:44:05AM -0700, E. Ornelas wrote:
> management, always
> crashing after a few moments.
> - Now I've implemented a message queue system. With a queue for the
> frequency values and
> one for the amplitude.
> - I've noticed that it became much slower (about 10 fft's per second, 2 per
> message)
> - I need to have a much higher number to do what I want in the final version
>
> Is there any way to speed up the process? (Have more values per message or
> more messages per second maybe..)

First off, depending on the machine you are running this on, you may
or may not have enough cycles to get the job done.

You want to do as little as possible in python.  That is, be sure that
there's no python on the critical path.

Consider writing a C++ block that scans the fft output vectors and
computes the value that you want.  It sounds like you want something
that looks at each fft output vector, and for each vector returns the
index of the bin with the most power, and the magnitude of that bin.

Eric
This topic is locked and can not be replied to.