# Gr-qtgui Waterfall Time Axis **URL:** \ **Category:** GNU Radio\ **Created:** [October 21, 2014, 10:01pm UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674 "2014-10-21T22:01:27Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![addis\_a](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/c4cdca/32.png) [@addis\_a](https://www.ruby-forum.com/u/addis_a)\ **Post date:** [October 21, 2014, 10:01pm UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/1 "2014-10-21T22:01:27Z")
I have some large spectrum records and I would like to use GR for analysis in a a sort of “off-line” mode. I want to visualize the data using the spectrogram and then identify subsets of interesting features (e.g. WiFi Packets in 2.4 GHz) by frequency and time coordinates. Essentially, I want to know what index in the file certain features correspond with so I can save them as separate files. To this end, I’ve found gr\_spectrogram\_plot very handy, but it doesn’t quite do what I want because the y-axis which displays time isn’t showing up correctly. My y-axis is always 0.0, and the cursor highlight also reads 0.0 (although the frequency is correct). However, the qt spectrogram appears to have correct time information when I make a simple file source -\> qt spectrogram sink. I’d like to fix this issue so I’ve been pouring over the code in gr-qtgui. I don’t have much experience with QWT or Python GUIs in general. From what I can tell, there are methods for frequency axis and intensity scaling, but not time. Is this true or am I missing something in the API? The historyExtent parameter in the WaterfallData constructor appears to get hardcoded to 200 in WaterfallDisplayPlot.cc (gr-qtgui/lib). It would also be nice to toggle between the time and sample number. I’m happy to do the work to implement this. Could someone point me in the right direction here? Thanks! PWG ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [October 30, 2014, 11:26am UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/2 "2014-10-30T11:26:55Z")
I’ve solved this and wanted to share my solutions, as well as improvements (to me, anyways) to GR’s off-line plotting tools (gr\_spectrogram\_plot). My overall goal here is to be able to do some analysis on wideband (25MSPS) data using GR since MATLAB isn’t really efficient for that. I’ve seen some interesting updates regarding gr-qtgui on the git commit log – I’d be very interested in comments from those folks so I know what’s in the pipeline and perhaps what you would want to pull into the baseline. Below is a list of issues I’ve encountered and what I’ve done to fix them. Problem:In gr\_spectrogram\_plot\_c, the time scale always reads 0. Solution: In waterfalldisplayform.cc, d\_update\_time is passed as the timePerFFT argument to plotNewData (WaterfallDisplayPlot object). This happens in WaterfallDisplayForm::newData(). However, gr\_spectrogram\_plot sets this to zero by calling set\_update\_time(0) on the waterfall\_sink\_c\_impl object. In waterfall\_sink\_c\_impl.cc it calls the postEvent() function with d\_last\_time, which is a variable representing the time between waterfall updates. Ultimately, this is regulated by d\_update\_time, which is set to zero by the set\_update\_time(0) call. The net result is that timePerFFT is always zero, resulting in an improperly set timescale. The proper thing to do here is to set timePerFFT based on the FFT size and sample rate in off-line mode. Obviously, this assumes you know the sample rate, but since I’ve modified gr\_spectrogram\_plot to read files with headers and autofill stuff (sample rate, frequency, data type, etc) appropriately, it always does. It appears that you won’t be necessarily processing every sample if you don’t set d\_update\_time=0, which you want to when running off-line. Problem:In gr\_spectrogram\_plot, the cursor pointer only gives readings to a 100th of a second. For analysis, it would be more useful to have scientific notation which scales with higher sample rates. Solution: In WaterfallDisplayPlot.cc, change .arg(secs,0,‘f’,2) to .arg(secs,0,‘e’,2) Problem: In gr\_spectrogram\_plot, the time scale is in in hundreds of a second. It may be more useful to have the plot in scientific notation. Solution: In WaterfallDisplayPlot.cc, change QwtText(QString(“”).sprintf(“%.1f”,secs)) to QwtText(QString(“”).sprintf(“%e”,secs)) Additional tasks: Problem: In gr\_spectrogram\_plot, the spectrogram is hardcoded to 200 FFTs. Make this a configurable parameter. Problem: In gr\_spectrogram\_plot, the timescale does not update properly with the file seek widget. * * * Message: 12 Date: Tue, 21 Oct 2014 16:00:15 -0400 (EDT) From: “Garver, Paul W” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Subject: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=utf-8 I have some large spectrum records and I would like to use GR for analysis in a a sort of “off-line” mode. I want to visualize the data using the spectrogram and then identify subsets of interesting features (e.g. WiFi Packets in 2.4 GHz) by frequency and time coordinates. Essentially, I want to know what index in the file certain features correspond with so I can save them as separate files. To this end, I’ve found gr\_spectrogram\_plot very handy, but it doesn’t quite do what I want because the y-axis which displays time isn’t showing up correctly. My y-axis is always 0.0, and the cursor highlight also reads 0.0 (although the frequency is correct). However, the qt spectrogram appears to have correct time information when I make a simple file source → qt spectrogram sink. I’d like to fix this issue so I’ve been pouring over the code in gr-qtgui. I don’t have much experience with QWT or Python GUIs in general. From what I can tell, there are methods for frequency axis and intensity scaling, but not time. Is this true or am I missing something in the API? The historyExtent parameter in the WaterfallData constructor appears to get hardcoded to 200 in WaterfallDisplayPlot.cc (gr-qtgui/lib). It would also be nice to toggle between the time and sample number. I’m happy to do the work to implement this. Could someone point me in the right direction here? Thanks! PWG ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [November 4, 2014, 12:18am UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/3 "2014-11-04T00:18:13Z")
On Wed, Oct 29, 2014 at 1:23 PM, Garver, Paul W [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Thanks for the feedback. Comments below. > result is that timePerFFT is always zero, resulting in an improperly set > timescale. The proper thing to do here is to set timePerFFT based on the > FFT size and sample rate in off-line mode. Obviously, this assumes you know > the sample rate, but since I’ve modified gr\_spectrogram\_plot to read files > with headers and autofill stuff (sample rate, frequency, data type, etc) > appropriately, it always does. It appears that you won’t be necessarily > processing every sample if you don’t set d\_update\_time=0, which you want to > when running off-line. That sounds fair. The update time is set to zero so it plots immediately. As for the autofill, are you using the file metadata format for this or your own? I’ve been wanting to make versions of these programs that read this data from our metadata files for just this reason. And if nothing is provided for the sample (the -r sets this on the command line) it defaults to 1.0, which should still operate well with what you are suggesting here, possibly with just meaningless numbers. > Problem:In gr\_spectrogram\_plot, the cursor pointer only gives readings to > a 100th of a second. For analysis, it would be more useful to have > scientific notation which scales with higher sample rates. > > Solution: In WaterfallDisplayPlot.cc, change .arg(secs,0,‘f’,2) to > .arg(secs,0,‘e’,2) I’d like to see what that really looks like, but I’m not opposed to it. > Problem: In gr\_spectrogram\_plot, the time scale is in in hundreds of a > second. It may be more useful to have the plot in scientific notation. > > Solution: In WaterfallDisplayPlot.cc, change > QwtText(QString(“”).sprintf(“%.1f”,secs)) to > QwtText(QString(“”).sprintf(“%e”,secs)) Same again. > Additional tasks: > > Problem: In gr\_spectrogram\_plot, the spectrogram is hardcoded to 200 FFTs. > Make this a configurable parameter. Yep, that seems wrong. > Problem: In gr\_spectrogram\_plot, the timescale does not update properly > with the file seek widget. Ah, interesting. That might be a bit more difficult fix. Can you put this work together into a git branch, probably on github forked off gnuradio/gnuradio.git so we can see what the commits would look like? I think these off-line analysis tools are useful, but they definitely need work. Glad someone’s interested in it! Tom ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [November 10, 2014, 5:48pm UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/4 "2014-11-10T17:48:06Z")
On Wed, Nov 5, 2014 at 10:24 PM, Garver, Paul W [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Does that mean you’ve changed the format of the header? These things are really only useful if we’re all using the same structure. The “extras” part of the header is designed to hold stuff like that, too, or is that what you meant? As for the extra parameters, you have samples/second, so why would you need seconds/sample? And with the samples/second and item count in the headers, you can calculate the length in seconds easily. > interesting features in the spectrogram and then be able to cut out a > section of a much larger file for analysis. > > PWG It doesn’t get the time info from a time tag, just from the system’s timer itself using the gr::high\_res\_timer\_type. So we’d have to pass this information through to the plotting widget. Tom ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [November 6, 2014, 4:25am UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/5 "2014-11-06T04:25:24Z")
Hey Tom, I’ll pull together the git branch and let you know when it’s ready. As for gr\_spectrogram\_plot, we do use the standard GR header format, with some “extras” such as center frequency. I think it would be useful in the metadata format to have center frequency, seconds/sample, and file length in seconds for the next metadata version. I should also mention I have some simple scripts to make headers for header-less data with command line args. I’m working on expanding them so they can do data type conversion and modification of headers as well. The time axis scaling is next on my list to do. Any suggestions on where to begin? I was thinking of adding some functions to the spectrogram plot to do this. Then in the plot reset function, manually set the scale properly. There are probably more elegant ways to do this, but I don’t want to break existing functionality with the on-line mode. I think the Y-Axis zero point is based on the timestamp, which makes total sense in “on-line” mode. I suppose we could keep that convention and generate fake timestamps for off-line mode. That would also work well when we have real timestamps in the headers. I also think it would be good to have a key toggle to switch between time and sample number. I want to identify interesting features in the spectrogram and then be able to cut out a section of a much larger file for analysis. PWG ----- Original Message ----- From: “Tom R.” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: “Paul W Garver” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Cc: “GNURadio D.ion List” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Sent: Monday, November 3, 2014 6:16:38 PM Subject: Re: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis On Wed, Oct 29, 2014 at 1:23 PM, Garver, Paul W \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> wrote: I’ve solved this and wanted to share my solutions, as well as improvements (to me, anyways) to GR’s off-line plotting tools (gr\_spectrogram\_plot). My overall goal here is to be able to do some analysis on wideband (25MSPS) data using GR since MATLAB isn’t really efficient for that. I’ve seen some interesting updates regarding gr-qtgui on the git commit log – I’d be very interested in comments from those folks so I know what’s in the pipeline and perhaps what you would want to pull into the baseline. Below is a list of issues I’ve encountered and what I’ve done to fix them. Thanks for the feedback. Comments below. > Problem:In gr\_spectrogram\_plot\_c, the time scale always reads 0. > > Solution: In waterfalldisplayform.cc, d\_update\_time is passed as the > timePerFFT argument to plotNewData (WaterfallDisplayPlot object). This > happens in WaterfallDisplayForm::newData(). However, gr\_spectrogram\_plot > sets this to zero by calling set\_update\_time(0) on the > waterfall\_sink\_c\_impl object. In waterfall\_sink\_c\_impl.cc it calls the > postEvent() function with d\_last\_time, which is a variable representing > the time between waterfall updates. Ultimately, this is regulated by > d\_update\_time, which is set to zero by the set\_update\_time(0) call. The > net result is that timePerFFT is always zero, resulting in an improperly > set timescale. The proper thing to do here is to set timePerFFT based on > the FFT size and sample rate in off-line mode. Obviously, this assumes > you know the sample rate, but since I’ve modified gr\_spectrogram\_plot to > read files with headers and autofill stuff (sample rate, frequency, data > type, etc) appropriately, it always does. It appears that you won’t be > necessarily processing every sample if you don’t set d\_update\_time=0, > which you want to when running off-line. That sounds fair. The update time is set to zero so it plots immediately. As for the autofill, are you using the file metadata format for this or your own? I’ve been wanting to make versions of these programs that read this data from our metadata files for just this reason. And if nothing is provided for the sample (the -r sets this on the command line) it defaults to 1.0, which should still operate well with what you are suggesting here, possibly with just meaningless numbers. > Problem:In gr\_spectrogram\_plot, the cursor pointer only gives readings to a 100th of a second. For analysis, it would be more useful to have scientific notation which scales with higher sample rates. > > Solution: In WaterfallDisplayPlot.cc, change .arg(secs,0,‘f’,2) to > .arg(secs,0,‘e’,2) I’d like to see what that really looks like, but I’m not opposed to it. > Problem: In gr\_spectrogram\_plot, the time scale is in in hundreds of a second. It may be more useful to have the plot in scientific notation. > > Solution: In WaterfallDisplayPlot.cc, change > QwtText(QString(“”).sprintf(“%.1f”,secs)) to > QwtText(QString(“”).sprintf(“%e”,secs)) Same again. > Additional tasks: > > Problem: In gr\_spectrogram\_plot, the spectrogram is hardcoded to 200 > FFTs. Make this a configurable parameter. Yep, that seems wrong. > Problem: In gr\_spectrogram\_plot, the timescale does not update properly with the file seek widget. Ah, interesting. That might be a bit more difficult fix. Can you put this work together into a git branch, probably on github forked off gnuradio/gnuradio.git so we can see what the commits would look like? I think these off-line analysis tools are useful, but they definitely need work. Glad someone’s interested in it! Tom > ------------------------------ > > Message: 12 > Date: Tue, 21 Oct 2014 16:00:15 -0400 (EDT) > From: “Garver, Paul W” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > Subject: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis > Message-ID: > \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > Content-Type: text/plain; charset=utf-8 > > I have some large spectrum records and I would like to use GR for > analysis in a a sort of “off-line” mode. I want to visualize the data > using the spectrogram and then identify subsets of interesting features > (e.g. WiFi Packets in 2.4 GHz) by frequency and time coordinates. > Essentially, I want to know what index in the file certain features > correspond with so I can save them as separate files. To this end, I’ve > found gr\_spectrogram\_plot very handy, but it doesn’t quite do what I > want because the y-axis which displays time isn’t showing up correctly. > My y-axis is always 0.0, and the cursor highlight also reads 0.0 > (although the frequency is correct). However, the qt spectrogram appears > to have correct time information when I make a simple file source → qt > spectrogram sink. > > I’d like to fix this issue so I’ve been pouring over the code in > gr-qtgui. I don’t have much experience with QWT or Python GUIs in > general. From what I can tell, there are methods for frequency axis and > intensity scaling, but not time. Is this true or am I missing something > in the API? The historyExtent parameter in the WaterfallData constructor > appears to get hardcoded to 200 in WaterfallDisplayPlot.cc > (gr-qtgui/lib). It would also be nice to toggle between the time and > sample number. > > I’m happy to do the work to implement this. Could someone point me in > the right direction here? Thanks! > > PWG > > * * * > > Discuss-gnuradio mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > > **[Discuss-gnuradio Info Page](https://lists.gnu.org/mailman/listinfo/discuss-gnuradio)** ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [November 10, 2014, 6:44pm UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/6 "2014-11-10T18:44:12Z")
Sorry my response was really confusing after re-reading it – let me try again. We use the standard metadata headers and simply store the center frequency in the ‘extras’ part . The metadata format has not been changed. I am not suggesting that we add redundant information to the header but rather have gr\_read\_file\_metadata calculate these things for us (based on the header) and display them. Of course, they are easily calculated by hand but it does simplify the analysis workflow. To that point, perhaps we should define combinations of item types and real/complex. For example, rather than displaying Complex T/F and item type float, just say Data Type: Complex Floats (CF) and have a table clearly identifying the types, sizes, enum values, etc in the documentation. I think a few refinements like this (and enhancing the plotting tools) could really help make GR a powerful analysis platform. PWG ----- Original Message ----- From: “Tom R.” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: “Paul W Garver” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Cc: “GNURadio D.ion List” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Sent: Monday, November 10, 2014 11:46:18 AM Subject: Re: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis On Wed, Nov 5, 2014 at 10:24 PM, Garver, Paul W \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> wrote: Hey Tom, I’ll pull together the git branch and let you know when it’s ready. As for gr\_spectrogram\_plot, we do use the standard GR header format, with some “extras” such as center frequency. I think it would be useful in the metadata format to have center frequency, seconds/sample, and file length in seconds for the next metadata version. I should also mention I have some simple scripts to make headers for header-less data with command line args. I’m working on expanding them so they can do data type conversion and modification of headers as well. Does that mean you’ve changed the format of the header? These things are really only useful if we’re all using the same structure. The “extras” part of the header is designed to hold stuff like that, too, or is that what you meant? As for the extra parameters, you have samples/second, so why would you need seconds/sample? And with the samples/second and item count in the headers, you can calculate the length in seconds easily. > The time axis scaling is next on my list to do. Any suggestions on where > to begin? I was thinking of adding some functions to the spectrogram > plot to do this. Then in the plot reset function, manually set the scale > properly. There are probably more elegant ways to do this, but I don’t > want to break existing functionality with the on-line mode. I think the > Y-Axis zero point is based on the timestamp, which makes total sense in > “on-line” mode. I suppose we could keep that convention and generate > fake timestamps for off-line mode. That would also work well when we > have real timestamps in the headers. I also think it would be good to > have a key toggle to switch between time and sample number. I want to > identify interesting features in the spectrogram and then be able to cut > out a section of a much larger file for analysis. > > PWG It doesn’t get the time info from a time tag, just from the system’s timer itself using the gr::high\_res\_timer\_type. So we’d have to pass this information through to the plotting widget. Tom > From: “Tom R.” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > To: “Paul W Garver” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > Cc: “GNURadio D.ion List” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > Sent: Monday, November 3, 2014 6:16:38 PM > Subject: Re: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis > > On Wed, Oct 29, 2014 at 1:23 PM, Garver, Paul W \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > wrote: > > > I've solved this and wanted to share my solutions, as well as improvements (to me, anyways) to GR's off-line plotting tools (gr\_spectrogram\_plot). My overall goal here is to be able to do some analysis on wideband (25MSPS) data using GR since MATLAB isn't really efficient for that. > > > > I’ve seen some interesting updates regarding gr-qtgui on the git commit > > log – I’d be very interested in comments from those folks so I know > > what’s in the pipeline and perhaps what you would want to pull into the > > baseline. > > > > Below is a list of issues I’ve encountered and what I’ve done to fix > > them. > > Thanks for the feedback. Comments below. > > > Problem:In gr\_spectrogram\_plot\_c, the time scale always reads 0. > > > > Solution: In waterfalldisplayform.cc, d\_update\_time is passed as the > > timePerFFT argument to plotNewData (WaterfallDisplayPlot object). This > > happens in WaterfallDisplayForm::newData(). However, gr\_spectrogram\_plot > > sets this to zero by calling set\_update\_time(0) on the > > waterfall\_sink\_c\_impl object. In waterfall\_sink\_c\_impl.cc it calls the > > postEvent() function with d\_last\_time, which is a variable representing > > the time between waterfall updates. Ultimately, this is regulated by > > d\_update\_time, which is set to zero by the set\_update\_time(0) call. The > > net result is that timePerFFT is always zero, resulting in an improperly > > set timescale. The proper thing to do here is to set timePerFFT based on > > the FFT size and sample rate in off-line mode. Obviously, this assumes > > you know the sample rate, but since I’ve modified gr\_spectrogram\_plot to > > read files with headers and autofill stuff (sample rate, frequency, data > > type, etc) appropriately, it always does. It appears that you won’t be > > necessarily processing every sample if you don’t set d\_update\_time=0, > > which you want to when running off-line. > > That sounds fair. The update time is set to zero so it plots > immediately. As for the autofill, are you using the file metadata format > for this or your own? I’ve been wanting to make versions of these > programs that read this data from our metadata files for just this > reason. And if nothing is provided for the sample (the -r sets this on > the command line) it defaults to 1.0, which should still operate well > with what you are suggesting here, possibly with just meaningless > numbers. > > > Problem:In gr\_spectrogram\_plot, the cursor pointer only gives readings to a 100th of a second. For analysis, it would be more useful to have scientific notation which scales with higher sample rates. > > > > Solution: In WaterfallDisplayPlot.cc, change .arg(secs,0,‘f’,2) to > > .arg(secs,0,‘e’,2) > > I’d like to see what that really looks like, but I’m not opposed to it. > > > Problem: In gr\_spectrogram\_plot, the time scale is in in hundreds of a second. It may be more useful to have the plot in scientific notation. > > > > Solution: In WaterfallDisplayPlot.cc, change > > QwtText(QString(“”).sprintf(“%.1f”,secs)) to > > QwtText(QString(“”).sprintf(“%e”,secs)) > > Same again. > > > Additional tasks: > > > > Problem: In gr\_spectrogram\_plot, the spectrogram is hardcoded to 200 > > FFTs. Make this a configurable parameter. > > Yep, that seems wrong. > > > Problem: In gr\_spectrogram\_plot, the timescale does not update properly with the file seek widget. > > Ah, interesting. That might be a bit more difficult fix. > > Can you put this work together into a git branch, probably on github > forked off gnuradio/gnuradio.git so we can see what the commits would > look like? I think these off-line analysis tools are useful, but they > definitely need work. Glad someone’s interested in it! > > Tom > > > ------------------------------ > > > > Message: 12 > > Date: Tue, 21 Oct 2014 16:00:15 -0400 (EDT) > > From: “Garver, Paul W” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > Subject: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis > > Message-ID: > > \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > Content-Type: text/plain; charset=utf-8 > > > > I have some large spectrum records and I would like to use GR for > > analysis in a a sort of “off-line” mode. I want to visualize the data > > using the spectrogram and then identify subsets of interesting features > > (e.g. WiFi Packets in 2.4 GHz) by frequency and time coordinates. > > Essentially, I want to know what index in the file certain features > > correspond with so I can save them as separate files. To this end, I’ve > > found gr\_spectrogram\_plot very handy, but it doesn’t quite do what I > > want because the y-axis which displays time isn’t showing up correctly. > > My y-axis is always 0.0, and the cursor highlight also reads 0.0 > > (although the frequency is correct). However, the qt spectrogram appears > > to have correct time information when I make a simple file source → qt > > spectrogram sink. > > > > I’d like to fix this issue so I’ve been pouring over the code in > > gr-qtgui. I don’t have much experience with QWT or Python GUIs in > > general. From what I can tell, there are methods for frequency axis and > > intensity scaling, but not time. Is this true or am I missing something > > in the API? The historyExtent parameter in the WaterfallData constructor > > appears to get hardcoded to 200 in WaterfallDisplayPlot.cc > > (gr-qtgui/lib). It would also be nice to toggle between the time and > > sample number. > > > > I’m happy to do the work to implement this. Could someone point me in > > the right direction here? Thanks! > > > > PWG > > > > * * * > > > > Discuss-gnuradio mailing list > > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > > > > **[Discuss-gnuradio Info Page](https://lists.gnu.org/mailman/listinfo/discuss-gnuradio)** ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [November 10, 2014, 7:00pm UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/7 "2014-11-10T19:00:18Z")
On Mon, Nov 10, 2014 at 12:43 PM, Garver, Paul W [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Sorry my response was really confusing after re-reading it – let me try > again. We use the standard metadata headers and simply store the center > frequency in the ‘extras’ part . The metadata format has not been changed. Gotcha, thanks for clearing that up. > PWG Ah, ok. Misunderstood that. The gr\_read\_file\_metadata spits out the header information for every segment, since the metadata files are segmented by default (for better reconstruction if a file error occurs and to make reading out smaller bits of large files easier). But I see your point, and I think it might be useful to have this tool keep track of all items throughout the file and have a summary dump at the end of the print out with the total number of items read in and possibly even the total length of time of the file since each segment might have a different sample rate. Also info like the total number of data segments in the file, etc. Good suggestions on the data type combining (right now, it’s just translating from what’s in the header to a print out), and the documentation. Can you roll these feature requests up into an Issue on [gnuradio.org](http://gnuradio.org)? (or better yet, send us a pull request on github…) Thanks, Tom ---
**Author:** ![GarverS\_Paul\_W](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/e19adc/32.png) [@GarverS\_Paul\_W](https://www.ruby-forum.com/u/GarverS_Paul_W)\ **Post date:** [November 12, 2014, 2:20pm UTC](https://www.ruby-forum.com/t/gr-qtgui-waterfall-time-axis/232674/8 "2014-11-12T14:20:39Z")
Hey Tom, I did a git pull request for my “olplottweaks” branch. Here’s the link for anyone else on the list who is interested: [https://github.com/garverp/gnuradio/tree/olplottweaks](https://github.com/garverp/gnuradio/tree/olplottweaks) Let me know if you like the direction this is going. I pulled numffts through the constructors of the spectrogram because it seems like a constructor type parameter to me. I wasn’t planning on supporting a change in the spectrogram length via widget. I can add some more stuff, particularly: - Have gr\_spectrogram\_plot respect -r sample rate flag (Currently can not tell if default of 1 or it simply wasn’t passed from optparse). - Add a box/key to toggle between sample number (e.g. rate=1) and time on the y-axis in gr\_spectrogram\_plot - Update the y-axis time with proper time/sample indices when using the file slider - Update y-axis time on spectrogram plot appropriately when fft size or sample rate changes. Is there currently any way to give the spectrogram an overlap to reduce the noise variance (Welch’s method)? I see there is an averaging setting, but the overlap would give us better plots too. On another note, some of the other features such as a more succinct gr\_read\_file\_metadata ouput are currently being worked on so I hope to be able to push those upstream once we have a more finished product. PWG ----- Original Message ----- From: “Tom R.” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: “Paul W Garver” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Cc: “GNURadio D.ion List” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Sent: Monday, November 10, 2014 12:52:39 PM Subject: Re: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis On Mon, Nov 10, 2014 at 12:43 PM, Garver, Paul W \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> wrote: Sorry my response was really confusing after re-reading it – let me try again. We use the standard metadata headers and simply store the center frequency in the ‘extras’ part . The metadata format has not been changed. Gotcha, thanks for clearing that up. > I am not suggesting that we add redundant information to the header but > rather have gr\_read\_file\_metadata calculate these things for us (based > on the header) and display them. Of course, they are easily calculated > by hand but it does simplify the analysis workflow. To that point, > perhaps we should define combinations of item types and real/complex. > For example, rather than displaying Complex T/F and item type float, > just say Data Type: Complex Floats (CF) and have a table clearly > identifying the types, sizes, enum values, etc in the documentation. I > think a few refinements like this (and enhancing the plotting tools) > could really help make GR a powerful analysis platform. > > PWG Ah, ok. Misunderstood that. The gr\_read\_file\_metadata spits out the header information for every segment, since the metadata files are segmented by default (for better reconstruction if a file error occurs and to make reading out smaller bits of large files easier). But I see your point, and I think it might be useful to have this tool keep track of all items throughout the file and have a summary dump at the end of the print out with the total number of items read in and possibly even the total length of time of the file since each segment might have a different sample rate. Also info like the total number of data segments in the file, etc. Good suggestions on the data type combining (right now, it’s just translating from what’s in the header to a print out), and the documentation. Can you roll these feature requests up into an Issue on [gnuradio.org](http://gnuradio.org) ? (or better yet, send us a pull request on github…) Thanks, Tom > From: “Tom R.” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > To: “Paul W Garver” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > Cc: “GNURadio D.ion List” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > Sent: Monday, November 10, 2014 11:46:18 AM > > Subject: Re: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis > > On Wed, Nov 5, 2014 at 10:24 PM, Garver, Paul W \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > wrote: > > > Hey Tom, > > > > I’ll pull together the git branch and let you know when it’s ready. As > > for gr\_spectrogram\_plot, we do use the standard GR header format, with > > some “extras” such as center frequency. I think it would be useful in > > the metadata format to have center frequency, seconds/sample, and file > > length in seconds for the next metadata version. I should also mention I > > have some simple scripts to make headers for header-less data with > > command line args. I’m working on expanding them so they can do data > > type conversion and modification of headers as well. > > Does that mean you’ve changed the format of the header? These things are > really only useful if we’re all using the same structure. The “extras” > part of the header is designed to hold stuff like that, too, or is that > what you meant? As for the extra parameters, you have samples/second, so > why would you need seconds/sample? And with the samples/second and item > count in the headers, you can calculate the length in seconds easily. > > > The time axis scaling is next on my list to do. Any suggestions on where > > to begin? I was thinking of adding some functions to the spectrogram > > plot to do this. Then in the plot reset function, manually set the scale > > properly. There are probably more elegant ways to do this, but I don’t > > want to break existing functionality with the on-line mode. I think the > > Y-Axis zero point is based on the timestamp, which makes total sense in > > “on-line” mode. I suppose we could keep that convention and generate > > fake timestamps for off-line mode. That would also work well when we > > have real timestamps in the headers. I also think it would be good to > > have a key toggle to switch between time and sample number. I want to > > identify interesting features in the spectrogram and then be able to cut > > out a section of a much larger file for analysis. > > > > PWG > > It doesn’t get the time info from a time tag, just from the system’s > timer itself using the gr::high\_res\_timer\_type. So we’d have to pass > this information through to the plotting widget. > > Tom > > > From: “Tom R.” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > To: “Paul W Garver” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > Cc: “GNURadio D.ion List” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > Sent: Monday, November 3, 2014 6:16:38 PM > > Subject: Re: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis > > > > On Wed, Oct 29, 2014 at 1:23 PM, Garver, Paul W \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > wrote: > > > > > I've solved this and wanted to share my solutions, as well as improvements (to me, anyways) to GR's off-line plotting tools (gr\_spectrogram\_plot). My overall goal here is to be able to do some analysis on wideband (25MSPS) data using GR since MATLAB isn't really efficient for that. > > > > > > I’ve seen some interesting updates regarding gr-qtgui on the git commit > > > log – I’d be very interested in comments from those folks so I know > > > what’s in the pipeline and perhaps what you would want to pull into the > > > baseline. > > > > > > Below is a list of issues I’ve encountered and what I’ve done to fix > > > them. > > > > Thanks for the feedback. Comments below. > > > > > Problem:In gr\_spectrogram\_plot\_c, the time scale always reads 0. > > > > > > Solution: In waterfalldisplayform.cc, d\_update\_time is passed as the > > > timePerFFT argument to plotNewData (WaterfallDisplayPlot object). This > > > happens in WaterfallDisplayForm::newData(). However, gr\_spectrogram\_plot > > > sets this to zero by calling set\_update\_time(0) on the > > > waterfall\_sink\_c\_impl object. In waterfall\_sink\_c\_impl.cc it calls the > > > postEvent() function with d\_last\_time, which is a variable representing > > > the time between waterfall updates. Ultimately, this is regulated by > > > d\_update\_time, which is set to zero by the set\_update\_time(0) call. The > > > net result is that timePerFFT is always zero, resulting in an improperly > > > set timescale. The proper thing to do here is to set timePerFFT based on > > > the FFT size and sample rate in off-line mode. Obviously, this assumes > > > you know the sample rate, but since I’ve modified gr\_spectrogram\_plot to > > > read files with headers and autofill stuff (sample rate, frequency, data > > > type, etc) appropriately, it always does. It appears that you won’t be > > > necessarily processing every sample if you don’t set d\_update\_time=0, > > > which you want to when running off-line. > > > > That sounds fair. The update time is set to zero so it plots > > immediately. As for the autofill, are you using the file metadata format > > for this or your own? I’ve been wanting to make versions of these > > programs that read this data from our metadata files for just this > > reason. And if nothing is provided for the sample (the -r sets this on > > the command line) it defaults to 1.0, which should still operate well > > with what you are suggesting here, possibly with just meaningless > > numbers. > > > > > Problem:In gr\_spectrogram\_plot, the cursor pointer only gives readings to a 100th of a second. For analysis, it would be more useful to have scientific notation which scales with higher sample rates. > > > > > > Solution: In WaterfallDisplayPlot.cc, change .arg(secs,0,‘f’,2) to > > > .arg(secs,0,‘e’,2) > > > > I’d like to see what that really looks like, but I’m not opposed to it. > > > > > Problem: In gr\_spectrogram\_plot, the time scale is in in hundreds of a second. It may be more useful to have the plot in scientific notation. > > > > > > Solution: In WaterfallDisplayPlot.cc, change > > > QwtText(QString(“”).sprintf(“%.1f”,secs)) to > > > QwtText(QString(“”).sprintf(“%e”,secs)) > > > > Same again. > > > > > Additional tasks: > > > > > > Problem: In gr\_spectrogram\_plot, the spectrogram is hardcoded to 200 > > > FFTs. Make this a configurable parameter. > > > > Yep, that seems wrong. > > > > > Problem: In gr\_spectrogram\_plot, the timescale does not update properly with the file seek widget. > > > > Ah, interesting. That might be a bit more difficult fix. > > > > Can you put this work together into a git branch, probably on github > > forked off gnuradio/gnuradio.git so we can see what the commits would > > look like? I think these off-line analysis tools are useful, but they > > definitely need work. Glad someone’s interested in it! > > > > Tom > > > > > ------------------------------ > > > > > > Message: 12 > > > Date: Tue, 21 Oct 2014 16:00:15 -0400 (EDT) > > > From: “Garver, Paul W” \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > > To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > > Subject: [Discuss-gnuradio] gr-qtgui Waterfall Time Axis > > > Message-ID: > > > \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> > > > Content-Type: text/plain; charset=utf-8 > > > > > > I have some large spectrum records and I would like to use GR for > > > analysis in a a sort of “off-line” mode. I want to visualize the data > > > using the spectrogram and then identify subsets of interesting features > > > (e.g. WiFi Packets in 2.4 GHz) by frequency and time coordinates. > > > Essentially, I want to know what index in the file certain features > > > correspond with so I can save them as separate files. To this end, I’ve > > > found gr\_spectrogram\_plot very handy, but it doesn’t quite do what I > > > want because the y-axis which displays time isn’t showing up correctly. > > > My y-axis is always 0.0, and the cursor highlight also reads 0.0 > > > (although the frequency is correct). However, the qt spectrogram appears > > > to have correct time information when I make a simple file source → qt > > > spectrogram sink. > > > > > > I’d like to fix this issue so I’ve been pouring over the code in > > > gr-qtgui. I don’t have much experience with QWT or Python GUIs in > > > general. From what I can tell, there are methods for frequency axis and > > > intensity scaling, but not time. Is this true or am I missing something > > > in the API? The historyExtent parameter in the WaterfallData constructor > > > appears to get hardcoded to 200 in WaterfallDisplayPlot.cc > > > (gr-qtgui/lib). It would also be nice to toggle between the time and > > > sample number. > > > > > > I’m happy to do the work to implement this. Could someone point me in > > > the right direction here? Thanks! > > > > > > PWG > > > > > > * * * > > > > > > Discuss-gnuradio mailing list > > > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > > > > > > **[Discuss-gnuradio Info Page](https://lists.gnu.org/mailman/listinfo/discuss-gnuradio)**