# GNU Radio Measurement Toolbox -- GSoC '14 Project **URL:** \ **Category:** GNU Radio\ **Created:** [May 22, 2014, 6:48pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303 "2014-05-22T18:48:09Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Detlef\_R](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/58f4c7/32.png) [@Detlef\_R](https://www.ruby-forum.com/u/Detlef_R)\ **Post date:** [May 22, 2014, 6:48pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303/1 "2014-05-22T18:48:09Z")
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, GR community! I was elected to do a Google Summer of Code project. Thanks for all the constructive feedback on my proposal, and all the support, ideas and hints I got the last weeks! As I just finished setting up my blog, I’m now happy to announce the beginning of the GNU Radio Measurement Toolbox. Its purposes are basically two-fold: 1. Ease the process of gathering data through changing flowgraph characteristics to get things like BER curves out of GNU Radio, and 2. Help optimizing GNU Radio and VOLK by offering the same automated data gathering for performance data. This covers generating a few measurement blocks, writing a framework to let developers run their flowgraphs through a pre-defined set of parametrizations, evaluating performance counters, dealing with the gathered data, visualization and automated task distribution. To not bore you to death here on the mailing list, I’ve made an introductory blog entry [1]. You may find it on [http://gsoc.hostalia.de](http://gsoc.hostalia.de) Then, there will be code; you will find that on my github page [2]. Looking forward to a load of criticism, and even more fun hacking, Greetings, Marcus [1] [http://gsoc.hostalia.de/posts/a-measurement-toolbox-for-gnu-radio-my-google-summer-of-code-project.html](http://gsoc.hostalia.de/posts/a-measurement-toolbox-for-gnu-radio-my-google-summer-of-code-project.html) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - [http://www.enigmail.net/](http://www.enigmail.net/) iQEcBAEBAgAGBQJTfioEAAoJEBQ6EdjyzlHtUSUH/3z2Ns/tZ1zpCJsOu6zXuhrw a2s5bdZFVoK7h9AwqD016iGziZX6/EPYouOL8J+zUY0PWfKmow8eaGMv+eypVwZy MmS2gvDhY4oedWdMChUl5rBA1iERmwnQmjyy06o1PQOp1rLzgImDrgXv69H/H0Xa ZDZV9fth8TERN79Tnb7n97S0IsNJFXk87R6ZpqkZ9Hd0q+3AqaeOXfyDuhcEF2BA ksvYNKb9BsdqJydXNhPVZdSITYyRUI8HUFE6PrYKRIWEtxvvg2Dqmi/2fDgLIK/J Xn3o+8Dnc77Vmr4bcDlK+DL32Gm1BFY8+Z2WeQjqIBJtY5VywEQ1Ow1JoR90eP8= =/xsI -----END PGP SIGNATURE----- ---
**Author:** ![Marcus\_MS\_ller](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/5daacb/32.png) [@Marcus\_MS\_ller](https://www.ruby-forum.com/u/Marcus_MS_ller)\ **Post date:** [May 25, 2014, 7:36pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303/2 "2014-05-25T19:36:30Z")
Hi Marcus, I like the approach you take by looking at what real life users will want from gnuradio when transitioning from an academic perspective to realtime system. Measurements are always not enough to understand the specifics of your system so I’m looking forward to see how your project provides measurements to the gnuradio user. Building on block computing performance measurements there is one thing I would like to see in gnuradio and that is a flowgraph optimizer. To be more specific, a flowgraph optimizer would try to adapt the parameters of the blocks (e.g. the data chunk passed to each block) in order to optimize one/more parameter(s) of a flowgraph (e.g. overall processing time). In a normal way this optimizer should be run just once to determine the optimum parameters that will be used subsequently. If we see the problem to solve from a general perspective the optimizer would fall in the category of multi-objective optimization which has a numerous solutions and has been thoroughly discussed in the academia and industry (gaming is usualy doing multi-bjective optimization through AI). Another real-life example would be the optimizer in the 4Nec2 antenna simulation program that uses AI to optimize the antenna when a set of objectives (variables) is set by the user, e.g. minimum SWR, Z close to a value, etc. In my opinion gnuradio will really benefit from such an optimizer as the values of block parameters can provide quite different end results. Not sure if this can be part of your GSOC project but I thought it worth mentioning to you and gnuradio users on this list. Maybe can be part of the next GSOC. Thanks, Bogdan On Thursday, May 22, 2014 7:49 PM, Marcus M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, GR community! I was elected to do a Google Summer of Code project. Thanks for all the constructive feedback on my proposal, and all the support, ideas and hints I got the last weeks! As I just finished setting up my blog, I’m now happy to announce the beginning of the GNU Radio Measurement Toolbox. Its purposes are basically two-fold: 1. Ease the process of gathering data through changing flowgraph characteristics to get things like BER curves out of GNU Radio, and 2. Help optimizing GNU Radio and VOLK by offering the same automated data gathering for performance data. This covers generating a few measurement blocks, writing a framework to let developers run their flowgraphs through a pre-defined set of parametrizations, evaluating performance counters, dealing with the gathered data, visualization and automated task distribution. To not bore you to death here on the mailing list, I’ve made an introductory blog entry [1]. You may find it on [http://gsoc.hostalia.de](http://gsoc.hostalia.de) Then, there will be code; you will find that on my github page [2]. Looking forward to a load of criticism, and even more fun hacking, Greetings, Marcus [1] [http://gsoc.hostalia.de/posts/a-measurement-toolbox-for-gnu-radio-my-google-summer-of-code-project.html](http://gsoc.hostalia.de/posts/a-measurement-toolbox-for-gnu-radio-my-google-summer-of-code-project.html) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - [http://www.enigmail.net/](http://www.enigmail.net/) iQEcBAEBAgAGBQJTfioEAAoJEBQ6EdjyzlHtUSUH/3z2Ns/tZ1zpCJsOu6zXuhrw a2s5bdZFVoK7h9AwqD016iGziZX6/EPYouOL8J+zUY0PWfKmow8eaGMv+eypVwZy MmS2gvDhY4oedWdMChUl5rBA1iERmwnQmjyy06o1PQOp1rLzgImDrgXv69H/H0Xa ZDZV9fth8TERN79Tnb7n97S0IsNJFXk87R6ZpqkZ9Hd0q+3AqaeOXfyDuhcEF2BA ksvYNKb9BsdqJydXNhPVZdSITYyRUI8HUFE6PrYKRIWEtxvvg2Dqmi/2fDgLIK/J Xn3o+8Dnc77Vmr4bcDlK+DL32Gm1BFY8+Z2WeQjqIBJtY5VywEQ1Ow1JoR90eP8= =/xsI -----END PGP SIGNATURE----- ---
**Author:** ![Marcus\_MS\_ller](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/5daacb/32.png) [@Marcus\_MS\_ller](https://www.ruby-forum.com/u/Marcus_MS_ller)\ **Post date:** [May 25, 2014, 8:52pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303/3 "2014-05-25T20:52:45Z")
Hi Marcus, I like the approach you take by looking at what real life users will want from gnuradio when transitioning from an academic perspective to realtime system. Measurements are always not enough to understand the specifics of your system so I’m looking forward to see how your project provides measurements to the gnuradio user. Building on block computing performance measurements there is one thing I would like to see in gnuradio and that is a flowgraph optimizer. To be more specific, a flowgraph optimizer would try to adapt the parameters of the blocks (e.g. the data chunk passed to each block) in order to optimize one/more parameter(s) of a flowgraph (e.g. overall processing time). In a normal way this optimizer should be run just once to determine the optimum parameters that will be used subsequently. If we see the problem to solve from a general perspective the optimizer would fall in the category of multi-objective optimization which has a numerous solutions and has been thoroughly discussed in the academia and industry (gaming is usualy doing multi-bjective optimization through AI). Another real-life example would be the optimizer in the 4Nec2 antenna simulation program that uses AI to optimize the antenna when a set of objectives (variables) is set by the user, e.g. minimum SWR, Z close to a value, etc. In my opinion gnuradio will really benefit from such an optimizer as the values of block parameters can provide quite different end results. Not sure if this can be part of your GSOC project but I thought it worth mentioning to you and gnuradio users on this list. Maybe can be part of the next GSOC. Thanks,Bogdan On Thursday, May 22, 2014 7:49 PM, Marcus M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, GR community! I was elected to do a Google Summer of Code project. Thanks for all the constructive feedback on my proposal, and all the support, ideas and hints I got the last weeks! As I just finished setting up my blog, I’m now happy to announce the beginning of the GNU Radio Measurement Toolbox. Its purposes are basically two-fold: 1. Ease the process of gathering data through changing flowgraph characteristics to get things like BER curves out of GNU Radio, and 2. Help optimizing GNU Radio and VOLK by offering the same automated data gathering for performance data. This covers generating a few measurement blocks, writing a framework to let developers run their flowgraphs through a pre-defined set of parametrizations, evaluating performance counters, dealing with the gathered data, visualization and automated task distribution. To not bore you to death here on the mailing list, I’ve made an introductory blog entry [1]. You may find it on [http://gsoc.hostalia.de](http://gsoc.hostalia.de) Then, there will be code; you will find that on my github page [2]. Looking forward to a load of criticism, and even more fun hacking, Greetings, Marcus [1] [http://gsoc.hostalia.de/posts/a-measurement-toolbox-for-gnu-radio-my-google-summer-of-code-project.html](http://gsoc.hostalia.de/posts/a-measurement-toolbox-for-gnu-radio-my-google-summer-of-code-project.html) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - [http://www.enigmail.net/](http://www.enigmail.net/) iQEcBAEBAgAGBQJTfioEAAoJEBQ6EdjyzlHtUSUH/3z2Ns/tZ1zpCJsOu6zXuhrw a2s5bdZFVoK7h9AwqD016iGziZX6/EPYouOL8J+zUY0PWfKmow8eaGMv+eypVwZy MmS2gvDhY4oedWdMChUl5rBA1iERmwnQmjyy06o1PQOp1rLzgImDrgXv69H/H0Xa ZDZV9fth8TERN79Tnb7n97S0IsNJFXk87R6ZpqkZ9Hd0q+3AqaeOXfyDuhcEF2BA ksvYNKb9BsdqJydXNhPVZdSITYyRUI8HUFE6PrYKRIWEtxvvg2Dqmi/2fDgLIK/J Xn3o+8Dnc77Vmr4bcDlK+DL32Gm1BFY8+Z2WeQjqIBJtY5VywEQ1Ow1JoR90eP8= =/xsI -----END PGP SIGNATURE----- ---
**Author:** ![Marcus\_MS\_ller](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/5daacb/32.png) [@Marcus\_MS\_ller](https://www.ruby-forum.com/u/Marcus_MS_ller)\ **Post date:** [May 25, 2014, 10:47pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303/4 "2014-05-25T22:47:54Z")
Hi Marcus, take your time 🙂 this is not a simple problem to solve and hence it does not have a simple solution. I like the idea of starting with changing the scheduler behaviour first to experiment with. Indeed it will take a lot of measurements, talks, experiments (not exactly in this order) to understand what is to be optimized. But this talk is touching the gnuradio scheduler area which is an area of its own and it brings me to one question: what happen with GRAS? Thanks, Bogdan On Sunday, May 25, 2014 10:14 PM, Marcus M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Bogdan, thanks for your comment 🙂 Such an optimizer would be really, really fancy. In a way, though, GNU Radio already does this when running a flow graph: It just asks blocks to compute a reasonable amount of items to fill up the downstream buffer. This actually (conceptually simple) approach is one great strength, because it just keeps the computer “busy” as much as possible. There might be space for optimization, though, I agree: Maybe it would be better for some blocks just to wait longer (and thus, not utilize the CPU) if it was computationally beneficial to work on larger chunks, as long as there are enough other blocks competing for CPU power. However, this leads to the problem of balancing average throughput with latency. What the GNU Radio infrastructure does to approach this is actually quite simple: 1. Although it might be “fastest” to process all data at once, buffer lengths set a natural limit to the chunk size, and thus latency. So we have an upper boundary. 2. It is best to be closest possible to that upper boundary. To achieve that, block developers are always encouraged to consume as many input items and produce as much output as possible, even if the overhead of having multiple (general\_)work calls is minute. This ensures that adjacent blocks don’t get asked to produce / consume small item chunks (which will happen if they were in a waiting state and a small number of items was produced or a small amount of output buffer was marked as read). Optimizing this will be hard. Maybe one could profile the same flowgraph with a lot of different settings of per-block maximum output chunk sizes, but I do believe this will only give as little more information than what the block developer already knew when he optimized the block in the first place. If he didn’t optimize, his main interest will be if his block poses a problem at at all; for that, standard settings should be employed. To give developers an API to inform the runtime of item amount preference, different methods exist, however. I’ll give a short rundown of them. 1. Most notable are the `fixed_rate properties` of `gr::block`, as implemented in `sync_block`, and the `decimator` and `interpolator` block types 2. If your block will only produce a multiples of a certain number of items, the `set_output_multiple` is a method that will potentially decrease overhead introduced by pointless calls to `forecast` and/or `work`. 3. In hardware optimization, alignment is often the performance critical factor. To account for that, set\_alignment was introduced. It’s working very similar to `set_output_multiple`, but does not _enforce_ the multiples, but sets an unaligned flag if non-multiple consumption occurred. The runtime will always try to achieve that the start of your current item chunk is memory-aligned to a certain item multiple. If however less was produced, your block might still be called, to keep the data flowing. To properly apply these flags, you’ll basically need a human understanding of what the block does. It may, nevertheless, be very helpful to understand how well your block performs with different item chunk sizes. To realize that, some mechanism to change scheduling behavior I will look into that; I think it should be possible to manipulate the `block_executors` to manipulate them into changing their forecasting/work calling behavior at runtime, but I’m quite sure that this will bring new code into the main tree[1]. All in all, right now I’m really stuck with what I actually want to improve with the performance analysis of GNU Radio flowgraphs offered by performance counters/gr-perf-monitorx, because they address many of these issues already. Your execution-per-item over chunk size idea is excellent! So long, Greetings, Marcus [1] I’ll really have to take a deeeep look at block\_executor and the tpb scheduler to tell; if I decide to add functionality that introduces significant runtime overhead or changes too much of internal behaviour, noone will be pleased, so I might take this slow and will have to discuss it with experienced core developers. I’m not very hesitant when it comes to fiddling with in-tree source code, but my workings almost never make it to the public, because I always figure they don’t address a problem properly or break too much in comparison to what they can possibly improve. On 25.05.2014 19:34, Bogdan D. wrote: > Building on block computing performance measurements there is one > perspective the optimizer would fall in the category of > results. > > all the constructive feedback on my proposal, and all the support, > > Then, there will be code; you will find that on my github page > \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Discuss-gnuradio > mailing list [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [Discuss-gnuradio Info Page](https://lists.gnu.org/mailman/listinfo/discuss-gnuradio) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - [http://www.enigmail.net/](http://www.enigmail.net/) iQEcBAEBAgAGBQJTgkEuAAoJEBQ6EdjyzlHtM54H/0cXRJiq7y0qdrm7N0jdeHVj vu9Tjbl6XfvPUwHnPejfi/OPskQVTfTF+3sMDtQ3QWXW5K+HBPob7ci7xpTOob3z zmkHBzXK+7X3Uvgfo0zV7zp1cnMWwCsnIdzK7aOelusK7unGZps5Zh7kUoSbqcto Dajf5RfG4YbCCzkv30Ykg5i3tqSPWtqDKqWsZnEC1wIBR7eLS9UgCn98CawhJYhB ittApa73nNcPtJl8Q6REzWufXRqa6asmxdZwXYqFguQrYUueqxUAtphwv1Sy4Ync BXAEyj48saXsvzmL2li0S4A8CC7dydPwAJnwTczS0cAlwuS2/c/J9BtwdOappUM= =A0bm -----END PGP SIGNATURE----- ---
**Author:** ![Marcus\_MS\_ller](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/5daacb/32.png) [@Marcus\_MS\_ller](https://www.ruby-forum.com/u/Marcus_MS_ller)\ **Post date:** [May 25, 2014, 9:16pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303/5 "2014-05-25T21:16:12Z")
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Bogdan, thanks for your comment 🙂 Such an optimizer would be really, really fancy. In a way, though, GNU Radio already does this when running a flow graph: It just asks blocks to compute a reasonable amount of items to fill up the downstream buffer. This actually (conceptually simple) approach is one great strength, because it just keeps the computer “busy” as much as possible. There might be space for optimization, though, I agree: Maybe it would be better for some blocks just to wait longer (and thus, not utilize the CPU) if it was computationally beneficial to work on larger chunks, as long as there are enough other blocks competing for CPU power. However, this leads to the problem of balancing average throughput with latency. What the GNU Radio infrastructure does to approach this is actually quite simple: 1. Although it might be “fastest” to process all data at once, buffer lengths set a natural limit to the chunk size, and thus latency. So we have an upper boundary. 2. It is best to be closest possible to that upper boundary. To achieve that, block developers are always encouraged to consume as many input items and produce as much output as possible, even if the overhead of having multiple (general\_)work calls is minute. This ensures that adjacent blocks don’t get asked to produce / consume small item chunks (which will happen if they were in a waiting state and a small number of items was produced or a small amount of output buffer was marked as read). Optimizing this will be hard. Maybe one could profile the same flowgraph with a lot of different settings of per-block maximum output chunk sizes, but I do believe this will only give as little more information than what the block developer already knew when he optimized the block in the first place. If he didn’t optimize, his main interest will be if his block poses a problem at at all; for that, standard settings should be employed. To give developers an API to inform the runtime of item amount preference, different methods exist, however. I’ll give a short rundown of them. 1. Most notable are the `fixed_rate properties` of `gr::block`, as implemented in `sync_block`, and the `decimator` and `interpolator` block types 2. If your block will only produce a multiples of a certain number of items, the `set_output_multiple` is a method that will potentially decrease overhead introduced by pointless calls to `forecast` and/or `work`. 3. In hardware optimization, alignment is often the performance critical factor. To account for that, set\_alignment was introduced. It’s working very similar to `set_output_multiple`, but does not _enforce_ the multiples, but sets an unaligned flag if non-multiple consumption occurred. The runtime will always try to achieve that the start of your current item chunk is memory-aligned to a certain item multiple. If however less was produced, your block might still be called, to keep the data flowing. To properly apply these flags, you’ll basically need a human understanding of what the block does. It may, nevertheless, be very helpful to understand how well your block performs with different item chunk sizes. To realize that, some mechanism to change scheduling behavior I will look into that; I think it should be possible to manipulate the `block_executors` to manipulate them into changing their forecasting/work calling behavior at runtime, but I’m quite sure that this will bring new code into the main tree[1]. All in all, right now I’m really stuck with what I actually want to improve with the performance analysis of GNU Radio flowgraphs offered by performance counters/gr-perf-monitorx, because they address many of these issues already. Your execution-per-item over chunk size idea is excellent! So long, Greetings, Marcus [1] I’ll really have to take a deeeep look at block\_executor and the tpb scheduler to tell; if I decide to add functionality that introduces significant runtime overhead or changes too much of internal behaviour, noone will be pleased, so I might take this slow and will have to discuss it with experienced core developers. I’m not very hesitant when it comes to fiddling with in-tree source code, but my workings almost never make it to the public, because I always figure they don’t address a problem properly or break too much in comparison to what they can possibly improve. On 25.05.2014 19:34, Bogdan D. wrote: > Building on block computing performance measurements there is one > perspective the optimizer would fall in the category of > results. > > all the constructive feedback on my proposal, and all the support, > > Then, there will be code; you will find that on my github page > \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Discuss-gnuradio > mailing list [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [Discuss-gnuradio Info Page](https://lists.gnu.org/mailman/listinfo/discuss-gnuradio) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - [http://www.enigmail.net/](http://www.enigmail.net/) iQEcBAEBAgAGBQJTgkEuAAoJEBQ6EdjyzlHtM54H/0cXRJiq7y0qdrm7N0jdeHVj vu9Tjbl6XfvPUwHnPejfi/OPskQVTfTF+3sMDtQ3QWXW5K+HBPob7ci7xpTOob3z zmkHBzXK+7X3Uvgfo0zV7zp1cnMWwCsnIdzK7aOelusK7unGZps5Zh7kUoSbqcto Dajf5RfG4YbCCzkv30Ykg5i3tqSPWtqDKqWsZnEC1wIBR7eLS9UgCn98CawhJYhB ittApa73nNcPtJl8Q6REzWufXRqa6asmxdZwXYqFguQrYUueqxUAtphwv1Sy4Ync BXAEyj48saXsvzmL2li0S4A8CC7dydPwAJnwTczS0cAlwuS2/c/J9BtwdOappUM= =A0bm -----END PGP SIGNATURE----- ---
**Author:** ![Marcus\_MS\_ller](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/5daacb/32.png) [@Marcus\_MS\_ller](https://www.ruby-forum.com/u/Marcus_MS_ller)\ **Post date:** [May 28, 2014, 12:25pm UTC](https://www.ruby-forum.com/t/gnu-radio-measurement-toolbox-gsoc-14-project/241303/6 "2014-05-28T12:25:23Z")
> **[History](https://github.com/guruofquality/gras/wiki/History)** > > GRAS: The GNU Radio Advanced Scheduler. Contribute to guruofquality/gras development by creating an account on GitHub. On Mon, May 26, 2014 at 6:46 AM, Bogdan D.