# Slow down rate of Python source block **URL:** \ **Category:** GNU Radio\ **Created:** [July 31, 2014, 6:23pm UTC](https://www.ruby-forum.com/t/slow-down-rate-of-python-source-block/231277 "2014-07-31T18:23:21Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![addis\_a](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/c4cdca/32.png) [@addis\_a](https://www.ruby-forum.com/u/addis_a)\ **Post date:** [July 31, 2014, 6:23pm UTC](https://www.ruby-forum.com/t/slow-down-rate-of-python-source-block/231277/1 "2014-07-31T18:23:21Z")
Dear All, I have a Python block that produces packets of size 1536 bytes. Due to various reasons, the latter parts of my flow graph are very slow (this is desired and cannot be changed). After producing 510 packets, I get the following error. “handler caught exception: operands could not be broadcast together with shapes (1527) (1536) Traceback (most recent call last): File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line 55, in eval try: self.\_callback() File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line 160, in \_\_gr\_block\_handle ) for i in self.\_\_out\_indexes], File “/usr/local/lib/python2.7/dist-packages/trl/blsd\_enc\_b.py”, line 198, in work out\_cA[0:len(cAm)] = cAm ValueError: operands could not be broadcast together with shapes (1527) (1536) thread[thread-per-block[1]: \]: caught unrecognized exception” Debugging more carefully, I can see that: len(cAm) = 1536 , len(out\_cA) = 32768 for the first 490 packets, and then the length of ‘out\_cA’ begins to reduce packet\_num = 490 len(cAm) = 1536 , len(out\_cA) = 32247 packet\_num = 491 len(cAm) = 1536 , len(out\_cA) = 30711 packet\_num = 492 len(cAm) = 1536 , len(out\_cA) = 29175 packet\_num = 493 len(cAm) = 1536 , len(out\_cA) = 27639 packet\_num = 494 len(cAm) = 1536 , len(out\_cA) = 26103 packet\_num = 495 len(cAm) = 1536 , len(out\_cA) = 24567 packet\_num = 496 len(cAm) = 1536 , len(out\_cA) = 23031 packet\_num = 497 len(cAm) = 1536 , len(out\_cA) = 21495 packet\_num = 498 len(cAm) = 1536 , len(out\_cA) = 19959 packet\_num = 499 len(cAm) = 1536 , len(out\_cA) = 18423 packet\_num = 500 len(cAm) = 1536 , len(out\_cA) = 16887 packet\_num = 501 len(cAm) = 1536 , len(out\_cA) = 15351 packet\_num = 502 len(cAm) = 1536 , len(out\_cA) = 13815 packet\_num = 503 len(cAm) = 1536 , len(out\_cA) = 12279 packet\_num = 504 len(cAm) = 1536 , len(out\_cA) = 10743 packet\_num = 505 len(cAm) = 1536 , len(out\_cA) = 9207 packet\_num = 506 len(cAm) = 1536 , len(out\_cA) = 7671 packet\_num = 507 len(cAm) = 1536 , len(out\_cA) = 6135 packet\_num = 508 len(cAm) = 1536 , len(out\_cA) = 4599 packet\_num = 509 len(cAm) = 1536 , len(out\_cA) = 3063 packet\_num = 510 len(cAm) = 1536 , len(out\_cA) = 1527 I believe this is because the latter parts of the flow graph are blocking, their buffers become full, and this backs up to my source block. This is confirmed by connecting my source block to null sinks, which then allows the source block to run infinitely (well, a long while!). Is there a way I can slow up this block? I tried using a throttle on the output, but this doesn’t help (in fact it seems to make it worse!). The full code of the block is attached. Thanks! David * * * NOTE: The information in this email and any attachments may be confidential and/or legally privileged. This message may be read, copied and used only by the intended recipient. If you are not the intended recipient, please destroy this message, delete any copies held on your system and notify the sender immediately. Toshiba Research Europe Limited, registered in England and Wales (2519556). Registered Office 208 Cambridge Science Park, Milton Road, Cambridge CB4 0GZ, England. Web: www.toshiba.eu/research/trl ---
**Author:** ![divotdave](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/bbe5ce/32.png) [@divotdave](https://www.ruby-forum.com/u/divotdave)\ **Post date:** [July 31, 2014, 8:13pm UTC](https://www.ruby-forum.com/t/slow-down-rate-of-python-source-block/231277/2 "2014-07-31T20:13:25Z")
On Thu, Jul 31, 2014 at 12:21 PM, David Halls [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line > thread[thread-per-block[1]: \]: caught unrecognized > exception" > > Debugging more carefully, I can see that: > > len(cAm) = 1536 , len(out\_cA) = 32768 Just a quick response without really studying the problem or your code. The dynamic scheduler in GR is getting in your way, and the throttle block is definitely not the right way to help you. You need to either tell the scheduler what you need it to send your block or handle it internally. There are three ways to solve these issues: 1. Use set\_output\_multiple in the constructor, which will only allow the scheduler to send you chunks of data of some multiple of the number you pass that. I’ve seen this slow down the scheduler in other situations, but it sounds like you’re going slow, anyways, so this shouldn’t cause a problem. 2. Make your input signature your packet\_length so each item will be a vector of that length. This would not be my preferred way, but we’ve played that game before. 3. Handle it internally. Buffer up the input until you have enough to produce what you need. You’d need to inherit from gr.basic\_block here and do more management of the data and buffers yourself. Tom ---
**Author:** ![divotdave](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/bbe5ce/32.png) [@divotdave](https://www.ruby-forum.com/u/divotdave)\ **Post date:** [August 1, 2014, 11:25am UTC](https://www.ruby-forum.com/t/slow-down-rate-of-python-source-block/231277/3 "2014-08-01T11:25:37Z")
* * * From: removed\_email\_address@domain.invalid [removed\_email\_address@domain.invalid] on behalf of Tom Rondeau [removed\_email\_address@domain.invalid] Sent: 31 July 2014 19:11 To: David Halls Cc: removed\_email\_address@domain.invalid Subject: Re: [Discuss-gnuradio] Slow down rate of Python source block On Thu, Jul 31, 2014 at 12:21 PM, David Halls \ wrote: Dear All, I have a Python block that produces packets of size 1536 bytes. Due to various reasons, the latter parts of my flow graph are very slow (this is desired and cannot be changed). After producing 510 packets, I get the following error. “handler caught exception: operands could not be broadcast together with shapes (1527) (1536) Traceback (most recent call last): File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line 55, in eval try: self.\_callback() File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line 160, in \_\_gr\_block\_handle ) for i in self.\_\_out\_indexes], File “/usr/local/lib/python2.7/dist-packages/trl/blsd\_enc\_b.py”, line 198, in work out\_cA[0:len(cAm)] = cAm ValueError: operands could not be broadcast together with shapes (1527) (1536) thread[thread-per-block[1]: \]: caught unrecognized exception” Debugging more carefully, I can see that: len(cAm) = 1536 , len(out\_cA) = 32768 Just a quick response without really studying the problem or your code. The dynamic scheduler in GR is getting in your way, and the throttle block is definitely not the right way to help you. You need to either tell the scheduler what you need it to send your block or handle it internally. There are three ways to solve these issues: 1. Use set\_output\_multiple in the constructor, which will only allow the scheduler to send you chunks of data of some multiple of the number you pass that. I’ve seen this slow down the scheduler in other situations, but it sounds like you’re going slow, anyways, so this shouldn’t cause a problem. 2. Make your input signature your packet\_length so each item will be a vector of that length. This would not be my preferred way, but we’ve played that game before. 3. Handle it internally. Buffer up the input until you have enough to produce what you need. You’d need to inherit from gr.basic\_block here and do more management of the data and buffers yourself. Tom * * * Tom, Thanks for your reply. Unfortunately as it is a Python block, I cannot use ‘set\_output\_multiple’. For 2. do you mean set the input signature of the blocks fed from the source block. This could be possible but quite a few different blocks (some standard GR blocks) are fed from it. Could you provide some more details, or a link to similar implementation for option 3. - again is this possible from within a Python block? Regards, David for the first 490 packets, and then the length of ‘out\_cA’ begins to reduce packet\_num = 490 len(cAm) = 1536 , len(out\_cA) = 32247 packet\_num = 491 len(cAm) = 1536 , len(out\_cA) = 30711 packet\_num = 492 len(cAm) = 1536 , len(out\_cA) = 29175 packet\_num = 493 len(cAm) = 1536 , len(out\_cA) = 27639 packet\_num = 494 len(cAm) = 1536 , len(out\_cA) = 26103 packet\_num = 495 len(cAm) = 1536 , len(out\_cA) = 24567 packet\_num = 496 len(cAm) = 1536 , len(out\_cA) = 23031 packet\_num = 497 len(cAm) = 1536 , len(out\_cA) = 21495 packet\_num = 498 len(cAm) = 1536 , len(out\_cA) = 19959 packet\_num = 499 len(cAm) = 1536 , len(out\_cA) = 18423 packet\_num = 500 len(cAm) = 1536 , len(out\_cA) = 16887 packet\_num = 501 len(cAm) = 1536 , len(out\_cA) = 15351 packet\_num = 502 len(cAm) = 1536 , len(out\_cA) = 13815 packet\_num = 503 len(cAm) = 1536 , len(out\_cA) = 12279 packet\_num = 504 len(cAm) = 1536 , len(out\_cA) = 10743 packet\_num = 505 len(cAm) = 1536 , len(out\_cA) = 9207 packet\_num = 506 len(cAm) = 1536 , len(out\_cA) = 7671 packet\_num = 507 len(cAm) = 1536 , len(out\_cA) = 6135 packet\_num = 508 len(cAm) = 1536 , len(out\_cA) = 4599 packet\_num = 509 len(cAm) = 1536 , len(out\_cA) = 3063 packet\_num = 510 len(cAm) = 1536 , len(out\_cA) = 1527 I believe this is because the latter parts of the flow graph are blocking, their buffers become full, and this backs up to my source block. This is confirmed by connecting my source block to null sinks, which then allows the source block to run infinitely (well, a long while!). Is there a way I can slow up this block? I tried using a throttle on the output, but this doesn’t help (in fact it seems to make it worse!). The full code of the block is attached. Thanks! David * * * NOTE: The information in this email and any attachments may be confidential and/or legally privileged. This message may be read, copied and used only by the intended recipient. If you are not the intended recipient, please destroy this message, delete any copies held on your system and notify the sender immediately. Toshiba Research Europe Limited, registered in England and Wales (2519556). Registered Office 208 Cambridge Science Park, Milton Road, Cambridge CB4 0GZ, England. Web: www.toshiba.eu/research/trl[http://www.toshiba.eu/research/trl](http://www.toshiba.eu/research/trl) * * * This email has been scanned for email related threats and delivered safely by Mimecast. For more information please visit [http://www.mimecast.com](http://www.mimecast.com) * * * * * * Discuss-gnuradio mailing list removed\_email\_address@domain.invalid[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) [https://lists.gnu.org/mailman/listinfo/discuss-gnuradio](https://lists.gnu.org/mailman/listinfo/discuss-gnuradio) * * * NOTE: The information in this email and any attachments may be confidential and/or legally privileged. This message may be read, copied and used only by the intended recipient. If you are not the intended recipient, please destroy this message, delete any copies held on your system and notify the sender immediately. Toshiba Research Europe Limited, registered in England and Wales (2519556). Registered Office 208 Cambridge Science Park, Milton Road, Cambridge CB4 0GZ, England. Web: www.toshiba.eu/research/trl ---
**Author:** ![divotdave](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/bbe5ce/32.png) [@divotdave](https://www.ruby-forum.com/u/divotdave)\ **Post date:** [August 14, 2014, 8:50pm UTC](https://www.ruby-forum.com/t/slow-down-rate-of-python-source-block/231277/4 "2014-08-14T20:50:03Z")
From: removed\_email\_address@domain.invalid [[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Tom R. Sent: 01 August 2014 14:31 To: David Halls Cc: removed\_email\_address@domain.invalid Subject: Re: [Discuss-gnuradio] Slow down rate of Python source block On Fri, Aug 1, 2014 at 5:24 AM, David Halls \ wrote: * * * From: removed\_email\_address@domain.invalid[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) [removed\_email\_address@domain.invalid[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] on behalf of Tom Rondeau [removed\_email\_address@domain.invalid[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Sent: 31 July 2014 19:11 To: David Halls Cc: removed\_email\_address@domain.invalid[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Subject: Re: [Discuss-gnuradio] Slow down rate of Python source block On Thu, Jul 31, 2014 at 12:21 PM, David Halls \\> wrote: Dear All, I have a Python block that produces packets of size 1536 bytes. Due to various reasons, the latter parts of my flow graph are very slow (this is desired and cannot be changed). After producing 510 packets, I get the following error. “handler caught exception: operands could not be broadcast together with shapes (1527) (1536) Traceback (most recent call last): File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line 55, in eval try: self.\_callback() File “/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gateway.py”, line 160, in \_\_gr\_block\_handle ) for i in self.\_\_out\_indexes], File “/usr/local/lib/python2.7/dist-packages/trl/blsd\_enc\_b.py”, line 198, in work out\_cA[0:len(cAm)] = cAm ValueError: operands could not be broadcast together with shapes (1527) (1536) thread[thread-per-block[1]: \]: caught unrecognized exception” Debugging more carefully, I can see that: len(cAm) = 1536 , len(out\_cA) = 32768 Just a quick response without really studying the problem or your code. The dynamic scheduler in GR is getting in your way, and the throttle block is definitely not the right way to help you. You need to either tell the scheduler what you need it to send your block or handle it internally. There are three ways to solve these issues: 1. Use set\_output\_multiple in the constructor, which will only allow the scheduler to send you chunks of data of some multiple of the number you pass that. I’ve seen this slow down the scheduler in other situations, but it sounds like you’re going slow, anyways, so this shouldn’t cause a problem. 2. Make your input signature your packet\_length so each item will be a vector of that length. This would not be my preferred way, but we’ve played that game before. 3. Handle it internally. Buffer up the input until you have enough to produce what you need. You’d need to inherit from gr.basic\_block here and do more management of the data and buffers yourself. Tom * * * Tom, Thanks for your reply. Unfortunately as it is a Python block, I cannot use ‘set\_output\_multiple’. For 2. do you mean set the input signature of the blocks fed from the source block. This could be possible but quite a few different blocks (some standard GR blocks) are fed from it. Could you provide some more details, or a link to similar implementation for option 3. - again is this possible from within a Python block? Regards, David David, set\_output\_multiple is exposed through the Python gateway code, so you should be able to call self.set\_output\_multiple inside the constructor… For 2, yes, I mean that you can adjust the io signature of the block, essentially taking in a vector so that 1 item is now a full packet. But yes, this makes it more difficult to integrate with other blocks; one reason I said this isn’t a preferred method. And for 3, the only think that I can think of is the qtgui sinks, like the freq sink that passes a chunk of data the length of the FFT size and buffers it internally. The work functions for these types of blocks tend to become more complex, though, and difficult to read. But you can certainly do it inside a Python block; you just may need to do hard copies out of the input buffers to make sure the data is actually moved and stored locally. Tom Hi Tom, I tried self.set\_output\_multiple and this doesn’t work in Python, I believe for the same reasons as "The “set\_min\_noutput\_items()” is not yet a supported python statement [1] as per March 06, 2014. Also, I believe this statement should be placed in the member functions (constructor, work function, callback function etc) of your custom block, don’t put it at the wrong place. In alternative, you may want to replace it with “set\_output\_multiple()” if this is more appropriate. 1. [http://gnuradio.4.n7.nabble.com/self-set-min-noutput-items-is-not-a-valid-python-command-in-gnuradio-td46731.html](http://gnuradio.4.n7.nabble.com/self-set-min-noutput-items-is-not-a-valid-python-command-in-gnuradio-td46731.html)" So I can’t do Option 1), for now, Option 3) sounds very tricky, so I’ve tried number 2) Fortunately my source block feeds only to ‘unpacked\_to\_packed’ blocks, a few of them, so I’ve created my own. I adjusted the io signature of the input to be …::make(1,1, packet\_len\*sizeof(unsigned char)) This gives Python runtime errors “itemsize mismatch: bsld\_enc\_b0:0 using 1, unpacked\_to\_packed1:0 using 1536”. I then try to set the io signature of my source block but if I try: out\_sig=[packet\_len\*numpy.uint8] I get TypeError: unsupported operand type(s) for \*: ‘int’ and ‘type’ and if I try out\_sig=[packet\_len] TypeError: data type not understood Can you help me further? Many thanks, David * * * NOTE: The information in this email and any attachments may be confidential and/or legally privileged. This message may be read, copied and used only by the intended recipient. If you are not the intended recipient, please destroy this message, delete any copies held on your system and notify the sender immediately. Toshiba Research Europe Limited, registered in England and Wales (2519556). Registered Office 208 Cambridge Science Park, Milton Road, Cambridge CB4 0GZ, England. Web: www.toshiba.eu/research/trl ---
**Author:** ![divotdave](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/bbe5ce/32.png) [@divotdave](https://www.ruby-forum.com/u/divotdave)\ **Post date:** [August 1, 2014, 3:32pm UTC](https://www.ruby-forum.com/t/slow-down-rate-of-python-source-block/231277/5 "2014-08-01T15:32:29Z")
On Fri, Aug 1, 2014 at 5:24 AM, David Halls [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > On Thu, Jul 31, 2014 at 12:21 PM, David Halls \< > Traceback (most recent call last): > (1536) > scheduler what you need it to send your block or handle it internally. > that game before. > Tom, > David David, set\_output\_multiple is exposed through the Python gateway code, so you should be able to call self.set\_output\_multiple inside the constructor.. For 2, yes, I mean that you can adjust the io signature of the block, essentially taking in a vector so that 1 item is now a full packet. But yes, this makes it more difficult to integrate with other blocks; one reason I said this isn’t a preferred method. And for 3, the only think that I can think of is the qtgui sinks, like the freq sink that passes a chunk of data the length of the FFT size and buffers it internally. The work functions for these types of blocks tend to become more complex, though, and difficult to read. But you can certainly do it inside a Python block; you just may need to do hard copies out of the input buffers to make sure the data is actually moved and stored locally. Tom