# CPU usage when hdlc framer is used **URL:** \ **Category:** GNU Radio\ **Created:** [June 16, 2015, 5:00pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334 "2015-06-16T17:00:41Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Detlef\_R](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/58f4c7/32.png) [@Detlef\_R](https://www.ruby-forum.com/u/Detlef_R)\ **Post date:** [June 16, 2015, 5:00pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/1 "2015-06-16T17:00:41Z")
Hi, CPU usage goes to 100% when the hdlc\_framer block is used! To verify this I did the following tests: 1. message\_strobe\_random ----\> message\_debug (CPU usage 2%) 2. message\_strobe\_random ----\> hdlc\_framer----\> null\_sink (CPU usage 100%) Any idea how can I avoid this overload? Thank you Thanasis ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 17, 2015, 11:57am UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/2 "2015-06-17T11:57:07Z")
On Tue, 2015-06-16 at 12:01 -0400, [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > From: Thanasis B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > To: “[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > Subject: [Discuss-gnuradio] CPU usage when hdlc framer is used > Hi, > > CPU usage goes to 100% when the hdlc\_framer block is used! > To verify this I did the following tests: > > 1. message\_strobe\_random ----\> message\_debug (CPU usage 2%) > 2. message\_strobe\_random ----\> hdlc\_framer----\> null\_sink (CPU usage 100%) > > Any idea how can I avoid this overload? Off the cuff, it appears the hdlc\_framer needs at least one fix: 1. If oidx == 0, it should use a blocking fetch of a message here: [gnuradio/gr-digital/lib/hdlc\_framer\_pb\_impl.cc at master · gnuradio/gnuradio · GitHub](https://github.com/gnuradio/gnuradio/blob/master/gr-digital/lib/hdlc_framer_pb_impl.cc#L132) If oidx != 0 then using the current delete\_head\_nowait() call is correct. This is probably the major contribution to the CPU spinning. Maybe some other changes to the hdlc\_framer block might help for high sample rates: 1. The block does a number of dynamic std:vector\<\> and pmt instantiations and destructions in a call to work(). If you use the oprofile tools, those likely generate some measureable overhead attributable to malloc() and atomic increments and decrements related to boost shared pointer for the PMTs. You might gain a few % CPU back by avoiding the constant allocation & deallocation by converting the block to use class variables where it makes sense. 2. The block also has 2 memcpy()'s in a call to work(). If you can figure out an way to optimize one away, you’ll get some savings. I don’t see an obvious way to do that right now. Also, your flowgraph does not have a throttle block to enforce a realistic timing for the sample rate you have chosen, so the hdlc\_framer is going to run as fast as it can. The null sink essentially ensures there is always output buffer space for the hdlc\_framer. > Thank you > Thanasis Regards, Andy ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 17, 2015, 12:19pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/3 "2015-06-17T12:19:55Z")
Issue #1 is likely the problem. Thanks for pointing that out, Andy! The other two issues should be pretty miniscule in overhead, but are probably good ideas nonetheless. The block was written for clarity rather than for speed, but even so it shouldn’t be particularly heavy-duty. –n On Tue, Jun 16, 2015 at 9:53 AM Andy W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 17, 2015, 3:29pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/4 "2015-06-17T15:29:46Z")
On Tue, 2015-06-16 at 12:27 -0700, Johnathan C. wrote: > On Tue, Jun 16, 2015 at 10:16 AM, Nick F. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > wrote: > > ``` > Issue #1 is likely the problem. Thanks for pointing that out, > Andy! The other two issues should be pretty miniscule in > overhead, but are probably good ideas nonetheless. The block > was written for clarity rather than for speed, but even so it > shouldn't be particularly heavy-duty. > > ``` > In the future, we plan to allow designating a non-empty message queue > as a prior condition to be satisfied before the scheduler calls > work(), thus it will won’t ever be necessary to block inside work > directly on the queue itself. This is a common issue with source > blocks that rely on input message ports for content. Hi John, That will help for blocks which can somehow guarantee that every incoming message will never generate more than noutput\_items in every call to work. Otherwise the block is going to have to save away the residual output items, to be output in a later call to work(). There is no need to block on message input to output those residual output items. I.e. in a call to work(): 0. output any residual items saved from last call to work() 1. grab message off queue (making a decision to block or not as appropriate) 2. generate items, which may number more than noutput\_items 3. store any residual items for output for next call to work() 4. return number of items generated up to noutput\_items The hdlc\_framer looks like it has this notion of a residual in d\_leftovers. Although different from GnuRadio internal messaging, some internal fixes I made to the gr-zeromq sub\_source\_impl._, to work reliably and at high throughput, also needed to deal with residual parts of messages that could not be sent out in the current call to work() and had to deal with context depending blocking when checking for messages. (The current sub\_source\_impl._ on the master branch discards the items in incoming ZeroMQ messages that are beyond noutput\_items - not a nice thing to do.) My $0.02 Regards, Andy ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 17, 2015, 2:31pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/5 "2015-06-17T14:31:21Z")
On Tue, Jun 16, 2015 at 10:16 AM, Nick F. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Issue #1 is likely the problem. Thanks for pointing that out, Andy! The > other two issues should be pretty miniscule in overhead, but are probably > good ideas nonetheless. The block was written for clarity rather than for > speed, but even so it shouldn’t be particularly heavy-duty. In the future, we plan to allow designating a non-empty message queue as a prior condition to be satisfied before the scheduler calls work(), thus it will won’t ever be necessary to block inside work directly on the queue itself. This is a common issue with source blocks that rely on input message ports for content. ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 17, 2015, 6:08pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/6 "2015-06-17T18:08:41Z")
On Tue, Jun 16, 2015 at 1:32 PM, Marcus Müller [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > if I understand you correctly, you think the situation calls for > something like a “wait for message flag” that a block would need to set > every work call? Actually, this is the idea–that, _when needed_, make a call to the scheduler just before ending work, that says, in in effect–don’t call me again until input message port X is not empty. Otherwise, call me again whenever you otherwise would, which, for a source block, is when there is room in the output buffer (subject to other constraints.) This would allow work() to be invoked one more times as needed, to exhaust any pending data that might exceed noutput\_items in a given call, yet then finally wait on a message for more content. The key point would be to not block in work, which prevents the scheduler from that block from processing any other events, but instead allow the scheduler itself to do so. ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 19, 2015, 9:45am UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/7 "2015-06-19T09:45:14Z")
On Wed, 2015-06-17 at 12:01 -0400, [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > On Tue, Jun 16, 2015 at 1:32 PM, Marcus M?ller [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > again until input message port X is not empty. > from that block from processing any other events, but instead allow the > scheduler itself to do so. Hi John, Yes, that seems right. The block needs to continually keep the gnuradio infrastructure informed about when to wait for a message or not, before calling work(). The infrastructure can’t possibly know otherwise. Regards, Andy ---
**Author:** ![Thanasis\_B](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/7ea924/32.png) [@Thanasis\_B](https://www.ruby-forum.com/u/Thanasis_B)\ **Post date:** [June 17, 2015, 3:35pm UTC](https://www.ruby-forum.com/t/cpu-usage-when-hdlc-framer-is-used/242334/8 "2015-06-17T15:35:37Z")
Hi Andy, if I understand you correctly, you think the situation calls for something like a “wait for message flag” that a block would need to set every work call? Best regards, Marcus