Forum: Ruby How use Profiling, a Quick Guide.

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
D812408537ac3a0fa2fec96eb8811559?d=identicon&s=25 John Carter (Guest)
on 2006-05-24 00:10
(Received via mailing list)
On Tue, 23 May 2006, Mark Somerville wrote:

> Can anyone recommend some good or tutorials on profiling and benchmarking?
> Although I have used them both (from the standard library), I feel I lack
> fundamental knowledge about how to go about it.

I don't know of any Good one but here are some things I do...

  * Always benchmark first, profile and optimize later. Otherwise you
don't
    know whether it was worth it. (Hint: Often the tweaks aren't worth
it, or
    make things worse)

  * Look at the top most items in the profile list. Usually they are
    standard library functions. Silly things like [] or +...
    Ask your self, "What could be calling these thing so often?
    Can I make it do it less often? eg. Cache the result etc.)

  * Scan down the list for the first few user written methods. Usually
the one
    that you spent the most time in has the most fat to trim.

  * Scan the list for the routine that took the longest (as opposed
    to the most time spent in it.) Try get that one to just do less.

  * Use strace, see what the system is doing at an OS level. I once
    improved another persons program (professional in production) by a
    factor of 40, yes FORTY times, by noting it was doing an lseek on
every
    record read and changing that to an mmap.

  * Always look out for surprises. The best moment in an optimizers life
is
    when you look at something and say "WTF! Why is _that_ routine
taking so
    long!?"

  * Large number of calls to .size and .length and .count_objects
methods are a
    clue some nidjit somewhere is doing something really N^2 stupid
like...
     for( i=0; i < container.count_objects(); i++) {
     }

  * Use top, vmstat, times as well.
     Large usertime means optimize program.
     Large system time means you really hammering the system calls (that
     was a clue in the lseek case I mentioned above)
     Large real time small user and system means you are waiting for
     input from user / disk / network /...? Find out what.
     Watch the page in page out stats. Watch the swap in swap out stats.
     If you are paging or swapping, time to switch to memory consumption
     optimization!

   * Use my memprofile /  newprofile tricks to try spot the
     http://wiki.rubygarden.org/Ruby/page/show/NewProfiler
     http://rubyforge.org/snippet/detail.php?type=snipp...


John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
Bb6ecee0238ef2461bef3416722b35c5?d=identicon&s=25 pat eyler (Guest)
on 2006-05-24 01:23
(Received via mailing list)
On 5/23/06, John Carter <john.carter@tait.co.nz> wrote:
> On Tue, 23 May 2006, Mark Somerville wrote:
>
> > Can anyone recommend some good or tutorials on profiling and benchmarking?
> > Although I have used them both (from the standard library), I feel I lack
> > fundamental knowledge about how to go about it.
>

Hmm, not sure how I missed the original message, but you might try:

http://www.ibm.com/developerworks/edu/os-dw-os-ruby2-i.html

(free subscription required)
D812408537ac3a0fa2fec96eb8811559?d=identicon&s=25 John Carter (Guest)
on 2006-05-24 02:49
(Received via mailing list)
On Wed, 24 May 2006, pat eyler wrote:

> http://www.ibm.com/developerworks/edu/os-dw-os-ruby2-i.html
>
> (free subscription required)

Sigh! Buggy Registration required.

I came, I tried to register, made one error, back, fixed error, pressed
continue, nothing happened. nothing. nothing. click. CLICK
CLICK_CLICK-CLICK!

Stupid @%$#%#! Buggy dumb registration form!! AARGH!!

I went away.

DeveloperWorks used to be quite a Good resource.

On the web, better, less frustrating things are always just one click
away.


John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
Bd0203dc8478deb969d72f52e741bd4f?d=identicon&s=25 Daniel Baird (Guest)
on 2006-05-24 06:34
(Received via mailing list)
On 5/24/06, John Carter <john.carter@tait.co.nz> wrote:
>
> On Wed, 24 May 2006, pat eyler wrote:
>
> > http://www.ibm.com/developerworks/edu/os-dw-os-ruby2-i.html
> >
> Stupid @%$#%#! Buggy dumb registration form!! AARGH!!
>
> I went away.


<OT>
you need bugmenot.com.. with the Firefox extension.
</OT>
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-05-24 09:09
(Received via mailing list)
From: John Carter [mailto:john.carter@tait.co.nz]
Sent: Wednesday, May 24, 2006 1:10 AM
> On Tue, 23 May 2006, Mark Somerville wrote:
> > Can anyone recommend some good or tutorials on profiling and
> benchmarking?
> > Although I have used them both (from the standard library), I feel I
> lack
> > fundamental knowledge about how to go about it.
>
> I don't know of any Good one but here are some things I do...

Outstanding point! Some things that are worth mentioning:

1. Unfortuantely, Windows users have lack of Unix power tools, which are
always helpful for analysys. But some of them can be found at
http://gnuwin32.sourceforge.net/

2.
>   * Look at the top most items in the profile list. Usually they are
>     standard library functions. Silly things like [] or +...
>     Ask your self, "What could be calling these thing so often?
>     Can I make it do it less often? eg. Cache the result etc.)

Here can help my Module#add_tracer, described here
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

Victor.
D812408537ac3a0fa2fec96eb8811559?d=identicon&s=25 John Carter (Guest)
on 2006-05-25 01:16
(Received via mailing list)
On Wed, 24 May 2006, Victor Shepelev wrote:

>>   * Look at the top most items in the profile list. Usually they are
>>     standard library functions. Silly things like [] or +...
>>     Ask your self, "What could be calling these thing so often?
>>     Can I make it do it less often? eg. Cache the result etc.)
>
> Here can help my Module#add_tracer, described here
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

Coool!! I _like_ it! Let that be a lesson to those who say Ruby can't do
AOP or Lispish Macros.

I'll just stuff in something that histograms the call path to a certain
depth and that will be really really powerful.

Coming soon to a Ruby snippet near you...



John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
D812408537ac3a0fa2fec96eb8811559?d=identicon&s=25 John Carter (johncarter)
on 2006-05-25 04:28
(Received via mailing list)
On Thu, 25 May 2006, John Carter wrote:

> Coool!! I _like_ it! Let that be a lesson to those who say Ruby can't do
> AOP or Lispish Macros.

LOVELY!

It works brilliantly! Just use -rprofile to find the hot spots, the
standard class.method that has been called many many many times and then
mixed in my little MethodTracer.rb

(Did you know Find.find invokes Kernel.catch once per file/directory?)

===MethodTracer.rb===============================================================

# Based on "Victor Shepelev" <vshepelev imho.com.ua> Ruby talk message.
# http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...
#
#Usage
#-----
# Very simple
# require 'MethodTracer'
#
#SomeClass.add_tracer(:some_method)
#call_my_long_routine()
#SomeClass.report_tracer( :some_method)
#

$__tracer_no_recurse = nil
$__tracer_histogram = {}

# User configurable, tweak this for how many call frames you want in the
profile.
$__tracer_histogram_depth = 3

class Module
   def report_tracer(meth)
     $__tracer_no_recurse = true
     trace_who = "#{to_s}.#{meth}"
     puts "Reporting hottest traces on '#{trace_who}'"
     raise "No tracer '#{trace_who}'" unless
       $__tracer_histogram.has_key? trace_who
     tracer = $__tracer_histogram[ trace_who]
     tracer.keys.sort_by{|k| -tracer[k]}.each do |k|
       printf "%s   %7d\n", k, tracer[k]
     end
     puts
     $__tracer_no_recurse = false
   end

   def add_tracer(meth)
     return if $__tracer_no_recurse
     $__tracer_no_recurse = true
     trace_who = "#{to_s}.#{meth}"
     puts "Adding tracer for '#{trace_who}'"

     raise "Already have a trace for #{trace_who}" if
       $__tracer_histogram.has_key? trace_who

     $__tracer_histogram[trace_who] = Hash.new(0)

     m_alias = case meth
       when :[] : "old_idx"
       when :+ : "old_plus"
       when :- : "old_minus"
       #and so on
       else ; "old_#{meth}"
     end

     to_eval = %Q{
       alias :#{m_alias} :#{meth}

       def #{meth}(*arg, &block)
         return #{m_alias}(*arg, &block) if $__tracer_no_recurse
         $__tracer_no_recurse = true
         begin
           path = caller[0..$__tracer_histogram_depth].join("\n")
           $__tracer_histogram["#{trace_who}"][path]+=1
           #{m_alias}(*arg, &block)
         ensure
           $__tracer_no_recurse = nil
         end
       end
     }

     #puts to_eval

     module_eval to_eval

     $__tracer_no_recurse = nil

   end
end

if $0 == __FILE__ then
   require 'test/unit'

   class TC_Tracer < Test::Unit::TestCase
     def try_a
       a = [1,2,3,4]
       puts a[2]
     end

     def try_b
       try_a
     end

     def try_c
       try_b
     end

     def foo
       try_b
       try_b
       try_b
       try_b
       try_b
       try_b
       try_b
       try_b
       try_b
       try_b
     end

     def test_trace
       Array.add_tracer(:[])
       try_c
       try_b
       try_a
       foo
       Array.report_tracer(:[])
     end
   end
end




John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
3a3fe02be09e3103aea5e22d62478f38?d=identicon&s=25 Pedro Côrte-Real (Guest)
on 2006-05-25 12:42
(Received via mailing list)
On 5/24/06, Victor Shepelev <vshepelev@imho.com.ua> wrote:
> Here can help my Module#add_tracer, described here
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

That's a nice idea. I made a small modification to count the totals of
each of the calling methods instead of printing each time:

$traces = Hash.new(0)

class Module
  def add_tracer(meth)
    m_alias = case meth
      when :[] : "old_idx"
      when :+ : "old_plus"
      when :- : "old_minus"
      #and so on
      else ; "old_#{meth}"
    end
    module_eval %Q{
      alias :#{m_alias} :#{meth}
      def #{meth}(*arg, &block)
        str = caller[0]+"-"+self.class.name+"-##{meth}"
        $traces[str] += 1
        #{m_alias}(*arg, &block)
      end
    }
  end
end

Array.add_tracer(:each)

....
Your code here
....

$traces.each {|name, value| puts "#{name} - #{value} calls"}

You can also sort it by the value in the hash to get an ordered list.
This will do for a hack but writing a tracing gem would be great.

Pedro.
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-05-25 15:32
(Received via mailing list)
From: Pedro Corte-Real [mailto:pedrocr@gmail.com]
Sent: Thursday, May 25, 2006 1:41 PM
>   def add_tracer(meth)
>         str = caller[0]+"-"+self.class.name+"-##{meth}"
> Your code here
> ....
>
> $traces.each {|name, value| puts "#{name} - #{value} calls"}
>
> You can also sort it by the value in the hash to get an ordered list.
> This will do for a hack but writing a tracing gem would be great.

Yep, it's a very good point!

BTW, it can have sense to gather all those "profiling hacks" (like this
#add_tracer, and John's NewProfile and memprofile) to some rubyforge
project?
3a3fe02be09e3103aea5e22d62478f38?d=identicon&s=25 Pedro Côrte-Real (Guest)
on 2006-05-25 15:39
(Received via mailing list)
On 5/25/06, Victor Shepelev <vshepelev@imho.com.ua> wrote:
> BTW, it can have sense to gather all those "profiling hacks" (like this
> #add_tracer, and John's NewProfile and memprofile) to some rubyforge
> project?

Yes, sure, it would be great to have some simple tracing tools that
one can use just by installing a gem.

Pedro.
7359ed44852399295c6247dd9719b81b?d=identicon&s=25 Ola Bini (Guest)
on 2006-05-25 15:42
(Received via mailing list)
----- Original Message -----
From: Pedro Côrte-Real <pedrocr@gmail.com>
Date: Thursday, May 25, 2006 12:41 pm
Subject: Re: How use Profiling, a Quick Guide.
To: ruby-talk@ruby-lang.org (ruby-talk ML)

>  def add_tracer(meth)
>        str = caller[0]+"-"+self.class.name+"-##{meth}"
> Your code here
> ....
>
> $traces.each {|name, value| puts "#{name} - #{value} calls"}
>
> You can also sort it by the value in the hash to get an ordered list.
> This will do for a hack but writing a tracing gem would be great.
>
> Pedro.
>
>



Hi, this is interesting, and I tried something like it. It works fairly
well, but DON'T try to trace :+ with this, for obvious reasons... =)

/O
Fcc5cdf0f0f3e1a3a39c11ed4bf8d5e5?d=identicon&s=25 Stephan Mueller (Guest)
on 2006-05-25 15:58
(Received via mailing list)
* Victor Shepelev <vshepelev@imho.com.ua> [060525 15:36]:

> BTW, it can have sense to gather all those "profiling hacks" (like this
> #add_tracer, and John's NewProfile and memprofile) to some rubyforge
> project?


code should go to rubyforge but the description of how to profile
brought up earlier in this thread should be on rubygarden:

http://wiki.rubygarden.org/Ruby/page/show/RubyOptimization

Hey, that's what a wiki is for! 8-)


Cheers,

Steph.
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-05-25 15:58
(Received via mailing list)
From: John Carter [mailto:john.carter@tait.co.nz]
Sent: Thursday, May 25, 2006 5:28 AM
> > Coool!! I _like_ it! Let that be a lesson to those who say Ruby can't do
> ===MethodTracer.rb========================================================
> =======

[skip]

John, it's really cool!
May be (as I've mentioned above) we can create "profile hacks" project
on
rubyforge?

Victor.
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-05-25 16:01
(Received via mailing list)
From: d454d@web.de [mailto:d454d@web.de]
Sent: Thursday, May 25, 2006 4:56 PM
> http://wiki.rubygarden.org/Ruby/page/show/RubyOptimization
>
> Hey, that's what a wiki is for! 8-)

Good point. Real problem is my very poor English. I'm afraid I can
easily
write something nobody can read :(

> Steph.

Victor.
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2006-05-25 17:35
(Received via mailing list)
2006/5/25, Pedro Côrte-Real <pedrocr@gmail.com>:
>   def add_tracer(meth)
>         str = caller[0]+"-"+self.class.name+"-##{meth}"
> Your code here
> ....
>
> $traces.each {|name, value| puts "#{name} - #{value} calls"}
>
> You can also sort it by the value in the hash to get an ordered list.
> This will do for a hack but writing a tracing gem would be great.


Note that a similar thing can be achieved with set_trace_func

$ ruby -e 'set_trace_func lambda {|*a| p a}; "".length'
["line", "-e", 1, nil, #<Binding:0x100f6920>, false]
["c-call", "-e", 1, :length, #<Binding:0x100f67b8>, String]
["c-return", "-e", 1, :length, #<Binding:0x100f6770>, String]

You just need to evaluate parameters appropriately.  See also

http://ruby-doc.org/docs/ProgrammingRuby/html/ospace.html

Kind regards

robert
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-05-25 17:38
(Received via mailing list)
From: Robert Klemme [mailto:shortcutter@googlemail.com]
Sent: Thursday, May 25, 2006 6:33 PM
> > class Module
> >       def #{meth}(*arg, &block)
> > ....
> > Your code here
> > ....
> >
> > $traces.each {|name, value| puts "#{name} - #{value} calls"}
> >
> > You can also sort it by the value in the hash to get an ordered list.
> > This will do for a hack but writing a tracing gem would be great.
>
>
> Note that a similar thing can be achieved with set_trace_func

Drawbacks of set_trace_func was discussed in the original topic here:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

Victor.
58479f76374a3ba3c69b9804163f39f4?d=identicon&s=25 Eric Hodel (Guest)
on 2006-05-25 21:47
(Received via mailing list)
On May 25, 2006, at 8:37 AM, Victor Shepelev wrote:

> From: Robert Klemme [mailto:shortcutter@googlemail.com]
>> Note that a similar thing can be achieved with set_trace_func
>
> Drawbacks of set_trace_func was discussed in the original topic here:
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

Use event_hook instead.  Its faster than both.

See zenprofile in ZenHacks for an example.

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com
D812408537ac3a0fa2fec96eb8811559?d=identicon&s=25 John Carter (johncarter)
on 2006-05-26 06:42
(Received via mailing list)
On Thu, 25 May 2006, Victor Shepelev wrote:

> May be (as I've mentioned above) we can create "profile hacks" project on
> rubyforge?

You welcome to.

I know unfortunately I'm somewhat overcommitted at the moment so I
probably won't be able to do much on it. But you're more than welcome to
grab my code and add it to such a project.


John Carter                             Phone : (64)(3) 358 6639
Tait Electronics                        Fax   : (64)(3) 359 4632
PO Box 1645 Christchurch                Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong
later."

From this principle, all of life and physics may be deduced.
0b561a629b87f0bbf71b45ee5a48febb?d=identicon&s=25 Dave Burt (Guest)
on 2006-05-26 08:47
(Received via mailing list)
On 5/25/06, Victor Shepelev wrote:
> [skip]
>
> John, it's really cool!
> May be (as I've mentioned above) we can create "profile hacks" project on
> rubyforge?

Can you call it "profiling" rather than "profile"? You could even
create such a project on RubyGarden instead of RubyForge initially.
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-05-26 08:53
(Received via mailing list)
From: dave.burt@gmail.com [mailto:dave.burt@gmail.com] On Behalf Of Dave
Burt
Sent: Friday, May 26, 2006 9:45 AM
> On 5/25/06, Victor Shepelev wrote:
> > [skip]
> >
> > John, it's really cool!
> > May be (as I've mentioned above) we can create "profile hacks" project
> on
> > rubyforge?
>
> Can you call it "profiling" rather than "profile"?

Thanks. And again, sorry for my English.

> You could even
> create such a project on RubyGarden instead of RubyForge initially.

Yes, makes sense. I'm planning to begin at Monday.

V.
0b561a629b87f0bbf71b45ee5a48febb?d=identicon&s=25 Dave Burt (Guest)
on 2006-05-26 09:17
(Received via mailing list)
Victor Shepelev wrote:
> From: Dave Burt
> Sent: Friday, May 26, 2006 9:45 AM
> > Can you call it "profiling" rather than "profile"?
>
> Thanks. And again, sorry for my English.

No problem with your English, profile is fine as a verb. It's just
mildly confusing because profile is often used in computing as a noun,
so if you see "profile", you assume it's talking about a profile
rather than to profile.

Cheers,
Dave
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2006-05-26 10:19
(Received via mailing list)
2006/5/25, Victor Shepelev <vshepelev@imho.com.ua>:

> > Note that a similar thing can be achieved with set_trace_func
>
> Drawbacks of set_trace_func was discussed in the original topic here:
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

Yes, but the other solution is more intrusive and will break if the
method is redefined.  If the sole purpose is counting of invocations
than I'd personally prefer the approach with set_trace_func.  Just my
0.02EUR...

Kind regards

robert
12271b6df73fe29930d65586be5a4a70?d=identicon&s=25 Dave Howell (Guest)
on 2006-06-01 21:38
(Received via mailing list)
"Boy!" I thought. "Now THIS sounds like a thread I want to read!"

The aforementioned "I" in this case being a hobbyist programmer
using/learning Ruby 1.8.4 on OSX 10.3 (w/Developer Tools installed, of
course)...

On May 23, 2006, at 15:10, John Carter wrote:

>  * Always benchmark first, profile and optimize later. Otherwise you
> don't
>    know whether it was worth it. (Hint: Often the tweaks aren't worth
> it, or
>    make things worse)

OK, that makes sense. Not that I have any idea *how* to benchmark my
programs, but I'll keep reading...

>  * Look at the top most items in the profile list. Usually they are
>    standard library functions. Silly things like [] or +...
>    Ask your self, "What could be calling these thing so often?
>    Can I make it do it less often? eg. Cache the result etc.)

Er, the profile list? Push "Where do I find/how do I generate a
'profile list'?" onto the question_stack

>  * Scan down the list for the first few user written methods. Usually
> the one
>    that you spent the most time in has the most fat to trim.

I understand that. Yay!
comprehension_counter = 1

>  * Scan the list for the routine that took the longest (as opposed
>    to the most time spent in it.) Try get that one to just do less.

I don't understand this. Took the longest != most time spent in it? But
what if it does the most work? If I make it do less, don't I have to
make something else do more?
question_stack << "huh? [Parse Error]"

>  * Use strace, see what the system is doing at an OS level. I once
>    improved another persons program (professional in production) by a
>    factor of 40, yes FORTY times, by noting it was doing an lseek on
> every
>    record read and changing that to an mmap.

question_stack << "[Error: File 'strace' Not Found]"

>  * Always look out for surprises. The best moment in an optimizers
> life is
>    when you look at something and say "WTF! Why is _that_ routine
> taking so
>    long!?"

class Fixnum
	def increment
		self = self + 1
	end
end

comprehension_counter.increment

>  * Large number of calls to .size and .length and .count_objects
> methods are a
>    clue some nidjit somewhere is doing something really N^2 stupid
> like...
>     for( i=0; i < container.count_objects(); i++) {
>     }

question_stack << "What exactly is stupid about that? And what's the
smart alternative? [Error: Unknown Language] [Error: C Allergy
Triggered] [Trap: C Antihistamine Enabled]"

>  * Use top, vmstat, times as well.
>     Large usertime means optimize program.
>     Large system time means you really hammering the system calls (that
>     was a clue in the lseek case I mentioned above)
>     Large real time small user and system means you are waiting for
>     input from user / disk / network /...? Find out what.
>     Watch the page in page out stats. Watch the swap in swap out stats.
>     If you are paging or swapping, time to switch to memory consumption
>     optimization!

question_stack << "My 'top' doesn't have 'usertime', 'system time' or
'realtime' values. Is this only for special kinds of top? [Error:
Potential OS-specific or otherwise specialized tools]"

question_stack << "[Error: File 'vmstat' Not Found]"

question_stack << "[Error: File 'times' Not Found]"

>   * Use my memprofile /  newprofile tricks to try spot the
>     http://wiki.rubygarden.org/Ruby/page/show/NewProfiler
>     http://rubyforge.org/snippet/detail.php?type=snipp...

question_stack << "I found them, but if I use them, what are they going
to tell me, and what kind of actions would I want to take in response?
[Error: Documentation Not Found]"

Fatal Error: Question Stack Overflow

Fatal Error: Clue Underrun. Initial Clue supply inadequate. Increase
Clues and re-run.

Fatal Error: Nuby Page Fault  Fatal Error: Nuby Chapter Fault  Fatal
Error: Nuby Volume Fault  Fatal Error: San Andreas Fault   Fatal Error:
Nobody's Fault. Please contact your system administrator to resolve the
problem. If you are the System Administrator, I guess you're screwed.
Ha. Ha.
0ca6e5c33d7e7ff901d75ff0b13d9e1c?d=identicon&s=25 Sam Roberts (Guest)
on 2006-06-01 22:19
(Received via mailing list)
On Fri, Jun 02, 2006 at 04:35:28AM +0900, Dave Howell wrote:
> > * Look at the top most items in the profile list. Usually they are
> >   standard library functions. Silly things like [] or +...
> >   Ask your self, "What could be calling these thing so often?
> >   Can I make it do it less often? eg. Cache the result etc.)
>
> Er, the profile list? Push "Where do I find/how do I generate a
> 'profile list'?" onto the question_stack

'profile', in the std lib

> question_stack << "[Error: File 'strace' Not Found]"

ktrace on BSD systems

> > * Large number of calls to .size and .length and .count_objects
> >methods are a
> >   clue some nidjit somewhere is doing something really N^2 stupid
> >like...
> >    for( i=0; i < container.count_objects(); i++) {
> >    }
>
> question_stack << "What exactly is stupid about that? And what's the
> smart alternative?
max=container.count_objects()
for( i=0; i < max; i++) {
}

container.each....

Sam
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-06-01 22:25
(Received via mailing list)
From: Dave Howell [mailto:groups@grandfenwick.net]
Sent: Thursday, June 01, 2006 10:35 PM
> To: ruby-talk ML
> Subject: Re: How use Profiling, a Quick Guide.
>
> Fatal Error: Nuby Page Fault  Fatal Error: Nuby Chapter Fault  Fatal
> Error: Nuby Volume Fault  Fatal Error: San Andreas Fault   Fatal Error:
> Nobody's Fault. Please contact your system administrator to resolve the
> problem. If you are the System Administrator, I guess you're screwed.
> Ha. Ha.

Most of your questions would go away after reading that:
http://ruby-doc.org/stdlib/libdoc/benchmark/rdoc/index.html
http://ruby-doc.org/docs/ProgrammingRuby/html/trou...
http://en.wikipedia.org/wiki/Strace
http://en.wikipedia.org/wiki/Top_(Unix)
http://en.wikipedia.org/wiki/Vmstat

V.
12271b6df73fe29930d65586be5a4a70?d=identicon&s=25 Dave Howell (Guest)
on 2006-06-01 22:38
(Received via mailing list)
On Jun 1, 2006, at 13:17, Sam Roberts wrote:

>> Er, the profile list? Push "Where do I find/how do I generate a
>> 'profile list'?" onto the question_stack
>
> 'profile', in the std lib

Alas, you have still underestimated how clueless I am. 'the std lib'?
Is that a Ruby standard library? A Unix standard library? And if I find
this library, what do I do with it? How would I invoke profile-omancy?

>> question_stack << "[Error: File 'strace' Not Found]"
>
> ktrace on BSD systems

Oo! My command line liked that much better. The man page even has
examples. Whether or not I can figure out what it's telling me, and if
so, what I'm supposed to do about it, remains to be seen, but it's
certainly a start.

> for( i=0; i < max; i++) {
> }
>
> container.each....

OK, so "container.each" is presumably the smart alternative, and what
I'd instinctively use anyway, but I don't understand what it is about
"for( i=0; i < max; i++)" that makes it "really n^2 stupid." (If n, or
N, = 1, then that wouldn't be all that stupid, but the use of the term
"nidjit" leads me to guess n is likely to be at least the age of said
nidjit, or some other suitably > 1 value. :)
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-06-01 22:51
(Received via mailing list)
From: Dave Howell [mailto:groups@grandfenwick.net]
Sent: Thursday, June 01, 2006 11:36 PM
> > for( i=0; i < max; i++) {
> > }
> >
> > container.each....
>
> OK, so "container.each" is presumably the smart alternative, and what
> I'd instinctively use anyway, but I don't understand what it is about
> "for( i=0; i < max; i++)" that makes it "really n^2 stupid."

The point was: if you use some value repeatedly, it is generally smarter
to
pre-calculate it.

Example:

def get_some_value
#some long calculation
end

(1..100).each{|i| puts i * get_some_value}

The above code would call get_some_value 100 times. If it rans slowly
and
would return 100 same values, the much more smarter way would be

v = get_some_value
(1..100).each{|i| puts i * v}

V.
12271b6df73fe29930d65586be5a4a70?d=identicon&s=25 Dave Howell (Guest)
on 2006-06-01 22:57
(Received via mailing list)
On Jun 1, 2006, at 13:47, Victor Shepelev wrote:

>>>> smart alternative?
> The point was: if you use some value repeatedly, it is generally
> smarter to
> pre-calculate it.

{brief scratching of head}

Oh! So BOTH of Sam's examples were 'non-stupid' rewrites of the initial
code. I completely failed to recognize the 'for' loop as being Ruby
code in the first place. Does Ruby really have an ++ operator? I missed
that somehow. Damn. As embarrassingly illustrated by the fact that I
coded my own ".increment" operator in the initial message instead of
using it.

my_clues++
E34b5cae57e0dd170114dba444e37852?d=identicon&s=25 Logan Capaldo (Guest)
on 2006-06-01 23:10
(Received via mailing list)
On Jun 1, 2006, at 4:55 PM, Dave Howell wrote:

> Oh! So BOTH of Sam's examples were 'non-stupid' rewrites of the
> initial code. I completely failed to recognize the 'for' loop as
> being Ruby code in the first place. Does Ruby really have an ++
> operator? I missed that somehow. Damn. As embarrassingly
> illustrated by the fact that I coded my own ".increment" operator
> in the initial message instead of using it.

Ruby does not have an increment operator (++). It does have #succ but
that is not in place.

as far as your increment goes:

% cat increment.rb
class Fixnum
   def increment
     self = self + 1
   end
end

a = 1
a.increment


% ruby increment.rb
-:13: Can't change the value of self
     self = self + 1
           ^
12271b6df73fe29930d65586be5a4a70?d=identicon&s=25 Dave Howell (Guest)
on 2006-06-01 23:19
(Received via mailing list)
On Jun 1, 2006, at 14:07, Logan Capaldo wrote:

>
> On Jun 1, 2006, at 4:55 PM, Dave Howell wrote:

> Ruby does not have an increment operator (++). It does have #succ but
> that is not in place.

Aha. So it *wasn't* Ruby code, and my initial confusion was justified.
Well, at least I understand it now. :)

> as far as your increment goes:
[snip]
> -:13: Can't change the value of self
>     self = self + 1
>           ^

Dang.

[thinks]

Nope. Can't figure this one out. Time to start a new thread.
2c7c807a1df0c76a8fc823c709b501a9?d=identicon&s=25 Victor Shepelev (Guest)
on 2006-06-01 23:42
(Received via mailing list)
From: Dave Howell [mailto:groups@grandfenwick.net]
Sent: Thursday, June 01, 2006 11:55 PM
> code. I completely failed to recognize the 'for' loop as being Ruby
> code in the first place.
No, it was something C-like.

> Does Ruby really have an ++ operator?
Nope. You can use +=1 instead.

V.
12271b6df73fe29930d65586be5a4a70?d=identicon&s=25 Dave Howell (Guest)
on 2006-06-02 01:46
(Received via mailing list)
On Jun 1, 2006, at 13:35, Dave Howell wrote:

> Alas, you have still underestimated how clueless I am. 'the std lib'?
> Is that a Ruby standard library? A Unix standard library? And if I
> find this library, what do I do with it? How would I invoke
> profile-omancy?

In case anybody else was wondering the same thing, the answer can be
found, among other places, in "Programming Ruby" aka the Pickaxe
manual, and starts with putting

	require 'profile'

in your Ruby code. I'll just go write a quick script to print "RTFM"
1000 times now....
12271b6df73fe29930d65586be5a4a70?d=identicon&s=25 Dave Howell (Guest)
on 2006-06-02 01:52
(Received via mailing list)
On Jun 1, 2006, at 13:25, Victor Shepelev wrote:

> Most of your questions would go away after reading that:
> http://ruby-doc.org/stdlib/libdoc/benchmark/rdoc/index.html

Hmm. OK, that explains how to benchmark, and I think explains what the
"std lib" is....

> http://en.wikipedia.org/wiki/Vmstat

Hmm again. The example shows a "CPU" column with subcolumns of "cs"
"us" "sy" "id" and "wa", which I might guess include "us" for user and
"sy" for system. It doesn't say, and apparently my Unix doesn't have
it.

Still, that makes the *real* question "How come I don't have vmstat?"
and that's a question for some other forum.

> http://ruby-doc.org/docs/ProgrammingRuby/html/trou...

Well, now, that's just embarrassing. RTFM, indeed.
Pop "Where do I find/how do I generate a 'profile list'?" off
question_stack

Thank you very much, Victor.
This topic is locked and can not be replied to.