Symbol.to_proc is way slower than invoking methods directly


#1

All,

I was recently introduced to the Symbol.to_proc trick where you can
invoke methods on collection elements in a map call with less syntax, as
in:

[“1”, “2”, “3”].map(&:to_i)

While this is very clever looking (albeit potentially harder to read for
Ruby/Rails noobs, which implies a human performance cost), I started
thinking about it and decided this had to be slower than calling methods
directly, as in:

[“1”, “2”, “3”].map {|x| x.to_i}

So I decided to benchmark it to see. My benchmark test shows that
calling the method through the Symbol.to_proc method (at least in this
case) is 8.5X slower than doing the more readable method invocation.

My benchmark code is below. Is it a valid test? And if so, why should
we use Symbol.to_proc unnecessarily when it is such a poor performer?

Thanks,
Wes

==========benchmark code============

#!/usr/bin/ruby

require ‘benchmark’
include Benchmark

LOOP_COUNT = 1_000_000
x = [“1”,“2”,“3”]
bmbm do |test|
test.report(“Method invoke”) do
LOOP_COUNT.times {x.map {|elem| elem.to_i}}
end

test.report(“Symbol.to_proc”) do
LOOP_COUNT.times {x.map(&:to_i)}
end
end
~


#2

One more thing - here’s how I ran it:

$ script/runner -e development benchmark_to_proc.rb Rehearsal

Method invoke 1.600000 0.000000 1.600000 ( 1.783758)
Symbol.to_proc 13.240000 0.070000 13.310000 ( 13.770514)
---------------------------------------- total: 14.910000sec

                 user     system      total        real

Method invoke 1.590000 0.010000 1.600000 ( 1.628473)
Symbol.to_proc 13.210000 0.060000 13.270000 ( 13.678768)


#3

Hi –

On Tue, 29 Jul 2008, Wes G. wrote:

Ruby/Rails noobs, which implies a human performance cost), I started
we use Symbol.to_proc unnecessarily when it is such a poor performer?
The good news, I guess, is that it seems to be only about twice as
slow in Ruby 1.9:

$ ruby19 to_proc.rb
Rehearsal --------------------------------------------------
Method invoke 1.480000 0.010000 1.490000 ( 1.498664)
Symbol.to_proc 2.640000 0.010000 2.650000 ( 2.656658)
----------------------------------------- total: 4.140000sec

                  user     system      total        real

Method invoke 1.470000 0.010000 1.480000 ( 1.483069)
Symbol.to_proc 2.640000 0.010000 2.650000 ( 2.668746)

$ ruby19 -v
ruby 1.9.0 (2008-07-20 revision 16244) [i686-darwin9.3.0]

David


Rails training from David A. Black and Ruby Power and Light:

  • Advancing With Rails August 18-21 Edison, NJ
  • Co-taught by D.A. Black and Erik Kastner
    See http://www.rubypal.com for details and updates!

#4

Still, twice as slow isn’t really acceptable.

Since I was told about this I’ve stopped using symbol to proc in
favour of typing it out. It may use a little bit more time, but that’s
time that my users won’t spend waiting for the page to process.


#5

Hi –

On Tue, 29 Jul 2008, Ryan B. wrote:

Still, twice as slow isn’t really acceptable.

I’m not thrilled either. I’m indifferent to the idiom itself, so if
there’s much of an admission price I’m unlikely to use it. We’ll see
how it plays out in further versions of 1.9.

David


Rails training from David A. Black and Ruby Power and Light:

  • Advancing With Rails August 18-21 Edison, NJ
  • Co-taught by D.A. Black and Erik Kastner
    See http://www.rubypal.com for details and updates!

#6

Hi –

On Mon, 28 Jul 2008, Phlip wrote:

Wes G. wrote:

My benchmark code is below. Is it a valid test? And if so, why should
we use Symbol.to_proc unnecessarily when it is such a poor performer?

You seem to have reproduced the research of Matz & Co. The next version of Ruby
will implement Symbol.to_proc in C - turning it from a hack into a /de-facto/
keyword.

See my 1.9.0 benchmarks, a few posts back. It’s still (as of then) a
good bit slower than the block version, at least in this test.

David


Rails training from David A. Black and Ruby Power and Light:

  • Advancing With Rails August 18-21 Edison, NJ
  • Co-taught by D.A. Black and Erik Kastner
    See http://www.rubypal.com for details and updates!

#7

Wes G. wrote:

My benchmark code is below. Is it a valid test? And if so, why should
we use Symbol.to_proc unnecessarily when it is such a poor performer?

You seem to have reproduced the research of Matz & Co. The next version
of Ruby
will implement Symbol.to_proc in C - turning it from a hack into a
/de-facto/
keyword.


Phlip


#8

Phlip wrote:

Do you have a guess why? It seems both expressions could resolve to the
same
opcodes…

Isn’t it as simply as two more method calls in the &:to_i case? One
call to Symbol.to_proc and another call to Object.send, in addition to
the actual method call that you want.

When I look at the &: idiom, the only good reason I can think of to use
it would be if you truly needed to invoke a method whose name you didn’t
know until runtime. Even then, I suspect that &: would still be slower,
again, because it inserts another (unnecessary) method call. In the
case of a truly dynamic method invocation, you would probably just use
Object.send, but with &:, you still have to do the call to
Symbol.to_proc before that.

This has been a good way to get me thinking about the potential cost of
the syntactic sugar that we may exploit all the time in Rails.

Thanks,
Wes


#9

David A. Black wrote:

See my 1.9.0 benchmarks, a few posts back. It’s still (as of then) a
good bit slower than the block version, at least in this test.

Do you have a guess why? It seems both expressions could resolve to the
same
opcodes…


Phlip


#10

Whenever you make a design decision there will be tradeoffs.
Symbol#to_proc trades an increase in readability for a decrease in
performance. It’s good to be aware of this when using it. On the other
hand, there are many things in your Rails app that will be orders of
magnitude slower than your uses of Symbol#to_proc, so I wouldn’t
necessarily reject it out of hand on performance grounds alone.

If you’re writing lower level libraries, I would definitely avoid
using Symbol#to_proc. For a Rails app, trying to eek out milliseconds
by replacing Symbol#to_proc with a block is probably a poor
optimization.

Rein


#11

Rein H. wrote:

Whenever you make a design decision there will be tradeoffs.
Symbol#to_proc trades an increase in readability for a decrease in
performance.

That’s interesting. I think that &: is way less readable than the
“regular” block syntax.

WG


#12

Wes G. wrote:

When I look at the &: idiom, the only good reason I can think of to use
it would be if you truly needed to invoke a method whose name you didn’t
know until runtime.

Premature optimization is the root of all evil. We cache both our
database hits
and our web pages hits. We value development time enough to not worry
about the
microseconds of each statement we write, so long as they save
mega-seconds of
coding time.

Even then, I suspect that &: would still be slower,
again, because it inserts another (unnecessary) method call. In the
case of a truly dynamic method invocation, you would probably just use
Object.send, but with &:, you still have to do the call to
Symbol.to_proc before that.

I can’t see a problem with making &: into a single operator. Thats how
constructs like x[0] ||= 0 work - the entire sequence from the [ to the
= goes
into an operator, with x, 0, and 0 as arguments.


Phlip


#13

Hi –

On Tue, 29 Jul 2008, Phlip wrote:

coding time.
Not all optimization is premature, though, and some is sort of
incidental. Every time you write code one way, it’s a decision not to
write it in n other possible ways. It’s even true that if I write:

s = “string”

I’ve chosen not to write it as:

s = “#{sleep 10; nil}string”

Of course, you could say that doing it that second way would be dumb
and so on. But if you happen to know that =~ is faster than
Regexp#match, or that a block is faster than Symbol#to_proc, then the
choice of the faster one becomes kind of instantaneous. (Unless of
course you have some specific reason to choose the other, like wanting
the MatchData object or wanting to squeeze your code down.)

In other words, we’re always making these choices, and if the way you
choose happens to run faster, that’s not necessarily the root of any
evil :slight_smile:

Even then, I suspect that &: would still be slower,
again, because it inserts another (unnecessary) method call. In the
case of a truly dynamic method invocation, you would probably just use
Object.send, but with &:, you still have to do the call to
Symbol.to_proc before that.

I can’t see a problem with making &: into a single operator. Thats how
constructs like x[0] ||= 0 work - the entire sequence from the [ to the = goes
into an operator, with x, 0, and 0 as arguments.

The &obj thing is more general, though. I guess it could be
special-cased for symbols, though then that means weird things like
that you wouldn’t be able to override #to_proc for a symbol (which
might not matter often but isn’t ideal).

David


Rails training from David A. Black and Ruby Power and Light:

  • Advancing With Rails August 18-21 Edison, NJ
  • Co-taught by D.A. Black and Erik Kastner
    See http://www.rubypal.com for details and updates!

#14

Hi –

On Tue, 29 Jul 2008, Wes G. wrote:

Phlip wrote:

Do you have a guess why? It seems both expressions could resolve to the
same
opcodes…

Isn’t it as simply as two more method calls in the &:to_i case? One
call to Symbol.to_proc and another call to Object.send, in addition to
the actual method call that you want.

I believe that’s right (though I haven’t dug into it). Basically, & is
now a unary operator that calls #to_proc on any object that implements
it:

david-blacks-macbook:~ dblack$ irb19 --simple-prompt

obj = Object.new
=> #Object:0x3b94f8

def obj.to_proc; Proc.new { puts “Hi!” }; end
=> nil

[1].each(&obj)
Hi!
=> [1]

David


Rails training from David A. Black and Ruby Power and Light:

  • Advancing With Rails August 18-21 Edison, NJ
  • Co-taught by D.A. Black and Erik Kastner
    See http://www.rubypal.com for details and updates!

#15

Enrico Thierbach wrote:

Besides, I found a hack that would decrease the runtime overhead of
using Symbol#to_proc, which is to cache the Proc object inside the
symbol:

class Symbol
def to_proc
@to_proc ||= Proc.new { |*args| args.shift.send(self, *args) }
end
end

With this I get runtimes only two times the original ones. But I still
don’t know if I should consider that one safe. Do you have any idea on
that?

I think it’s a good optimization. I still don’t like the idea of
slowing down stuff any more than necessary, no matter how you do it.

Wes


#16

Wes G. wrote:

All,

I was recently introduced to the Symbol.to_proc trick where you can
invoke methods on collection elements in a map call with less syntax, as
in:

[“1”, “2”, “3”].map(&:to_i)

While this is very clever looking (albeit potentially harder to read for
Ruby/Rails noobs, which implies a human performance cost), I started
thinking about it and decided this had to be slower than calling methods
directly, as in:

[“1”, “2”, “3”].map {|x| x.to_i}

So I decided to benchmark it to see. My benchmark test shows that
calling the method through the Symbol.to_proc method (at least in this
case) is 8.5X slower than doing the more readable method invocation.

My benchmark code is below. Is it a valid test? And if so, why should
we use Symbol.to_proc unnecessarily when it is such a poor performer?

Thanks,
Wes

==========benchmark code============

#!/usr/bin/ruby

require ‘benchmark’
include Benchmark

LOOP_COUNT = 1_000_000
x = [“1”,“2”,“3”]
bmbm do |test|
test.report(“Method invoke”) do
LOOP_COUNT.times {x.map {|elem| elem.to_i}}
end

test.report(“Symbol.to_proc”) do
LOOP_COUNT.times {x.map(&:to_i)}
end
end
~

Hey wes,

you are testing “invoking explicit blocks” vs “creating a temporary Proc
object and invoking via that”. What is missing here would be “create a
proc once and invoke via that.” This would be the more real-life
approach, because you usually invoke a Proc/block/whatever on a
structure with more than three entries.

Besides, I found a hack that would decrease the runtime overhead of
using Symbol#to_proc, which is to cache the Proc object inside the
symbol:

class Symbol
def to_proc
@to_proc ||= Proc.new { |*args| args.shift.send(self, *args) }
end
end

With this I get runtimes only two times the original ones. But I still
don’t know if I should consider that one safe. Do you have any idea on
that?

/eno

I have a post on my blog that goes into that in more detail:
http://1rad.wordpress.com/2008/11/10/0x0a-some-optimization-hacks/