Some optimization hacks: your opinion?

Rails
1

Hi everyone,

I found and tried two optimization hacks; however, I don’t feel too
comfortable with them. Therefore I would appreciate your opinion:

  • Caching proc objects, i.e.

    class Symbol
    def to_proc
    @to_proc ||= Proc.new { |*args| args.shift.send(self, *args) }
    end
    end

  • and disabling the GC during an action:

    around_filter do |controller, action|
    GC.disable
    begin
    action.call
    ensure
    GC.enable
    GC.start
    end
    end

On some of our machines and with some requests they boost performance
by as much as 50%. however, as I don’t know if I shuld feel ssave with
them I would like to hear your opinions.

BTW: A longer version of this text, with some explanation, is here:

/eno

2

Some questions about disabling GC:

  • Memory consuming action - will it kill server?

  • Very small (not memory consuming) action - won’t GC.start slow it
    down significantly? (it’s called every time)

  • Lets assume that one of your libraries used by the action is calling
    GC.start explicitly - the library “knows” it consumes a lot of memory
    and tries to free it in the right time - what will happen?

These are just some random questions about “bad” scenarios.

On Nov 12, 6:02 pm, “Enrico Thierbach”

3

Hi reHa,

On Wed, Nov 12, 2008 at 9:49 PM, reHa [email protected] wrote:

Some questions about disabling GC:

  • Memory consuming action - will it kill server?

  • Lets assume that one of your libraries used by the action is calling
    GC.start explicitly - the library “knows” it consumes a lot of memory
    and tries to free it in the right time - what will happen?

Hell, for sure those are dangerous situations. But then I guess I
would know about the memory consumption of my app, or measure it in a
high usage scenario.

I guess this is an issue for all apps, GC running or not: If you are
running some code that needs much RAM all at the same time (so even
the GC kicking in couldn’t help) this can already kill your server.
(And I have seen ActiveRecord queries taking up up to somewhere close
to 1 GByte) So you should always know what memory requirements your
app has. And if you are know that and know that you are on the right
side of the tracks, then, I guess, you should be fine.

In our app we have quite a strict upper limit on how many objects we
have to deal with, so I guess I might feel safe :slight_smile:

  • Very small (not memory consuming) action - won’t GC.start slow it
    down significantly? (it’s called every time)

Yes, certainly. On my developer machine a GC run takes ~200msecs,
which would turn fast actions in pretty slow ones :slight_smile: So I would like
to have the GC run somewhere in the rails kernel - after the
response would be sent back to the client. But that, of course, can
only be done by patching rails. And of course I wouldn’t run that code
on all actions, only on some anyways.

These are just some random questions about “bad” scenarios.

Thanks for your response. Did anyone use techniques like that before?

/eno

A wee piece of ruby every monday: http://1rad.wordpress.com/

4

Hi philip,

restart (hence the need to passing a lambda into your namedscope).
as far as I can see, my code would never be called in those
circumstances, as 5.days doesn’t evaluate to a symbol, does it?

Comparing Rails’ original implementation

def to_proc
Proc.new { |*args| args.shift.send(self, *args) }
end

with the improved(?) one:

def to_proc
@to_proc ||= Proc.new { |*args| args.shift.send(self, *args) }
end

you have the same values “going” into the Proc, namely the symbol
(i.e. self). What I don’t know The question is more if there is
something “invisible” going on in there?

Speaking of questions: does anyone know in what circumstances the
*args array can hold any values at all?

regards,
/eno

5

would know about the memory consumption of my app, or measure it in a
In our app we have quite a strict upper limit on how many objects we
have to deal with, so I guess I might feel safe :slight_smile:

Be careful here… I wrote an app awhile back that used the
ultraviolet syntax highlighter. Didn’t even occur to me it might chew
up a lot of ram and in all my usage it didn’t. Until I sent it a
couple pages of horribly formatted html and it ballooned into a couple
hundred megs. Kind of surprised me.

Point being… just be sure to double check your assumptions about
memory usage… everyone knows to take another look at imagemagick,
but some others suffer in odd cases as well…

Also, as to the Proc enhancement, I haven’t thought about it and my
head hurts…

But what happens if one of the arguments is say Time.now? Will that
get evaluated once or each time? That is, will you have the same
issue you’d have if you wrote:

named_scope :foo, :conditions => [“created_at >= ?”, 5.days.ago]

Where 5.days.ago gets evaulated once and never changes until you
restart (hence the need to passing a lambda into your namedscope).

-philip

6

On Nov 13, 10:31 am, “Enrico Thierbach”
[email protected] wrote:

def to_proc
Proc.new { |*args| args.shift.send(self, *args) }
end

with the improved(?) one:

def to_proc
@to_proc ||= Proc.new { |*args| args.shift.send(self, *args) }
end

This is interesting.
I can’t remember quite what it is that makes .map(&:name) slower
than .map{|o| o.name}

But I have a feeling that it was to do with the cost of instantiating
a new Proc object each time.

With your solution,
I’d be interested in seeing some benchmarks.

7

Sure: there you go, in a somewhat :slight_smile: clinical testcase.

----------------- snip -----------------
require “benchmark”

CNT=200000

Benchmark.bm { |x|
x.report “V1” do
(1…CNT).each do [1].each { |i| i.to_i } end
end
}

class Symbol
def to_proc; Proc.new { |*args| args.shift.send(self, *args) };
end
end

Benchmark.bm { |x|
x.report “V2” do
(1…CNT).each do [].each (&:to_i) end
end
}

class Symbol
def to_proc; Proc.new { |obj| obj.send(self) }; end
end

Benchmark.bm { |x|
x.report “V3” do
(1…CNT).each do [1].each (&:to_i) end
end
}

class Symbol
def to_proc; @proc ||= Proc.new { |*args| args.shift.send(self,
*args) }; end
end

Benchmark.bm { |x|
x.report “V4” do
(1…CNT).each do [1].each (&:to_i) end
end
}
----------------- snap ----------------

The result is on my machine as follows:

  user     system      total        real

V1 0.200000 0.000000 0.200000 ( 0.201815)
user system total real
V2 0.860000 0.000000 0.860000 ( 0.870993)
user system total real
V3 0.950000 0.000000 0.950000 ( 0.953523)
user system total real
V4 0.680000 0.010000 0.690000 ( 0.685361)

The fastest, V1, doesn’t build proc objects at all. V2 is rails’
default implementation, V4 caches the Proc objects inside the symbol,
i.e. generates the Proc object only.

Of course, invoking 200000 Proc objects this way is usually not what
you are doing in any application; we, however, had a test case where
we did it 9000 times per action and it matched badly with the garbage
collector. Ruby enterprise behaved much much better :slight_smile:

/eno