Forum: Ruby on Rails Some optimization hacks: your opinion?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
27127e68480c1fbf62e6d98a331c3fc6?d=identicon&s=25 Enrico Thierbach (Guest)
on 2008-11-12 19:14
(Received via mailing list)
Hi everyone,

I found and tried two optimization hacks; however, I don't feel too
comfortable with them. Therefore I would appreciate your opinion:

- Caching proc objects, i.e.

  class Symbol
    def to_proc
      @to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) }
    end
  end

- and disabling the GC during an action:

  around_filter do |controller, action|
    GC.disable
    begin
      action.call
    ensure
      GC.enable
      GC.start
    end
  end

On some of our machines and with some requests they boost performance
by as much as 50%. however, as I don't know if I shuld feel ssave with
them I would like to hear your opinions.

BTW: A longer version of this text, with some explanation, is here:
http://1rad.wordpress.com/2008/11/10/0x0a-some-opt...

/eno
5acf8e6be07793d22e24d8eb24a0e043?d=identicon&s=25 reHa (Guest)
on 2008-11-12 21:50
(Received via mailing list)
Some questions about disabling GC:

* Memory consuming action - will it kill server?

* Very small (not memory consuming) action - won't GC.start slow it
down significantly? (it's called every time)

* Lets assume that one of your libraries used by the action is calling
GC.start explicitly - the library "knows" it consumes a lot of memory
and tries to free it in the right time - what will happen?


These are just some random questions about "bad" scenarios.


On Nov 12, 6:02 pm, "Enrico Thierbach"
27127e68480c1fbf62e6d98a331c3fc6?d=identicon&s=25 Enrico Thierbach (Guest)
on 2008-11-12 23:43
(Received via mailing list)
Hi reHa,

On Wed, Nov 12, 2008 at 9:49 PM, reHa <preszke@gmail.com> wrote:
>
> Some questions about disabling GC:
>
> * Memory consuming action - will it kill server?
>
> * Lets assume that one of your libraries used by the action is calling
> GC.start explicitly - the library "knows" it consumes a lot of memory
> and tries to free it in the right time - what will happen?

Hell, for sure those are dangerous situations. But then I guess I
would know about the memory consumption of my app, or measure it in a
high usage scenario.

I guess this is an issue for all apps, GC running or not: If you are
running some code that needs much RAM all at the same time (so even
the GC kicking in couldn't help) this can already kill your server.
(And I have seen ActiveRecord queries taking up up to somewhere close
to 1 GByte) So you should always know what memory requirements your
app has. And if you are know that and know that you are on the right
side of the tracks, then, I guess, you should be fine.

In our app we have quite a strict upper limit on how many objects we
have to deal with, so I guess I might feel safe :)

> * Very small (not memory consuming) action - won't GC.start slow it
> down significantly? (it's called every time)

Yes, certainly. On my developer machine a GC run takes ~200msecs,
which would turn fast actions in pretty slow ones :) So I would like
to have the GC run somewhere in the rails kernel - *after* the
response would be sent back to the client. But that, of course, can
only be done by patching rails. And of course I wouldn't run that code
on all actions, only on some anyways.

> These are just some random questions about "bad" scenarios.

Thanks for your response. Did anyone use techniques like that before?

/eno
====================================================================
A wee piece of ruby every monday: http://1rad.wordpress.com/
2505b282d57c29be797dc35b245adb4c?d=identicon&s=25 Philip Hallstrom (Guest)
on 2008-11-13 00:19
(Received via mailing list)
> would know about the memory consumption of my app, or measure it in a
> In our app we have quite a strict upper limit on how many objects we
> have to deal with, so I guess I might feel safe :)

Be careful here... I wrote an app awhile back that used the
ultraviolet syntax highlighter.  Didn't even occur to me it might chew
up a lot of ram and in all my usage it didn't.  Until I sent it a
couple pages of horribly formatted html and it ballooned into a couple
hundred megs.  Kind of surprised me.

Point being.. just be sure to double check your assumptions about
memory usage... everyone knows to take another look at imagemagick,
but some others suffer in odd cases as well...

Also, as to the Proc enhancement, I haven't thought about it and my
head hurts...

But what happens if one of the arguments is say Time.now?  Will that
get evaluated *once* or each time?  That is, will you have the same
issue you'd have if you wrote:

named_scope :foo, :conditions => ["created_at >= ?", 5.days.ago]

Where 5.days.ago gets evaulated *once* and never changes until you
restart (hence the need to passing a lambda into your namedscope).

-philip
27127e68480c1fbf62e6d98a331c3fc6?d=identicon&s=25 Enrico Thierbach (Guest)
on 2008-11-13 11:38
(Received via mailing list)
Hi philip,

> restart (hence the need to passing a lambda into your namedscope).
as far as I can see, my code would never be called in those
circumstances, as 5.days doesn't evaluate to a symbol, does it?

Comparing Rails' original implementation

def to_proc
  Proc.new { |*args| args.shift.__send__(self, *args) }
end

with the improved(?) one:

def to_proc
  @to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) }
end

you have the same values "going" into the Proc, namely the symbol
(i.e. self). What I don't know The question is more if there is
something "invisible" going on in there?

Speaking of questions: does anyone know in what circumstances the
*args array can hold any values at all?

regards,
/eno
E60b2dc57668b5662ce3f07781e41710?d=identicon&s=25 Matthew Rudy Jacobs (matthewrudy)
on 2008-11-13 14:41
(Received via mailing list)
On Nov 13, 10:31 am, "Enrico Thierbach"
<enrico.thierb...@googlemail.com> wrote:

> def to_proc
>   Proc.new { |*args| args.shift.__send__(self, *args) }
> end
>
> with the improved(?) one:
>
> def to_proc
>   @to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) }
> end

This is interesting.
I can't remember quite what it is that makes .map(&:name) slower
than .map{|o| o.name}

But I have a feeling that it was to do with the cost of instantiating
a new Proc object each time.

With your solution,
I'd be interested in seeing some benchmarks.
27127e68480c1fbf62e6d98a331c3fc6?d=identicon&s=25 Enrico Thierbach (Guest)
on 2008-11-13 16:48
(Received via mailing list)
Sure: there you go, in a somewhat :) clinical testcase.

----------------- snip -----------------
require "benchmark"

CNT=200000

Benchmark.bm { |x|
  x.report "V1" do
    (1..CNT).each do [1].each { |i| i.to_i } end
  end
}

class Symbol
  def to_proc; Proc.new { |*args| args.shift.__send__(self, *args) };
end
end

Benchmark.bm { |x|
  x.report "V2" do
    (1..CNT).each do [].each (&:to_i) end
  end
}

class Symbol
  def to_proc; Proc.new { |obj| obj.__send__(self) }; end
end

Benchmark.bm { |x|
  x.report "V3" do
    (1..CNT).each do [1].each (&:to_i) end
  end
}

class Symbol
  def to_proc; @proc ||= Proc.new { |*args| args.shift.__send__(self,
*args) }; end
end

Benchmark.bm { |x|
  x.report "V4" do
    (1..CNT).each do [1].each (&:to_i) end
  end
}
----------------- snap ----------------

The result is on my machine as follows:

      user     system      total        real
V1  0.200000   0.000000   0.200000 (  0.201815)
      user     system      total        real
V2  0.860000   0.000000   0.860000 (  0.870993)
      user     system      total        real
V3  0.950000   0.000000   0.950000 (  0.953523)
      user     system      total        real
V4  0.680000   0.010000   0.690000 (  0.685361)

The fastest, V1, doesn't build proc objects at all. V2 is rails'
default implementation, V4 caches the Proc objects inside the symbol,
i.e. generates the Proc object only.

Of course, invoking 200000 Proc objects this way is usually not what
you are doing in any application; we, however, had a test case where
we did it 9000 times per action and it matched badly with the garbage
collector. Ruby enterprise behaved much much better :)

/eno
This topic is locked and can not be replied to.