Backporting Enumerator.new { ... } to Ruby 1.8.7

Ruby
1

Hello, good people of ruby-talk.

I’m trying to backport my code from Ruby 1.9 to Ruby 1.8.7 so that I can
profile it sanely with ruby-prof or zenprofile (neither of which works
sensibly under Ruby 1.9 as of now). Unfortunately, my code heavily uses
lazy-evaluated, nested Enumerator objects. What is the sanest way to
backport the following example so that it works more-or-less in the same
way under Ruby 1.8.7 as it does under Ruby 1.9?

def multiplier param
Enumerator.new do |yielder|
(1…10).each do |elem|
sleep 1 # simulates a long-running process
yielder.yield param * elem
end
end
end

multiplier(3).each do |e|
puts e
end

I think I need a proxy object that can emulate Ruby 1.9’s Enumerator.new
by taking a block and then being able to evaluate it in the original
calling place’s context (so that the ‘param’ above is available to it
upon subsequent Enumerator#each calls), but my still-limited Ruby-fu
fails to come up with a solution for this.

— Shot

2

I’m trying to backport my code from Ruby 1.9 to Ruby 1.8.7 so that I can
profile it sanely with ruby-prof or zenprofile (neither of which works
sensibly under Ruby 1.9 as of now).

I heard some rumors ruby-prof would work soon…
http://twitter.com/methodmissing

-r

3

Roger P.:

I’m trying to backport my code from Ruby 1.9 to Ruby 1.8.7 so that I can
profile it sanely with ruby-prof or zenprofile (neither of which works
sensibly under Ruby 1.9 as of now).

I heard some rumors ruby-prof would work soon…
http://twitter.com/methodmissing

ruby-prof does work in 1.9, there are even two versions that work.
Unfortunately, one of them does not aggregate the results (and
you end up with mutli-megabyte reports and thousands of calls like
Set#hash-1139), while the other, which does aggregate the results,
does this in insane amounts of time (I waited for an hour to get
a minute-long run results aggregated).

ZenProfile, in turn, depends on event_hook, which does not work
in 1.9 and it’s not sure whether it’ll get ported anytime soon:
http://groups.google.com/group/ruby-core-google/browse_thread/thread/afaf0aad1e20d9c0/f09084047cbe04d4

— Shot

4

Roger P.:

What is the sanest way to backport the following example so that it
works more-or-less in the same way under Ruby 1.8.7 as it does under
Ruby 1.9?

def multiplier param
Enumerator.new do |yielder|

This might help:

http://talklikeaduck.denhaven2.com/2006/10/15/enumeration-improvement-in-ruby-1-9
How do Enumerators work in Ruby 1.9.1? - Stack Overflow
http://anthonylewis.com/2007/11/09/ruby-generators

Hm. I read these and – given that it lacks Fibers – can’t figure
out a way to come up with a simple lazy enumerator in Ruby 1.8. :expressionless:
What am I missing?

— Shot

5

sensibly under Ruby 1.9 as of now). Unfortunately, my code heavily uses
lazy-evaluated, nested Enumerator objects. What is the sanest way to
backport the following example so that it works more-or-less in the same
way under Ruby 1.8.7 as it does under Ruby 1.9?

looks like there’s a couple ways
one
require ‘backports’

leave the original code as is

or this one

require ‘generator’
def multiplier param
Generator.new do |yielder|
(1…10).each do |elem|
sleep 1 # simulates a long-running process
yielder.yield param * elem
end
end
end

multiplier(3).each do |e|
puts e
end

I’m not sure if that will actually satisfy your desire (nor have I
figured out if enumerables are really “chainable” in 1.8 like they
apparently are in 1.9).
Thanks!
-r

6

On Oct 3, 2009, at 6:50 AM, Shot (Piotr S.) wrote:

http://talklikeaduck.denhaven2.com/2006/10/15/enumeration-improvement-in-ruby-1-9
How do Enumerators work in Ruby 1.9.1? - Stack Overflow
http://anthonylewis.com/2007/11/09/ruby-generators

Hm. I read these and – given that it lacks Fibers – can’t figure
out a way to come up with a simple lazy enumerator in Ruby 1.8. :expressionless:
What am I missing?

#!/usr/bin/env ruby -wKU

require “generator”

fib = Generator.new do |g|
g.yield(n2 = 0)
g.yield(n1 = 1)
loop do
n2, n1 = n1, n2 + n1
g.yield(n1)
end
end

p Array.new(10) { fib.next }

Or with a closure:

#!/usr/bin/env ruby -wKU

def fibonacci
n2 = n1 = nil
lambda {
if n2.nil?
n2 = 0
elsif n1.nil?
n1 = 1
else
n2, n1 = n1, n2 + n1
n1
end
}
end

fib = fibonacci
p Array.new(10) { fib.call }

Or with an object:

#!/usr/bin/env ruby -wKU

class Fibonacci
def initialize
@sequence = [ ]
end

 def next
   if @sequence.size < 2
     @sequence << @sequence.size
   else
     @sequence << @sequence[-2] + @sequence[-1]
     @sequence.unshift
   end
   @sequence[-1]
 end

end

fib = Fibonacci.new
p Array.new(10) { fib.next }

James Edward G. II

7

On Oct 3, 2009, at 9:46 AM, James Edward G. II wrote:

   n1 = 1
 else
   n2, n1 = n1, n2 + n1
   n1
 end

}
end

fib = fibonacci
p Array.new(10) { fib.call }

I forgot to mention, there are some other good resources for code like
this:

http://moonbase.rydia.net/software/lazy.rb/

http://blog.grayproductions.net/articles/infinite_streams

James Edward G. II

8

Hm. I read these and – given that it lacks Fibers – can’t figure
out a way to come up with a simple lazy enumerator in Ruby 1.8. :expressionless:

You could also use “poor man’s fibers” for 1.8 :slight_smile:
-r

9

Roger P.:

Unfortunately, my code heavily uses lazy-evaluated, nested Enumerator
objects. What is the sanest way to backport the following example
so that it works more-or-less in the same way under Ruby 1.8.7 as
it does under Ruby 1.9?

looks like there’s a couple ways
one
require ‘backports’

leave the original code as is

Oh, wow! How could I have missed this? (Does not show up on ‘Ruby
backporting’ Google queries, but still…) This plus some amazingly
minimal monkey-patching got me a backport in two hours or so. :slight_smile:

or this one

require ‘generator’
def multiplier param
Generator.new do |yielder|

…AND I have missed Ruby 1.8’s stdlib’s Generator! A double shame,
but I’m happy as a clam nevertheless – I’ll finally be able to nicely
profile my PhD code!

(Even if the results do not necessarily reflect Ruby 1.9’s bottlenecks,
I guess they will be close enough, and being able to profile my code
will most probably turn the rest of the month into a very interesting
adventure with RuDy¹.)

¹
http://app.euruko2009.org/talks/21-announcing-rudy-write-ruby-native-extensions-in-d-programming-language

Thanks again, Roger, I can’t say enough how happy I am with the above.

— Shot

10

James Edward G. II:

Shot (Piotr S.) wrote:

What is the sanest way to backport the following example so that it
works more-or-less in the same way under Ruby 1.8.7 as it does under
Ruby 1.9?

def multiplier param
Enumerator.new do |yielder|

require “generator”

[…]

Or with a closure:

[…]

Or with an object:

[…]

I forgot to mention, there are some
other good resources for code like this:
http://moonbase.rydia.net/software/lazy.rb/
http://blog.grayproductions.net/articles/infinite_streams

Thanks a lot, James! As I just wrote in the reply to Roger P.,
I ended up using the backports gem, and if I didn’t know about it
I’d most probably expose Generator.new as Enumerator.new, but the
closure and object examples are very enlightening.

Even more so is your ‘Higher-Order Ruby’ series, which I always split
into the ‘understandable’ part on Iterators, Caching and Memoization,
and the ‘still a bit over my head’ rest.

I think I’m finally ready to dive deep and wrap my
head around all of the concepts mentioned there. :slight_smile:

— Shot

11

Shot (Piotr S.) wrote:

What is the sanest way to
backport the following example so that it works more-or-less in the same
way under Ruby 1.8.7 as it does under Ruby 1.9?

I backported this form of Enumerator a while back, you can find it in
the ‘facets’ library.

12

What is the sanest way to

backport the following example so that it works more-or-less in the same
way under Ruby 1.8.7 as it does under Ruby 1.9?

def multiplier param
Enumerator.new do |yielder|

This might help:

http://talklikeaduck.denhaven2.com/2006/10/15/enumeration-improvement-in-ruby-1-9

http://anthonylewis.com/2007/11/09/ruby-generators

GL.
-r