Article: An Exercise in Metaprogramming with Ruby


#1

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

My editor said it “didn’t do that well” in terms of page views. And I
said,
well, I should have posted it to ruby-talk. And she said: Do that now,
and we’ll see what effect it has.

So there you have it. No bots or artificial inflation, please. :wink:

This article, by the way, was adapted from a talk given in January
to the Austin on Rails group.

Cheers,
Hal


#2

Interesting article… just about the level I needed. A decent
example, not uselessly trivial, but not terribly complex either, so I
can follow enough of the metaprogramming to truly understand what’s
going on.

Thanks…


#3

Hal wrote:

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

Wow. Very nice. I’m doing a fair amount of meta programing myself these
days and having resources like this for reference and information is
great.

I like the way you dynamically create the class using Object#const_set.
Metaprogramming sure beats generating code the traditional way.


#4

On 3/24/06, removed_email_address@domain.invalid removed_email_address@domain.invalid wrote:

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

My editor said it “didn’t do that well” in terms of page views. And I
said,
well, I should have posted it to ruby-talk. And she said: Do that now,
and we’ll see what effect it has.

Nice article Hal. It’s a shame ZD requires registration to comment
there though.

So there you have it. No bots or artificial inflation, please. :wink:

I’ll be sharing it around at work, just a bit of natural inflation :slight_smile:


#5

On 3/24/06, Matthew M. removed_email_address@domain.invalid wrote:

Interesting article… just about the level I needed. A decent
example, not uselessly trivial, but not terribly complex either, so I
can follow enough of the metaprogramming to truly understand what’s
going on.

+1

Great stuff Hal, thanks much.


#6

On Mar 24, 2006, at 1:38 PM, removed_email_address@domain.invalid wrote:

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

That’s so darn cool, I think I’m just going to have to add it to
FasterCSV… :wink:

Excellent article Hal!

James Edward G. II


#7

Nice article. As others have said, a clear survey of some intermediate
techniques. A couple of suggestions:

  1. More sample output or interactive session information. You build up
    the
    People class kind of piecemeal without giving intermediate output along
    the
    way. I think that might be hard for someone new to metaprogramming to
    follow.
  2. You use some advanced syntax that might confuse others (example:
    splat
    operator for attributes). There is a lot packed into that article - I
    would
    reduce tricks like that as much as possible.

Good luck with page views!


#8

I’m a Ruby newbie. So far I am loving everything I learn about
Ruby. I’m trying to find a real app to create with it. I have need
for a client program that talks to an LDAP server and that makes
calls to an ONC/RPC server that we wrote here at my job in C++. Do
these exist for Ruby?

Thanks!

Ernest


#9

Of course, un-pedagogically, it could be compressed a tad:

class DataRecord

def self.make(file_name)
  header = File.open(file_name){|f|f.gets}.split(/,/)
  struct = File.basename(file_name,'.txt').capitalize
  record = Struct.new(struct, *header)

  class<<record;self;end.send :define_method, :read do
    File.open(file_name) do |f|
      f.gets
      f.inject([]){|a,l| a << record.new(*eval("[#{l}]")) }
    end
  end

  record
end

end

data = DataRecord.make(‘people.txt’)
list = data.read # or: Struct::People.read
person = list[2]
puts person.name
if person.age < 18
puts “under 18”
else
puts “over 18”
end
puts “Weight is: %.2f kg” % kg = person.weight / 2.2

cheers,
andrew


#10

Question along these lines, suppose you add an attribute to the
‘People’ class after the initial creation (say by adding another
column to the people.txt file), do the ‘old’ people classes get the
new attribute as well? If so, what’s the initial value? I suspect it
would be nil.

thanks,


#11

On 3/24/06, removed_email_address@domain.invalid removed_email_address@domain.invalid wrote:

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

[…]

i like the article!

yeah, and even if code using the fields is coupled tightly to the
created classes, the solution is highly reusable.
– henon


#12

On 24 Mar 2006, at 19:38, removed_email_address@domain.invalid wrote:

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

Thanks very much, you’ve inspired me to clean up some crusty code
I’ve been ignoring for a while!


#13

On Sun, 26 Mar 2006, Joel VanderWerf wrote:

could be stored in a list kept in class instance variable…

class Person
field :name, :favorite_ice_cream, …
end

In this way, some rudimentary error checking can be performed when reading
the file, rather than failing later when trying to serve ice cream to the
person. (I just hate it when my ruby-scripted robo-fridge serves me passion
fruit and rabbit dropping ice cream.) This is not always the best way to go
(what if, as the article points out, fields get added to the file?), but one
more thing to keep in mind.

i’m totally with you on this joel. still, i think one can have a bit of
both:

 harp:~ > cat a.rb
 require "arrayfields"
 require "csv"

 csv = <<-csv
 latitude,longitude,description
 47.23,59.34,Omaha
 32.17,39.24,New York City
 73.11,48.91,Carlsbad Caverns
 csv


 class CSVTable < ::Array
   attr "fields"
   def initialize arg
     CSV::parse(arg) do |row|
       row.map!{|c| c.to_s}
       if @fields
         self << row
       else
         @row_class = Class::new(::Array) do
           define_method("initialize") do |a|
             self.fields = row
             replace a
           end
         end
         @fields = row
       end
     end
     @fields.each{|field| column_attr field}
   end
   def << row
     super @row_class::new(row)
   end
   def column_attr(ca)
     singleton_class = class << self; self; end
     singleton_class.module_eval{ define_method(ca){ map{|r| r[ca]}} 

}
end
def [](*a, &b)
m = a.first
return(send(m)) if [String, Symbol].map{|c| c === m}.any? &&
respond_to?(m)
super
end
end

 table = CSVTable::new csv

 p table
 puts

 p table.fields
 puts

 table.fields.each{|f| puts "#{ f }: #{ table[f].join(', ') }"}
 puts

 table.each{|row| puts row.fields.map{|f| "#{ f }: #{ row[f] 

}"}.join(’, ') }
puts

 harp:~ > ruby a.rb
 [["47.23", "59.34", "Omaha"], ["32.17", "39.24", "New York City"], 

[“73.11”, “48.91”, “Carlsbad Caverns”]]

 ["latitude", "longitude", "description"]

 latitude: 47.23, 32.17, 73.11
 longitude: 59.34, 39.24, "48.91
 description: Omaha, New York City, Carlsbad Caverns

 latitude: 47.23, longitude: 59.34, description: Omaha
 latitude: 32.17, longitude: 39.24, description: New York City
 latitude: 73.11, longitude: 48.91, description: Carlsbad Caverns

regards.

-a


#14

Meinrad R. wrote:

created classes, the solution is highly reusable.
– henon

The point about coupling (mentioned in the second-to-last paragraph of
the article) is important, and I feel it is dismissed to easily in the
article. There are some tradeoffs to consider, though perhaps they are
out of the scope of the article, which is intended as an exercise, not
as a complete guide:

  1. Suppose your code needs to discover what fields are in the file?
    You can use #instance_methods(false), but that is not perfect: you have
    to filter out “to_s” and “inspect”, which were added by #make. And what
    if you add a new method in addition to the ones generated by #make? The
    field names could be stored in a list kept in class instance variable…

  2. Once you have discovered the field names, you have to use
    send(fieldname) and send("#{fieldname}=") to access them. That’s more
    awkward and (at least in the second case) less efficient than Hash#[]
    and #[]=. Who’s the “second class citizen” in this case?

  3. If you really know the field names “in advance” (that is, you have
    enough information to hard code them into your program), rather than “by
    discovery”, then maybe it is better to use a different metaprogramming
    style in which the fields are declared using class methods:

class Person
field :name, :favorite_ice_cream, …
end

In this way, some rudimentary error checking can be performed when
reading the file, rather than failing later when trying to serve ice
cream to the person. (I just hate it when my ruby-scripted robo-fridge
serves me passion fruit and rabbit dropping ice cream.) This is not
always the best way to go (what if, as the article points out, fields
get added to the file?), but one more thing to keep in mind.

  1. Is it always a good idea to couple the class name with the file name?
    Maybe the class’s identity should be associated with the set of fields
    defined in the header? Why not reuse the anonymous class if the header
    is the same as those in some other file you imported earlier? (This
    could be done using a hash mapping sorted lists of column names to
    classes.) That would make it possible to use == to compare objects read
    from different files. Further, it would let you use x.class == y.class
    to determine if x and y came from files with compatible formats (same
    fields, but maybe in a different order).

  2. Maybe Struct would serve just as well, since it takes care of
    everything in the class_eval block. For example:

klass = Struct.new(*names.map{|s|s.intern})

None of these are necessarily problems, depending on what you are trying
to do, but alternate solutions (for example, using hashes) are worth
considering. Metaprogramming is not always the best solution, though it
is good to have it in your pocket.

Some minor quibbles:

  1. In DataRecord.make, if the file happens to be empty, data.gets.chomp
    will raise an exception and the file will not be closed. Similarly in
    the #read method of the generated class. Why not use a block with
    File.open?

  2. The second way of referring to the class:

    require ‘my-csv’
    DataRecord.make(“people.txt”) # Ignore the return value and
    list = People.read # refer to the class by name.

should raise the hackles on a ruby programmer’s neck. It’s a violation
of DRY: you have typed the string “people” in two places, and your
program breaks if (a) the filename changes or (b) the way “people.txt”
is transformed into “People” changes. Maybe you want that breakage
(maybe you want the program to fail if someone tries to run it on
“other-people.txt” or on “places.txt”). Or maybe not: it’s another
tradeoff. (To be fair, the article doesn’t claim that the version with
People hard-coded can read places.txt.)

  1. Is it really a good idea to encourage people to eval("[#{line}]") ???

#15

removed_email_address@domain.invalid wrote:

  1. Suppose your code needs to discover what fields are in the file?
    and send("#{fieldname}=") to access them. That’s more awkward and (at
    in which the fields are declared using class methods:
    fruit and rabbit dropping ice cream.) This is not always the best way
    to go
    (what if, as the article points out, fields get added to the file?),
    but one
    more thing to keep in mind.

i’m totally with you on this joel. still, i think one can have a bit of
both:

class CSVTable < ::Array
attr “fields”

Sure, that’s more or less what I meant by storing the fields, but I was
thinking of using a class instance variable to keep that information at
the class level (assuming you might want to reuse one table class and
one row class for several files).

Using arrayfields is nice, since you have the symbolic #[] and #[]=
interfaces, as with hashes, as well as the array interface. But you have
neither a declared list of what fields should be in the file (what I was
suggesting for error checking purposes), nor the ability to refer to
fields directly with a “first class citizen” method (what Hal’s article
was advocating):

p table[1].latitude

Nothing wrong with any of these approaches, it’s just good to be aware
of all of them.

Btw, you can use a block with #any? :

    return(send(m)) if [String, Symbol].map{|c| c === m}.any? &&

respond_to?(m)

return(send(m)) if [String, Symbol].any? {|c| c === m} && 

respond_to?(m)


#16

Joel VanderWerf wrote:

[snip snip snip]

Joel,

Thanks for your comments. I have read them with interest
(and everyone else’s) but am too busy to reply at length.

In short, yes, there are flaws in the approach I took, and
there is more than one way to do it. For the most part, it’s
just an exercise.

Thanks,
Hal


#17

On Mar 24, 2006, at 2:28 PM, James Edward G. II wrote:

On Mar 24, 2006, at 1:38 PM, removed_email_address@domain.invalid wrote:

I failed to post this link before, so here it is now:

http://www.devsource.com/article2/0,1895,1928561,00.asp

That’s so darn cool, I think I’m just going to have to add it to
FasterCSV… :wink:

Developer at play:

$ irb -r lib/faster_csv.rb

class FullName < Struct.new(:first, :last)
def initialize( first, last, other = Hash.new )
super(first, last)
@middle, @suffix = other.values_at(:middle, :suffix)
end
attr_accessor :middle, :suffix
end
=> nil

names = [ FullName.new(“Santa”, “Clause”),
?> FullName.new(“James”, “Gray”, :middle =>
“Edward”, :suffix => “II”),
?> FullName.new(“Easter”, “Bunny”) ]
=> [#<struct FullName first=“Santa”, last=“Clause”>, #<struct
FullName first=“James”, last=“Gray”>, #<struct FullName
first=“Easter”, last=“Bunny”>]

csv = FasterCSV.dump(names)
=> “class,FullName\n@middle,@suffix,first=,last=\n,Santa,Clause
\nEdward,II,James,Gray\n,Easter,Bunny\n”

puts csv
class,FullName
@middle,@suffix,first=,last=
,Santa,Clause
Edward,II,James,Gray
,Easter,Bunny
=> nil

reloaded = FasterCSV.load(csv)
=> [#<struct FullName first=“Santa”, last=“Clause”>, #<struct
FullName first=“James”, last=“Gray”>, #<struct FullName
first=“Easter”, last=“Bunny”>]

reloaded.find { |name| name.first == “James” }.middle
=> “Edward”

That’s using the development version of FasterCSV. Thanks for the
idea Hal! :wink:

James Edward G. II