Records and Arrays (#170)

mansfiem · July 18, 2008, 6:12pm

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

The three rules of Ruby Q. 2:

Please do not post any solutions or spoiler discussion for this
quiz until 48 hours have passed from the time on this message.
Support Ruby Q. 2 by submitting ideas as often as you can! (A
permanent, new website is in the works for Ruby Q. 2. Until then,
please visit the temporary website at

http://splatbang.com/rubyquiz/.
Enjoy!

Suggestion: A [QUIZ] in the subject of emails about the problem
helps everyone on Ruby T. follow the discussion. Please reply to
the original quiz message, if you can.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Records and Arrays (#170)

In a typical application, you might use what is commonly referred to
as an “array of records.” Essentially, this is a single array
containing multiple objects that represent the same kind of structured
data, either as defined class or OpenStruct instances. Take, for
example, this bit of code that reads in some data regarding people:
their names, ages and favorite colors.

require 'ostruct'
data = []
File.open("data.txt").each do |line|
  name, age, color = line.chomp.split(/,/)
  d = OpenStruct.new
  d.name = name
  d.age = age
  d.color = color
  data << d
end

The variable records is considered an “array of records”, since to
get any particular piece of data, you must first access it as an array
(to get a particular record), then access it as a record (to get a
particular field).

> p data[3].name
=> "Matthew"

> p data[3].age
=> 36

However, at times, it is more convenient to store data as a “record of
arrays”. Instead of one array containing multiple records, you have
one object (i.e. record) containing multiple, parallel arrays. Access
to data then is done first as a record, then as an array.

> p data.name[3]
=> "Matthew"

> p data.age[3]
=> 36

This sort of arrangement is useful when you want to access series of
data at a time. For example, if I have a graphing component that takes
two arrays – one for the domain axis, and another for the range axis
– a “record of arrays” will make accessing that data trivial.

Your task this week is to define two functions that move data
between “array of records” storage and “record of arrays” storage.

def aor_to_roa(arr)
  # This method accepts an array of records, and
  # should return a single record of arrays.
  #
  # This is your task!
end

def roa_to_aor(rec)
  # This method accepts a record of arrays, and
  # should return a single array of records.
  #
  # This is also your task!
end

You should make this work with OpenStruct; do not limit yourself
to the example records shown above.

There are two, optional extra-credits for this week.

Extend these two functions to accept arbitrary classes.
As an alternative to these two functions, create an adapter class
that can wrap around “array of records” data to provide a “record of
arrays” interface without actually moving data about.

mansfiem · July 18, 2008, 6:27pm

The variable records is considered an “array of records”, since to

Changed the code slightly, but forgot to change the reference made
here. This should refer to the variable data, not records.

mansfiem · July 20, 2008, 7:00pm

On Jul 18, 1:03 pm, Matthew M. [email protected] wrote:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Records and Arrays (#170)

Here’s aor → roa adapter that works with any class, it just sends the
methods to the base class as they come, so the behaviour for
exceptional conditions should be consistent with it, it also has a
persistent proxy, so the client can define extra functionality on it
if needed.
http://pastie.org/236858

This is an on the fly converter, it returns an cached array that gets
created the first time a “column” is accessed, this one verifies that
at least one record has the requested attribute.
http://pastie.org/236948

Then, a roa → aor adapter, using the same principles as the first
case.
http://pastie.org/237062

And finally an “on the fly” converter, in this case I could’t return a
proper record wothout knowing in advance the name of all the
attributes.
http://pastie.org/237081

Lucas.

mansfiem · July 20, 2008, 6:55pm

On Jul 18, 2008, at 11:03 AM, Matthew M. wrote:

Records and Arrays (#170)

Neat idea. I was working with jQuery a lot last week and thinking
along similar lines.

As an alternative to these two functions, create an adapter class
that can wrap around “array of records” data to provide a “record of
arrays” interface without actually moving data about.

I chose to run with this idea:

class MapToEnum
instance_methods.each { |meth| undef_method(meth) unless meth =~ /
\A__/ }

def initialize(enum)
@enum = enum
end

def enum
@enum
end

def method_missing(meth, *args, &block)
@enum.map { |o| o.send(meth, *args, &block) }
end
end

def aor_to_roa(arr)
MapToEnum.new(arr)
end

def roa_to_aor(mte)
mte.enum
end

if FILE == $PROGRAM_NAME
Person = Struct.new(:name, :age)
people = [ Person.new(“James”, 32),
Person.new(“Dana”, 33),
Person.new(“Matthew”, 36) ]

wrapped = aor_to_roa(people)
p wrapped.name
p wrapped.name[2]

p roa_to_aor(wrapped)
end

END

James Edward G. II

mansfiem · July 20, 2008, 7:45pm

Matthew M. ha scritto:

Records and Arrays (#170)
File.open("data.txt").each do |line|
(to get a particular record), then access it as a record (to get a
one object (i.e. record) containing multiple, parallel arrays. Access
two arrays – one for the domain axis, and another for the range axis
end

Hi all,

here’s my solution:

require ‘ostruct’

def aor_to_roa(arr)
hash = { }
arr.each do |record|
record.marshal_dump.each_key { |field| (hash[field] ||= []) <<
record.send(field) }
end
OpenStruct.new(hash)
end

def roa_to_aor(rec)
result = []
rec.marshal_dump.each do |field, array|
array.each_with_index { |value, index| (result[index] ||=
OpenStruct.new).send("#{field}=", value) }
end
result
end

mansfiem · July 20, 2008, 7:31pm

I’ve gone for a fairly simple proxy based approach that doesn’t do any
real converting. When converting an array of records we return a
wrapper object.
When you call (for example) name on that object you get back a proxy
object that just stores that name.
When you then call [3] on that proxy object, the proxy retrieves the
object at index 3 and calls the previously remembered method name on it.

The reverse conversion is similar. the conversion function just
returns a wrapper object, and wrapper[2] is just a proxy that
remembers that index. When you call a method on the proxy
it forwards it to the record of arrays to obtain the appropriate array
and retrieves the index element. A limitation is that the wrapper
object just understands the [] method. It doesn’t for example know how
many elements it contains.

Nothing particular about OpenStruct is relied upon (eg activerecord
objects work just fine, or just any old objects for example eg,
roa_to_aor(aor_to_roa(data)) works)
My wrappers also allow assigning (the assignment gets passed through
to the original object)

usage example:
#setup some test data
require ‘ostruct’
data = []
data << OpenStruct.new( :name => ‘fred’, :color => ‘red’, :age => 22)
data << OpenStruct.new( :name => “paul”, :color => “blue”, :age => 30)
data << OpenStruct.new( :name => ‘henry’, :color => ‘pink’, :age => 23)
data << OpenStruct.new( :name => ‘claire’, :color => ‘orange’, :age =>
19)

rec = OpenStruct.new
rec.name = %w(fred paul henry)
rec.age = [10,20,30]
rec.food = %w(pizza cake burger)

#real stuff begins here
x = aor_to_roa data
x.name[2] #=> “henry”
x.name[2] = ‘sarah’
data[2] #> => #<OpenStruct name=“sarah”, color=“pink”, age=23>

y = roa_to_aor rec
y[1].age #=> 20
y[2].food = ‘ice-cream’
rec.food #=> [“pizza”, “cake”, “ice-cream”]

#Composing:

x = roa_to_aor(aor_to_roa(data))
x[1].name #=> “paul”
x[1].name = “charlie”
data[1].name => “charlie”

y= aor_to_roa(roa_to_aor(rec))
y.name[1] #=> “paul”
y.name[1] = “alice”
rec.name #=> [“fred”, “alice”, “henry”]

With aor_to_roa I succumbed to a desire for cuteness, so for example

x.name #=> [“fred”, “paul”, “henry”, “claire”]
x.name.class #=> Array (Even though it isn’t)
x.name.collect {|s| s.upcase} #=> [“FRED”, “PAUL”, “HENRY”, “CLAIRE”]

ie it can really look like an array if you want it to (but this isn’t
very efficient as it’s materializing the array over and over again.
this could probably be improved on). I didn’t do the same for
roa_to_aor.
If one was feeling particularly enthusiastic, sticking some
pluralization rules into method missing would allow one to do (given
the above data) x.names[2] rather than x.name[2] which reads slightly
nicer

Fred

CODE:

def roa_to_aor(rec)
RecordOfArraysWrapper.new(rec)
end

def aor_to_roa(arr)
ArrayOfRecordsWrapper.new(arr)
end

class ArrayOfRecordsWrapper
def initialize(array)
@array = array
end

def method_missing(name, *args)
FieldProxy.new(@array, name)
end

class FieldProxy
instance_methods.each { |m| undef_method m unless m =~ /(^__)/ }

 def initialize array, name
   @array, @name = array, name
 end

 def [](index)
   @array[index].send @name
 end

 def []=(index,value)
   @array[index].send(@name.to_s + '=', value)
 end

 def to_a
   @field_array = @array.collect {|a| a.send @name}
 end

 def method_missing(*args, &block)
   to_a.send *args, &block
 end

end
end

class RecordOfArraysWrapper
def initialize(record)
@record = record
end

def
RecordProxy.new(@record, index)
end

class RecordProxy
def initialize(record, index)
@record, @index = record, index
end

 def method_missing(name, *args)
   if name.to_s =~ /(.*)=$/
     @record.send($1)[@index]=args.first
   else
     @record.send(name)[@index]
   end
 end

end
end

mansfiem · July 21, 2008, 12:36am

Here’s my solution - playing w/ anonymous classes for the adapter
solutions, probably not ideal since it doesn’t provide all the methods
a proper Array or arbitrary record would provide, but was fun to try
out

require ‘ostruct’

def extract_attributes(rec)
if rec.kind_of?(OpenStruct)
attributes = rec.marshal_dump.keys
else
attributes = rec.public_methods
attributes.delete_if do |a|
!(a =~ /^[a-z_]*$/i && attributes.include?("#{a}="))
end
end

attributes
end

def aor_to_roa(arr)
attributes = extract_attributes(arr.first)
roa = arr.first.class.new
attributes.each { |a| roa.send("#{a}=", []) }

arr.each do |rec|
attributes.each do |a|
roa.send(a).push(rec.send(a))
end
end

roa
end

def roa_to_aor(roa)
attributes = extract_attributes(roa)
arr = []

roa.send(attributes.first).size.times do |i|
arr[i] = roa.class.new
attributes.each do |a|
arr[i].send("#{a}=", roa.send(a)[i])
end
end

arr
end

class ROAAdapter
def initialize(arr)
@arr = arr
end

def method_missing(a)
arr = @arr
Class.new do
define_method(:[]) { |i| arr[i].send(a) }
end.new
end
end

class AORAdapter < Array
def initialize(roa)
@roa = roa
end

def
roa = @roa
Class.new do
define_method(:method_missing) { |a| roa.send(a)[i] }
end.new
end
end

-Dustin

mansfiem · July 21, 2008, 1:41am

class AORAdapter < Array

oops, this should’ve just been:

class AORAdapter

complete (revised) source below:

require ‘ostruct’

def extract_attributes(rec)
if rec.kind_of?(OpenStruct)
attributes = rec.marshal_dump.keys
else
attributes = rec.public_methods
attributes.delete_if do |a|
!(a =~ /^[a-z_]*$/i && attributes.include?("#{a}="))
end
end

attributes
end

def aor_to_roa(arr)
attributes = extract_attributes(arr.first)
roa = arr.first.class.new
attributes.each { |a| roa.send("#{a}=", []) }

arr.each do |rec|
attributes.each do |a|
roa.send(a).push(rec.send(a))
end
end

roa
end

def roa_to_aor(roa)
attributes = extract_attributes(roa)
arr = []

roa.send(attributes.first).size.times do |i|
arr[i] = roa.class.new
attributes.each do |a|
arr[i].send("#{a}=", roa.send(a)[i])
end
end

arr
end

class ROAAdapter
def initialize(arr)
@arr = arr
end

def method_missing(a)
arr = @arr
Class.new do
define_method(:[]) { |i| arr[i].send(a) }
end.new
end
end

class AORAdapter
def initialize(roa)
@roa = roa
end

def
roa = @roa
Class.new do
define_method(:method_missing) { |a| roa.send(a)[i] }
end.new
end
end

mansfiem · July 21, 2008, 3:35am

Mine:
require ‘ostruct’

class AryOfRecs

def initialize
@ary = []
end

def <<(record)
@ary << record
keys = ( record.respond_to?(“marshal_dump”) ?
record.marshal_dump.keys :
record.instance_variables)
keys.each { |key|
roa_way(key.to_s.sub("@","")) if
record.respond_to?(key.to_s.sub("@",""))
}
end

def
return @ary[index]
end

def to_a
@ary
end

private

def roa_way(attrib)
instance_eval(
%(def #{attrib}
@ary.map { |rec| rec.#{attrib} }
end))
end
end

I only made the wrapper for an array of records, I’t can be filled as in
the
example:

data = AryOfRecs.new
File.open(“data.txt”).each do |line|
name, age, color = line.chomp.split(/,/)
d = OpenStruct.new
d.name = name
d.age = age
d.color = color
data << d
end

puts data[2].name
puts data.name[2]

Or with a class:

class Person
attr_accessor :name, :age, :color
end

data = AryOfRecs.new
File.open(“data.txt”).each do |line|
name, age, color = line.chomp.split(/,/)
d = Person.new
d.name = name
d.age = age
d.color = color
data << d
end

puts data[2].name
puts data.name[1]

mansfiem · July 20, 2008, 7:47pm

my solution (including a simple test)

require ‘ostruct’
require ‘test/unit’

solution by rolando abarca - [email protected]

module Quiz170
def self.aor_to_roa(arr)
os = OpenStruct.new
arr.each do |o|
# another way to get the keys:
# keys = (o.methods - OpenStruct.new.methods).delete_if { |m|
m.match(/=$/) }
o.instance_variable_get("@table").keys.each { |k|
(os.send(k).nil? ? os.send("#{k}=", []) : os.send(k)) <<
o.send(k)
}
end
os
end

def self.roa_to_aor(rec)
arr = []
table = rec.instance_variable_get("@table")
size = table[table.keys.first].size
0.upto(size-1) { |i|
obj = OpenStruct.new
table.keys.each { |k| obj.send("#{k}=", table[k][i]) }
arr << obj
}
arr
end
end

class TestQuiz170 < Test::Unit::TestCase
def setup
@data = []
1.upto(100) { |i|
_t = OpenStruct.new
_t.name = “t#{i}”
_t.data = (rand * 1000).round
@data << _t
}
@roa = Quiz170.aor_to_roa(@data)
end

def test_001
1.upto(100) {
idx = (rand * 100).to_i
assert_equal @data[idx].name, @roa.name[idx]
assert_equal @data[idx].data, @roa.data[idx]
}
end

def test002
assert_equal @data, Quiz170.roa_to_aor(@roa)
end
end

regards,
rolando./

mansfiem · July 21, 2008, 5:32am

On Jul 18, 2008, at 10:03 AM, Matthew M. wrote:

Records and Arrays (#170)

cfp:~ > cat quiz-170.rb

quiz-170.rb

aor = [ Record[0,1,2], Record[3,4,5], Record[6,7,8] ]

y ‘aor’ => aor
y ‘aor2roa(aor)’ => aor2roa(aor)
y ‘roa2aor(aor2roa(aor))’ => roa2aor(aor2roa(aor))

BEGIN {

require ‘yaml’
require ‘rubygems’
require ‘arrayfields’

class Record < Array.struct(%w( a b c ))
def to_yaml(*a, &b) to_hash.to_yaml(*a, &b) end
end

module Inline
def to_yaml_style() :inline end
end

Array.send :include, Inline
Record.send :include, Inline

def aor2roa aor
fields = aor.first.fields
rclass = aor.first.class
roa = rclass[]
aor.each do |record|
fields.each do |field|
(roa[field] ||= [] ) << record[field]
end
end
roa
end

def roa2aor roa
fields = roa.fields
rclass = roa.class
aor = []
n = fields.map{|field| roa[field].size}.max
n.times do |i|
values = fields.map{|field| roa[field][i]}
aor << rclass[*values]
end
aor
end

}

cfp:~ > ruby quiz-170.rb

aor: [{a: 0, b: 1, c: 2}, {a: 3, b: 4, c: 5}, {a: 6, b: 7, c: 8}]

aor2roa(aor):
a: [0, 3, 6]
b: [1, 4, 7]
c: [2, 5, 8]

roa2aor(aor2roa(aor)): [{a: 0, b: 1, c: 2}, {a: 3, b: 4, c: 5}, {a: 6,
b: 7, c: 8}]

a @ http://codeforpeople.com/

mansfiem · July 22, 2008, 9:01am

at last, I get track of this quiz(#170)…(^_^;)

On 2008/07/19, at 1:03, Matthew M. wrote:

Your task this week is to define two functions that move data
between “array of records” storage and “record of arrays” storage.

wow, interesting(^_^)
it sounds like rewriting a C-code into BASIC-code (and back).
and You may be saying Ruby provides a way
like Worm-Hole-jump: very-short cut.

I don’t think I could find a best answer,
but i’ll try pushing data into RDB and
pull things out in transverse of Row&Column…

=============================
ContextSwitcher

     Shindo  Motoakira
<[email protected]>

=============================

mansfiem · July 22, 2008, 1:58am

On 7/18/08, Matthew M. [email protected] wrote:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Your task this week is to define two functions that move data
between “array of records” storage and “record of arrays” storage.

You should make this work with [OpenStruct][1]; do not limit yourself
to the example records shown above.

Here’s my solution which works with OpenStruct only, I didn’t do any
extra credit.
-Adam

require ‘ostruct’
def aor_to_roa(arr)
# This method accepts an array of records, and
# returns a single record of arrays.
rec = OpenStruct.new
return rec if ! arr[0] #empty array => empty record
vars = (arr[0].methods - rec.methods)
setters = vars.select{|m|m[-1]==?=}.sort
getters = (vars-setters).sort!
vars = getters.zip setters
vars.each{|get,set| rec.send(set, Array.new)}
arr.each{|item|
vars.each{|get,set|
rec.send(get)<< item.send(get)
}
}
rec
end

def roa_to_aor(rec)

This method accepts a record of arrays, and

returns a single array of records.

arr=[]
vars = (rec.methods - OpenStruct.new.methods)
setters = vars.select{|m|m[-1]==?=}.sort
getters = (vars-setters).sort!
vars = getters.zip setters
vars.each {|get,set|
  rec.send(get).each_with_index{|value,i|
    arr[i]||=OpenStruct.new
    arr[i].send(set,value)
  }
}
arr

end

Records and Arrays (#170)

Records and Arrays (#170)

Records and Arrays (#170)

Records and Arrays (#170)

Records and Arrays (#170)

solution by rolando abarca - [email protected]

Records and Arrays (#170)

quiz-170.rb

cfp:~ > ruby quiz-170.rb

aor: [{a: 0, b: 1, c: 2}, {a: 3, b: 4, c: 5}, {a: 6, b: 7, c: 8}]

aor2roa(aor): a: [0, 3, 6] b: [1, 4, 7] c: [2, 5, 8]

This method accepts a record of arrays, and

returns a single array of records.

aor2roa(aor):
a: [0, 3, 6]
b: [1, 4, 7]
c: [2, 5, 8]