Forum: Ruby using YAML as config

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Ed0addfc66596995a471e91c15e3c49a?d=identicon&s=25 Larry Edelstein (ledelste)
on 2007-02-12 02:21
Hi all -

I want to write a config file in YAML.  In this file I will store a
small graph of objects in an array:

- !mydomain,mydate/Company
  name: TuesdayCo
  subthings:
    - !mydomain,mydate/Person
      name: Ruby Tuesday
    - !mydomain,mydate/Person
      name: Tuesday Weld
- !mydomain,myDate
  name: FridayCo
  subthings:
    etc.

Then I want to read in this file and get an array of Company objects,
each of which has an array of Person objects.  I'm defining these
classes myself.  I've tried using add_domain_type and hoped to get
instances of my classes, but instead I got instances of
YAML::DomainType.

How should I do this?

-larry
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2007-02-12 04:45
(Received via mailing list)
Larry Edelstein wrote:
>     - !mydomain,mydate/Person
> YAML::DomainType.
>
> How should I do this?

Here's the only way I know to do this (I'm not sure if add_domain_type
is another way):

require 'yaml'

# This assumes that all your objects will serialize as hashes ("maps")
module Base
   def to_yaml( opts = {} )
     YAML::quick_emit( object_id, opts ) do |out|
       out.map( taguri, to_yaml_style ) do |map|
         to_yaml_properties.each do |m|
           map.add( m, send( m ) )
         end
       end
     end
   end

   module BaseModuleMethods
     def yaml_new( klass, tag, val )
       unless Hash === val
         raise YAML::TypeError, "Invalid #{self}: " + val.inspect
         ## do more error checking here, if desired
       end

       name, type = YAML.read_type_class( tag, self )
       st = type.new
       val.each do |k,v|
         st.send( "#{k}=", v )
       end
       st
     end
   end

   def self.included mod
     mod.extend BaseModuleMethods
   end
end

class Company
   include Base

   attr_accessor :name, :subthings
   def initialize name = nil
     @name = name
   end

   yaml_as "tag:my.domain,2007:Company"

   def to_yaml_properties
     ["name", "subthings"]
   end
end

class Person
   include Base

   attr_accessor :name
   def initialize name = nil
     @name = name
   end

   yaml_as "tag:my.domain,2007:Person"

   def to_yaml_properties
     ["name"]
   end
end

co = Company.new "TuesdayCo"
co.subthings = [
   Person.new("Ruby Tuesday"),
   Person.new("Tuesday Weld")
]

co_yaml = co.to_yaml
puts co_yaml
# ==>
# --- !my.domain,2007/Company
# name: TuesdayCo
# subthings:
# - !my.domain,2007/Person
#   name: Ruby Tuesday
# - !my.domain,2007/Person
#   name: Tuesday Weld

co2 = YAML.load(co_yaml)
p co2

# ==>
#<Company:0xb7ce3cfc @subthings=[#<Person:0xb7ce44cc @name="Ruby
Tuesday">, #<Person:0xb7ce3ef0 @name="Tuesday Weld">],
@name="TuesdayCo">


# Check that error-checking works by
# giving the parser an array instead of a hash:
s = <<END
--- !my.domain,2007/Company
- 1
- 2
END

begin
   p YAML.load(s)
rescue YAML::TypeError => ex
   puts ex.message   # Invalid Company: [1, 2]
end
19fdf8bd123216b5056fb856cf1a5771?d=identicon&s=25 _why (Guest)
on 2007-02-12 05:31
(Received via mailing list)
On Mon, Feb 12, 2007 at 10:21:54AM +0900, Larry Edelstein wrote:
>   subthings:
>     etc.
> [...]
>
> How should I do this?

Heya, larry!  The easy way is like this:

  class Company
    yaml_as 'tag:mydomain.com,2007:Company'
  end

  class Person
    yaml_as 'tag:mydomain.com,2007:Person'
  end

_why
Ed0addfc66596995a471e91c15e3c49a?d=identicon&s=25 Larry Edelstein (ledelste)
on 2007-02-12 23:10
Wow Joel - that's a lot of code and a lot to digest.  I'm going through
it.  Your response and _why's use the yaml_as attribute - I've never
even seen that before.
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2007-02-12 23:22
(Received via mailing list)
Larry Edelstein wrote:
> Wow Joel - that's a lot of code and a lot to digest.  I'm going through
> it.  Your response and _why's use the yaml_as attribute - I've never
> even seen that before.

It's nice to know that _why's much simpler code works. The other stuff I
wrote is how I've handled validation and other customization before. I
wonder if _why has a better way to do that too...
61396c69040bbc56dbad1c5c71ac27d2?d=identicon&s=25 Rover Rhubarb (rhubarb)
on 2007-05-07 21:14
So I come to the part of my little app where I want to load and save a
config file. I want it to be human read/writeable too though - it's not
just for serialization. In fact, usually a human will write it, my
little app will read it, and may adjust it and write it out anew.

So in xml it would look like this
<channels>
  <channel>
    <name>foo</name>
    <id>3</id>
  </channel>
  <channel>
 …
 </channels>

I don't care about channels that's just a random root node, and I might
do it with attbs instead of elements, so:

<channel name="foo" id="3"/>

So I thought, lets use YAML instead - it's even easier to read and
write. Google, google, read blogs, docs, google, google, more docs, more
blogs, google to_yaml, fxri: hack hack, google to_yaml_type, fxri: hack,
hack, hack, google, google, yaml_as, hack.

Argh! Seems like day later, and I'm ready to go back to xml. But I'm
sure there's a simple way to do what I want, so I'll post here first and
hope. See, my class is Channel, like so:

class Channel; attr_accessor :name, :id; end

and want my array of channels to look something like this

- Channel
  name: foo
  id: 3
- Channel
  name: bar
  id: 4
…

But I just can't work out how to simply the yaml output to this stage.
And I don't want to override a dozen methods and do my own hash mapping.
The easiest way - thanks to why for posting this in the forum - seems to
be to use yaml_as:

class Channel
  yaml_as "tag:foobar,2007:Channel"
end

but then I get each entry as

 --- !foolbar,2007/Channel
   name: foo
   id: 3

This is fairly simple, but remember that I want my entries to be
human-writeable the first time. That line of "---!foobar,2007…" is just
way too machiney for us poor humans to get right every time.

I can simplify the taguri like this
  yaml_as "tag::Channel"

to get

 --- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML::DynamicType instead - which is what it always gives if
it doesn't recognize the object.
I guess the load method needs a valid taguri

(Side note - I'd never heard of a taguri before, but when I went to read
about it at http://www.taguri.org/ I was _very_ pleasantly surprised at
how quickly I was able to grok the whole thing by reading Sandro Hawke's
four-paragraph tutorial about his dog Taiko. Way to go Sandro - if I'd
been directed to the RFC first it'd be all over)

So anyway, the fact is that while taguris are great for distinguishing
Taiko from his master's descendents' pooches, and while it's very
simple, it's still overkill for me. I don't need a unique ID. I know
what "--- Channel" means in my little app. I don't want to submit my
Channel types to some great panoply of random little types each with
their own special pet name. I just want to damn well read and write it.

I've even considered parsing the text in to_yaml and YAML.load and
gsubbing "!foobar, 2007/" out and in again.  But there's got to be a
better way. So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?
61396c69040bbc56dbad1c5c71ac27d2?d=identicon&s=25 Rover Rhubarb (rhubarb)
on 2007-05-07 21:29
Let me add:

- I meant DomainType, not dynamic type

- The use of yaml_as is redundant since I can get the same effect
without it - where my objects are dumped as

 - !ruby/object:Channel

instead of my own silly name.

Still not good enough: how do I get rid of the !ruby/object
machine-talk. (Is it machine-talk, or was YAML designed by bushmen of
the Kalahari, where the scripting language of choice is "click-ruby")
19fdf8bd123216b5056fb856cf1a5771?d=identicon&s=25 _why (Guest)
on 2007-05-07 23:26
(Received via mailing list)
On Tue, May 08, 2007 at 04:14:24AM +0900, Rover Rhubarb wrote:
> give ma a YAML::DynamicType instead - which is what it always gives if
> it doesn't recognize the object.

Try:

  class Channel
    yaml_as "x-private:Channel"
  end

And you get:

  --- !!Channel
  id: 3
  name: foo

G'luck!!

_why
1bac2e65d64faf472cf2ebc94f0f5ee0?d=identicon&s=25 Ara.T.Howard (Guest)
on 2007-05-07 23:32
(Received via mailing list)
On Tue, 8 May 2007, Rover Rhubarb wrote:

>  </channel>
> write. Google, google, read blogs, docs, google, google, more docs, more
>
> The easiest way - thanks to why for posting this in the forum - seems to
>   id: 3
> --- !Channel
> how quickly I was able to grok the whole thing by reading Sandro Hawke's
> I've even considered parsing the text in to_yaml and YAML.load and
> gsubbing "!foobar, 2007/" out and in again.  But there's got to be a
> better way. So far that better way is looking like xml. but there's got
> to be a better rubyful way.
>
> What is it?


there are simpler ways, but here is one that quite simple and also quite
extensible:



cfp:~ > ruby a.rb
- Foo:
     a: 4
     b: 2
- Foo::Bar:
     a: 40
     b: 1
     c: 1
#<Foo:0xb75aadc4 @b=2, @a=4>
#<Foo::Bar:0xb75aacc0 @b=1, @c=1, @a=40>



cfp:~ > cat a.rb
#
# define some classes which impliment the required '.from_hash' and
'#to_hash'
# duckface our ObjectList class will expect
#
   class Foo
     def self.from_hash hash
       new hash['a'], hash['b']
     end
     def initialize a, b
       @a, @b = a, b
     end
     def to_hash
       { 'a' => @a, 'b' => @b }
     end
   end
   class Foo::Bar
     def self.from_hash hash
       new hash['a'], hash['b'] , hash['c']
     end
     def initialize a, b, c
       @a, @b, @c = a, b, c
     end
     def to_hash
       { 'a' => @a, 'b' => @b, 'c' => @c }
     end
   end
#
# an ObjectList class adapted to store any objects which respond_to
'to_hash'
# and whose class responds_to 'from_hash'
#
   require 'yaml'
   class ObjectList < ::Array
     attr 'path'
     def initialize path
       load path
     end
     def load path
       @path = path
       replace YAML.load(( IO.read @path rescue '[]' ))
       map!{|entry| obj_from_entry entry}
     end
     def save path = @path
       yaml = map{|obj| entry_from_obj obj}.to_yaml
       open(path, 'w'){|fd| fd.write yaml}
     end
     def obj_from_entry entry
       classname, hash = entry.to_a.first
       constant_get(classname).from_hash hash
     end
     def entry_from_obj obj
       classname, hash = obj.class.name, obj.to_hash
       { classname => hash }
     end
     def constant_get hierachy
       ancestors = hierachy.split %r/::/
       parent = Object
       while((child = ancestors.shift))
         klass = parent.const_get(child) and parent = klass
       end
       klass
     end
   end
#
# setup a object_list object
#
   object_list = ObjectList.new 'object_list'
#
# store a few instances in the object_list
#
   object_list << Foo.new(4,2)
   object_list << Foo::Bar.new(40, 1, 1)
#
# save the object_list
#
   object_list.save
#
# show that save works
#
   puts IO.read(object_list.path)
#
# show that loading works
#
   object_list = ObjectList.new 'object_list'
   p object_list.first
   p object_list.last




-a
4299e35bacef054df40583da2d51edea?d=identicon&s=25 James Gray (bbazzarrakk)
on 2007-05-07 23:50
(Received via mailing list)
On May 7, 2007, at 2:14 PM, Rover Rhubarb wrote:

> and want my array of channels to look something like this
>
> - Channel
>   name: foo
>   id: 3
> - Channel
>   name: bar
>   id: 4

Hope this helps:

class Channel < Struct.new(:name, :id)
   def self.parse(io)
     io.inject(Array.new) do |channels, line|
       if line =~ /\bChannel\b/
         channels << new
       elsif not channels.empty? and line =~ /\A\s+(name|id):\s+(.+)/
         channels.last.send("#{$1}=", $2)
       end
       channels
     end
   end
end

if __FILE__ == $PROGRAM_NAME
   p Channel.parse(DATA)
end

__END__
- Channel
   name: foo
   id: 3
- Channel
   name: bar
   id: 4

James Edward Gray II
Aad37b5f7116c8d1f547d23b37566032?d=identicon&s=25 Greg Donald (destiney)
on 2007-05-08 00:38
(Received via mailing list)
On 5/7/07, Rover Rhubarb <rover.rhubarb@gmail.com> wrote:
> So far that better way is looking like xml. but there's got
> to be a better rubyful way.
>
> What is it?

json?

http://json.rubyforge.org/
61396c69040bbc56dbad1c5c71ac27d2?d=identicon&s=25 Rover Rhubarb (rhubarb)
on 2007-05-08 12:17
Thanks all, for the effort made in replying. I'm pleased to see some
solutions, and don't want to seem ungrateful, but I'm still a little
disillusioned that xml is looking more straightforward even now.

_why: thanks for the x-private tip, but I still have to put
"click-click-Channel". Simple enough to read, but honestly to the
average config hand-coder the xml <Channel> is easier to remember and
understand than the arcane "--- !!Channel". YAML should beat XML hands
down - this is one case where it doesn’t. I know it's not the same
because I'm going to have to parse the XML and build the objects -
whereas YAML is going to deserialize into lovely objects right there on
the spot - but still…

James: thanks for giving me a solution that does what I asked exactly.
But to be honest here I wanted to use an existing format, not invent
one. (I guess it could be argued that by wanting to customize my YAML to
get rid of the ---!! I am trying to invent one - but I really want that
to be an existing feature of YAML)

Greg: funny you mention json because I had a similar experience with it
recently. I was coding some ajax for the first time (building a progress
indicator) and I decided to pass my simple values as properties of a
javascript object using json. Hours of tricky debugging later (very hard
to find those bugs in the json code) I got it working but decided next
time I'd use XML. The extra parsing step might cost more, but the bugs
would be easier to find.

I guess _why has the closest to what I was looking for. x-private is
clearly even intended to be used for this.  (BTW sorry to bellyache
about docs, since I know its enough effort to code the libs in the first
place, but this can't be an uncommon request so maybe the x-private
could get more airtime in the docs somewhere). And I guess what I'd
really like is for the x-private tag in yaml_as to allow the --- !! to
be skipped to.

thanks again all
4299e35bacef054df40583da2d51edea?d=identicon&s=25 James Gray (bbazzarrakk)
on 2007-05-08 14:41
(Received via mailing list)
On May 8, 2007, at 5:17 AM, Rover Rhubarb wrote:

> YAML should beat XML hands down - this is one case where it doesn’t.

I think mine is an unpopular opinion among Rubyists, but I find XML
easier to write, as a human, than YAML.  YAML just has too many rules
I have to remember.  XML is definitely wordier, but I can keep the
rules in my head as I work.  My editor also makes managing those tags
fairly easy.

If all I'm doing is a simple edit here and there though, YAML is
easier on the eyes, for sure.

> I know it's not the same because I'm going to have to parse the XML
> and build the objects

I'm pretty sure there's a library or two that handles that:

http://raa.ruby-lang.org/cat.rhtml?
category_major=Library;category_minor=XML

> James: thanks for giving me a solution that does what I asked exactly.
> But to be honest here I wanted to use an existing format, not invent
> one. (I guess it could be argued that by wanting to customize my
> YAML to
> get rid of the ---!! I am trying to invent one - but I really want
> that
> to be an existing feature of YAML)

This is where we start to disagree.

You asked for simple and even showed examples of what you had in
mind.  I wrote code that parses it.  Simple pays off at both ends.

You wanted to drop the computerese; my version did.  In fact, mine
doesn't need those dashes.  With a one byte change, you could also
make indenting the attributes optional.  It gets better and better
for the human who needs to write it and it's still not complex on
your end.

Finally, I think it's easier to read or write than XML or YAML.

When your needs are big, of course you want to go with a proven
format.  What you showed is simple though and parsed just fine with a
few lines of code.  I don't see any shame in that.

James Edward Gray II
61396c69040bbc56dbad1c5c71ac27d2?d=identicon&s=25 Rover Rhubarb (rhubarb)
on 2007-05-09 16:19
James Gray wrote:

> I think mine is an unpopular opinion among Rubyists, but I find XML
> easier to write, as a human, than YAML.  YAML just has too many rules
> I have to remember.  XML is definitely wordier, but I can keep the
> rules in my head as I work.  My editor also makes managing those tags
> fairly easy.
>
> If all I'm doing is a simple edit here and there though, YAML is
> easier on the eyes, for sure.

Amen. I might not have agreed with you had I just read about YAML, but
after going through this process I'm a born again xml convert.

I also agree YAML is easier on the eyes. If were just ser-deser I was
after, I'd probably go with YAML.

>
>> I know it's not the same because I'm going to have to parse the XML
>> and build the objects
>
> I'm pretty sure there's a library or two that handles that:
>
> http://raa.ruby-lang.org/cat.rhtml?
> category_major=Library;category_minor=XML
>

Thanks for this tip.
Of these, http://clabs.org/clxmlserial.htm looks the most promising.

Note, btw, the home page of myxml
(http://raa.ruby-lang.org/project/myxml/), where it states:
    (* use YAML instead, it's 100% better for small apps ;-))

> This is where we start to disagree.
>
> You asked for simple and even showed examples of what you had in
> mind.  I wrote code that parses it.  Simple pays off at both ends.
>
> You wanted to drop the computerese; my version did.  In fact, mine
> doesn't need those dashes.  With a one byte change, you could also
> make indenting the attributes optional.  It gets better and better
> for the human who needs to write it and it's still not complex on
> your end.
>
> Finally, I think it's easier to read or write than XML or YAML.
>
> When your needs are big, of course you want to go with a proven
> format.  What you showed is simple though and parsed just fine with a
> few lines of code.  I don't see any shame in that.
>
> James Edward Gray II

Fair point.
This topic is locked and can not be replied to.