Serialization ruby 1.8 vs ruby 1.9

odo · January 7, 2010, 11:16am

Hi,

I am trying to marshal objects with ruby 1.8 and read them with 1.9 but
I get an error on date objects:
ruby 1.8:
f=File.new(’/tmp/date’, ‘w+’); f.puts Marshal.dump(Date.today); f.close

ruby 1.9:
m=File.read(’/tmp/date’); Marshal.load(m)

gives me:
class Date needs to have method `_load’ (TypeError)

The versions of Marshal are identical, but in 1.8, Date has a _dump and
_load method, in 1.9 it does not.

Am I missing something or is this a bug?
Any hints are appreciated.

Thanks,
Florian

odo · January 7, 2010, 11:41am

On 2010-01-07 05:16:38 -0500, Florian O. said:

gives me:
class Date needs to have method `_load’ (TypeError)

The versions of Marshal are identical, but in 1.8, Date has a _dump and
_load method, in 1.9 it does not.

Am I missing something or is this a bug?
Any hints are appreciated.

Thanks,
Florian

Marshal isn’t constant, it can only be expected to work when you are
using the same version on Ruby.

I’d suggest using YAML.

odo · January 7, 2010, 2:11pm

On 01/07/2010 11:16 AM, Florian O. wrote:

gives me:
class Date needs to have method `_load’ (TypeError)

The versions of Marshal are identical, but in 1.8, Date has a _dump and
_load method, in 1.9 it does not.

Am I missing something or is this a bug?
Any hints are appreciated.

James is right: the format of Marshal is not guaranteed to be portable
across Ruby versions. Nevertheless some types do work.

Btw, Marshal’s format is binary. Which means two things

you should open files in binary mode,
printing methods like #puts are not guaranteed to work.

If you observe these rules you can make it work with Time…

robert@fussel:~$ ruby1.8 -e
‘File.open(“x”,“wb”){|i|Marshal.dump(Time.now,i)}’
robert@fussel:~$ ls -l x
-rw-r–r-- 1 robert robert 18 2010-01-07 13:44 x
robert@fussel:~$ ruby19 -e ‘p(File.open(“x”,“rb”){|i|Marshal.load(i)})’
2010-01-07 13:44:40 +0100
robert@fussel:~$

… but apparently neither Date nor DateTime:

robert@fussel:~$ ruby1.8 -r date -e
‘File.open(“x”,“wb”){|i|Marshal.dump(Date.today,i)}’
robert@fussel:~$ ruby19 -r date -e
‘p(File.open(“x”,“rb”){|i|Marshal.load(i)})’
-e:1:in load': class Date needs to have method _load’ (TypeError)
from -e:1:in block in <main>' from -e:1:in open’
from -e:1:in `’
robert@fussel:~$

robert@fussel:~$ ruby1.8 -r date -e
‘File.open(“x”,“wb”){|i|Marshal.dump(DateTime.now,i)}’
robert@fussel:~$ ruby19 -r date -e
‘p(File.open(“x”,“rb”){|i|Marshal.load(i)})’
-e:1:in load': class DateTime needs to have method _load’ (TypeError)
from -e:1:in block in <main>' from -e:1:in open’
from -e:1:in `’
robert@fussel:~$

It does work for Date and DateTime when only using one version

robert@fussel:~$ ruby19 -r date -e
‘File.open(“x”,“wb”){|i|Marshal.dump(DateTime.now,i)}’
robert@fussel:~$ ruby19 -r date -e
‘p(File.open(“x”,“rb”){|i|Marshal.load(i)})’
#<DateTime: 2010-01-07T13:54:15+01:00
(2356995876177376393/960000000000,1/24,2299161)>
robert@fussel:~$ ruby19 -r date -e
‘File.open(“x”,“wb”){|i|Marshal.dump(Date.today,i)}’
robert@fussel:~$ ruby19 -r date -e
‘p(File.open(“x”,“rb”){|i|Marshal.load(i)})’
#<Date: 2010-01-07 (4910407/2,0,2299161)>
robert@fussel:~$

You have quite a few options:

use a type that works, e.g. String:

robert@fussel:~$ ruby1.8 -r date -e
‘File.open(“x”,“wb”){|i|Marshal.dump(Date.today.strftime,i)}’
robert@fussel:~$ ruby19 -r date -e
‘p(Date.strptime(File.open(“x”,“rb”){|i|Marshal.load(i)}))’
#<Date: 2010-01-07 (4910407/2,0,2299161)>
robert@fussel:~$

create custom Marshalling and demarshalling methods which use types
that work.
create your custom date type which encapsulates a Date but uses
customized Marshal serialization.

You can see an example for customized persistence in section “Custom
Persistence” on
http://blog.rubybestpractices.com/posts/rklemme/018-Complete_Class.html
.

you use another serialization mechanism, like Yaml:

robert@fussel:~$ ruby1.8 -r date -r yaml -e
‘File.open(“x”,“wb”){|i|YAML.dump(Date.today,i)}’
robert@fussel:~$ ruby19 -r date -r yaml -e
‘p(File.open(“x”,“rb”){|i|YAML.load(i)})’
#<Date: 2010-01-07 (4910407/2,0,2299161)>
robert@fussel:~$

use yet another completely different format.

There are probably more options…

Kind regards

robert

odo · January 7, 2010, 2:56pm

On 01/07/2010 02:32 PM, Florian O. wrote:

Thank you for your very informative answers.

You’re welcome!

I was missing the thing with the binary mode. In the real use case I am
writing to and reading from Memcache.

I tried to serialize the data with YAML and JSON which turned out to be
far too slow.

I followed your suggestion and am now converting dates to strings and
parsing them at the receiving end which works fine.

If performance is crucial you should probably check parsing performance.
I remember I did the parsing myself once because #strptime was too
slow. Ah, found it:

github.com

rklemme/muppet-laboratories/blob/master/bin/sample-animal.rb

#!/usr/local/bin/ruby19 -w

# Implementation of the parser for the sample
# generator script test-gen.rb.
$: << File.join(File.dirname(File.dirname($0)), "lib")

require 'animal'

# main defines the custom parser class!
Animal.main do

  attr_accessor :year
  attr_reader :interaction_id, :time_stamp

  def parse(line)
    if %r{
       ^
       ( \d{4} ) - ( \d{2} ) - ( \d{2} ) 
       \s
       ( \d{2} ) : ( \d{2} ) : ( \d{2}(?:\.\d+)? )

This file has been truncated. show original

If you can better use a numeric type like Time#to_i or Time#to_f. It
may be that you can get at the rational inside Date.

irb(main):001:0> d=Date.today
=> #<Date: 2010-01-07 (4910407/2,0,2299161)>
irb(main):002:0> d.instance_variables
=> [:@sg, :@of, :@ajd, :@ca]
irb(main):003:0> d.instance_variables.map {|iv|
d.instance_variable_get(iv)}
=> [2299161, 0, (4910407/2), {252120=>2455204, 258504=>[2010, 1, 7]}]

Then you “only” need to transform that back into a Date.

Kind regards

robert

odo · January 7, 2010, 2:32pm

Thank you for your very informative answers.

I was missing the thing with the binary mode. In the real use case I am
writing to and reading from Memcache.

I tried to serialize the data with YAML and JSON which turned out to be
far too slow.

I followed your suggestion and am now converting dates to strings and
parsing them at the receiving end which works fine.

Cheers,
Florian