Forum: Ruby Marshaling NArray objects.

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
2ec38e324983babdd5c926c56f9e0c80?d=identicon&s=25 Alex Polite (Guest)
on 2006-05-25 05:37
(Received via mailing list)
I'm trying to marshal narray objects by first converting them to
ordinary arrays but I can't seem to get it right. I understand that I
have to implement marshal_dump and marshal_load for NArray so I've
tried this:

class NArray
  def marshal_dump
    self.to_a
  end

 def marshal_load(array)
   NArray.to_narray(array)
 end
end

And lots of other stuff. I won't bore you with the error messages,
suffice to say that I can't seem to get it right.

alex
E34b5cae57e0dd170114dba444e37852?d=identicon&s=25 Logan Capaldo (Guest)
on 2006-05-25 06:02
(Received via mailing list)
On May 24, 2006, at 11:35 PM, Alex Polite wrote:

> I'm trying to marshal narray objects by first converting them to
> ordinary arrays but I can't seem to get it right. I understand that I
> have to implement marshal_dump and marshal_load for NArray so I've
> tried this:
>
> class NArray
>  def marshal_dump
>    self.to_a
>  end


It's this part.
> def marshal_load(array)
>   NArray.to_narray(array)
> end
> end
>



You need something like
replace(NArray.to_narray(array))

Unfortunately, replace isn't a method for NArray.
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2006-05-25 06:02
(Received via mailing list)
Alex Polite wrote:
> def marshal_load(array)
>   NArray.to_narray(array)
> end
> end
>
> And lots of other stuff. I won't bore you with the error messages,
> suffice to say that I can't seem to get it right.

One problem is that #marshal_load needs to populate self with the data,
rather than just instantiating a new object. In other words, don't
bother returning a new NArray from marshal_load--it won't be used.

Example:

class C
  attr_accessor :x, :y

  def marshal_dump
    [x, y]
  end

  def marshal_load(ary)
    p ary
    @x, @y = ary # <-- populate self with persisted data
  end
end

c = C.new
c.x = "foo"
c.y = "bar"

s = Marshal.dump(c)
p Marshal.load(s)

__END__

Output:

["foo", "bar"]
#<C:0xb7dbc220 @y="bar", @x="foo">


Is there a NArray method like Array#replace ? That would probably do the
trick, if you called it in marshal_load with the narray as its argument.
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2006-05-25 06:30
(Received via mailing list)
Joel VanderWerf wrote:
> Is there a NArray method like Array#replace ? That would probably do the
> trick, if you called it in marshal_load with the narray as its argument.

This seems to work as a replace method:

a[] = b[]

Ye cats, NArray is nice. Why isn't it in the stdlib, again?
2ec38e324983babdd5c926c56f9e0c80?d=identicon&s=25 Alex Polite (Guest)
on 2006-05-25 07:05
(Received via mailing list)
On 5/25/06, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:
>   def marshal_load(ary)
>     p ary
>     @x, @y = ary # <-- populate self with persisted data
>   end

>
> Is there a NArray method like Array#replace ? That would probably do the
> trick, if you called it in marshal_load with the narray as its argument.
>

There doesn't seem to be anything like replace in narray. Maybe I
should resort to using a wrapper class around my narrays?

class NArrayWrapper
  def initialize(narray)
     @narray = narray
  end

   def marshal_dump
      @narray.to_a
   end

   def marshal_load(array)
     @narray = NArray.to_narray(array)
   end
end

But then I'd have to write a whole bunch of accessor methods and stuff.
I bet there's some C/ruby/narray wizardry that will replicate replace.

alex
2ec38e324983babdd5c926c56f9e0c80?d=identicon&s=25 Alex Polite (Guest)
on 2006-05-25 07:54
(Received via mailing list)
On 5/25/06, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:

> This seems to work as a replace method:
>
> a[] = b[]
>

===============
     1  require 'narray'
     2  class NArray
     3    def marshal_dump
     4      self.to_a
     5    end
     6
     7    def marshal_load(array)
     8      newnarray = NArray.to_narray(array)
     9      self[] = newnarray[]
    10    end
    11  end
    12
    13  na = NArray.float(10)
    14
    15  Marshal.load(Marshal.dump(na))
===============

Results in

marshaling_narrays.rb:9:in `[]=': wrong argument type NArray (expected
Data) (TypeError)
        from marshaling_narrays.rb:9:in `marshal_load'
        from marshaling_narrays.rb:15
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2006-05-25 07:57
(Received via mailing list)
On Thu, 25 May 2006, Alex Polite wrote:

>
>  end
>
>  def marshal_load(array)
>    @narray = NArray.to_narray(array)
>  end
> end
>
> But then I'd have to write a whole bunch of accessor methods and stuff.
> I bet there's some C/ruby/narray wizardry that will replicate replace.

are you sure need it?  you can write/read narrays from file quite
easily:


     harp:~ > cat a.rb
     require 'narray'

     na = NArray.int 2, 3

     na[0, true] = 42

     p na

     open('dat', 'w'){|f| f.write na.to_s}

     buf = IO.read('dat')

     na = NArray.to_na buf, NArray::INT, 2, 3

     p na


     harp:~ > ruby a.rb
     NArray.int(2,3):
     [ [ 42, 0 ],
       [ 42, 0 ],
       [ 42, 0 ] ]
     NArray.int(2,3):
     [ [ 42, 0 ],
       [ 42, 0 ],
       [ 42, 0 ] ]


so, assuming that you do, it's easy enough:

     harp:~ > cat a.rb
     require 'narray'

     class NArray
       def _dump *ignored
         Marshal.dump :typecode => typecode, :shape => shape, :data =>
to_s
       end
       def self._load buf
         h = Marshal.load buf
         typecode = h[:typecode]
         shape = h[:shape]
         data = h[:data]
         to_na data, typecode, *shape
       end
     end

     na = NArray.int 2, 3

     na[0, true] = 42

     dumped = Marshal.dump na

     na = Marshal.load dumped

     p na



     harp:~ > ruby a.rb
     NArray.int(2,3):
     [ [ 42, 0 ],
       [ 42, 0 ],
       [ 42, 0 ] ]


regards.


-a
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2006-05-25 10:29
(Received via mailing list)
Alex Polite wrote:
>     3    def marshal_dump
>    14
>
That's a bug in NArray. It should define an allocator function by
calling rb_define_alloc_func(), otherwise the T_OBJECT allocator is used
instead of an allocator that returns a T_DATA.

In this case, #marshal_load is being called on an object that is already
corrupt, before we even try to replace its contents: it is marked as
T_OBJECT, and so NArray's methods will fail on it, expecting T_DATA.

In any case, with the bug fixed or not, this won't work as written. The
typecode and shape would have to be added to the persisted data, as
Ara's solution points out. But more that that, there would be no way for
NArray to correctly allocate storage without knowing the typecode, so
the load method has to be a class method (_load), not an instance method
(marshal_load).
2ec38e324983babdd5c926c56f9e0c80?d=identicon&s=25 Alex Polite (Guest)
on 2006-05-26 21:26
(Received via mailing list)
On 5/25/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:
>
> are you sure need it?

yeah. I'm marshaling objects that have narrays as a component among
others.
>          typecode = h[:typecode]
>          shape = h[:shape]
>          data = h[:data]
>          to_na data, typecode, *shape
>        end
>      end
>

Thanks a bunch. That did the trick. Just one more thing. When I load a
marshalled object I can access the attributes on that object but I can
call it's methods. This is from irb. I've googled for stuff like
"marshaling methods" to no avail.

Alex
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2006-05-26 21:36
(Received via mailing list)
On Sat, 27 May 2006, Alex Polite wrote:

>>        def _dump *ignored
>>
>
> Thanks a bunch. That did the trick. Just one more thing. When I load a
> marshalled object I can access the attributes on that object but I can
> call it's methods. This is from irb. I've googled for stuff like
> "marshaling methods" to no avail.
>
> Alex

you must have something else going on, you needn't do anything special
to
marshal methods:

     harp:~ > cat a.rb
     require 'narray'
     class NArray
       def _dump *ignored
         Marshal.dump :typecode => typecode, :shape => shape, :data =>
to_s
       end
       def self._load buf
         h = Marshal.load buf
         typecode = h[:typecode]
         shape = h[:shape]
         data = h[:data]
         to_na data, typecode, *shape
       end
     end

     na = Marshal.load(Marshal.dump(NArray.int(4,2)))

     p na
     na[true, 0] = 42
     p na
     p na.size
     p na.shape


     harp:~ > ruby a.rb
     NArray.int(4,2):
     [ [ 0, 0, 0, 0 ],
       [ 0, 0, 0, 0 ] ]
     NArray.int(4,2):
     [ [ 42, 42, 42, 42 ],
       [ 0, 0, 0, 0 ] ]
     8
     [4, 2]


you'll have to show us your problem - probably you are returning the
wrong kind
of object from the class _load method.

regards.

-a
This topic is locked and can not be replied to.