# Can you explain what this repo (yaml/Marshal) code do?

**URL:** <https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957>\
**Category:** Ruby\
**Created:** [August 2, 2006, 3:17am UTC](https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957 "2006-08-02T03:17:54Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\
**Post date:** [August 2, 2006, 3:17am UTC](https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957/1 "2006-08-02T03:17:54Z")

</div>

repositiory is a hash keyed on some kind of check sum derived from the  
file.  
But what is the use of Marshal? something to do with memory management,

instead of writing to file?  
What is the format of what is spewed out?

REPO\_FILE = “repo.bin”.freeze

class Repository  
attr\_accessor :main\_dir, :duplicate\_dir, :extensions

def initialize(extensions = %w{mp3 ogg})  
@extension = extensions  
@repository = {}  
end

def process\_dir(dir)  
# find all files with the extensions we support  
Dir[File.join(dir, “\*.{#{extensions.join(‘,’)}}”)].each do |f|  
process\_file( File.join(dir, f) )  
end  
end

def process\_file(file)  
digest = digest(file)  
name = @repository[digest]

```
if name
  target = duplicate_dir
  # ...
else
  target = main_dir
  # ...
end

FileUtils.cp( file, File.join( target, File.basename( file ) ) )

```

end

def digest(file)  
Digest::MD5.hexdigest( File.open(file, ‘rb’) {|io| io.read})  
end

def self.load(file)  
File.open(file, ‘rb’) {|io| Marshal.load(io)}  
end

def save(file)  
File.open(file, ‘wb’) {|io| Marshal.dump(self, io)}  
end  
end

repo = begin  
Repository.load( REPO\_FILE )  
rescue Exception =\> e

# not there =\> create

r = Repository.new  
r.main\_dir = “foo”  
r.duplicate\_dir = “bar”  
r  
end

ARGV.each {|dir| repo.process\_dir(dir)}

repo.save( REPO\_FILE )  
[http://groups.google.com/group/comp.lang.ruby/browse\_frm/thread/88c20ef88239c54a/fa358e55b7f86841?lnk=gst&q=repo&rnum=2#fa358e55b7f86841](http://groups.google.com/group/comp.lang.ruby/browse_frm/thread/88c20ef88239c54a/fa358e55b7f86841?lnk=gst&q=repo&rnum=2#fa358e55b7f86841)

---

<div class="post-metadata">

**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\
**Post date:** [August 2, 2006, 3:49am UTC](https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957/2 "2006-08-02T03:49:30Z")

</div>

On 8/1/06, anne001 [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> repositiory is a hash keyed on some kind of check sum derived from the  
> file.  
> But what is the use of Marshal? something to do with memory management,

Marshal is a way to save the state of your Ruby objects to a file.  
Unlike YAML, it outputs in binary format. It is written in C, so it  
can serialize and restore objects very quickly.

It looks like in this code, this is simply how the save and load  
functions are implemented, and since ‘self’ is being passed, it will  
just serialize the Repository object to file during save and restore  
it during load.

If you need readable, but much slower serialization, you could use  
YAML in place of the Marshal calls.

---

<div class="post-metadata">

**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\
**Post date:** [August 3, 2006, 12:22pm UTC](https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957/3 "2006-08-03T12:22:51Z")

</div>

> It looks like in this code, this is simply how the save and load  
> functions are implemented, and since ‘self’ is being passed, it will  
> just serialize the Repository object to file during save and restore  
> it during load.

but why do you need to save and load the objects,  
I have never seen code like this before. What do you gain?  
what is the problem that it resolves

---

<div class="post-metadata">

**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\
**Post date:** [August 3, 2006, 12:27pm UTC](https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957/4 "2006-08-03T12:27:38Z")

</div>

anne001 wrote:

> > It looks like in this code, this is simply how the save and load  
> > functions are implemented, and since ‘self’ is being passed, it will  
> > just serialize the Repository object to file during save and restore  
> > it during load.
> 
> but why do you need to save and load the objects,  
> I have never seen code like this before. What do you gain?  
> what is the problem that it resolves

It can be used to store objects on disk for future use (e.g. web  
application sessions) or to send objects between Ruby interpreters  
(only works with same interpreter version & object class loaded on  
both).

---

<div class="post-metadata">

**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\
**Post date:** [August 3, 2006, 12:37pm UTC](https://www.ruby-forum.com/t/can-you-explain-what-this-repo-yaml-marshal-code-do/100957/5 "2006-08-03T12:37:31Z")

</div>

On Aug 2, 2006, at 5:35 AM, anne001 wrote:

> > It looks like in this code, this is simply how the save and load  
> > functions are implemented, and since ‘self’ is being passed, it will  
> > just serialize the Repository object to file during save and restore  
> > it during load.
> 
> but why do you need to save and load the objects,  
> I have never seen code like this before. What do you gain?  
> what is the problem that it resolves

Well, let’s pretend you had some wiki class in your code:

```
 # a mock wiki object...
 class WikiPage
   def initialize( page_name, author, contents )
     @page_name = page_name
     @revisions = Array.new

     add_revision(author, contents)
   end

   attr_reader :page_name

   def add_revision( author, contents )
     @revisions << { :created => Time.now,
                     :author => author,
                     :contents => contents }
   end

   def wiki_page_references
     [@page_name] + @revisions.last[:contents].scan(/\b(?:[A-Z]+

```

[a-z]+){2,}/)  
end

```
   # ...
 end

```

Now, let’s assume you have a Hash of these things you are using to  
run your wiki:

```
 wiki = Hash.new
 [["HomePage", "James", "A page about the SillyEmailExamples..."],
   ["SillyEmailExamples", "James", "Blah, blah, blah..."] ].each

```

do |page|  
new\_page = WikiPage.new(\*page)  
wiki[new\_page.page\_name] = new\_page  
end

When your script runs you will need to save these pages to a disk  
somehow, so you don’t lose the site contents between runs. You have  
a ton of options here, of course, including using a database or  
rolling some method that can write these pages out to files.

Writing them out is a pain though because page contents can be pretty  
much anything, so you’ll need to come up with a good file format that  
allows you to tell where each revision starts and stops. This  
probably means handling some escaping characters of some kind, at the  
minimum.

Or, you can just use Marshal/YAML. With these helpers, saving the  
entire wiki is reduced to the trivial:

```
 File.open("wiki.dump", "w") { |file| Marshal.dump(wiki, file) }

```

When needed, you can load that back with:

```
 wiki = File.open("wiki.dump") { |file| Marshal.load(file) }

```

Those files will be stored in a binary format for Ruby to read. If  
you would prefer a human-readable format, replace the word Marshal  
with YAML above and make sure your script does a:

require “yaml”

See how easy it is to get instant saving/loading of entire Ruby  
structures?

James Edward G. II