GC oddness

regularfry · July 6, 2006, 5:30pm

Hi all,

Can someone smarter tell me why this fails? I’m expecting the TestDir
to get GC’d when I call GC.start in the test, but something’s obviously
keeping a reference lying around. I just can’t see what it is…

require ‘test/unit’
require ‘fileutils’

class TestDir
def TestDir.set_finalizer(testdir, dir)
ObjectSpace.define_finalizer(testdir, proc{|id|
FileUtils.rm_rf(dir)
}
end
def initialize(dir)
FileUtils.mkdir(dir)
TestDir.set_finalizer(self, dir)
end
end

class DestroyTest < Test::Unit::TestCase
def test_destroy
a = TestDir.new(‘test_dir’)
a = nil
GC.start
assert !File.directory?(‘test_dir’)
end
end

It’s got me a bit stumped.

Thanks,

regularfry · July 6, 2006, 5:33pm

Hi,

At Thu, 6 Jul 2006 23:58:27 +0900,
Alex Y. wrote in [ruby-talk:200382]:

Can someone smarter tell me why this fails? I’m expecting the TestDir
to get GC’d when I call GC.start in the test, but something’s obviously
keeping a reference lying around. I just can’t see what it is…

The finalizer proc refers testdir.

require ‘test/unit’
require ‘fileutils’

class TestDir
def TestDir.finalizer(dir)
proc{|id|
FileUtils.rm_rf(dir)
}
end
def initialize(dir)
FileUtils.mkdir(dir)
ObjectSpace.define_finalizer(self, TestDir.finalizer(dir))

regularfry · July 6, 2006, 5:50pm

[email protected] wrote:

The finalizer proc refers testdir.
I thought it might be that, but this also fails:

require ‘test/unit’
require ‘fileutils’

class TestDir
@@dir = nil
@@clearup = proc {|id| FileUtils.rm_rf(@@dir)}

def TestDir.set_finalizer(testdir, dir)
@@dir = dir
ObjectSpace.define_finalizer(testdir, @@clearup)
end
def initialize(dir)
FileUtils.mkdir(dir)
TestDir.set_finalizer(self, dir)
end
end

class DestroyTest < Test::Unit::TestCase
def test_destroy
a = TestDir.new(‘test_dir’)
a = nil
GC.start
assert !File.directory?(‘test_dir’)
end
end

testdir hasn’t come into existence when the proc’s created this time, so
I don’t see how it can be keeping a reference to it, or how a finalizer
proc keeping a reference to the object that’s being finalized can stop
GC - if that’s the case, how does the object ever get GC’d? What have I
missed?

regularfry · July 7, 2006, 1:44am

On Jul 6, 2006, at 8:46 AM, Alex Y. wrote:

ObjectSpace.define_finalizer(testdir, @@clearup)
a = nil
GC.start
assert !File.directory?('test_dir')
end
end

testdir hasn’t come into existence when the proc’s created this
time, so I don’t see how it can be keeping a reference to it

Ruby’s garbage collector is conservative, so their may be references
to TestDir lying around on the C stack.

or how a finalizer proc keeping a reference to the object that’s
being finalized can stop GC - if that’s the case, how does the
object ever get GC’d? What have I missed?

Procs contain the environment of their creation. In other words,
they keep references to all variables visible at their creation time
including self. When using a finalizer proc you must ensure that
finalizer doesn’t refer to the object it is supposed to clean up after.

weakref.rb contains an example of use of finalizers, including a map
between object ids and objects to be cleaned up after.

PS: What you want to do is easier with blocks:

def testdir(dir)
FileUtils.mkdir dir
begin
yield
ensure
FileUtils.rm_rf dir
end
end

testdir ‘mytempdir’ do

…

end

–
Eric H. - [email protected] - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

regularfry · July 7, 2006, 9:58am

Yohanes S. wrote:

 @@dir = dir
 a = TestDir.new('test_dir')
Loaded suite ./gc
Started
Cleaned
GC invoked
Done
.
Finished in 0.015623 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
That’s… odd.

$ ruby -v
ruby 1.8.4 (2005-12-24) [i386-linux]

Which OS and ruby are you using?

$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

It’s on Ubuntu 5.10.

regularfry · July 7, 2006, 10:11am

Eric H. wrote:

On Jul 6, 2006, at 8:46 AM, Alex Y. wrote:

>> ObjectSpace.define_finalizer(testdir, @@clearup) >> a = nil >> GC.start >> assert !File.directory?('test_dir') >> end >> end >> ---------------------------- >> >> or how a finalizer proc keeping a reference to the object that's >> being finalized can stop GC - if that's the case, how does the object >> ever get GC'd? What have I missed? > > Procs contain the environment of their creation. In other words, they > keep references to all variables visible at their creation time > including self. When using a finalizer proc you must ensure that > finalizer doesn't refer to the object it is supposed to clean up after. That's what the code above ensures, I *think*...

weakref.rb contains an example of use of finalizers, including a map
between object ids and objects to be cleaned up after.
Thanks, I’ll take a look at that.

end

testdir ‘mytempdir’ do

…

end
Unfortunately, that’s not going to work here. What I’m trying to do is
create a drop-in replacement for a small subset of Array’s functionality
which serialises its contents to disk rather than keeping it in memory,
for some existing code. The idea is that a new instance creates a
directory for itself on creation, and cleans up after itself when it
gets GC’d. The existing code structure doesn’t lend itself well to
blocking. Of course, since everything gets GC’d on exit anyway, it’s
not exactly a huge problem, but it is making testing a tad tricky, and
surprised me somewhat…

regularfry · July 7, 2006, 2:15am

Alex Y. [email protected] writes:

I thought it might be that, but this also fails:
It does not fail on me:

require ‘test/unit’
require ‘fileutils’

class TestDir
@@dir = nil
@@clearup = proc {|id| FileUtils.rm_rf(@@dir); puts “Cleaned”}

def TestDir.set_finalizer(testdir, dir)
@@dir = dir
ObjectSpace.define_finalizer(testdir, @@clearup)
end
def initialize(dir)
FileUtils.mkdir(dir)
TestDir.set_finalizer(self, dir)
end
end

class DestroyTest < Test::Unit::TestCase
def test_destroy
a = TestDir.new(‘test_dir’)
a = nil
GC.start
puts “GC invoked”
assert !File.directory?(‘test_dir’)
puts “Done”
end
end

$ ruby ./gc.rb
Loaded suite ./gc
Started
Cleaned
GC invoked
Done
.
Finished in 0.015623 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

$ ruby -v
ruby 1.8.4 (2005-12-24) [i386-linux]

Which OS and ruby are you using?

YS.

regularfry · July 7, 2006, 10:14am

fr Alex:

That’s… odd.

> $ ruby -v

> ruby 1.8.4 (2005-12-24) [i386-linux]

>

> Which OS and ruby are you using?

$ ruby -v

ruby 1.8.4 (2005-12-24) [i486-linux]

It’s on Ubuntu 5.10.

can’t help, but fwiw,

:~# ruby -w gc.rb
Loaded suite gc
Started
GC invoked
F
Finished in 0.102665 seconds.

Failure:
test_destroy(DestroyTest) [gc.rb:24]:
is not true.

1 tests, 1 assertions, 1 failures, 0 errors
Cleaned
:~# uname -an
Linux pc4all 2.6.15-25-686 #1 SMP PREEMPT Wed Jun 14 11:34:19 UTC 2006
i686 GNU/Linux
:~# cat /etc/issue
Ubuntu 6.06 LTS \n \l

kind regards -botp :~#

regularfry · July 7, 2006, 12:59pm

Peña wrote:

It’s on Ubuntu 5.10.

can’t help, but fwiw,

Failure:
test_destroy(DestroyTest) [gc.rb:24]:
is not true.

1 tests, 1 assertions, 1 failures, 0 errors

Thanks for that, it’s good to know it’s not just me… Ok, next
question: which is the correct behaviour?

regularfry · July 7, 2006, 4:25pm

Alex Y. [email protected] writes:

> 1) Failure: > test_destroy(DestroyTest) [gc.rb:24]: > is not true. > 1 tests, 1 assertions, 1 failures, 0 errors
Thanks for that, it’s good to know it’s not just me… Ok, next
question: which is the correct behaviour?

Mine has the correct behaviour.

The finaliser has no reference to the the TestDir instance being
finalised.

YS.

regularfry · July 7, 2006, 4:35pm

Yohanes S. wrote:

Alex Y. [email protected] writes:

Thanks for that, it’s good to know it’s not just me… Ok, next

question: which is the correct behaviour?

Mine has the correct behaviour.

The finaliser has no reference to the the TestDir instance being
finalised.

That’s what I thought. Which OS are you using? Did you build your Ruby
yourself?

regularfry · July 7, 2006, 5:13pm

Alex Y. [email protected] writes:

That’s what I thought. Which OS are you using? Did you build your
Ruby yourself?

$ ruby -v
ruby 1.8.4 (2005-12-24) [i386-linux]

Debian 3.1. The ruby was locally build from debian testing’s
ruby-1.8.4 source package.

YS.

regularfry · July 8, 2006, 7:31am

On Fri, Jul 07, 2006 at 12:46:50AM +0900, Alex Y. wrote:

class DestroyTest < Test::Unit::TestCase
I don’t see how it can be keeping a reference to it, or how a finalizer
proc keeping a reference to the object that’s being finalized can stop
GC - if that’s the case, how does the object ever get GC’d? What have I
missed?

–
Alex

To get slightly off the track, I don’t think using finalizers is the
best way to tackle your problem.

Are you per chance coming from C++? The RAI-pattern (Resource
Aquisition is Initialisation) is very C+±specific and doesn’t work
well for any language where destructors (or finalizers) don’t get to
run at fixed times. Consider instead something more rubyish like this:

def with_testdir(dir)
raise “needs a block” unless block_given?
setup(dir)
yield dir # calls your given block
ensure
cleanup(dir) # is executed right after your block finishes
end

with_test_dir("/tmp/test") do |dir|
puts “now working in #{dir}”

your code

end

-JÃ¼rgen

regularfry · July 8, 2006, 6:36pm

Eric H. wrote:

On Jul 7, 2006, at 1:08 AM, Alex Y. wrote:

Unfortunately, that’s not going to work here. What I’m trying to do
is create a drop-in replacement for a small subset of Array’s
functionality which serialises its contents to disk rather than
keeping it in memory, for some existing code.

Have you looked at PStore?
Yes. The serialisation isn’t the problem - it’s the tidying up
afterwards. While I’m actually using YAML for the serialisation dump of
the array entries, I could quite easily swap it out for PStore if it
turns out that transactions become vital. I might do that anyway -
PStore’s mature, and my code isn’t

regularfry · July 7, 2006, 10:42pm

On Jul 7, 2006, at 1:08 AM, Alex Y. wrote:

Unfortunately, that’s not going to work here. What I’m trying to
do is create a drop-in replacement for a small subset of Array’s
functionality which serialises its contents to disk rather than
keeping it in memory, for some existing code.

Have you looked at PStore?

–
Eric H. - [email protected] - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

regularfry · July 10, 2006, 1:41pm

Joel VanderWerf wrote:

Yes. The serialisation isn’t the problem - it’s the tidying up
afterwards. While I’m actually using YAML for the serialisation dump
of the array entries, I could quite easily swap it out for PStore if
it turns out that transactions become vital. I might do that anyway -
PStore’s mature, and my code isn’t

You might find fsdb an interesting alternative to pstore:

http://raa.ruby-lang.org/project/fsdb/

That looks quite intriguing. I’ll have to take a look at that.

regularfry · July 9, 2006, 2:25am

Alex Y. wrote:

afterwards. While I’m actually using YAML for the serialisation dump of
the array entries, I could quite easily swap it out for PStore if it
turns out that transactions become vital. I might do that anyway -
PStore’s mature, and my code isn’t

You might find fsdb an interesting alternative to pstore:

http://raa.ruby-lang.org/project/fsdb/

GC oddness

class DestroyTest < Test::Unit::TestCase def test_destroy a = TestDir.new(‘test_dir’) a = nil GC.start assert !File.directory?(‘test_dir’) end end

class DestroyTest < Test::Unit::TestCase def test_destroy a = TestDir.new(‘test_dir’) a = nil GC.start assert !File.directory?(‘test_dir’) end end

end end

…

…

That’s… odd.

> $ ruby -v

> ruby 1.8.4 (2005-12-24) [i386-linux]

>

> Which OS and ruby are you using?

$ ruby -v

ruby 1.8.4 (2005-12-24) [i486-linux]

It’s on Ubuntu 5.10.

It’s on Ubuntu 5.10.

your code

class DestroyTest < Test::Unit::TestCase
def test_destroy
a = TestDir.new(‘test_dir’)
a = nil
GC.start
assert !File.directory?(‘test_dir’)
end
end

class DestroyTest < Test::Unit::TestCase
def test_destroy
a = TestDir.new(‘test_dir’)
a = nil
GC.start
assert !File.directory?(‘test_dir’)
end
end

end
end