We have had a really great integrated javascripting testing in our very
large, very javascripty Sinatra application. Our testing setup uses a
custom version of harmony (GitHub - baccigalupi/harmony: Javascript + DOM in your ruby, the simple way), that
reduces the out of control memory we were seeing in the original gem.
The trade off has been performance, but it has been worth it since
harmony without these modifications get above 2G of memory consumption.
That was bringing our development box to its knees. The custom version
of harmony creates individual window objects with each request which can
then be garbage collected at the end of usage. It worked great in rspec
1.x. Here was our setup:
describe ‘some javascript class’ do
before :all do @dom = Harmony::Page.new(my_ruby_view) @dom.load(some_js_files)
end
it ‘should do something’ do @dom.execute_js(‘javascript here’).should == what_we_expect
end
end
We are upgrading to RSpec 2, which has been a lot more involved and
undocumented than we had hoped.
Our biggest issue though is that the memory reduction measures that we
added to the harmony gem are no longer working. Presumably this is
because RSpec 2 is hanging on to the variables somewhere that we cannot
find. Setting our harmony Page objects to nil is not working:
in the spec_helper Rspec.configure block:
config.after(:all) do
puts ‘about to cleanup’ @dom = nil
GC.start # trying to cleanup via Ruby
puts Johnson.evaluate <<-JS
Johnson.runtime.gc(); // trying to cleanup via JS
JS
end
We have tried a lot of ordering combinations in our garbage collection
to see if anything will work, but instead the memory is climbing out of
control with each suite. In version RSpec 1.x we didn’t have to do any
manual garbage collection.
Does anyone have an idea of where the variable might be referenced in
RSpec 2 and how we can demand cleanup? For now we are going to have to
make a rake task that runs each spec separately, which will lead to a
not very useful testing task. Better than nothing, but it will cost us a
lot in developer time, going through all the output to find the
failures.
puts Johnson.evaluate <<-JS
Johnson.runtime.gc(); // trying to cleanup via JS
JS
end
We have tried a lot of ordering combinations in our garbage collection
to see if anything will work, but instead the memory is climbing out of
control with each suite. In version RSpec 1.x we didn’t have to do any
manual garbage collection.
Nor should you have to. Can you use after(:each) instead of after all?
puts Johnson.evaluate <<-JS
Johnson.runtime.gc(); // trying to cleanup via JS
JS
end
We have tried a lot of ordering combinations in our garbage collection
to see if anything will work, but instead the memory is climbing out of
control with each suite. In version RSpec 1.x we didn’t have to do any
manual garbage collection.
Nor should you have to. Can you use after(:each) instead of after all?
We set up our @dom in a before :all block and reuse it through out the
file. We don’t want to eliminate it after each test, just once per
block/file/whatever we set up.
The problem isn’t that the block isn’t getting called at the right time.
The problem is that dereferencing the instance variable doesn’t seem to
be releasing the variable for garbage collection like it was happening
in version 1.x. We don’t know why, and what might be different about
RSpec 2 variable references. More important, we are looking for a good
solution, any solution, to the memory problem.
The instance vars in after(:all) are copies, so setting them to nil
there has no real effect. This was true in rspec 1 as well, so what you
are experiencing is unrelated.
As a workaround, how about setting a global? Not a perm solution, but
might get your suite working for the moment.
We didn’t have to release our @dom variable in version 1.x, whether the
Harmony page object was defined in an :all or an :each block. The
variable just got released for garbage collection on its own at the end
of the file, and now it is not. Is it an impossibility to dereference
these variables if they are defined initially in an :all block?
We can try a global, but it isn’t an ideal situation.
We can try a global, but it isn’t an ideal situation.
So, we did replace @dom with $dom. That meant we could go back to doing
no manual garbage collection, and in fact we didn’t even have to set the
global to nil between test files.
Doesn’t that imply that RSpec 2 is holding on to stuff that could lead
to memory leaks in big tests for other folks too?
I am happy to get back to you about the documentation a little later. I
want to take the time to fully describe the problems we had.
Much appreciated.
JS
We set up our @dom in a before :all block and reuse it through out the
file. We don’t want to eliminate it after each test, just once per
block/file/whatever we set up.
The problem isn’t that the block isn’t getting called at the right time.
The problem is that dereferencing the instance variable doesn’t seem to
be releasing the variable for garbage collection like it was happening
in version 1.x. We don’t know why, and what might be different about
RSpec 2 variable references.
The instance vars in after(:all) are copies, so setting them to nil
there has no real effect. This was true in rspec 1 as well, so what you
are experiencing is unrelated.
As a workaround, how about setting a global? Not a perm solution, but
might get your suite working for the moment.
We can try a global, but it isn’t an ideal situation.
So, we did replace @dom with $dom. That meant we could go back to doing
no manual garbage collection, and in fact we didn’t even have to set the
global to nil between test files.
Doesn’t that imply that RSpec 2 is holding on to stuff that could lead
to memory leaks in big tests for other folks too?
