I’m working on a program that manipulates and stores a lot of data in
a database via ActiveRecord. I’m having a heck of a time trying to
write unit and functional tests for this program because I don’t have
a reasonable way to reload the test db. I’d like to use the fixtures
facility from Rails but so far I’ve failed at ripping it out and
moving it to a non-Rails environment.
(BTW, I’m using the MockFS gem to test the filesystem portions of the
code and I have to say it is working brilliantly. Many thanks to
the author!)
Anyone have experience with this type of work? Care to share any tips
on getting this to work?
Thanks very much.
cr
On May 30, 2006, at 2:33 PM, [email protected] wrote:
Anyone have experience with this type of work? Care to share any
tips on getting this to work?
Thanks very much.
cr
Hey-
I had to do the same thing to test an activerecord extension plugin
outside of rails. You can feel free to steal the test harness from
my plugin:
http://opensvn.csie.org/ezra/rails/ez_where_two/test/test_helper.rb
Thats the test_helper that loads activerecord with a database.yml
and then defines some models and loads the fixtures as well. Have a
look around the plugin for the way I made this happen.
-Ezra
On May 30, 2006, at 8:34 PM, Ezra Z. wrote:
(BTW, I’m using the MockFS gem to test the filesystem portions of
look around the plugin for the way I made this happen.
You’re a life saver! Thank you!
cr
Chuck R.
[email protected]
www.familyvideovault.com (not yet live!)
On May 31, 2006, at 9:47 AM, Pistos C. wrote:
problems using AR outside of Rails. I have been writing an
application
(IRC game) which uses AR with PostgreSQL, and I have had uncountable
problems due to the way AR connects to the database. I am constantly
hitting the maximum connection limit, and stray PostgreSQL child
processes are always being left around.
What DB are you using?
I’m using mysql 5.018 (latest AR, ruby 1.8.4 all on OSX 10.4.6) and
so far I haven’t run into any problems like you describe. But since
you’ve mentioned them, I’ll make sure my tests include detecting and
handling those types of AR exceptions.
BTW, using Ezra’s sample code I’ve already gotten the fixtures
loading in my tests. Excellent stuff. I’ll do a quick write-up after
I’ve worked out all the kinks.
cr
unknown wrote:
I’m working on a program that manipulates and stores a lot of data in
a database via ActiveRecord. I’m having a heck of a time trying to
Anyone have experience with this type of work? Care to share any tips
on getting this to work?
A bit away from your original topic (and perhaps this belongs on the
Rails mailing list), but: I’d be interested to hear if you run into any
problems using AR outside of Rails. I have been writing an application
(IRC game) which uses AR with PostgreSQL, and I have had uncountable
problems due to the way AR connects to the database. I am constantly
hitting the maximum connection limit, and stray PostgreSQL child
processes are always being left around.
What DB are you using?
Pistos
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Pistos C. wrote:
Rails mailing list), but: I’d be interested to hear if you run into any
problems using AR outside of Rails. I have been writing an application
(IRC game) which uses AR with PostgreSQL, and I have had uncountable
problems due to the way AR connects to the database. I am constantly
hitting the maximum connection limit, and stray PostgreSQL child
processes are always being left around.
What DB are you using?
Let me guess, your IRC game is multi-threaded? AR stores connections
based on the Thread.current_thread.id (or at least it does in
1.13.2, things changed slightly in 1.14.2). I ran into this problem as
well. If you are using AR 1.13.2 I can give you a fix,
otherwise you’ll have to do it yourself or wait about a week for me to
patch 1.14.2 (if it needs it).
Here is the code:
Attempt to load active-record version, but if this dies, then don’t
complain
Newer versions of rails seem to include VERSION by default, however
activerecord-1.13.2 doesn’t.
begin ; require ‘active_record/version’ ; rescue LoadError; end
This includes changes to ActiveRecord 1.13.2
class ActiveRecord::Base
# This is overridden because ActiveRecord 1.13.2 stores database
connections based on the
# Thread id. The processor uses a new thread for each new set of
files to process.
# We want the same connection to be used.
#
# Version 1.14.2 no longer has the class variable
@@connection_cache, so we should
# not redefine this method if we are at or above that version.
if ActiveRecord::VERSION::STRING < ‘1.14.2’
def self.connection
@@connection_cache[123456][name] ||= retrieve_connection
end
end
end
How this helps,
Zach
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFEffr/Myx0fW1d8G0RAkJLAJ9YeHhrlNCce4HwQGC+StwhAv4xEgCeLMX9
LgMa1ecsMSCxynmCRs0BOYc=
=Igay
-----END PGP SIGNATURE-----
zdennis wrote:
Let me guess, your IRC game is multi-threaded? AR stores connections
I think you’re onto something there… 
based on the Thread.current_thread.id (or at least it does in
1.13.2, things changed slightly in 1.14.2). I ran into this problem as
well. If you are using AR 1.13.2 I can give you a fix,
otherwise you’ll have to do it yourself or wait about a week for me to
patch 1.14.2 (if it needs it).
I happen to be specifically require_gem’ing 1.13.2 for the game, solely
because stuff after 1.13.2 caused me even worse problems…
Here is the code:
How this helps,
Zach
I put the code in place, and it seems to be helping, based on
preliminary testing. It’s too early to tell for sure, though, so I will
keep you posted. Thanks for this patch. If it works, it will surely be
a great boon to the game.
Pistos
zdennis wrote:
You can actually test this in a unit test. =)
There is also a method on ActiveRecord::Base called ‘active_connections’
There is another method on ActiveRecord::Base called
‘threaded_connections’ which returns true or false (by default true). I
[much snippage above]
Well, unit test or not, the evidence is here:
ps aux | grep postgres | grep idle | grep -v grep | wc -l
22
template1=# select count(*) from pg_stat_activity where current_query =
‘’ and query_start < NOW() - ‘2 hours’::INTERVAL;
count
18
The good news is, some of the processes DO seem to go away on their own.
The bad news is, some stay around for some time:
template1=# select count(*) from pg_stat_activity where current_query =
‘’ and query_start < NOW() - ‘20 hours’::INTERVAL;
count
2
As well, I haven’t yet actually gotten the “too many connections” error
from PostgreSQL, so that is somewhat encouraging.
Furthermore, we should bear in mind that some of these connections are
from the companion Rails site that works with the game (shared DB). I’m
not sure how to distinguish Rails connections from game connections.
I’m using FastCGI, and, as far as I know, that keeps processes alive.
If you ever find time, maybe we can work together on the problem via
IRC? I’m on FreeNode often with the nick Pistos.
Pistos
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Pistos C. wrote:
well. If you are using AR 1.13.2 I can give you a fix,
Zach
I put the code in place, and it seems to be helping, based on
preliminary testing. It’s too early to tell for sure, though, so I will
keep you posted. Thanks for this patch. If it works, it will surely be
a great boon to the game. 
Pistos,
You can actually test this in a unit test. =)
def test_connection_count
2.times do
Thread.new{ Job.find_all }
end
assert_equal 1, ActiveRecord::Base.module_eval( “@@connection_cache”
).size, “Too many connections are being created!”
end
There is also a method on ActiveRecord::Base called ‘active_connections’
and it seems to always return 1, however if you actually
do the module_eval for @@connection_cache, you will see there are more
connections (that are potentially active) then
‘active_connections’ returns.
There is another method on ActiveRecord::Base called
‘threaded_connections’ which returns true or false (by default true). I
was
hoping that if you set it to false, it would fix the problem, but
setting it to false doesn’t seem to do anything for this
particular problem.
I am gone for the weekend, if you find anything else out please post so
I can read when I get back, otherwise I’ll dive in over
the weekend to AR and find out what I can (and to see if AR 1.14.2 has
this problem)
Have a good weekend Pistos!
Zach
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFEf6oRMyx0fW1d8G0RAlMEAJ0Q3nfhiYUGfJsjjPbZQEgvqlcf4gCbB6Hv
jsb6ZJpOF+cFnzBjJ0sC3z0=
=V9ES
-----END PGP SIGNATURE-----