In-memory SQLite for testing

fxn · December 22, 2005, 11:12pm

I am following the rails book, and arrived to the section about
testing. The test database is configured as:

test:
adapter: sqlite3
database: “:memory:”

and I get this error with the simple product test:

% ruby test/unit/product_test.rb
Loaded suite test/unit/product_test
Started
E/usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/lib/sqlite3/
errors.rb:94:in check': cannot rollback - no transaction is active (SQLite3::SQLException) from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/ lib/sqlite3/resultset.rb:76:incheck’
from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/
lib/sqlite3/resultset.rb:68:in commence' from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/ lib/sqlite3/resultset.rb:61:ininitialize’
from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/
lib/sqlite3/statement.rb:158:in execute' from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/ lib/sqlite3/database.rb:211:inexecute’
from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/
lib/sqlite3/database.rb:186:in prepare' from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/ lib/sqlite3/database.rb:210:inexecute’
from /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.1.0/
lib/sqlite3/database.rb:620:in rollback' ... 11 levels... from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:200:inrun’
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:13:in
`run’
from /usr/local/lib/ruby/1.8/test/unit.rb:285
from test/unit/product_test.rb:7

Any ideas?

– fxn

fxn · December 25, 2005, 7:16pm

Hi,

On 12/25/05, Daniel H. [email protected] wrote:

In addition, the sqlite adapter needs the
‘dbfile:’ argument rather than ‘database:’.

On Rails 1.0 and Rails trunk, you can use either “dbfile” or “database”.
See changeset [2825] for details.

Lastly, use “:memory”
instead of “:memory:”, omit the second colon. That made it work for
me.

No, this tells SQLite to use a file on your filesystem named “:memory”.

| $ ls
| $ sqlite :memory: ‘CREATE TABLE test (id INTEGER)’
| $ ls
| $ sqlite :memory ‘CREATE TABLE test (id INTEGER)’
| $ ls
| :memory

–
sam

fxn · December 25, 2005, 11:37pm

yeah, whoops. I just assumed when it stopped giving me errors that it
had worked. I guess I should examine things more carefully next time.
This means that I still do not have it working.

Does anyone have any insights?

Thanks and sorry for any confusion,
Daniel H.

fxn · December 29, 2005, 6:51pm

Also keep in mind that SQLite does not support nested transactions.
So if your test starts a transaction (the default behavior) and then
your production code under test starts another transaction, the test
transaction rollback probably won’t work.

Scott

fxn · December 29, 2005, 8:24pm

On Dec 29, 2005, at 18:50, Scott W. wrote:

Also keep in mind that SQLite does not support nested transactions.
So if your test starts a transaction (the default behavior) and
then your production code under test starts another transaction,
the test transaction rollback probably won’t work.

Ah, this might be relevant.

My test is a pretty simple unit test for the Product model of Depot,
but maybe ActiveRecord is using some transaction behind the scenes.

If that was the case, why surprises me is that SQLite in-memory for
test is what rails(1) generates. If it does not work even in the
simplest of the tests it shouldn’t be there.

– fxn

fxn · January 3, 2006, 7:16pm

But it might work if you don’t use transactional fixtures. Try changing
the
line in test/test_helper.rb to

self.use_transactional_fixtures = false

Since this works with SQLite-on-disk.

fxn · December 25, 2005, 2:42pm

I have not been able to get the ‘sqlite3’ adapter to work, just using
‘sqlite’ should work. In addition, the sqlite adapter needs the
‘dbfile:’ argument rather than ‘database:’. Lastly, use “:memory”
instead of “:memory:”, omit the second colon. That made it work for
me.

Dan

fxn · December 25, 2005, 11:58pm

The problem occurs when Rails attempts to rollback the transaction. It
seems like sqlite may be dropping the database (i.e. removing it from
memory) earlier than it should and thus when it comes time to close
the transaction, the database is already gone.

Could this be it?

Daniel H.

fxn · January 4, 2006, 2:41am

Matthew P. wrote:

On Tue, Jan 03, 2006 at 01:02:29PM -0500, Andrew S. wrote:

But it might work if you don’t use transactional fixtures. Try changing the
line in test/test_helper.rb to

self.use_transactional_fixtures = false

As several people have reported in the past, this does not work.
The problem isn’t the transactions, it’s the complete lack of tables in
the
in-memory database. Please stop suggesting it as a solution.

Matt

What do you mean by ‘the complete lack of tables in the in-memory
database’? SQLite in-memory databases do support tables…

For an article about SQLite in-memory databases from the PHP
perspective:
http://www.filipdewaard.com/21_SQLite_inmemory_databases.html

fxn · January 4, 2006, 6:20pm

Helmut Sedding wrote:

Am 04.01.2006 um 16:11 schrieb Andrew S.:

It would appear that the in-memory sqlite database example in
database.yml is just a cruel hoax…

I assume if it worked, the speed wouldn’t be better. The OS caches
filesystem accesses anyway.

No, SQLite calls fsync() when a transaction is committed.

One alternative: use a ramdisk.

That improves performance considerably.

fxn · January 3, 2006, 10:59pm

On Tue, Jan 03, 2006 at 01:02:29PM -0500, Andrew S. wrote:

But it might work if you don’t use transactional fixtures. Try changing the
line in test/test_helper.rb to

self.use_transactional_fixtures = false

As several people have reported in the past, this does not work.
The problem isn’t the transactions, it’s the complete lack of tables in
the
in-memory database. Please stop suggesting it as a solution.

Matt

fxn · January 4, 2006, 6:12pm

So as I understand it, because the test database is generated in a
separate
rake task, and it disappears once that task completes, the in-memory
database you get when you tests start to run is in fact a new, empty
database.

It would appear that the in-memory sqlite database example in
database.ymlis just a cruel hoax…

fxn · January 4, 2006, 6:14pm

Am 04.01.2006 um 16:11 schrieb Andrew S.:

It would appear that the in-memory sqlite database example in
database.yml is just a cruel hoax…

I assume if it worked, the speed wouldn’t be better. The OS caches
filesystem accesses anyway.

One alternative: use a ramdisk.

regards,
Helmut

fxn · January 4, 2006, 6:35pm

On Jan 4, 2006, at 18:20, Andreas S. wrote:

One alternative: use a ramdisk.

That improves performance considerably.

Nevertheless the thread is deviating here.

We are not talking about performance, the summary is:

 1. rails generated database.yml has :memory: for test

 2. Nobody has been known to succeed using that

Therefore,

 1. Either we are missing something,

 2. or else that should NOT be generated at all

– fxn