Does Berkeley DB and Rails seem like a good match?

Rails
1

I’ve been reading up on Berkeley DB and I’ve been refelcting on the
Rails philosophy and it just seemed to me like the two should go
together hand in glove. Rails seeks to abstract the programer from
squirrely database coding which is great. My only problem is that
you’re still left with a large component, namely the databsase server
(MySQL, PostgreSQL…etc) which is treated like a black box…these
databases are hugely complex and maybe shouldn’t be treated like blck
boxes, maybe a small light-weight database, essentially a giant hashing
table (i.e. Berkeley DB) would result in lower complexity and better
performance?

what does everyone think? I’m happy to be shot out of the water by
those with more experience than me :slight_smile:

2

SQLite works very well in this regard.

http://www.sqlite.org/
http://wiki.rubyonrails.com/rails/pages/SQLite

Stephen G.

3

Andrew G. wrote:

what does everyone think? I’m happy to be shot out of the water by
those with more experience than me :slight_smile:

Rails needs a database that supports SQL.

SQLite might be the lightweight database you are looking for.

http://www.sqlite.org/

Not sure of the quality of the information, but there’s advice on using
SQLite with Rails on the wiki here:

http://wiki.rubyonrails.org/rails/pages/HowtoUseSQLite

regards

Justin

4

On 7/7/06, Andrew G. [email protected] wrote:

what does everyone think? I’m happy to be shot out of the water by
those with more experience than me :slight_smile:

I don’t know about Berkeley DB, but SQLite works fine. As long as you
have few updates relative to your reads, and you don’t need
super-performance, it’s a decent choice. I’ve had a Rails site
running for several months now using that db. It’s certainly simple.

http://www.sqlite.org/

Honestly though, I’ll be using Postgres from now on for anything
except the simplest of apps.

– James

5

Andrew G. wrote:

Ok, just out of curiosity, why do you want/need postgres over sqlite?

what isn’t sqlite giving you?

Take a look at the concurrency issues mentioned here:

http://www.sqlite.org/faq.html#q7

It appears that Sqlite locks the entire database for any writes, while
other
databases lock individual rows or the entire table, allowing more
clients to
update the database simultaneously.

Regards,
Blair

6

Fair point, which brings me back to Berkeley DB, you can check out the
info here:

I’m not trying to be stubborn here, I would realy like to try and start
a dicussion about this…

it’s not an SQL based data solution but it sounds like it’s got top
notch performance (256 terrabytes addressable,

Take a look at the concurrency issues mentioned here:

SQLite Frequently Asked Questions

It appears that Sqlite locks the entire database for any writes, while
other
databases lock individual rows or the entire table, allowing more
clients to
update the database simultaneously.

Regards,
Blair

7

Ok, just out of curiosity, why do you want/need postgres over sqlite?

what isn’t sqlite giving you?

http://www.sqlite.org/

Honestly though, I’ll be using Postgres from now on for anything
except the simplest of apps.

– James

8

Andrew G. wrote:

it’s not an SQL based data solution but it sounds like it’s got top
notch performance (256 terrabytes addressable,

…but as I wrote earlier, Rails (ActiveRecord) requires a database with
SQL support.

You could put a different back end behind ActionPack, but you would lose
an awful lot.

regards

Justin

9

Andrew G. wrote:

Fair point, which brings me back to Berkeley DB, you can check out the
info here:
Oracle Berkeley DB Downloads
Berkeley DB - Wikipedia

I’m not trying to be stubborn here, I would realy like to try and start
a dicussion about this…

I’ve got a little bit of anecdotal information about this topic. Assaph
Mehr wrote an ActiveRecord adapter for KirbyBase, called Ackbar. Now,
KirbyBase does not use SQL as it’s query language; it uses Ruby blocks.
So, inside the guts of Ackbar, Assaph had to do all kinds of strange
morphing to convert the SQL string received from ActiveRecord into a
Ruby block that KirbyBase could understand.

He got this working with Rails 1.0. Unfortunately, Rails 1.1 broke it
and I’m not sure Assaph is going to attempt to get Ackbar working with
Rails 1.1. It is not a trivial task to keep Ackbar in sync with
ActiveRecord changes.

Now, writing an ActiveRecord adapter might be easier for Berkeley DB,
since it’s query language is much simpler than KirbyBase’s.

As a side note, just for fun, a few days ago I started playing around
with building a Rails app and using KirbyBase as the database back-end,
but not using ActiveRecord (i.e. my model did not inherit from
ActiveRecord’s Base class). It was fairly easy to get basic #find
functionality working all the way through the controller to the view.
However, when I attempted to do #updates, what I started to miss was all
of the cool stuff, like errors being propagated from the model back to
the view and showing up all nice and pretty.

It’s definitely do-able to use a non-ActiveRecord backend in Rails, but
you will either have to live without a lot of the niceties that the
ActiveRecord-Rails integration gives you, or try to duplicate it in your
own code.

Maybe ActiveResource will prove to be another alternative.

Jamey C.

Confidentiality Notice: This email message, including any attachments,
is for the sole use of the intended recipient(s) and may contain
confidential and/or privileged information. If you are not the intended
recipient(s), you are hereby notified that any dissemination,
unauthorized review, use, disclosure or distribution of this email and
any materials contained in any attachments is prohibited. If you receive
this message in error, or are not the intended recipient(s), please
immediately notify the sender by email and destroy all copies of the
original message, including attachments.

10

Now we’re cookin’ with gas here:)

In terms of interacting with the databse, there are already a couple of
Ruby libs for talking to Berkeley DB… you can find them on rubyforge
and also on Darwin Ports

the reason why I started thinking along these lines was that it just
seemed like an awful lot of overhead to create the ORM layer to wrap
around an entire seperate database program, especially when as rubyists
we want to code as much as possible in Ruby…so why not find a data
storage solution which allows to always stay in Ruby-land and that acts
as a giant hash-table?

I guess I should stop running off my mouth at this point though and
maybe roll of my sleeves and try to integrate one of these libs into
Rails and see how it all works out

Ron M wrote:

Justin F. wrote:

Andrew G. wrote:

I’ve been reading up on Berkeley DB and I’ve been refelcting …

Rails needs a database that supports SQL.

You exaggerate.

I’ve been doing fine with an application that uses a web service
(the “solr” front end to lucene) for all its models rather than
a database.

AFAIK the only subset of Rails that wants a SQL database is the
ActiveRecord part. And even when I do use a database with Rails,
I tend not to use ActiveRecord stuff, since I find ActiveRecord
rather annoying (relationships between tables defined in
the ruby class instead of by foreign key constraints; can’t
understand user-defined types in the database; etc).

11

Ron M wrote:

a database.

AFAIK the only subset of Rails that wants a SQL database is the
ActiveRecord part. And even when I do use a database with Rails,
I tend not to use ActiveRecord stuff, since I find ActiveRecord
rather annoying (relationships between tables defined in
the ruby class instead of by foreign key constraints; can’t
understand user-defined types in the database; etc).

You are quite right that it is ActiveRecord that requires SQL - I made
this clearer in my second post in this thread. I would be happier if
Rails separated its persistence solution from its domain modelling
features, but at the moment they come in one bundle, ActiveRecord.
That’s why I said, in my other post, that Andrew would be losing a lot
by discarding ActiveRecord.

regards

Justin

12

Justin F. wrote:

Andrew G. wrote:

I’ve been reading up on Berkeley DB and I’ve been refelcting …

Rails needs a database that supports SQL.

You exaggerate.

I’ve been doing fine with an application that uses a web service
(the “solr” front end to lucene) for all its models rather than
a database.

AFAIK the only subset of Rails that wants a SQL database is the
ActiveRecord part. And even when I do use a database with Rails,
I tend not to use ActiveRecord stuff, since I find ActiveRecord
rather annoying (relationships between tables defined in
the ruby class instead of by foreign key constraints; can’t
understand user-defined types in the database; etc).

13

Now we’re cookin’ with gas here:)

agricultural methane, i’d hope…

the reason why I started thinking along these lines was that it just
seemed like an awful lot of overhead to create the ORM layer to wrap
around an entire seperate database program

dynamic object persistence is still a research topic… i dont remember
SQL being proclaimed god and ORM his right hand, so much as it was the
most pragmatic solution at the current time

, especially when as rubyists
we want to code as much as possible in Ruby…

one can write an entire fully featured and functional site, without
touching SQL, or HTML, or bash, or CSS, or JavaScript. to me this seems
like a pretty cool achievement. finally web development can complete
with things like Tcl/Tk and QT and Smalltalk in terms of what it offers
to the developer.

so why not find a data
storage solution which allows to always stay in Ruby-land and that acts
as a giant hash-table?

go for it… the name ActiveHash isnt taken, btw ;). it would be great to
have this option well integrated.

14

carmen wrote:

Now we’re cookin’ with gas here:)

agricultural methane, i’d hope…

???

one can write an entire fully featured and functional site, without
touching […] HTML […]

Really?

Joe

15

You do know that MySQL has the BDB table type, don’t you? The only thing
I use the BDB database for is SVN.

Joe

16

On 7-jul-2006, at 21:08, Andrew G. wrote:

the reason why I started thinking along these lines was that it just
seemed like an awful lot of overhead to create the ORM layer to wrap
around an entire seperate database program, especially when as
rubyists
we want to code as much as possible in Ruby…so why not find a data
storage solution which allows to always stay in Ruby-land and that
acts
as a giant hash-table?

Three words: queries, joins and concurrency.

When you have joins (what SQL databases do very well) you can depend
on the DB for managing
the datastore in multiple related tables. When you have BDB (or such)
you have to join the rows yourself,
because all convenience that you got is a Giant Hashtable. An SQL
database is an airplane, BDB is a hot air balloon
(but a very robust one). Another problem is typing.

Try that with BDB (pseudocode):

find_all_entries whose date_created is between (today and 3 days ago)

The problem with joins is that you end up pulling lots of data out of
the DB to do most simple comparisons/computations
as soon as you need more than one entity.

MovableType uses BDB because it’s the only choice for hosts where an
SQL database is not available. Granted, it allowed
me to use it back in the day when all I had was my student account,
but I would not go with BDB for most of data-driven apps now.

You can take a look at all the scaffolding that is used to support
BDB in MT and you will realise that with many tables it becomes
more trouble than it’s worth.