Berger, Daniel wrote:
The dbm/gdbm/sdbm libraries are toys, imho. I wouldn’t trust them to
store more than a few thousand records, let alone a million. I’ve seen
them fall down horribly on Windows. Just ask Hal F… As far as I
can tell, the dbm/gdbm/sdbm wrappers were included because thats what
Thanks, Dan. That’s the kind of info I was looking for before I spent a
lot of time testing.
I’m very curious Jamey - why aren’t you using KirbyBase?
Well, here’s the deal.
I’m looking at trying to improve KirbyBase’s performance, to make it
scale better. One way I am doing this is I have started playing around
with using Skiplists for KirbyBase indexes, rather than arrays. For a
lot of queries, this has greatly speeded up the part of the query that
grabs all matching keys.
But, I’m still running into the bottleneck of taking those keys from the
query and using them to pull the actual records in from the flat text
file where KirbyBase stores the actual data.
So, I’ve been looking at ways to get around this. I would basically
look at replacing the back-end flat file format of KirbyBase with
something faster. The user wouldn’t see any difference in how they
interacted with KirbyBase. One of the possibilities was to store the
data records in some kind of dbm-like file (i.e. gdbm, qdbm, etc.) and
sdbm looked attractive because it is included with Ruby. The downside
to this is that the data is no longer kept in easily-editable plain-text
Another option would be to keep the entire table in memory as a hash and
use some kind of journal file to handle updates/deletes. This might be
an issue on memory-constrained systems, but I’m thinking this is less
and less of an issue on modern pcs.
I’m just kind of brainstorming right now. KirbyBase performs pretty
well, but I would like to see it scale better.
Confidentiality Notice: This email message, including any attachments,
is for the sole use of the intended recipient(s) and may contain
confidential and/or privileged information. If you are not the intended
recipient(s), you are hereby notified that any dissemination,
unauthorized review, use, disclosure or distribution of this email and
any materials contained in any attachments is prohibited. If you receive
this message in error, or are not the intended recipient(s), please
immediately notify the sender by email and destroy all copies of the
original message, including attachments.