Strange document loss


#1

Hi all :slight_smile:

I’m experiencing strange document loss with Ferret 0.11.4 / Ruby 1.8.4.
The index handles around 3000 documents, and sometimes during a
query_update operation, the delete is done but not the add. That’s very
random and happens after a lot of queries (many K).

I’ve tried to look into Ferret’s C code, everything looks like going
well, the add LOOKS LIKE being done, so it’s probably deeper in the
internals…

Does anybody had this problem too ?
Is there any known solution ? :confused:

Cheers,
Jérémie


Jérémie -ahFeel- BORDIER
http://www.unixaumonde.com


#2

On 2007-07-09, at 10:11 AM, Jérémie Bordier wrote:

I’ve tried to look into Ferret’s C code, everything looks like going
well, the add LOOKS LIKE being done, so it’s probably deeper in the
internals…

Does anybody had this problem too ?
Is there any known solution ? :confused:

hey …

i’ve have a similar problem but did not investigate further… i got some
50k records indexed… and sometimes specific documents are gone from
the index… i’m currently doing a reindex every now and then… but i
need
to check this behavior … i try to take a look today, maybe we have the
same problem …

and to be more precisely… i’m not sure if the documents are really gone
from the index, but they are no longer a part of search results…

Ben


#3

Interesting… That’s very strange, what kind of usage do you do of
Ferret ? most reading / searching operations ? We have a lot of writes
in our requests, which isn’t common in ferret’s usage (ppl generally use
only act_as_ferret).

We are able to reproduce these issues with a copy of our index and a log
of queries to replay, and i’ve asked D. Balmain for some help, but he
doesn’t answer actually.

It looks like it’s not just a search bug, because a strings index/* |
grep myword doesn’t get any result.

Waiting on your answer, and happy to see we’re not alone with that,
Cheers,
Jérémie


Jérémie -ahFeel- BORDIER
http://www.unixaumonde.com


#4

Hey …

Interesting… That’s very strange, what kind of usage do you do of
Ferret ? most reading / searching operations ? We have a lot of writes
in our requests, which isn’t common in ferret’s usage (ppl
generally use
only act_as_ferret).

I’m running omdb.org … and we have a lot of indexing requests, acutally
we have a backgroundrb server accepting indexing requests and
queue’ing them… so we’re basically using a mechanism similar to
acts_as_ferrets index-server.

but our implementation is based on ferret and has nothing to do with
acts_as_ferret (beside the fact, that Jens write large parts of the
indexing/searching code) :wink:

We are able to reproduce these issues with a copy of our index and
a log
of queries to replay, and i’ve asked D. Balmain for some help, but he
doesn’t answer actually.

What i’m experiencing is, that a certain important record is missing
after a few days. You can add casts/crew member to movies on omdb,
and the Job “Director of Photography” is part of the ferret index… but
for some reason, it’s gone after a few days… and you’re no longer able
to select “director of photography” e.g. in such a dialog:

http://www.omdb.org/movie/1832/cast/edit_crew

Waiting on your answer, and happy to see we’re not alone with that,

as i said, i will try to investigate further in that topic… it’s
about time to
identify a few problems with the ferret indexing…

Ben