Hi,
I have been using Ferret on ruby-forum.com for a few days. While the
searching works with reasonable performance, there are a lot of problems
related to indexing. Sometimes the process hangs with 100% CPU usage,
sometimes it leaves Lockfiles in the directory (causing other processes
to fail while one ist still working), sometimes it even crashes with a
segfault.
I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.
Andreas
Andreas S. wrote:
I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.
You must’ve tried this already, just checking - did you set :auto_flush
=> true while creating your index? The documentation says it prevents
locking when multiple processes are accessing the index.
Vamsee.
Vamsee K. wrote:
Andreas S. wrote:
I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.
You must’ve tried this already, just checking - did you set :auto_flush
=> true while creating your index?
Yes.
The documentation says it prevents
locking when multiple processes are accessing the index.
Locking usually works correctly, but sometimes I get lockfiles that
are never removed. What I don’t understand is why one of the proceses is
still able to write to the index then, while the others aren’t.
Hi, Andreas,
maybe a stupid guess: Maybe because this one process is the locking one?
Regards
Jan
Jan P. wrote:
Hi, Andreas,
maybe a stupid guess: Maybe because this one process is the locking one?
Regards
Jan
I don’t know. I have two fastcgi processes, and it seems that they are
actually both able to write to the index (at least I never got
exceptions when I created a post in the forum), only the third process
(mail receiver) isn’t.
Andreas
Hi Andreas,
I am using a modified version of your Ferret code in my application,
which is still in development mode, so no actual user testing has been
done. I also encountered the locking problem. What I did was remove the
auto_flush => true and handle flushing myself. So in the create_doc
method I placed:
index << doc
index.flush
at the end and removed your last line ‘doc’. Which also made the update
method absolete, so I made update an alias of create_doc . I haven’t had
any locks during my testing, but I haven’t done any real world testing
yet. So I’m eager to know how you managed to fix it, once it is fixed.
Kind regards,
Nick
Andreas S. wrote:
Hi,
I have been using Ferret on ruby-forum.com for a few days. While the
searching works with reasonable performance, there are a lot of problems
related to indexing. Sometimes the process hangs with 100% CPU usage,
sometimes it leaves Lockfiles in the directory (causing other processes
to fail while one ist still working), sometimes it even crashes with a
segfault.
I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.
Andreas
Nick S. wrote:
Hi Andreas,
I am using a modified version of your Ferret code in my application,
which is still in development mode, so no actual user testing has been
done. I also encountered the locking problem. What I did was remove the
auto_flush => true and handle flushing myself.
I tried that too, but had the same problem as before.
at the end and removed your last line ‘doc’. Which also made the update
method absolete, so I made update an alias of create_doc . I haven’t had
any locks during my testing, but I haven’t done any real world testing
yet. So I’m eager to know how you managed to fix it, once it is fixed.
I am now using a process that checks periodically for new/changed posts
and updates the index. This seems to be the best solution. You can find
the code in RForum SVN.
Andreas