Problems with Ferret in RForum

Hi,

I have been using Ferret on ruby-forum.com for a few days. While the
searching works with reasonable performance, there are a lot of problems
related to indexing. Sometimes the process hangs with 100% CPU usage,
sometimes it leaves Lockfiles in the directory (causing other processes
to fail while one ist still working), sometimes it even crashes with a
segfault.

I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.

Andreas

Andreas S. wrote:

I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.

You must’ve tried this already, just checking - did you set :auto_flush
=> true while creating your index? The documentation says it prevents
locking when multiple processes are accessing the index.

Vamsee.

Vamsee K. wrote:

Andreas S. wrote:

I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.

You must’ve tried this already, just checking - did you set :auto_flush
=> true while creating your index?

Yes.

The documentation says it prevents
locking when multiple processes are accessing the index.

Locking usually works correctly, but sometimes I get lockfiles that
are never removed. What I don’t understand is why one of the proceses is
still able to write to the index then, while the others aren’t.

Hi, Andreas,

maybe a stupid guess: Maybe because this one process is the locking one?

Regards
Jan

Jan P. wrote:

Hi, Andreas,

maybe a stupid guess: Maybe because this one process is the locking one?

Regards
Jan

I don’t know. I have two fastcgi processes, and it seems that they are
actually both able to write to the index (at least I never got
exceptions when I created a post in the forum), only the third process
(mail receiver) isn’t.

Andreas

Hi Andreas,

I am using a modified version of your Ferret code in my application,
which is still in development mode, so no actual user testing has been
done. I also encountered the locking problem. What I did was remove the
auto_flush => true and handle flushing myself. So in the create_doc
method I placed:

index << doc
index.flush

at the end and removed your last line ‘doc’. Which also made the update
method absolete, so I made update an alias of create_doc . I haven’t had
any locks during my testing, but I haven’t done any real world testing
yet. So I’m eager to know how you managed to fix it, once it is fixed.

Kind regards,

Nick

Andreas S. wrote:

Hi,

I have been using Ferret on ruby-forum.com for a few days. While the
searching works with reasonable performance, there are a lot of problems
related to indexing. Sometimes the process hangs with 100% CPU usage,
sometimes it leaves Lockfiles in the directory (causing other processes
to fail while one ist still working), sometimes it even crashes with a
segfault.

I’m going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.

Andreas

Nick S. wrote:

Hi Andreas,

I am using a modified version of your Ferret code in my application,
which is still in development mode, so no actual user testing has been
done. I also encountered the locking problem. What I did was remove the
auto_flush => true and handle flushing myself.

I tried that too, but had the same problem as before.

at the end and removed your last line ‘doc’. Which also made the update
method absolete, so I made update an alias of create_doc . I haven’t had
any locks during my testing, but I haven’t done any real world testing
yet. So I’m eager to know how you managed to fix it, once it is fixed.

I am now using a process that checks periodically for new/changed posts
and updates the index. This seems to be the best solution. You can find
the code in RForum SVN.

Andreas

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs