Forum: Ferret Problems with Ferret in RForum

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
3a83969376c805ef5b6042191fdb0ff3?d=identicon&s=25 Andreas S. (andreas)
on 2006-01-05 19:53
Hi,

I have been using Ferret on ruby-forum.com for a few days. While the
searching works with reasonable performance, there are a lot of problems
related to indexing. Sometimes the process hangs with 100% CPU usage,
sometimes it leaves Lockfiles in the directory (causing other processes
to fail while one ist still working), sometimes it even crashes with a
segfault.

I'm going to try to use only one indexing process to solve the locking
problems. A process that looks periodically for posts that need to be
indexed is probably the most robust solution.

Andreas
Bcfc926d36e15709d7e6c70b9791211a?d=identicon&s=25 Vamsee Kanakala (Guest)
on 2006-01-06 06:01
(Received via mailing list)
Andreas S. wrote:

>I'm going to try to use only one indexing process to solve the locking
>problems. A process that looks periodically for posts that need to be
>indexed is probably the most robust solution.
>
>
You must've tried this already, just checking - did you set :auto_flush
=> true while creating your index? The documentation says it prevents
locking when multiple processes are accessing the index.

Vamsee.
3a83969376c805ef5b6042191fdb0ff3?d=identicon&s=25 Andreas S. (andreas)
on 2006-01-06 13:32
Vamsee Kanakala wrote:
> Andreas S. wrote:
>
>>I'm going to try to use only one indexing process to solve the locking
>>problems. A process that looks periodically for posts that need to be
>>indexed is probably the most robust solution.
>>
>>
> You must've tried this already, just checking - did you set :auto_flush
> => true while creating your index?

Yes.

> The documentation says it prevents
> locking when multiple processes are accessing the index.

Locking usually works correctly, but *sometimes* I get lockfiles that
are never removed. What I don't understand is why one of the proceses is
still able to write to the index then, while the others aren't.
E48d29dc8fedb2878fa518d41cc63d88?d=identicon&s=25 Jan Prill (Guest)
on 2006-01-06 14:05
(Received via mailing list)
Hi, Andreas,

maybe a stupid guess: Maybe because this one process is the locking one?

Regards
Jan
3a83969376c805ef5b6042191fdb0ff3?d=identicon&s=25 Andreas S. (andreas)
on 2006-01-06 15:38
Jan Prill wrote:
> Hi, Andreas,
>
> maybe a stupid guess: Maybe because this one process is the locking one?
>
> Regards
> Jan

I don't know. I have two fastcgi processes, and it seems that they are
actually both able to write to the index (at least I never got
exceptions when I created a post in the forum), only the third process
(mail receiver) isn't.

Andreas
91308e9bc88cb069fd1bcf88e910d042?d=identicon&s=25 Nick Snels (nicksnels)
on 2006-01-07 20:46
Hi Andreas,

I am using a modified version of your Ferret code in my application,
which is still in development mode, so no actual user testing has been
done. I also encountered the locking problem. What I did was remove the
auto_flush => true and handle flushing myself. So in the create_doc
method I placed:

index << doc
index.flush

at the end and removed your last line 'doc'. Which also made the update
method absolete, so I made update an alias of create_doc . I haven't had
any locks during my testing, but I haven't done any real world testing
yet. So I'm eager to know how you managed to fix it, once it is fixed.

Kind regards,

Nick


Andreas S. wrote:
> Hi,
>
> I have been using Ferret on ruby-forum.com for a few days. While the
> searching works with reasonable performance, there are a lot of problems
> related to indexing. Sometimes the process hangs with 100% CPU usage,
> sometimes it leaves Lockfiles in the directory (causing other processes
> to fail while one ist still working), sometimes it even crashes with a
> segfault.
>
> I'm going to try to use only one indexing process to solve the locking
> problems. A process that looks periodically for posts that need to be
> indexed is probably the most robust solution.
>
> Andreas
3a83969376c805ef5b6042191fdb0ff3?d=identicon&s=25 Andreas S. (andreas)
on 2006-01-07 21:34
Nick Snels wrote:
> Hi Andreas,
>
> I am using a modified version of your Ferret code in my application,
> which is still in development mode, so no actual user testing has been
> done. I also encountered the locking problem. What I did was remove the
> auto_flush => true and handle flushing myself.

I tried that too, but had the same problem as before.

> at the end and removed your last line 'doc'. Which also made the update
> method absolete, so I made update an alias of create_doc . I haven't had
> any locks during my testing, but I haven't done any real world testing
> yet. So I'm eager to know how you managed to fix it, once it is fixed.

I am now using a process that checks periodically for new/changed posts
and updates the index. This seems to be the best solution. You can find
the code in RForum SVN.

Andreas
This topic is locked and can not be replied to.