Acts_as_ferret and auto-flush

Ruby Apps Ferret
1

Hi,

I’m using acts_as_ferret in with a mongrel and I’ m getting locking
errors that after a while result in a corrupt database.

I know about the problem with different processes writing to the
index but I haven’t been able to get the DRB server working properly
yet. I read on this list that another solution is to set :auto_flush
to true but I’m not sure how to do this. As I understand it I have to
do this for the Index class and not for the model that
acts_as_ferret, right? How exactly do I do this? Do I just have to
make a new Index model?

I hope someone can help me out.

–
Jeroen J.

2

On Tue, Mar 13, 2007 at 05:20:51PM +0100, jeroen janssen wrote:

Hi,

I’m using acts_as_ferret in with a mongrel and I’ m getting locking
errors that after a while result in a corrupt database.

what version of ferret do you use? the latest ferret versions (0.11.x)
should show a much better behaviour with shared index access.

I know about the problem with different processes writing to the
index but I haven’t been able to get the DRB server working properly
yet. I read on this list that another solution is to set :auto_flush
to true but I’m not sure how to do this. As I understand it I have to
do this for the Index class and not for the model that
acts_as_ferret, right? How exactly do I do this? Do I just have to
make a new Index model?

acts_as_ferret already uses auto_flush => true in normal use (but not in
rebuild_index, because it would be really dumb to flush the index after
every record while batch indexing all records).

cheers,
Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

3

I’m using acts_as_ferret in with a mongrel and I’ m getting locking
errors that after a while result in a corrupt database.

what version of ferret do you use? the latest ferret versions (0.11.x)
should show a much better behaviour with shared index access.

Thanks… that seems to work a little better. unfortunately I’m still
having some problems.

The best solution would probably be to get DRB server to work, but I
haven’t had much luck with that yet. As a temporary solution I was
thinking of just not let the model index itself on create but do a
scheduled rebuild every hour or something. Is there any way to have a
acts_as_ferret model not update de index automatically?

–
Jeroen

4

On Wed, Mar 14, 2007 at 07:06:09PM +0100, jeroen janssen wrote:

haven’t had much luck with that yet.
what were your problems?

As a temporary solution I was
thinking of just not let the model index itself on create but do a
scheduled rebuild every hour or something. Is there any way to have a
acts_as_ferret model not update de index automatically?

yeah, override the ferret_enabled? instance method to return false so
the automatic indexing is skipped. In aaf trunk this method has an
optional boolean parameter that indicates if it is called from
rebuild_index (true) or not (false, default). Before that it has not
been called when the index was rebuilt.

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

5

The best solution would probably be to get DRB server to work, but I
haven’t had much luck with that yet.

what were your problems?

If the load on the server is not too high it works alright, but after
a while I get exceptions after which the index gets corrupted, I
think. I have included some of these errors at the end of this e-mail.

My problem with the DRB server is simply that I can’t get the script
to run. when I run script/runner vendor/plugins/acts_as_ferret/script/
ferret_server I get a ‘undefined local variable or method `vendor’
for #Object:0x1f2968 (NameError)’

This happens on my server and on my local machine. (Rails 1.2.1)

If I copy the script to RAILS_ROOT/lib and do script/runner “require
‘ferret_server’” as you suggested earlier I don’t get an error but I
also don’t get any feedback that something is running. When I try to
search I get a ‘druby://localhost:9009 - #<Errno::ECONNREFUSED:
Connection refused - connect(2)>’ error.

As a temporary solution I was
thinking of just not let the model index itself on create but do a
scheduled rebuild every hour or something. Is there any way to have a
acts_as_ferret model not update de index automatically?

yeah, override the ferret_enabled? instance method to return false so
the automatic indexing is skipped. In aaf trunk this method has an
optional boolean parameter that indicates if it is called from
rebuild_index (true) or not (false, default). Before that it has not
been called when the index was rebuilt.

Ok… thanks, I will try if this helps for now…

Here are some of the errors I’m getting now:

–

A IOError occurred in search#weblogs:

IO Error occured at <except.c>:93 in xraise
Error occured in index.c:886 - sis_find_segments_file
Error reading the segment infos. Store listing was

/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/
index.rb:706:in `initialize’

A EOFError occurred in weblog#show_by_login:

End-of-File Error occured at <except.c>:117 in xpop_context
Error occured in store.c:216 - is_refill
current pos = 0, file length = 0

/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/
index.rb:285:in `delete’

A FileNotFoundError occurred in weblog#show_by_login:

File Not Found Error occured at <except.c>:93 in xraise
Error occured in fs_store.c:329 - fs_open_input
tried to open “/www/wnf.dma.nl/rails_app/config/…/index/production/
user/_1ez_0.f5” but it doesn’t exist:

A FileNotFoundError occurred in weblog#show_by_login:

File Not Found Error occured at <except.c>:117 in xpop_context
Error occured in fs_store.c:329 - fs_open_input
tried to open “/www/wnf.dma.nl/rails_app/config/…/index/production/
user/_50t_w.del” but it doesn’t exist:

/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/
index.rb:285:in `delete’

A FileNotFoundError occurred in account#logout:

File Not Found Error occured at <except.c>:117 in xpop_context
Error occured in fs_store.c:329 - fs_open_input
tried to open “/www/wnf.dma.nl/rails_app/config/…/index/production/
user/_50t_w.del” but it doesn’t exist:

/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/
index.rb:285:in `delete’

6

could you please try

./script/runner “load ‘ferret_server’”

instead? this seems to fix the problem. Hope I’ll find the time to
integrate the start/stop scripts that recently were posted to the list
soon.

Hey, thanks! That seems to do the trick, on my local server at least.
I’m going to try it out soon on the production server.

did you rebuild your index after upgrading to 0.11.3?

Yeah, I did.

Besides that, I don’t know what to do about these Errors - maybe
Dave can comment on
these?

No problem, I will try if it works better with the DRB server. Thanks
a lot for your help and for the excellent work on acts_as_ferret.

7

On Thu, Mar 15, 2007 at 09:44:04AM +0100, jeroen janssen wrote:

On Wed, Mar 14, 2007 at 07:06:09PM +0100, jeroen janssen wrote:
[…]
also don’t get any feedback that something is running. When I try to
search I get a ‘druby://localhost:9009 - #<Errno::ECONNREFUSED:
Connection refused - connect(2)>’ error.

could you please try

./script/runner “load ‘ferret_server’”

instead? this seems to fix the problem. Hope I’ll find the time to
integrate the start/stop scripts that recently were posted to the list
soon.

[…]

Here are some of the errors I’m getting now:

did you rebuild your index after upgrading to 0.11.3? Besides that, I
don’t know what to do about these Errors - maybe Dave can comment on
these?

Jens.

index.rb:706:in `initialize’
/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/

/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/

/usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.3/lib/ferret/
index.rb:285:in `delete’

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa