Forum: Ferret Ferret falling through the Monit cracks

I have a standard AAF Ferret installation running over DRB, serving
several thousand searches daily. I use the monit script as provided in
the AAF package to ensure the DRB server stays up.

I also use the totalmem line which is commented out by default
if totalmem > 60.0 MB for 5 cycles then restart

Twice in the last two months the DRB server has failed, but monit thinks
it's fine so it has required manual intervention to solve (both times I
have just restarted the server)

In both occasions it directly follows the following actions in the monit
log, ferret exceeds the memory limits and is restarted, seemingly
successfully.

07:41:54] error    : 'ferret' total mem amount of 63140kB matches
resource limit [total mem amount>61440kB]
07:44:54] error    : 'ferret' total mem amount of 63140kB matches
resource limit [total mem amount>61440kB]
07:47:54] error    : 'ferret' total mem amount of 63140kB matches
resource limit [total mem amount>61440kB]
07:50:54] error    : 'ferret' total mem amount of 63140kB matches
resource limit [total mem amount>61440kB]
07:53:54] error    : 'ferret' total mem amount of 63140kB matches
resource limit [total mem amount>61440kB]
07:53:54] info     : 'ferret' trying to restart
07:53:54] info     : 'ferret' stop: /bin/su
07:53:57] info     : 'ferret' start: /bin/su
07:53:57] error    : 'ferret' failed, cannot open a connection to
INET[localhost:9010] via TCP
07:53:57] error    : 'ferret' failed, cannot open a connection to
INET[localhost:9010] via TCP
07:56:57] info     : 'ferret' resource passed
07:56:57] info     : 'ferret' connection passed to INET[localhost:9010]
via TCP

So at this stage ferret should be up and fine but...
Now any attempt to use ferret gets the error (Entry is the model)

A ActsAsFerret::IndexNotDefined occurred in search#index:
 entry
 (druby:/localhost:9010)
/var/www/releases/20080922044701/vendor/plugins/acts_as_ferret/lib/acts_as_ferret.rb:264:in
`get_index'


I've tried to simulate this in the staging environment with an
artificially low memory limit but everything seems to reset fine.

Anyone see anything similar themselves, do anything different or have
any suggestions?

Sam

> So at this stage ferret should be up and fine but...
> Now any attempt to use ferret gets the error (Entry is the model)
> 
> A ActsAsFerret::IndexNotDefined occurred in search#index:
>  entry
>  (druby:/localhost:9010)
> /var/www/releases/20080922044701/vendor/plugins/acts_as_ferret/lib/acts_as_ferret.rb:264:in
> `get_index'
> 
> 
> I've tried to simulate this in the staging environment with an
> artificially low memory limit but everything seems to reset fine.
> 
> Anyone see anything similar themselves, do anything different or have
> any suggestions?

Hey,

I'm also getting the same error from my app almost every time the DRB 
has been restarted by monit (either due to the DRB exceeding the memory 
limits configured in monit, or after the DRB has segfaulted).

I can reproduce this error by:
* running my app & DRB in production mode
* stopping the DRB
* doing something with the app that updates / searches the ferret index
* starting the DRB
* running a search / or updating the index will yield this error now...

Restarting mongrel when this error occurs fixes the issue, but it's not 
an ideal solution...

I'm using the latest aaf from github and ferret gem version 0.11.6.

Wondering if anyone else is getting the same IndexNotDefined exception?

Cheers,

Deb