Constant 0.11.4 Errors

Ruby Apps Ferret
1

Folks,

Since upgrading to ferret 0.11.4 on Mac OS X Tiger, I’ve been running
into constant crashes when running my Rails tests. Here’s an example of
the output.

– snip –

  1. Error:
    test_published_blog_can_be_ferreted(BlogTest):
    Ferret::FileNotFoundError: File Not Found Error occured at
    <except.c>:117 in xpop_context
    Error occured in fs_store.c:329 - fs_open_input
    tried to open
    “/Users/mark/Sites/www.site.com/config/…/index/test/blog/_17_0.del” but
    it doesn’t exist: 
/opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:285:in

delete' /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:285:in<<’
/opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:8:in
synchrolock' /opt/local/lib/ruby/1.8/monitor.rb:238:insynchronize’
/opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:8:in
synchrolock' /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:267:in<<’
/Users/mark/Sites/www.site.com/config/…/vendor/plugins/acts_as_ferret/lib/local_index.rb:140:in
<<' /Users/mark/Sites/www.site.com/config/../vendor/plugins/acts_as_ferret/lib/instance_methods.rb:73:inferret_create’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/callbacks.rb:333:in
send' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/callbacks.rb:333:incallback’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/callbacks.rb:330:in
each' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/callbacks.rb:330:incallback’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/callbacks.rb:255:in
create_without_timestamps' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/timestamp.rb:39:increate’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/base.rb:1789:in
create_or_update_without_callbacks' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/callbacks.rb:242:increate_or_update’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/base.rb:1545:in
save_without_validation' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/validations.rb:752:insave_without_transactions’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/transactions.rb:129:in
save' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/database_statements.rb:59:intransaction’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/transactions.rb:95:in
transaction' /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/transactions.rb:121:intransaction’
/opt/local/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/transactions.rb:129:in
save' ./test/unit/blog_test.rb:277:intest_published_blog_can_be_ferreted’

– end snip –

Anyone run into a similar problem or have some suggestions? Thanks in
advance.

2

On 4/8/07, Mark [email protected] wrote:

Folks,

Since upgrading to ferret 0.11.4 on Mac OS X Tiger, I’ve been running
into constant crashes when running my Rails tests. Here’s an example of
the output.

Hi Mark,

Which version did you upgrade from. I can’t recall any changes between
0.11.3 and 0.11.4 that could possibly have caused this. If you
upgraded from an earlier version it is possible that you may need to
rebuild your test index. Otherwise I can’t think what the problem
might be.

Cheers,
Dave

3

Dave,

I was running 0.11.3. I upgraded to 0.11.4. After deleting the “index”,
my tests automatically rebuilt the index as you would imagine. When
running the same tests using the 0.11.4 version, it consistently
produces the errors above. By uninstalling 0.11.4 and using 0.11.3,
everything works again.

Any other suggestions on what the issue might be?

David B. wrote:

On 4/8/07, Mark [email protected] wrote:

Folks,

Since upgrading to ferret 0.11.4 on Mac OS X Tiger, I’ve been running
into constant crashes when running my Rails tests. Here’s an example of
the output.

Hi Mark,

Which version did you upgrade from. I can’t recall any changes between
0.11.3 and 0.11.4 that could possibly have caused this. If you
upgraded from an earlier version it is possible that you may need to
rebuild your test index. Otherwise I can’t think what the problem
might be.

Cheers,
Dave

4

On 4/10/07, Levent A. [email protected] wrote:

Hi Mark,

Which version did you upgrade from. I can’t recall any changes between
0.11.3 and 0.11.4 that could possibly have caused this. If you
upgraded from an earlier version it is possible that you may need to
rebuild your test index. Otherwise I can’t think what the problem
might be.

Cheers,
Dave

I’m seeing similar errors now, too, while trying to upgrade and
reindex everything from 0.10.

-ryan

5

I’m experiencing the same thing exactly

6

On 4/12/07, Ryan K. [email protected] wrote:

everything works again.

Any other suggestions on what the issue might be?

I’m seeing similar errors now, too, while trying to upgrade and
reindex everything from 0.10.

Hey guys,

Any chance one of you could distill this problem to a simple script. I
don’t seem to be able to replicate the problem here on my brother’s
mac.

Cheers,
Dave

7

On 4/12/07, David B. [email protected] wrote:

produces the errors above. By uninstalling 0.11.4 and using 0.11.3,
don’t seem to be able to replicate the problem here on my brother’s
mac.

I don’t have a script to replicate this, but I’ve narrowed the problem
down to an issue with concurrency.

Currently I’m running AAF’s rebuild_index (against ferret_server) on
a model and when I try to save an item in that model from another
process, I get this error.

I was under the impression that ferret takes care of these concurrency
problems, but maybe AAF needs to do some mutual exclusion, too?

-ryan

8

On 4/13/07, Ryan K. [email protected] wrote:

Currently I’m running AAF’s rebuild_index (against ferret_server) on
a model and when I try to save an item in that model from another
process, I get this error.

I was under the impression that ferret takes care of these concurrency
problems, but maybe AAF needs to do some mutual exclusion, too?

Sorry, I don’t know enough about AAF and FerretServer to know why this
error would pop up in Ferret 0.11.4. Hopefully Jens or someone else
will be able to shed some light on this.

9

Ryan,

I’m doing the exact same thing, so that seems to be a likely area to
investigate in order to re-produce.

Ryan K. wrote:

On 4/12/07, David B. [email protected] wrote:

produces the errors above. By uninstalling 0.11.4 and using 0.11.3,
don’t seem to be able to replicate the problem here on my brother’s
mac.

I don’t have a script to replicate this, but I’ve narrowed the problem
down to an issue with concurrency.

Currently I’m running AAF’s rebuild_index (against ferret_server) on
a model and when I try to save an item in that model from another
process, I get this error.

I was under the impression that ferret takes care of these concurrency
problems, but maybe AAF needs to do some mutual exclusion, too?

-ryan

10

Jens K. wrote:

On Thu, Apr 12, 2007 at 03:10:37PM -0700, Ryan K. wrote:

running the same tests using the 0.11.4 version, it consistently
Any chance one of you could distill this problem to a simple script. I
don’t seem to be able to replicate the problem here on my brother’s
mac.

I don’t have a script to replicate this, but I’ve narrowed the problem
down to an issue with concurrency.

Currently I’m running AAF’s rebuild_index (against ferret_server) on
a model and when I try to save an item in that model from another
process, I get this error.

Yeah, that’s a problem. As of now, rebuild_index is not safe for
concurrent operations on the same index. I plan to rework this so
rebuild_index will build the new index in another index directory,
leaving intact the original index dir where normal index operations like
add/update/delete work on.

However I dont know why the same problem would not show up with earlier
versions of ferret.

Jens

I’m seeing this as well with a brand new installation 0.11.4 using AAF.
Also, should I be actively running rebuild_index? I must have missed
that somewhere if that’s the case. I’ve simply included the standard
acts_as_ferret :fields => […] to my models and that’s really it (which
seems to work fine). So, this error is causing my app to throw internal
server errors. What can I do about this?

11

On Thu, Apr 12, 2007 at 03:10:37PM -0700, Ryan K. wrote:

running the same tests using the 0.11.4 version, it consistently
Any chance one of you could distill this problem to a simple script. I
don’t seem to be able to replicate the problem here on my brother’s
mac.

I don’t have a script to replicate this, but I’ve narrowed the problem
down to an issue with concurrency.

Currently I’m running AAF’s rebuild_index (against ferret_server) on
a model and when I try to save an item in that model from another
process, I get this error.

Yeah, that’s a problem. As of now, rebuild_index is not safe for
concurrent operations on the same index. I plan to rework this so
rebuild_index will build the new index in another index directory,
leaving intact the original index dir where normal index operations like
add/update/delete work on.

However I dont know why the same problem would not show up with earlier
versions of ferret.

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

12

On 4/16/07, El Gato [email protected] wrote:

Currently I’m running AAF’s rebuild_index (against ferret_server) on
versions of ferret.
server errors. What can I do about this?
Rebuild index won’t work with Drb servers if you’re try to write to
the index from another process besides the one running
‘rebuild_index’.

For now, I’d suggest hacking AAF to not automatically rebuild indexes.
I just commented out line #23 of local_index.rb (which should read
“rebuild_index”).

Then, to build/rebuild my index I’m running something like:

model = MyModel

0.step(model.count, 1000){|i| model.find(:all, :limit=>1000,
:offset=>i).each{|r| r.ferret_update}; }

This should allow concurrent writes, though it will be slower than the
existing rebuld_model.

-ryan

13

Update - Reverting back to 0.10.14 seems to fix this problem.

Kyle wrote:

I too am getting this error, I was not getting it before with 0.10.4.

AAF 0.4.0
Ferret 0.11.4

I have:
Stopped the webserver (lighttpd-1.4.15 (ssl)), removed all the indexes
by hand.

14

Just wanted to add that I had a similar error. Just updated from 11.3 to
11.4 and started getting the FileNotFound errors as stated above.

The only change to the application was an update from 11.3 to 11.4.

I have reverted back to 11.3 and it seems to be working again.

15

I too am getting this error, I was not getting it before with 0.10.4.

AAF 0.4.0
Ferret 0.11.4

I have:
Stopped the webserver (lighttpd-1.4.15 (ssl)), removed all the indexes
by hand.

Rebuilt the indexes with ‘./script/console production’
Person.rebuild_index.

Start the site and DRB back up, and bwam anytime it tries to write a new
person to the index, or a new photo I get the following errors.

Any solution to this? yet?

{{{
Ferret::FileNotFoundError (File Not Found Error occured at
<except.c>:117 in xpo p_context
Error occured in fs_store.c:329 - fs_open_input
tried to open
“/var/www/site/public/…/config/…/index/production/ph
oto/_3z1_5.del” but it doesn’t exist:

):
/usr/lib64/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:285:in
dele te' /usr/lib64/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:285:in <<’
/usr/lib64/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:8:in
`synchr olock’

}}}

Ryan K. wrote:

On 4/16/07, El Gato [email protected] wrote:

Currently I’m running AAF’s rebuild_index (against ferret_server) on
versions of ferret.
server errors. What can I do about this?
Rebuild index won’t work with Drb servers if you’re try to write to
the index from another process besides the one running
‘rebuild_index’.

For now, I’d suggest hacking AAF to not automatically rebuild indexes.
I just commented out line #23 of local_index.rb (which should read
“rebuild_index”).

Then, to build/rebuild my index I’m running something like:

model = MyModel

0.step(model.count, 1000){|i| model.find(:all, :limit=>1000,
:offset=>i).each{|r| r.ferret_update}; }

This should allow concurrent writes, though it will be slower than the
existing rebuld_model.

-ryan

16

Joey G. wrote:

Just wanted to add that I had a similar error. Just updated from 11.3 to
11.4 and started getting the FileNotFound errors as stated above.

The only change to the application was an update from 11.3 to 11.4.

I have reverted back to 11.3 and it seems to be working again.

Hmm… not wanting to make a mistake here so allow me to ask a silly
question. Do I just sudo gem install ferret, pick the 0.11.3 version,
and then erase the 0.11.4 gem?

17

Vince W. wrote:

Hmm… not wanting to make a mistake here so allow me to ask a silly
question. Do I just sudo gem install ferret, pick the 0.11.3 version,
and then erase the 0.11.4 gem?

$ sudo gem uninstall ferret

Select gem to uninstall:

  1. ferret-0.11.3
  2. ferret-0.11.4
  3. All versions

2
Successfully uninstalled ferret version 0.11.4
Remove executables and scripts for
‘ferret-browser’ in addition to the gem? [Yn] n
Executables and scripts will remain installed.

18

I am getting the same error using Ubuntu 6.10 and acts_as_ferret.

I can reindex just fine, but when I resave some records I get the
fs_store, File Not Found Error.

downgrading to 11.3 seems to be working.

  1. Error:
    test_published_blog_can_be_ferreted(BlogTest):
    Ferret::FileNotFoundError: File Not Found Error occured at
    <except.c>:117 in xpop_context
    Error occured in fs_store.c:329 - fs_open_input
    tried to open
    “/Users/mark/Sites/www.site.com/config/…/index/test/blog/_17_0.del” but
    it doesn’t exist:
19

Joey G. wrote:

Just wanted to add that I had a similar error. Just updated from 11.3 to
11.4 and started getting the FileNotFound errors as stated above.

The only change to the application was an update from 11.3 to 11.4.

I have reverted back to 11.3 and it seems to be working again.

Had the same problem, solved it the same way.

One interesting thing perhaps: this problem only occurred when running
entire test suites, never when running individual tests (in TextMate).

20

I actually still get this error in 0.11.3, I have to revert to 0.10.14
to get it remotely working.

Does anyone have any advice on what exactly the issue is? or how it can
be worked around. I would love to be as close to the recent devel as
possible =).

j. weir wrote:

I am getting the same error using Ubuntu 6.10 and acts_as_ferret.

I can reindex just fine, but when I resave some records I get the
fs_store, File Not Found Error.

downgrading to 11.3 seems to be working.

  1. Error:
    test_published_blog_can_be_ferreted(BlogTest):
    Ferret::FileNotFoundError: File Not Found Error occured at
    <except.c>:117 in xpop_context
    Error occured in fs_store.c:329 - fs_open_input
    tried to open
    “/Users/mark/Sites/www.site.com/config/…/index/test/blog/_17_0.del” but
    it doesn’t exist: