Concurrency / #search_each problem / segfault

Hello everyone,

I was stress-testing my application (running on Rails via FastCGI) by
letting two concurrent users (not human … an app called ‘siege’)
a) save an Article and b) search for all Articles.

I am searching via
Article.ferret_index.search_each( …) do |doc_id,score|
doc = index[doc_id]

end

and writing via
Article.ferret_index << self.to_doc

where Article.ferret_index is implemented as in ‘act_as_ferret’:
@@ferret_index = nil
def Article::ferret_index
@@ferret_index ||= Ferret::Index::Index.new( :path => ferret_path,
:auto_flush => true, :create_if_missing => false )
end

The 2 errors I got (when I do “doc = index[doc_id]”) were :
ArgumentError (:12250 is out of range [0…12243] for IndexWriter#[]):
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
[]' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in[]’
/usr/lib/ruby/1.8/monitor.rb:229:in synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in[]’

Ferret::StateError (State Error occured at <except.c>:79 in xraise
Error occured in index.c:3404 - sr_get_lazy_doc
Document 0 has already been deleted

):
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
[]' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in[]’
/usr/lib/ruby/1.8/monitor.rb:229:in synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in[]’

So, obviously the index changed after #search_each and before “doc =
index[doc_id]”.
Is this expected behaviour? How to avoid this?

Then I did the same thing without a webserver, just 2 consoles. One for
saving, one for searching.
The one searching now just ends with
“/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:
[BUG] Segmentation fault”

As for now, to avoid the problem, I use an IndexReader that I newly
create for every search.
But I guess that is not the best approach?

I am using ferret 0.10.8 and ruby 1.8.4 on Debian Sarge.

Best regards

josh

On 9/27/06, Josh D. [email protected] wrote:

end

/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in

"/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:

josh

Hi Josh,

Just thought I’d let you know that I’m working on fixing this. Expect
a solution in the next release.

Cheers,
Dave

On 9/27/06, Josh D. [email protected] wrote:

end
Hi Josh,

Just to you this time. What is the rest of the code in this loop
(above). ie, what is the “…”. It should help me sort out the problem.

By the way, you should upgrade to Ferret-0.10.9.

Cheers,
Dave

Hi Dave,

Just to you this time. What is the rest of the code in this loop
(above). ie, what is the “…”. It should help me sort out the problem.

here are the complete stack traces:

ArgumentError (:12250 is out of range [0…12243] for IndexWriter#[]):
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
[]' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in[]’
/usr/lib/ruby/1.8/monitor.rb:229:in synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in[]’
/app/models/article.rb:150:in fulltext_search' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:364:insearch_each’
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:in
search_each' /usr/lib/ruby/1.8/monitor.rb:229:insynchronize’
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:359:in
search_each' /app/models/article.rb:149:infulltext_search’
/app/controllers/public/article_controller.rb:91:in search' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:941:inperform_action_without_filters’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:368:in
perform_action_without_benchmark' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:inperform_action_without_rescue’
/usr/local/lib/site_ruby/benchmark.rb:300:in measure' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:inperform_action_without_rescue’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.rb:82:in
perform_action' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:408:inprocess_without_filters’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:377:in
. process_without_session_management_support' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_management.rb:117:inprocess’
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in
dispatch' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:150:inprocess_request’
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:54:in
process!' /usr/lib/ruby/1.8/fcgi.rb:600:ineach_cgi’
/usr/lib/ruby/1.8/fcgi.rb:597:in each_cgi' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:53:inprocess!’
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:23:in
`process!’

Ferret::StateError (State Error occured at <except.c>:79 in xraise
Error occured in index.c:3404 - sr_get_lazy_doc
Document 0 has already been deleted

):
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
[]' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in[]’
/usr/lib/ruby/1.8/monitor.rb:229:in synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in[]’
/app/models/article.rb:150:in fulltext_search' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:364:insearch_each’
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:in
search_each' /usr/lib/ruby/1.8/monitor.rb:229:insynchronize’
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:359:in
search_each' /app/models/article.rb:149:infulltext_search’
/app/controllers/public/article_controller.rb:91:in search' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:941:inperform_action_without_filters’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:368:in
perform_action_without_benchmark' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:inperform_action_without_rescue’
/usr/local/lib/site_ruby/benchmark.rb:300:in measure' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:inperform_action_without_rescue’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.rb:82:in
perform_action' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:408:inprocess_without_filters’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:377:in
. process_without_session_management_support' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_management.rb:117:inprocess’
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in
dispatch' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:150:inprocess_request’
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:54:in
process!' /usr/lib/ruby/1.8/fcgi.rb:600:ineach_cgi’
/usr/lib/ruby/1.8/fcgi.rb:597:in each_cgi' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:53:inprocess!’
/usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:23:in
`process!’

Hope that helps.

By the way, you should upgrade to Ferret-0.10.9.

Ok, will do that.

Thank you

josh

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs