Using custom stem analyzer giving mongrel errors

Ruby Apps Ferret
1

I’m using the custom stem analyzer:

require ‘rubygems’
require ‘ferret’
include Ferret
module Ferret::Analysis
class FerretAnalyzer
def initialize(stop_words = FULL_ENGLISH_STOP_WORDS)
@stop_words = stop_words
end
def token_stream(field, text)
StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(text)),
@stop_words))
end
end
end

and I’m simply setting the :analyzer option in AAF.

However, I get odd behavior. The first search that I do will go through
and display the proper results, but any subsequent request starts to
produce odd behavior. For example when you are redirected, mongrel does
not redirect the browser but instead displays the message ‘You are being
redirected’. Here is the error produced in the Mongrel logs:

Thu Dec 07 17:49:06 PST 2006: Error calling Dispatcher.dispatch
#<NameError: cannot remove Object::QueryParser>
/Users/raymond/Rails/cart/config/…/vendor/rails/activerecord/lib/…/…/activesupport/lib/active_support/dependencies.rb:416:in
remove_const' /Users/raymond/Rails/cart/config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:416:inremove_constant’
/Users/raymond/Rails/cart/config/…/vendor/rails/activerecord/lib/…/…/activesupport/lib/active_support/dependencies.rb:270:in
remove_unloadable_constants!' /Users/raymond/Rails/cart/config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:270:inremove_unloadable_constants!’
/Users/raymond/Rails/cart/config/…/vendor/rails/activerecord/lib/…/…/activesupport/lib/active_support/dependencies.rb:73:in
clear' /Users/raymond/Rails/cart/config/../vendor/rails/railties/lib/dispatcher.rb:60:inreset_application!’
/Users/raymond/Rails/cart/config/…/vendor/rails/railties/lib/dispatcher.rb:116:in
reset_after_dispatch' /Users/raymond/Rails/cart/config/../vendor/rails/railties/lib/dispatcher.rb:51:indispatch’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/rails.rb:84:in
process' /Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/1.8/sync.rb:229:insynchronize’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/rails.rb:83:in
process' /Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:580:inprocess_client’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:579:in
process_client' /Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:inrun’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:686:in
run' /Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel.rb:673:inrun’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/configurator.rb:267:in
run' /Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/configurator.rb:266:inrun’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:127:in
run' /Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/lib/mongrel/command.rb:211:inrun’
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/lib/ruby/gems/1.8/gems/mongrel-0.3.13.4/bin/mongrel_rails:231
/Applications/Locomotive2/Bundles/standardRailsSept2006.locobundle/i386/bin/mongrel_rails:18

2

Oh I forgot to add, that these errors all go away and everything works
fine when I remove the :analyzer option from aaf.

Thanks,
Ray

3

Well after a few hours of playing with it, I finally gave up and decided
to try a different stem analyzer from another post. Specifically:
require ‘rubygems’
require ‘ferret’

class StemmedAnalyzer < Ferret::Analysis::Analyzer
include Ferret::Analysis
def initialize(stop_words = ENGLISH_STOP_WORDS)
@stop_words = stop_words
end
def token_stream(field, str)
StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(str)),
@stop_words))
end
end

The error seems to have disappeared now. Very strange indeed.

4

looks like inheriting from Analyzer is essential…

Not necessarily. I had a similar problem when I upgraded to Rails 1.2,
and the solution seems to have been putting the ‘include’ statement
inside the class, instead of outside. It looks like the same change
happened here, in addition to the change in inheritance.

5

On Fri, Dec 08, 2006 at 06:20:11AM +0100, Raymond O’connor wrote:

Well after a few hours of playing with it, I finally gave up and decided
to try a different stem analyzer from another post. Specifically:
require ‘rubygems’
require ‘ferret’

class StemmedAnalyzer < Ferret::Analysis::Analyzer

[…]

The error seems to have disappeared now. Very strange indeed.

looks like inheriting from Analyzer is essential…

Jens

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66