Slow acts as ferret indexing - taking 7 days for 10Million rows (2GB db)

The Ferret Indexing is taking very 7 days to complete (table
with10Million row - DB size is about 2 gb). I could see my memory is
used only 40% and cpu is utilized only 10%. but still the indexing is
slow.

I am running ferret through DRB

My acts_as_ferret index definition looks something like this.

module Ferret::Analysis
class MyAnalyzer
def token_stream(field,text)
StemFilter.new(LowerCaseFilter.new(StandardTokenizer.new(text)))
end
end
end

ActsAsFerret::define_index(‘my_index’,
:models => {
Product => {
:fields => {
:field_one => {:boost => 0.0576},
:field_two => {:boost => 0.0432},
:field_three => {:boost => 0.0432},
:field_four => {:boost => 0.0432},
:field_five => {:boost => 0.0144},
:field_six => :boost => 0.0144},
:field_seven => {:boost => 0.0036},
:field_eight => {:boost => 0.0036},
:field_nine => {:boost => 0.0036},
:field_ten => {:boost => 0.0009},
:field_eleven => {:boost => 0.0006},
:field_twelve => {:boost => 0.0002},
:field_thirteen => {:boost => 0.0000001}
}
}
},
:ferret => { :or_default => true,
:analyzer => Ferret::Analysis::MyAnalyzer.new
},
:remote => true,
:max_buffer_memory => 536870912, #setting to 512 MB for faster
indexing. default value is 16 Mb.
:merge_factor => 5 #default value is 10. Lower number will speed up
search but slow down indexing
)

Is there anyway I can improve the performance of indexing ? I really
love ferret and acts_as_ferret and and don’t want to give-up to sphinx
if there is a way.

appreciate your reply

regards
madhu

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs