Forum: Ferret Can't remove duplicate

573817a2b8cd7ddc3db11628920f7faf?d=identicon&s=25 Femto Zheng (femto)
on 2009-08-18 08:28
(Received via mailing list)
Hello all, I can't remove duplicate,I'm using ferret
to index log file in order to monitor application activity,
what I want to do is index data based on the uniqueness of
[filename,line](actullay should be [host,filename,line],
the code is following:

if !$indexer

      field_infos = Ferret::Index::FieldInfos.new(:index =>
:untokenized_omit_norms,
                                   :term_vector => :no)
      field_infos.add_field(:content, :store => :yes, :index => :yes)

      $indexer = Ferret::I.new(:path => index_dir,
                               :field_infos => field_infos,
                               :key => [:filename, :line],
                               :max_buffered_docs=>100)

      #$indexer ||= Ferret::I.new(:path=>index_dir, :key => ['filename',
'line'], :max_buffered_docs=>100) #unique host,file_name,line
      #$indexer.field_infos.add_field(:time,
      #                               #:default_boost => 20,
      #                               :store => :yes,
      #                               :index => :untokenized,
      #                               :term_vector => :no)
    end
but the problem is, I will index a new datum even if the [filename,line]
is same, even I change :key => ["filename", "line"], it also doesn't
work,
what's the problem? Thanks.
This topic is locked and can not be replied to.