Forum: Ferret Grouping with ferret

Hi list,

I have a problem grouping with ferret.

I'm using the filter_proc from Dave's book as seen below

results = {}
group_by_proc = lambda do |doc_id, score, searcher|
  doc = searcher[doc_id]
    (results[doc[:pk_file_id]]||=[]) << doc[:filename] << doc[:path]
  next true
end


The problem is that if I use this it ignores my limit clause.

I set limit on 10 and I still get 5995 results and it takes several
seconds.

How come the limit clause is ignored when using a filter_proc? How can
I change this behaviour?


term = "wi"

bool1 = Ferret::Search::BooleanQuery.new()
bool1.add_query(Ferret::Search::PrefixQuery.new(:filename, "#{term}"))
bool1.add_query(Ferret::Search::PrefixQuery.new(:path, "#{term}"))

index.search(bool1, :limit => 10, :filter_proc => group_by_proc)

puts result.size
5995


Cheers,
Henke

Hi!

On 28.07.2008, at 13:29, Henrik wrote:

>   next true
> end
>
>
> The problem is that if I use this it ignores my limit clause.
>
> I set limit on 10 and I still get 5995 results and it takes several  
> seconds.
>
> How come the limit clause is ignored when using a filter_proc? How  
> can I change this behaviour?

Filters are applied by Ferret before the result is limited, that's why
your filter gets to see all possible results regardless of the limit
you specify. If it was implemented the other way around, first
limiting and then filtering, you would possibly end up with less than
limit results in case your filter would actually filter out any
results. Of course in your case this wouldnt happen as your filter
does no filtering but always returns true.

If you really only want the first 10 results, why dont you just use
the results you get back and do your result
collecting there like this?

results = {}
hit_count = index.search_each(query, :limit => 10, :filter_proc =>
group_by_proc) do |doc, score|
   (results[doc[:pk_file_id]]||=[]) << doc[:filename] << doc[:path]
end

You could of course also return false in your filter_proc for every
possible hit once your results collection has reached the desired size
to save the time collecting all results.

cheers,
Jens


--
Jens Krämer
Finkenlust 14, 06449 Aschersleben, Germany
VAT Id DE251962952
http://www.jkraemer.net/ - Blog
http://www.omdb.org/     - The new free film database