Back in September (http://rubyforge.org/pipermail/ferret-talk/2006-
September/001210.html) Dave said:
“Another use-case might be to have multiple indexes to speed up
indexing. If for example you are scraping websites it is a very good
idea to have multiple scraping processes. The best way to do this is
to have each process indexing to its own index. You could then search
all indexes at once using a MultiReader or you could also merge all
indexes into a single index.”
Does anyone have any recommendations as to whether one method is
superior to the other? We have a system where we build multiple
indexes from scraping websites and need to expose the indexes as a
web application–the web app will only search the index and will
never modify it. If we have 30 or 40 indexes, would it be better to
combine them or use a MultiReader? How would we combine them? Thanks
in advance.
Erik