I’m trying to index ~130,000 documents [soon to grow to about 500,000
documents] and I’m wondering if its possible to combine ferret databases
or in some other way split up the building process.
Normally, indexing 130k documents wouldn’t be that painful except that
there are different types of links between these documents and they are
not absolute (so for example doc a refers to a document b but there are
multiple different documents laballed document a and document b and to
prevent false links I have to use some fairly computationally intensive
If its not possible to split up the building of a ferret index I’ll
probably resolve the links into absolute links as a separate part of the
process [which I can split up] and then build the ferret index one one
machine after that.