I want to use Ferret to index openstreetmap.org’s (OSM) node data: 400M
pairs of latitudes and longitudes.
Has anyone already done this?
What optimizations can I use? I don’t need to access the data while
it’s indexing, if that helps.
Can grid computing help here? Can I index chunks of data separately
and then efficiently merge the indexes?
I realize OSM has lots more data (ways, tags, relations, etc), but
indexing the nodes (to find all nodes within a given
latitude/longitude range) would be my first step.
I also realize MySQL or PostgreSQL could do this, but I’m looking for
an embedded/serverless solution and sqlite3 indexing is too slow.
–
We’re just a Bunch Of Regular Guys, a collective group that’s trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.