I wrote this application using a MySQL DB and Solr Server (updated via
acts_as_solr plugin)
I need to load tons of documents to test it out, the app is a
newspaper article collection repository that I upload via SGML corpus.
Now the strange is…that after loading about 30K articles…the app
started to act weird: it does not load anymore or it does after a lot
of minutes…yesterday I’ve kept loading…it reached the 56K
articles…but now is stuck again…
In testing this on a different machine that I use from remote… any
clue?
of minutes…yesterday I’ve kept loading…it reached the 56K
articles…but now is stuck again…
In testing this on a different machine that I use from remote… any
clue?
(the solr server is perfectly reachable…)
I have a development system with over 5 million rows in one table. The
string
field is typically 5-50 characters, the rest is a half dozen integers of
different sizes. The production system has over 6 million rows.
That said, ActiveRecord is convenient and memory greedy. For that afore
mentioned table, I just use straight SQL (inserts and selects can run to
thousands of rows). For the other tables, I continue to use
ActiveRecord.
of minutes…yesterday I’ve kept loading…it reached the 56K
articles…but now is stuck again…
In testing this on a different machine that I use from remote… any
clue?
(the solr server is perfectly reachable…)
I have a development system with over 5 million rows in one table. The
string
field is typically 5-50 characters, the rest is a half dozen integers of
different sizes. The production system has over 6 million rows.
That said, ActiveRecord is convenient and memory greedy. For that afore
mentioned table, I just use straight SQL (inserts and selects can run to
thousands of rows).
Have you tried ar-extensions? This should let you do bulk queries
properly.
For the other tables, I continue to use
ActiveRecord.
