I’m experimenting with optimizing some areas that involve long
times on some SQL selects (or any background task, really). This is in
Rails 4.0.0 with Ruby 2.0.0p247, PostgreSQL 9.2.4.
In this case, I can do intelligent dividing of selects to even the
amongst N processes for the DB backend.
I currently am trying Spawnling, and this works for exactly one hit
I have to restart the web service:
spawns = 
ABC.each do # criteria for selects to divide load
spawns << Spawnling.new do
# portion of big SQL select here via ActiveRecord, and store in
wait for all N blocks of code to finish running
assemble result from memcache and deliver
Works for one hit and then future hits log correctly but fail to deliver
any content to the browser. There seems to be a number of issues with
Spawnling anyways as it seems to be in disrepair.
How are folks parallelizing tasks these days for lots of CPUs per hit?
What’s the Rails 4/Ruby 2 way to do it?
Thanks for any thoughts.