RE: What to do with HUGE instance variables in Rails?

Problem is that the DB has about 260,000 lines which

My first answer is: don’t do it.
Second: Never ever.
Third: are you serious?

Well lets qualify Stefan’s comments a bit. A task processing 260K rows
in a database is not a job to be done during a web request. If you were
doing a mail merge for example then I’d put that off as a scheduled
task. Which ever way you go this is going to take some considerable
processing. You want a web page to respond in a couple of seconds not a
few minutes.

In a web page reading 260K rows into memory at once is bad (long time to
process and consumes LOTs of ram). Reading 260K rows one at a time (find
:first/ find :next) is also bad as it takes a long time to process.

So my first answer is: don’t do it in Rails, but you could do it in Ruby
still using ActiveRecord scheduled with cron.

Second: Never do something like this in any Web framework. Even if you
process 1000 rows/sec that’s 4 1/2 minutes, who’d wait that long for a
web page?

Third: do it as an offline or overnight batch job. Perhaps using a web
page to trigger the start of processing and an email notifying it’s
completion so you can download from the server.


This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs