# How much "records" can development env stands?!

**URL:** <https://www.ruby-forum.com/t/how-much-records-can-development-env-stands/183060>\
**Category:** Rails\
**Created:** [February 7, 2010, 10:37am UTC](https://www.ruby-forum.com/t/how-much-records-can-development-env-stands/183060 "2010-02-07T10:37:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![adedip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/e495f1/32.png) [@adedip](https://www.ruby-forum.com/u/adedip)\
**Post date:** [February 7, 2010, 10:37am UTC](https://www.ruby-forum.com/t/how-much-records-can-development-env-stands/183060/1 "2010-02-07T10:37:52Z")

</div>

I didn’t know how to title this question sorry 😛

I wrote this application using a MySQL DB and Solr Server (updated via  
acts\_as\_solr plugin)

I need to load tons of documents to test it out, the app is a  
newspaper article collection repository that I upload via SGML corpus.

Now the strange is…that after loading about 30K articles…the app  
started to act weird: it does not load anymore or it does after a lot  
of minutes…yesterday I’ve kept loading…it reached the 56K  
articles…but now is stuck again…

In testing this on a different machine that I use from remote… any  
clue?

(the solr server is perfectly reachable…)

thx,  
Andrea

---

<div class="post-metadata">

**Author:** ![adedip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/e495f1/32.png) [@adedip](https://www.ruby-forum.com/u/adedip)\
**Post date:** [February 7, 2010, 8:41pm UTC](https://www.ruby-forum.com/t/how-much-records-can-development-env-stands/183060/2 "2010-02-07T20:41:50Z")

</div>

Quoting adedip [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> of minutes…yesterday I’ve kept loading…it reached the 56K  
> articles…but now is stuck again…
> 
> In testing this on a different machine that I use from remote… any  
> clue?
> 
> (the solr server is perfectly reachable…)

I have a development system with over 5 million rows in one table. The  
string  
field is typically 5-50 characters, the rest is a half dozen integers of  
different sizes. The production system has over 6 million rows.

That said, ActiveRecord is convenient and memory greedy. For that afore  
mentioned table, I just use straight SQL (inserts and selects can run to  
thousands of rows). For the other tables, I continue to use  
ActiveRecord.

HTH,  
Jeffrey

---

<div class="post-metadata">

**Author:** ![adedip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/e495f1/32.png) [@adedip](https://www.ruby-forum.com/u/adedip)\
**Post date:** [February 7, 2010, 8:47pm UTC](https://www.ruby-forum.com/t/how-much-records-can-development-env-stands/183060/3 "2010-02-07T20:47:23Z")

</div>

Jeffrey L. Taylor wrote:

> Quoting adedip [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):
> 
> > of minutes…yesterday I’ve kept loading…it reached the 56K  
> > articles…but now is stuck again…
> > 
> > In testing this on a different machine that I use from remote… any  
> > clue?
> > 
> > (the solr server is perfectly reachable…)
> 
> I have a development system with over 5 million rows in one table. The  
> string  
> field is typically 5-50 characters, the rest is a half dozen integers of  
> different sizes. The production system has over 6 million rows.
> 
> That said, ActiveRecord is convenient and memory greedy. For that afore  
> mentioned table, I just use straight SQL (inserts and selects can run to  
> thousands of rows).

Have you tried ar-extensions? This should let you do bulk queries  
properly.

> For the other tables, I continue to use  
> ActiveRecord.
> 
> HTH,  
> Jeffrey

Best,  
–Â  
Marnen Laibow-Koser  
[http://www.marnen.org](http://www.marnen.org)  
removed\_email\_address@domain.invalid