Has anyone experienced an ActiveRecord query being much much slower than
running the equivalent
query in the MySql console?
I am running a simple “group by” query with one condition on a table
million records. When
I run EXPLAIN it says that it will have to scan over all the records -
that’s OK. In the MySQL
query it takes 1min 40secs which is OK for me - this is not a web app.
when I run exactly
the same query in a ruby script using ActiveRecord it takes about 45
far too long.
I am running all this on my work PC, 3 gig of memory. Windows PerfMon
I have 2 gig available
and 1 gig on standyby. I am using ruby 1.8.7; rails 3; mysql2 gem;
7 and MySql 5.5.
I have also set the INNODB buffer pool size to 480M.
What seems to be happening is that the MySQL process uses all the memory
to 500M and
there is a lot of page swapping. PerfMon shows that the disk activity
very high (99%). So
that explains the bottleneck. But I still don’t know why this happens
I run the code from
Ruby and yet from the MySQL console it is fine.
I even tried just using the MySQL2 gem directly in my ruby code using
but still very very slow.
I know this probably doesn’t explain everything but I just wondered if
anyone had a similar experience.
Thanks, nice idea for installing ruby-prof. Never used it before and
extremely easy to install and use.
I’m on Windows so I needed to install with this command:
gem install ruby-prof --platform=mswin32 -v 0.8.1
I tried it on a simpler query and I give the top lines of the profile
You can see that it takes 5.9 seconds with virtually all the time spend
Mysql2::Client#query. When I run the same query through
the console it takes 812ms.
Yes I am pretty sure, because I write it out using to_sql before I run.
the moment is seems like a Windows memory problem rather than a ruby
problem. As I commented on the StackOverflow thread I now have a
for my immediate problem. Since I am calculating averages over a date
over three years - and since the dates are actually end-of-month dates -
am really calculating an overage over 36 distinct months. I therefore
query in two steps. First, I calculate averages and counts per month.
Second I aggregate up to get averages over 36 months. Runs pretty
But I haven’t solved the problem and I haven’t been able to find
from anyone else experiencing the same thing. So the mystery remains.
Ruby-prof, though. Thanks for that. You made me install it and its
actually rather nice.
Yes I do. But still the problem is the difference in speed between
the query in the MySql console and using MySql2 gem. At the moment I
I might also be having a problem with a gem file. So slowly trying to
remove them to see if I can see one causing a problem.