Log analyzer

Hi,

What log analyzing tool do you use? I have found
wvanbergen-request-log-analyzer
to be pretty cool for that purpose. It’s a ruby gem. Here is its github
page: GitHub - wvanbergen/request-log-analyzer: Create reports based on your log files. Supports Rails, Apache, MySQL, Delayed::Job, and other formats.

My needs are:

  • which pages people visit, i.e: which controller and its actions are
    most visited
  • where should I refactor my code to improve performance

Are there any other cool tools like this?

I don’t have any experience with general log analyzers. For performance,
one
option is New Relic’s RPM [ http://newrelic.com/ ]. They offer free and
paid
services. I’ve also used two nice exception loggers, Hoptoad [
http://hoptoadapp.com/ ] and Exceptional [ http://getexceptional.com/ ].
I
like Exceptional better right now because I can comment on why I’ve
closed/ignored an exception.

Regards,
Craig

Thank you for the links. Newrelic is exactly what I was looking for.

Does anyone use rawk? I am trying to get the group by action feature,
but it doesn’t work in Rails 2.1.1 as redefining the Logger class no
longer works as demonstrated in the help.

Any idea on how to do that today?

Cool I found a patch to do that!

Here is the code: http://pastie.org/171954

Credits go to Peter de Bert.

Now rawk is a fantastic Rails log analyzing tool.

I am playing with newrelic, but there is something bothering me: the
graphs don’t take into account the number of times an action has been
called, it only looks at its average response time.

Well this is a wrong approach. I have an action that takes like 3x more
time than any other action, should I optimize it? According to newrelic,
the answer is obviously yes, but in reality no! Why? Because this action
is a special admin only action used only a few times a day.

However I have noticed that another relatively slow action is my root!
And it is the most often called one! That really surprised me. But the
free edition of newrelic doesn’t enable to profile the action in detail,
and the screenshots are not very clear, so I don’t want to pay $40 for a
poor result.

Using rawk I was able to figure out that most of the time spent on my
root is in rendering, I need to optimize this or it will kill my app.

On Mon, Oct 20, 2008 at 4:55 PM, Fernando P. <
[email protected]> wrote:

I am playing with newrelic, but there is something bothering me: the
graphs don’t take into account the number of times an action has been
called, it only looks at its average response time.

Well this is a wrong approach. I have an action that takes like 3x more
time than any other action, should I optimize it? According to newrelic,
the answer is obviously yes, but in reality no! Why? Because this action
is a special admin only action used only a few times a day.

I disagree that the approach is wrong per se. Rather, it’s just not what
you want. RPM can’t know that any of your actions are “special.” It’s
just
telling you that, of all the requests that it has seen, some of them are
slower than others. I’m not defending RPM because I think it’s the best
solution for anyone, I just don’t think that you’re being fair in your
judgment. Also, consider making a feature request if it doesn’t do
something
that you’d like.

However I have noticed that another relatively slow action is my root!
And it is the most often called one! That really surprised me. But the
free edition of newrelic doesn’t enable to profile the action in detail,
and the screenshots are not very clear, so I don’t want to pay $40 for a
poor result.

I’ve only tried New Relic’s RPM Lite on a staging server so far.
Hopefully,
I’ll be trying it in production soon. Another performance monitoring
tool is
offered by FiveRuns [ http://www.fiveruns.com/ ]. I haven’t tried theirs
yet.

Using rawk I was able to figure out that most of the time spent on my

root is in rendering, I need to optimize this or it will kill my app.

I’m glad you found something useful.

Regards,
Craig

I guess it is fairly easy for RPM to count the number of times an action
was triggered. It’s probably part of their paid service.

I am still in search for a good profiling tool that can tell me in a
clear manner, where my action is spending its time.

On my root page, I guess it is all these link_to I have in my
application layout that are slowing everything down, but I’d like to
have proof of it before editing all the code.

On Mon, Oct 20, 2008 at 6:30 PM, Fernando P. <
[email protected]> wrote:

I guess it is fairly easy for RPM to count the number of times an action
was triggered. It’s probably part of their paid service.

I am still in search for a good profiling tool that can tell me in a
clear manner, where my action is spending its time.

On my root page, I guess it is all these link_to I have in my
application layout that are slowing everything down, but I’d like to
have proof of it before editing all the code.

Yeah. Don’t want to spend time in the wrong place. :slight_smile:

When I played w/ RPM Lite on a staging server, it allowed me to drill
quite
far down into the SQL. Thus, I’d be surprised if there wasn’t a way to
drill
into the Ruby/Rails aspect of slow actions.

Craig