How to filter db column(s) from log

Hi, All!

I have been looking for a way to eliminate the contents of certain
large (e.g. pdf files) database columns from my development (and
production) logs using RoR 3 with no luck. I found many posts, etc.
about eliminating parameters (e.g. password) from the log, but nothing
about the contents of the db.

I have two reasons for this, 1) the verbosity and size of the log slow
down my development/debugging, and 2) writing these columns definitely
slows the operation of the site itself.

Can anyone help?

TIA

Donz

I believe you’re looking for config.filter_parameters in
application.rb. Note that filter_parameters is an array, so you can add
to it with <<

Hope this helps,
Garrett L.

Filesystem is write-only.

Don Z. wrote in post #973810:

Hi, All!

I have been looking for a way to eliminate the contents of certain
large (e.g. pdf files) database columns from my development (and
production) logs using RoR 3 with no luck.

Wait, you’re storing PDF files in your DB? Stop doing that. Data like
that belongs in the filesystem.

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

OOPS… Filesystem is READ-only :-[

On Jan 11, 2011, at 7:57 AM, Donald R. Ziesig wrote:

OOPS… Filesystem is READ-only :-[

So use Paperclip and the S3 plugin, and have at it. You can write
anything to S3, probably more files than you will ever have patience
for.

Walter

Garrett,

I have config.filter_parameters = [:password, :document_file] in
application.rb. :document_file is the name of the big column. I
“puts” the array.inspect and confirmed that both password and
document_file are there. The contents of the big column are still
showing up in the log.

BTW, I was advised to store these data in the file system instead of the
db, Unfortunately, the file system will be read-only once I deploy and
I want to test in the environment that I will be using, so I set the
rails tree to read only during some of my development and test
operations (which is how I discovered that I needed to use the database
for all dynamic data in the first place).

Thanks,

Donz

Hi All (again)!

I started this thread with a set of requirements and conditions that I
did not state in my original post. Among these are:

1) The ultimate host will have a read-only file system which
precludes use of paperclip's default filesystem storage, and;

2) My client's business/legal reasons preclude the use of
paperclip's S3 option.

3) I have added the name(s) of the big columns in
config.filter_parameters but that doesn't seem to change anything.

Pat Shaughnessy’s attachment-in-database-column(s) update to paperclip
(which works great, incidentally) does the job except for the cruft in
the log when storing large files.

The cruft tends to make my life difficult when analyzing the log and
seems to impact the speed during the time that the logfile is being
written.

If I have to, I’ll continue with the current implementation and put up
with the aggravation hoping that production mode will stop most of it
(have to figure out where to put the log file, too, due to the read-only
fs).

Thanks,

Don Z.

I have two reasons for this, 1) the verbosity and size of the log slow
down my development/debugging, and 2) writing these columns definitely
slows the operation of the site itself.

Can anyone help?

You want it removed from the “SQL (1.2ms) INSERT…” line in the logs?
filter_params will remove it from the “Parameters: {…” line.

I think you’re only option for removing it from the SQL in the log
itself is to silence the active record logger whenever you do something
that generates those queries.

It’s not an issue in production as production doesn’t log the sql
queries…

-philip

After a long interlude, I found my problem with
config.filter_parameters. I can’t count how many hours I have wasted
due to a typo :frowning: The original line was:

config.filter_parameters = [:password, :document_file]

the correct line is:

config.filter_parameters = [:password, :documents_file]

This isn’t the first time that I have gotten burned by rails’
pluralization.

Many thanks for all who commented and pointed me in the right direction.

Don Z.