Which is faster: searching in a file or in a database

Rails
1

Hello,

I have ten CVS file with about 20000 rows in each.
If I want to search in these files for example get every row that has
the word “apple” i a column, which is faster:

If I read all these ten files into the database and then making search
in the database

or

If I search directly in the files for this inforamtion?

Thank you! Please motivate your answer and include a code to make the
search.

Best regards,

Mark

2

On 26 Dec 2007, at 16:04, Mark T. wrote:

or

If I search directly in the files for this inforamtion?

Thank you! Please motivate your answer and include a code to make the
search.

Assuming you aren’t processing a fresh set of csv files, I’d be very
surprised if parsing the csv files in ruby each time wasn’t slower
than sticking it in the database. It’s what databases are supposed to
do. That’s just my hunch though, and I’m afraid I’ve got enough of my
own work to do before doing yours :slight_smile:

Fred

3

Frederick C. wrote:

On 26 Dec 2007, at 16:04, Mark T. wrote:

or

If I search directly in the files for this inforamtion?

Thank you! Please motivate your answer and include a code to make the
search.

Assuming you aren’t processing a fresh set of csv files, I’d be very
surprised if parsing the csv files in ruby each time wasn’t slower
than sticking it in the database. It’s what databases are supposed to
do. That’s just my hunch though, and I’m afraid I’ve got enough of my
own work to do before doing yours :slight_smile:

Fred

Thank you for your answer.
Just another quick question. I am using fasterCSV to read the file and a
column looks like:

This is the first line
This is the “second” line
This is the third line

I get an error on the second line from fasterCSV since it includes "
char. Can you help me how I can avvoid this problem?

Thanks,

Mark

4

You have to do the validation and have a valid CSV file to parse it
correctly. Correct the file and try again.
On Dec 26, 2007 11:20 AM, Mark T. [email protected]
wrote:

Exactly I get this error:

“Unclosed quoted field on line 21”

–
http://www.rubyplus.org/
Free Ruby and Rails Screencasts

5

Bala P. wrote:

You have to do the validation and have a valid CSV file to parse it
correctly. Correct the file and try again.
On Dec 26, 2007 11:20 AM, Mark T. [email protected]
wrote:

Exactly I get this error:

“Unclosed quoted field on line 21”

–
http://www.rubyplus.org/
Free Ruby and Rails Screencasts

Well, I have a correct CSV file, but with some small changes, for
example I didn´t use " for the strings. I have this column:

This is the first line
This is the “second” line
This is the third line

How can I read this information to my database? Is there any command to
replace the " char in the whole file with nothing?

6

Exactly I get this error:

“Unclosed quoted field on line 21”

7

On 12/27/07, Mark T. [email protected] wrote:

How can I read this information to my database? Is there any command to
replace the " char in the whole file with nothing?

In bash, using a temp file, it’d be something like:

cp foo.csv foo.csv.tmp
sed -e ‘s/"//g’ foo.csv.tmp >foo.csv
rm foo.csv.tmp

–
Greg D.
http://destiney.com/

8

On Dec 27, 5:30 am, Mark T. [email protected]
wrote:

How can I read this information to my database? Is there any command to
replace the " char in the whole file with nothing?

There’s always a text editor. :slight_smile:

Seriously, using the IO class to 1) read each line of the input file,
2) use gsub to remove the quotes, and 3) write each line of the file,
sounds to me like a useful exercise.

///ark