Parsing text with while loop

Ruby
1

I am parsing a csv file and I am unable to end my loop as I want it.

When I run my ruby file, I get this :

feelx@tbiweb:~/public_html$ ruby csv_to_sql.rb
/usr/lib/ruby/1.8/date/format.rb:1055:in dup': can't dup NilClass (TypeError) from /usr/lib/ruby/1.8/date/format.rb:1055:in _parse’
from /usr/lib/ruby/1.8/date.rb:999:in `parse’
from csv_to_sql.rb:43

Link to my code : http://pastie.org/806415

Thanks in advance.

Felix

2

Felix F. wrote:

I am parsing a csv file and I am unable to end my loop as I want it.

When I run my ruby file, I get this :

feelx@tbiweb:~/public_html$ ruby csv_to_sql.rb
/usr/lib/ruby/1.8/date/format.rb:1055:in dup': can't dup NilClass (TypeError) from /usr/lib/ruby/1.8/date/format.rb:1055:in _parse’
from /usr/lib/ruby/1.8/date.rb:999:in `parse’
from csv_to_sql.rb:43

Link to my code : http://pastie.org/806415

Thanks in advance.

Felix

Why wouldn’t you trust JEGII’s CSV library (or ‘FasterCSV’ if you are on
1.8’) ? It does all the hard work of parsing CSV for you.

3

Aldric G. wrote:

Felix F. wrote:

I am parsing a csv file and I am unable to end my loop as I want it.

When I run my ruby file, I get this :

feelx@tbiweb:~/public_html$ ruby csv_to_sql.rb
/usr/lib/ruby/1.8/date/format.rb:1055:in dup': can't dup NilClass (TypeError) from /usr/lib/ruby/1.8/date/format.rb:1055:in _parse’
from /usr/lib/ruby/1.8/date.rb:999:in `parse’
from csv_to_sql.rb:43

Link to my code : http://pastie.org/806415

Thanks in advance.

Felix

Why wouldn’t you trust JEGII’s CSV library (or ‘FasterCSV’ if you are on
1.8’) ? It does all the hard work of parsing CSV for you.

Because the content of the csv is not constant. It’s not built as a
database file…

4

2010/2/2 Felix F. [email protected]:

Aldric G. wrote:

Why wouldn’t you trust JEGII’s CSV library (or ‘FasterCSV’ if you are on
1.8’) ? It does all the hard work of parsing CSV for you.

Because the content of the csv is not constant. It’s not built as a
database file…

I don’t see how that rules out a CSV lib for reading CSV formatted
input.

Also, I am not sure what the purpose of your exercise is. Isn’t there
a CSV import in MySQL available? What about
http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html ? I do not
see any data manipulation in your code so the regular tools should be
sufficient. Did I miss anything?

Kind regards

robert

5

I edited your code and just changed a couple lines:
http://pastie.org/807473

I changed this:
input_file_stream = File.open(input_filename, “r”)

To this:
input_file_stream = IO.readlines(input_filename)

Using IO.readlines will just read the entire contents of the file and
store
each line as an element in an array. You can then iterate through the
array
rather than using a while loop.

I changed your loop and changed the line where you split the lines of
the
csv file from this:
input_file_line = input_file_stream.gets.to_s.split(/,/)

To this:
input_file_line = line.split(/,/)

This should work, I obviously did not test it. Hopefully this will
point
you in the right direction.

Bryan

6

Felix F. wrote:

Aldric G. wrote:

Felix F. wrote:

I am parsing a csv file and I am unable to end my loop as I want it.

When I run my ruby file, I get this :

feelx@tbiweb:~/public_html$ ruby csv_to_sql.rb
/usr/lib/ruby/1.8/date/format.rb:1055:in dup': can't dup NilClass (TypeError) from /usr/lib/ruby/1.8/date/format.rb:1055:in _parse’
from /usr/lib/ruby/1.8/date.rb:999:in `parse’
from csv_to_sql.rb:43

Link to my code : http://pastie.org/806415

Thanks in advance.

Felix

Why wouldn’t you trust JEGII’s CSV library (or ‘FasterCSV’ if you are on
1.8’) ? It does all the hard work of parsing CSV for you.

Because the content of the csv is not constant. It’s not built as a
database file…

Why doing :

begin

end until (blop[24] == “Total:”)

if blop[24] contains a string would give an error saying “can’t dup
NilClass”?

7

Felix F. wrote:

That is why […] (I think) using FasterCSV wouldn’t help that much either.

This CSV library at least parses the file correctly for you. CSV has
some quoting rules and merely spliting by “,” isn’t very robust.

BTW, I find your code a bit hard to read. Maybe because of all the long
variable names.

Tip: instead of referring to some_variable[14], some_variable[7], etc.,
you can do:

created, name, place, duration = some_variable.values_at(14, 7, 6,
20)

(I think it’s more readable.)

8

Bryan W. wrote:

I edited your code and just changed a couple lines:
http://pastie.org/807473

I changed this:
input_file_stream = File.open(input_filename, “r”)

To this:
input_file_stream = IO.readlines(input_filename)

Using IO.readlines will just read the entire contents of the file and
store
each line as an element in an array. You can then iterate through the
array
rather than using a while loop.

I changed your loop and changed the line where you split the lines of
the
csv file from this:
input_file_line = input_file_stream.gets.to_s.split(/,/)

To this:
input_file_line = line.split(/,/)

This should work, I obviously did not test it. Hopefully this will
point
you in the right direction.

Bryan

Thanks everyone for helping me out. I really appreciate your help.

As I said previously, the CSV file is not constant — no used as a
database-like file. Here’s an exemple of the data :

http://pastie.org/807687

Every single line begins by the same string. However, the first and the
last lines differ from each other. Also, the middle lines are also
different from the first.

That is why I read the first line as being the headline and then going
through a loop till EOF. I achieved this part. It’s working great the
way I did it. However, I would like my loop to stop on the string
“Total:” which is part of the last line only.

The reason why I need to stop on that particular word is that I will
have many of the first lines, middles lines and last lines for specific
people in the same file.

That is why I could not use MySQL to import the CSV file and that (I
think) using FasterCSV wouldn’t help that much either. I also want to
automate processes the more I can.

Hope this can help to help me :slight_smile: Maybe I am not doing it the way I
should be. I have to mention that I’m new with Ruby but, I love it.

Have a good day everyone!

Felix

9

Robert K. wrote:

On 02/03/2010 10:00 PM, Albert S. wrote:

Felix F. wrote:

That is why […] (I think) using FasterCSV wouldn’t help that much either.

This CSV library at least parses the file correctly for you. CSV has
some quoting rules and merely spliting by “,” isn’t very robust.

Absolutely agree! Felix, I believe it is not clear to you what CSV
really does. Try this:

require ‘csv’

CSV.foreach(‘pastie-807687.txt’) do |row|
p row
break if /Total/ =~ row[22]
end

Cheers

robert

Thanks everyone, I found my problem.

:wink:

Felix

10

On 02/03/2010 10:00 PM, Albert S. wrote:

Felix F. wrote:

That is why […] (I think) using FasterCSV wouldn’t help that much either.

This CSV library at least parses the file correctly for you. CSV has
some quoting rules and merely spliting by “,” isn’t very robust.

Absolutely agree! Felix, I believe it is not clear to you what CSV
really does. Try this:

require ‘csv’

CSV.foreach(‘pastie-807687.txt’) do |row|
p row
break if /Total/ =~ row[22]
end

Cheers

robert