Couldn't understand how to use the option :skip_lines

Ruby
#1

Here is a CSV data in one of my file has :

“DE”,“Klasse”,“Deutsch”, “x”
“EN”,“Class”,“Carpenter”,
“DE”,“Klasse”,“Mathe”,
,

Code I wrote :

require ‘csv’

input_file_path = File.expand_path(‘input.csv’,File.dirname(FILE))
output_file_path = File.expand_path(‘output.csv’,File.dirname(FILE))

option = { :skip_blanks => true,
:quote_char => “’”,
:converters => CSV::Converters[:remove_quotes] = lambda do
|field|
field.to_s.tr(’"’,’’)
end
}

CSV.open(output_file_path,‘w’,:force_quotes => true) do |out_row|
CSV.foreach(input_file_path, option) do |in_row|
row_to_add = in_row.reject(&:empty?)
unless row_to_add.empty?
row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0…-2]) :
out_row.puts(row_to_add)
end
end
end

output ( also expected )

“EN”,“Class”,“Carpenter”
“DE”,“Klasse”,“Mathe”

But my question is -

How can I skip lines like , using option :skip_lines (
http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new)

Thanks

#2

On Sun, Feb 23, 2014 at 4:30 PM, Arup R. [email protected]
wrote:

“EN”,“Class”,“Carpenter”
“DE”,“Klasse”,“Mathe”

But my question is -

How can I skip lines like , using option :skip_lines (
http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new)

The :skip_lines option receives an object that responds to match (for
example a Regex). The lines for which this object returns false are
skipped. For example:

:skip_lines => /,/

(untested)

Jesus.

#3

Hi Arup,

Jess has guided you to the right way! :wink:
(I’ve tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the “right” way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr(’"’,’’)
}

options = { :skip_blanks => true,
:quote_char => “’”,
:converters => :remove_quotes,
:skip_lines => /,/
}

Your approach only works because the assignment

CSV::Converters[:remove_quotes] = lambda do |field|
field.to_s.tr(’"’,’’)
end

returns the lambda itself.

And, as stated at the documentation,
“A single converter doesn’t have to be in an Array.”

So, the fragment …
:converters => CSV::Converters[:remove_quotes] = lambda do
|field|
field.to_s.tr(’"’,’’)
end

… acts exactly the same as
:converters => lambda do |field|
field.to_s.tr(’"’,’’)
end

Abinoam Jr.

On Mon, Feb 24, 2014 at 3:40 AM, Jess Gabriel y Galn

#4

Abinoam Jr. wrote in post #1138813:

Hi Arup,

Jess has guided you to the right way! :wink:
(I’ve tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the “right” way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr(’"’,’’)
}

Thanks, I got your point. One doubt, Are you suggesting me to use
{..} instead of do..end ? I don’t think so. But if yes, then why so
?

#5

… more

With :skip_line working you can now strip off the “unless
row_to_add.empty?” because all (/,/) rows will be skipped.

Abinoam Jr.

#6

Hi Arup,

Although there’s some stylistic common pratice, you can use what you
prefer.

My suggestion is not about that.

You are setting the CSV::Converters[:remove_
quotes] inside the options Hash.

I’m suggesting you to do the “right” way.
Set it outside and reference it with its index (Symbol
:remove_quotes).

Best regards,
Abinoam Jr.

#7

… another thing…

It seems all the Converters harassment you are dealing with it’s
because of a malformed CSV file.

You have a line with an extra space
“DE”,“Klasse”,“Deutsch”, “x” # But it should be
“DE”,“Klasse”,“Deutsch”,“x” # without the space before “x”

If you fix this before parsing as csv, the code get much simpler.

I called the “fixed” version of the file “input_fixed.csv”

input_file_path =
File.expand_path(‘input_fixed.csv’,File.dirname(FILE))
output_file_path = File.expand_path(‘output.csv’,File.dirname(FILE))

options = { :skip_blanks => true, :skip_lines => /,/ }

CSV.open(output_file_path,‘w’,:force_quotes => true) do |out_row|
CSV.foreach(input_file_path, options) do |in_row|
row_to_add = in_row.compact
row_to_add.pop if row_to_add.last =~ /\A\sx\s\Z/i
out_row << row_to_add
end
end

The last column of the some lines will be yielded as nil, not as an
empty string “” as before.

[“EN”, “Class”, “Carpenter”, nil]

So you can simply call Array#compact! on it to have

[“EN”, “Class”, “Carpenter”]

And I improved the regexp and the logic to identify the “x” column a
little bit (to fit my taste).

Best regards,
Abinoam Jr.

#8

Dear Arup,

Well… did you edited your original post at the forum after sending it?

The code at https://www.ruby-forum.com/topic/4423224 and the code at
my mail inbox are differents.

unless row_to_add.empty?
  row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0..-2]) :

out_row.puts(row_to_add)
end

versus

out_row.puts row_to_add unless row_to_add.empty? or

row_to_add.last[/^\s+x$/]

They do different things.

For this “second option” you could do

out_row << row_to_add unless row_to_add.last =~ /\A\s*x\s*\Z/i

instead of

row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
out_row << row_to_add

Keep Rubying,
Abinoam Jr.