Couldn't understand how to use the option :skip_lines

Here is a CSV data in one of my file has :

“DE”,“Klasse”,“Deutsch”, “x”
“EN”,“Class”,“Carpenter”,
“DE”,“Klasse”,“Mathe”,
,

Code I wrote :

require ‘csv’

input_file_path = File.expand_path(‘input.csv’,File.dirname(FILE))
output_file_path = File.expand_path(‘output.csv’,File.dirname(FILE))

option = { :skip_blanks => true,
:quote_char => “’”,
:converters => CSV::Converters[:remove_quotes] = lambda do
|field|
field.to_s.tr(’"’,’’)
end
}

CSV.open(output_file_path,‘w’,:force_quotes => true) do |out_row|
CSV.foreach(input_file_path, option) do |in_row|
row_to_add = in_row.reject(&:empty?)
unless row_to_add.empty?
row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0…-2]) :
out_row.puts(row_to_add)
end
end
end

output ( also expected )

“EN”,“Class”,“Carpenter”
“DE”,“Klasse”,“Mathe”

But my question is -

How can I skip lines like , using option :skip_lines (
http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new)

Thanks

On Sun, Feb 23, 2014 at 4:30 PM, Arup R. [email protected]
wrote:

“EN”,“Class”,“Carpenter”
“DE”,“Klasse”,“Mathe”

But my question is -

How can I skip lines like , using option :skip_lines (
http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new)

The :skip_lines option receives an object that responds to match (for
example a Regex). The lines for which this object returns false are
skipped. For example:

:skip_lines => /,/

(untested)

Jesus.

Hi Arup,

Jess has guided you to the right way! :wink:
(I’ve tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the “right” way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr(’"’,’’)
}

options = { :skip_blanks => true,
:quote_char => “’”,
:converters => :remove_quotes,
:skip_lines => /,/
}

Your approach only works because the assignment

CSV::Converters[:remove_quotes] = lambda do |field|
field.to_s.tr(’"’,’’)
end

returns the lambda itself.

And, as stated at the documentation,
“A single converter doesn’t have to be in an Array.”

So, the fragment …
:converters => CSV::Converters[:remove_quotes] = lambda do
|field|
field.to_s.tr(’"’,’’)
end

… acts exactly the same as
:converters => lambda do |field|
field.to_s.tr(’"’,’’)
end

Abinoam Jr.

On Mon, Feb 24, 2014 at 3:40 AM, Jess Gabriel y Galn

Abinoam Jr. wrote in post #1138813:

Hi Arup,

Jess has guided you to the right way! :wink:
(I’ve tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the “right” way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr(’"’,’’)
}

Thanks, I got your point. One doubt, Are you suggesting me to use
{..} instead of do..end ? I don’t think so. But if yes, then why so
?

… more

With :skip_line working you can now strip off the “unless
row_to_add.empty?” because all (/,/) rows will be skipped.

Abinoam Jr.

Hi Arup,

Although there’s some stylistic common pratice, you can use what you
prefer.

My suggestion is not about that.

You are setting the CSV::Converters[:remove_
quotes] inside the options Hash.

I’m suggesting you to do the “right” way.
Set it outside and reference it with its index (Symbol
:remove_quotes).

Best regards,
Abinoam Jr.

… another thing…

It seems all the Converters harassment you are dealing with it’s
because of a malformed CSV file.

You have a line with an extra space
“DE”,“Klasse”,“Deutsch”, “x” # But it should be
“DE”,“Klasse”,“Deutsch”,“x” # without the space before “x”

If you fix this before parsing as csv, the code get much simpler.

I called the “fixed” version of the file “input_fixed.csv”

input_file_path =
File.expand_path(‘input_fixed.csv’,File.dirname(FILE))
output_file_path = File.expand_path(‘output.csv’,File.dirname(FILE))

options = { :skip_blanks => true, :skip_lines => /,/ }

CSV.open(output_file_path,‘w’,:force_quotes => true) do |out_row|
CSV.foreach(input_file_path, options) do |in_row|
row_to_add = in_row.compact
row_to_add.pop if row_to_add.last =~ /\A\sx\s\Z/i
out_row << row_to_add
end
end

The last column of the some lines will be yielded as nil, not as an
empty string “” as before.

[“EN”, “Class”, “Carpenter”, nil]

So you can simply call Array#compact! on it to have

[“EN”, “Class”, “Carpenter”]

And I improved the regexp and the logic to identify the “x” column a
little bit (to fit my taste).

Best regards,
Abinoam Jr.

Dear Arup,

Well… did you edited your original post at the forum after sending it?

The code at https://www.ruby-forum.com/topic/4423224 and the code at
my mail inbox are differents.

unless row_to_add.empty?
  row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0..-2]) :

out_row.puts(row_to_add)
end

versus

out_row.puts row_to_add unless row_to_add.empty? or

row_to_add.last[/^\s+x$/]

They do different things.

For this “second option” you could do

out_row << row_to_add unless row_to_add.last =~ /\A\s*x\s*\Z/i

instead of

row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
out_row << row_to_add

Keep Rubying,
Abinoam Jr.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs