Forum: Ruby Couldn't understand how to use the option :skip_lines

249c7fd851c5c5ac5a1abdb756472ae1?d=identicon&s=25 Arup Rakshit (my-ruby)
on 2014-02-23 16:30
Here is a CSV data in one of my file has :

"DE","Klasse","Deutsch", "x"
"EN","Class","Carpenter",
"DE","Klasse","Mathe",
,,,

Code I wrote :

require 'csv'

input_file_path = File.expand_path('input.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

option = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => CSV::Converters[:remove_quotes] = lambda do
|field|
             field.to_s.tr('"','')
           end
         }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, option) do |in_row|
    row_to_add = in_row.reject(&:empty?)
    unless row_to_add.empty?
      row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0..-2]) :
out_row.puts(row_to_add)
    end
  end
end

**output** ( also expected )

"EN","Class","Carpenter"
"DE","Klasse","Mathe"

But my question is -

How can I skip lines like **,,,** using option **:skip_lines** (
http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc...)

Thanks
E088bb5c80fd3c4fd02c2020cdacbaf0?d=identicon&s=25 Jesús Gabriel y Galán (Guest)
on 2014-02-24 07:41
(Received via mailing list)
On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit <lists@ruby-forum.com>
wrote:
>
>
> "EN","Class","Carpenter"
> "DE","Klasse","Mathe"
>
> But my question is -
>
> How can I skip lines like **,,,** using option **:skip_lines** (
> http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc...)

The :skip_lines option receives an object that responds to match (for
example a Regex). The lines for which this object returns false are
skipped. For example:

:skip_lines => /,,,/

(untested)

Jesus.
09a32175057418748822c587ac08c429?d=identicon&s=25 Abinoam Jr. (abinoampraxedes_m)
on 2014-03-05 00:08
(Received via mailing list)
Hi Arup,

Jess has guided you to the right way! ;-)
(I've tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','')
}

options = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => :remove_quotes,
           :skip_lines => /,,,/
         }

Your approach only works because the assignment

CSV::Converters[:remove_quotes] = lambda do |field|
             field.to_s.tr('"','')
           end

returns the lambda itself.

And, as stated at the documentation,
"A single converter doesn't have to be in an Array."

So, the fragment ...
           :converters => CSV::Converters[:remove_quotes] = lambda do
|field|
             field.to_s.tr('"','')
           end


... acts exactly the same as
           :converters => lambda do |field|
             field.to_s.tr('"','')
           end

Abinoam Jr.

On Mon, Feb 24, 2014 at 3:40 AM, Jess Gabriel y Galn
249c7fd851c5c5ac5a1abdb756472ae1?d=identicon&s=25 Arup Rakshit (my-ruby)
on 2014-03-05 00:34
Abinoam Jr. wrote in post #1138813:
> Hi Arup,
>
> Jess has guided you to the right way! ;-)
> (I've tested and it worked).
>
> But I would give you some suggestions (not directly related to your
> question).
>
> I think the "right" way to deal with the converters is:
>
> CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','')
> }

Thanks,, I got your point. One doubt, Are you suggesting me to use
`{..}` instead of `do..end` ? I don't think so. But if yes, then why so
?
09a32175057418748822c587ac08c429?d=identicon&s=25 Abinoam Jr. (abinoampraxedes_m)
on 2014-03-05 00:37
(Received via mailing list)
... more

With :skip_line working you can now strip off the "unless
row_to_add.empty?" because all (/,,,/) rows will be skipped.

Abinoam Jr.
09a32175057418748822c587ac08c429?d=identicon&s=25 Abinoam Jr. (abinoampraxedes_m)
on 2014-03-05 00:40
(Received via mailing list)
Hi Arup,

Although there's some stylistic common pratice, you can use what you
prefer.

My suggestion is not about that.

You are setting the CSV::Converters[:remove_
quotes] __inside__ the options Hash.

I'm suggesting you to do the "right" way.
Set it __outside__ and reference it with its index (Symbol
:remove_quotes).

Best regards,
Abinoam Jr.
09a32175057418748822c587ac08c429?d=identicon&s=25 Abinoam Jr. (abinoampraxedes_m)
on 2014-03-05 00:57
(Received via mailing list)
... another thing...

It seems all the Converters harassment you are dealing with it's
because of a malformed CSV file.

You have a line with an extra space
"DE","Klasse","Deutsch", "x"  # But it should be
"DE","Klasse","Deutsch","x" # without the space before "x"

If you fix this before parsing as csv, the code get much simpler.

# I called the "fixed" version of the file "input_fixed.csv"

input_file_path =
File.expand_path('input_fixed.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

options = { :skip_blanks => true, :skip_lines => /,,,/  }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, options) do |in_row|
    row_to_add = in_row.compact
    row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
    out_row << row_to_add
  end
end

The last column of the some lines will be yielded as nil, not as an
empty string "" as before.

["EN", "Class", "Carpenter", nil]

So you can simply call Array#compact! on it to have

["EN", "Class", "Carpenter"]

And I improved the regexp and the logic to identify the "x" column a
little bit (to fit my taste).

Best regards,
Abinoam Jr.
09a32175057418748822c587ac08c429?d=identicon&s=25 Abinoam Jr. (abinoampraxedes_m)
on 2014-03-05 01:07
(Received via mailing list)
Dear Arup,

Well... did you edited your original post at the forum after sending it?

The code at https://www.ruby-forum.com/topic/4423224 and the code at
my mail inbox are differents.

    unless row_to_add.empty?
      row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0..-2]) :
out_row.puts(row_to_add)
    end

versus

    out_row.puts row_to_add unless row_to_add.empty? or
row_to_add.last[/^\s+x$/]

They do different things.

For this "second option" you could do

    out_row << row_to_add unless row_to_add.last =~ /\A\s*x\s*\Z/i

instead of

    row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
    out_row << row_to_add

Keep Rubying,
Abinoam Jr.
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.