Beginner question - CSV error (undefined local variable or method `csv' for main:Object )

Ruby
1

Still learning Ruby and trying to get a web scraper that is writing to a
csv file. I am able to get everything but the writing of the array to
the csv file. Getting the following error message when it runs.

search.rb:108:in block in find_att_value': undefined local variable or method csv’ for main:Object (NameError)
from search.rb:71:in each' from search.rb:71:in find_att_value’
from search.rb:120:in block in <main>' from /home/bob/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/csv.rb:1273:in open’
from search.rb:17:in `’

Below is my code. How can I fix this?
Thanks in advance.
Bob

require ‘rubygems’
require ‘nokogiri’
require ‘open-uri’
require ‘mechanize’
require ‘csv’

BASEURL = “http://www.pbm-erson.com”
BASEURL1 =
“http://www.pbm-erson.com/Catalog/PBM/Camshafts_PBM/Performance_CPBM?&sortby=2&pageSize=5&page=”
HSH = Hash.new
RESULTS = []
BRY = Array.new #attribute name array from products
LAST_PAGE_NUMBER = 2 #last page to stop on
ARR1 = []
$bsv

#go through and get links for each product page
CSV.open(“results-test.csv”,“w”) do |csv|
#$bsv = csv
#def csvwrite

csv << $bsv

#end

def find_link
puts " Finding Links"
item_info_css = “.item-info”
product_item_row = “.item-row”
product_num = “span.item-num”

for current_page_number in 1..LAST_PAGE_NUMBER
  url = "#{BASEURL1}#{current_page_number}"
  puts url
  doc = Nokogiri::HTML(open(url))
  products = doc.css(item_info_css)
  products.each do |product|
    product_nodes = product.css(product_num)
    links = product.css('a')
    linkref = links[0]["href"]
    raw_results = BASEURL + linkref
    RESULTS << raw_results
    sleep 1
  end

end
puts RESULTS#finding links

end

def build_att_name_array
BRY.push(“ProductID:”)
BRY.push(“Name:”)
BRY.push(“Short Description:”)
BRY.push(“HTML Content:”)#building attribute name list
puts “Building Table Headers”

RESULTS.each do |link|
  url = link
  doc = Nokogiri::HTML(open(url))
  doc.css(".attribute-list-item").each do |attrib|
    att = attrib.at_css("span.attrib-name").text
    BRY << att
  end
  sleep 1
end

end
#need to verify the lining up of the code from here down
def find_att_value
puts “Matching attributes to attribute name”
hry = BRY.uniq

RESULTS.each do |link|
  url = link
  doc = Nokogiri::HTML(open(url))
  erp_num = doc.at_css("span.erp-num").text.strip #our part number

from Sxe
short_desc = doc.at_css(“.pd-header”).text.strip # short
description
long_desc = doc.at_css(“.product-cm”).text.strip #html content
id = doc.at_css(“input#CurrentProductId”)[‘value’]

  hry.each do |ab|
    if ab == "ProductID:"
      HSH[ab]= id
      elsif ab == "Name:"
        HSH[ab]= erp_num
      elsif ab == "Short Description:"
        HSH[ab]= short_desc
      elsif ab == "HTML Content:"
        HSH[ab]= long_desc
      else
        begin
          els = doc.search"[text()*='#{ab}']"
          el = els.first
          eln = el.next_element.text.strip
          HSH[ab]= eln
        rescue
          HSH[ab]= ""
        end
    end
  end
  puts "-----------------------"
  puts HSH

  HSH.each_value do |val|
    ARR1 << val
  end

#$bsv << ARR1
#csvwrite
csv << ARR1 # <— HERE IS WHERE THE ERROR IS
ARR1.clear
HSH.clear
puts “------------------------”
end

end

find_link
build_att_name_array
bry = BRY.uniq
csv << bry
find_att_value
end

2

The ‘csv’ variable is defined at the top-level context and you want to
use it inside a method (defined at the top-level as well). However in
Ruby, variables with outer scope are not available to the inner scope.

What programming language did you use anyway? This style reminds me
Pascal :slight_smile:
You define globals everywhere, which makes your code very hard to debug.
I don’t want to steer you in the direction of functional programming I
just advise that you should avoid global variables, write methods and
pass everything to the methods they need as arguments and use these
methods’ return values to call other methods or save the output to
disk).

This way you will have a bunch of methods that work using ONLY
arguments, which means you can easily debug a single method by providing
mock data (even in REPL).

3

Földes László wrote in post #1178178:

The ‘csv’ variable is defined at the top-level context and you want to
use it inside a method (defined at the top-level as well). However in
Ruby, variables with outer scope are not available to the inner scope.

What programming language did you use anyway? This style reminds me
Pascal :slight_smile:
You define globals everywhere, which makes your code very hard to debug.
I don’t want to steer you in the direction of functional programming I
just advise that you should avoid global variables, write methods and
pass everything to the methods they need as arguments and use these
methods’ return values to call other methods or save the output to
disk).

This way you will have a bunch of methods that work using ONLY
arguments, which means you can easily debug a single method by providing
mock data (even in REPL).

New to Ruby. But currently at work I use Progress DB language to fix
issues and it is more functional.Funny thing is that this was my 4th or
5th revision after researching using Ruby for web scraping.

Yeah others in the Ruby IRC said I was abusing the constants and I
agree. Just trying to get something working and I am working on cleaning
up my bad coding, while learning Ruby.

How can I get the hash written into the csv file without an error?

Thank you for your time looking at my mess and responding.

4

Bob Biehl wrote in post #1178196:

How can I get the hash written into the csv file without an error?

Thank you for your time looking at my mess and responding.

That ‘csv’ exist outside of the method, so just to make it work, you
have two options:
a.) return the value of ARR at the end of the method (instead of ‘csv <<
ARR’), and where you call the method ‘find_att_value’, use that return
value to put it into ‘csv’ → like: ‘csv << find_att_value’.

b.) The ‘find_att_value’ method is not really a method, as

  1. it has no arguments and no return value,
  2. it is called once
    So it is just a bunch of code in a separate scope, and this ‘separate’
    scope is what hurts you. Simply replace the ‘find_att_value’ method call
    with the code inside the ‘find_att_value’ method (and throw away the
    ‘def find_att_value’ and ‘end’ at the end. This way you remove the scope
    problem and csv will be available.

A few Ruby tips:

  1. Don’t create method inside a loop. Every time the loop runs, it will
    try to create the method, over and over again. Ruby is smart and will
    figure out that the method already exists, but avoid this. And as a side
    note:
  2. Ruby has no nested methods (Jörg Mittag tells this a few times:
    ruby - Is it possible to have Methods inside Methods? - Stack Overflow)

def hoge
crapcrap
def fuge
a = 10
end
end
puts “dobedobedo”

This is not a nested method. Ruby is interpreted, but when it
‘interprets’ it does not goes into methods. When the interpreter hits
the “dobedobedo” line, the #hoge method would exist (it is callable) but
its internals are still unknown, so #fuge method wouldn’t exist up until
you call #hoge, which would then stop with an error, as ‘crapcrap’ does
not exist. But if you don’t call #hoge Ruby won’t throw an error as it
knows nothing about the method’s code.

  1. This is a ‘mechanical’ approach to fill an array:
    HSH.each_value do |val|
    ARR1 << val
    end

If you want to know Ruby better, knowing the Enumerables module is a
must. This module is mixed in everywhere, and provides tons of useful
methods. Like iterating over a Hash or an Array doing some magic with
each element and creating an Array from the result (with the same size).
For that, I would use the #map method. However, there is a specific
method for this task (turning a Hash into an Array of its values). And
there are tons of other ‘specific’ methods as well, so you don’t have to
reinvent them :slight_smile:

The above is the same as:
ARR1 = HSH.values

When using Ruby, always suspect that what you do, can be done at least 3
different ways. There are the ones that produce smaller code and look
functional (as in functional languages), the ones that run faster but
are totally unreadable or look like C, and there is always a one-liner
:slight_smile:

5

Földes,
Thanks for the response. I will now use that information to improve my
coding skills, especially since they need much improving.

Thanks again.