trying to take scraped info business name address and phone# i can get
the values into a variable but it writes to 1 line in csv i,m not
concerned with headers but do want to write columns corectly. I’m new
to butchering ruby but really like all the functions I’ve found/used so
far.
here is the code i’m working with
require ‘rubygems’
require ‘fastercsv’
require ‘nokogiri’
require ‘open-uri’
page =
Nokogiri::HTML(open(“Best 30 Auto Repair in FL with Reviews”))
bname = page.css(“h3.business-name.fn.has-menu.org”).text
addy = page.css(“span.street-address”).text
csz = page.css(“span.city-state”).text
count = page.css(“p.result-count”).text
count = count.gsub(/Results 1-30 of /,‘’)
phone = Array.new
phone = page.css(“span.business-phone.phone”).text
phone= phone.gsub(/\n/,‘’)
phone= phone.gsub(/ /,‘’)
phone= phone.gsub(/(/,’ ‘)
phone= phone.gsub(/)/,’-')
puts phone
Writing data into a csv file
FasterCSV.open(“test.csv”, “w”) do |csv|
csv << [bname,phone]
end
On Mon, May 28, 2012 at 9:21 PM, steven gre [email protected]
wrote:
require ‘nokogiri’
phone= phone.gsub(/\n/,‘’)
phone= phone.gsub(/ /,‘’)
phone= phone.gsub(/(/,’ ‘)
phone= phone.gsub(/)/,’-')
You could make you life easier by
- using String#gsub!
- combining regexps which have the same replacement
puts phone
Writing data into a csv file
FasterCSV.open(“test.csv”, “w”) do |csv|
csv << [bname,phone]
end
And what exactly was your question?
Regards
robert
Robert K. wrote in post #1062423:
On Mon, May 28, 2012 at 9:21 PM, steven gre [email protected]
wrote:
require ‘nokogiri’
phone= phone.gsub(/\n/,‘’)
phone= phone.gsub(/ /,‘’)
phone= phone.gsub(/(/,’ ‘)
phone= phone.gsub(/)/,’-')
You could make you life easier by
- using String#gsub!
- combining regexps which have the same replacement
puts phone
Writing data into a csv file
FasterCSV.open(“test.csv”, “w”) do |csv|
csv << [bname,phone]
end
And what exactly was your question?
Regards
robert
trying to send all the data scraped to csv file
business name, address, phone
i either get all 1 record 1 row or just phone #'s and sometimes nothing
been messing with this for some time, i’m a neub so this is probably
most of the problem
trying to make it each record row bname,addy,phone
steven gre wrote in post #1062429:
trying to send all the data scraped to csv file
business name, address, phone
i either get all 1 record 1 row or just phone #'s and sometimes nothing
been messing with this for some time, i’m a neub so this is probably
most of the problem
trying to make it each record row bname,addy,phone
This code:
FasterCSV.open(“test.csv”, “w”) do |csv|
csv << [bname,phone]
end
opens the file for write, truncates it to zero size if it already
exists, and then writes one row consisting of two fields, bname and
phone.
If you want to write three fields, hopefully the solution is clear.
If you want to write multiple rows, then you either open the file for
append (“a”) instead of write (“w”), or you open the csv file outside of
a loop which iterates over the data you want to generate.
If you are getting blank fields, then check the source data:
puts "bname=#{bname.inspect}"
puts "phone=#{phone.inspect}"
I suspect you will find that one of these is nil or empty string, in
which case the problem is in parsing the data, not in writing it out to
the CSV. (Sorry I’m not a Nokogiri user so I can’t give you any hints on
that part)
Thanks for the response im getting data with puts ill dig deeper today.
It seems as if i need to loop thru. In php its call while so i need to
figure out a do when writing to csv. Ill post wen im back at my desk.
What exactly do you see from these two lines? (Copy-paste it please.
You can XXX out the names and phone numbers but please don’t change
anything else)
puts "bname=#{bname.inspect}"
puts "phone=#{phone.inspect}"
The reason: I’m guessing that bname and phone may be arrays, not single
strings. Is that the case? If so you can iterate over the arrays when
writing the CSV.
steven@steven:~/Desktop$ ruby test.rb
bname=[“Firestone Complete Auto CareQuality Imports IncSunshine Auto
Sales And ServiceA & I Auto Complex IncWray’s Auto Service IncForeign
Car Clinic & Parts IncFrank’s Guaranteed A/C & Auto RepairJJ’s Custom
Import Repair IncBill’s TransmissionGordon’s AutomotiveSpeedy Muffler &
Auto RepairBeach Auto CareAmerican AutoMr TransmissionE & M Motors
IncLeasure AutomotiveAll Pro AutomotiveB & S Auto SalvageGoodyear
Certified Auto & Tire CentersAction Transmission ServiceAAA
TransmissionAa AutomotiveMike Ferran IncParadise Transmission
ServiceApex TexacoGoodyear Certified Auto & Tire CenterJerry Bush Auto
RepairReilly Auto & Electric RepairAamco TransmissionsSal’s TowingA & A
Auto & Truck CenterMeineke Car Care CentersACDelcoMeineke Car Care
CentersAsk an Auto Mechanic Online - Just AnswerSafelite Auto GlassTire
KingdomGoodyear Auto Service CenterCar-X Auto ServiceCottman
Transmission and Total Auto CareSears Auto CenterAAMCO Transmissions &
Auto RepairMaaco Collision Repair & Auto PaintingDaytona Converter”, "
“]
phone=” 850-863-2161 904-548-0331 850-785-7229 727-937-2902 321-259-3661
561-684-6920 904-721-6646 850-763-1977 904-644-0446 321-267-5000
850-249-8416 386-428-1941 850-478-4050 772-283-4610 813-684-1553
904-241-4800 877-536-6254 954-755-5584 386-668-8744 305-944-0509
904-739-9977 904-390-0993 305-293-0923 561-746-8069 561-488-1822
352-378-4525 386-254-8660 321-268-2626 954-566-5155 407-982-3002"
there should be 30 business names and 30 phone numbers.and i can’t seem
to get a space at the end of each business but thats whole other thing
Brian C. wrote in post #1062462:
steven gre wrote in post #1062429:
trying to send all the data scraped to csv file
business name, address, phone
i either get all 1 record 1 row or just phone #'s and sometimes nothing
been messing with this for some time, i’m a neub so this is probably
most of the problem
trying to make it each record row bname,addy,phone
This code:
FasterCSV.open(“test.csv”, “w”) do |csv|
csv << [bname,phone]
end
opens the file for write, truncates it to zero size if it already
exists, and then writes one row consisting of two fields, bname and
phone.
If you want to write three fields, hopefully the solution is clear.
If you want to write multiple rows, then you either open the file for
append (“a”) instead of write (“w”), or you open the csv file outside of
a loop which iterates over the data you want to generate.
If you are getting blank fields, then check the source data:
puts "bname=#{bname.inspect}"
puts "phone=#{phone.inspect}"
I suspect you will find that one of these is nil or empty string, in
which case the problem is in parsing the data, not in writing it out to
the CSV. (Sorry I’m not a Nokogiri user so I can’t give you any hints on
that part)
Brian i checked and i’m pulling data and i gave some though about you
said opening the csv outside the loop… i’m a little lost as to how to
do that i’m playing with mostly sample code and changing as needed i
tried with an “a” but still getting the same results prints all the
business names and then on next line all phone #'s on the next tried
bname,phone and it’s all on 1 row business names and then phones "bname
bname bname phone phone phone…
steven gre wrote in post #1062601:
steven@steven:~/Desktop$ ruby test.rb
bname=[“Firestone Complete Auto CareQuality Imports IncSunshine Auto
Sales And ServiceA & I Auto Complex IncWray’s Auto Service IncForeign
Car Clinic & Parts IncFrank’s Guaranteed A/C & Auto RepairJJ’s Custom
Import Repair IncBill’s TransmissionGordon’s AutomotiveSpeedy Muffler &
Auto RepairBeach Auto CareAmerican AutoMr TransmissionE & M Motors
IncLeasure AutomotiveAll Pro AutomotiveB & S Auto SalvageGoodyear
Certified Auto & Tire CentersAction Transmission ServiceAAA
TransmissionAa AutomotiveMike Ferran IncParadise Transmission
ServiceApex TexacoGoodyear Certified Auto & Tire CenterJerry Bush Auto
RepairReilly Auto & Electric RepairAamco TransmissionsSal’s TowingA & A
Auto & Truck CenterMeineke Car Care CentersACDelcoMeineke Car Care
CentersAsk an Auto Mechanic Online - Just AnswerSafelite Auto GlassTire
KingdomGoodyear Auto Service CenterCar-X Auto ServiceCottman
Transmission and Total Auto CareSears Auto CenterAAMCO Transmissions &
Auto RepairMaaco Collision Repair & Auto PaintingDaytona Converter”, "
“]
phone=” 850-863-2161 904-548-0331 850-785-7229 727-937-2902 321-259-3661
561-684-6920 904-721-6646 850-763-1977 904-644-0446 321-267-5000
850-249-8416 386-428-1941 850-478-4050 772-283-4610 813-684-1553
904-241-4800 877-536-6254 954-755-5584 386-668-8744 305-944-0509
904-739-9977 904-390-0993 305-293-0923 561-746-8069 561-488-1822
352-378-4525 386-254-8660 321-268-2626 954-566-5155 407-982-3002"
there should be 30 business names and 30 phone numbers.and i can’t seem
to get a space at the end of each business but thats whole other thing
still havn’t got any help with this and tried several things anyone,
here is what is on my script now
#!/usr/bin/env ruby
require ‘rubygems’
require ‘sendfile’
require ‘fastercsv’
require ‘nokogiri’
require ‘open-uri’
page =
Nokogiri::HTML(open(“Best 30 Auto Repair in FL with Reviews”))
#Best 30 Auto Repair in FL with Reviews
#addy = page.css(“span.street-address”).text
#csz = page.css(“span.city-state”).text
#count = page.css(“p.result-count”).text
#count = count.gsub(/Results 1-30 of /,‘’)
pho=[]
page.css(“span.business-phone.phone”).each do |pho|
phone=[]
phone = pho.text
bname = []
page.css(“a.no-tracks.url”).each do |biz|
bname = biz.text
FasterCSV.open(“test.csv”, “a”) do |csv|
csv << [phone]
end
end
end