Mechanize problem "..absolute URL needed (not "").."

Detlef_R · February 26, 2014, 4:30pm

Hello All!
This little (crappy ) code worked fine for a while and stopped
working. I hope somebody out there can give me a hint of what could be
wrong.

The Site table in the db has url in format “http://www.ruby-forum.com”
(without quotes). When I try to puts the url in irb I see the correct
url and the code still writes values to my txt file but doesnt insert
new values to the db.

The error that I get is this:
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:
607:in `resolve': absolute URL needed (not "") (ArgumentError) from C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/h ttp/agent.rb:223:in`fetch’
from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize.r
b:440:in `get' from Stats_testDB2.rb:41:in`block (2 levels) in ’
from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/colle
ction.rb:508:in `block in each' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo rt/lazy_array.rb:411:in`block in each’
from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `each' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo rt/lazy_array.rb:411:in`each’
from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/colle
ction.rb:505:in `each' from Stats_testDB2.rb:39:in`block in ’
from Stats_testDB2.rb:38:in `open' from Stats_testDB2.rb:38:in`’

This is my code:

require ‘rubygems’
require ‘mechanize’
require ‘data_mapper’

DataMapper.setup(:default,‘sqlite3:\Dropbox\Ruby\Stats\StatsDB.sqlite’)

class Site
include DataMapper::Resource
property :id, Serial
property :description, String
property :url, String

#has n, :Statistics
end

class Statistic
include DataMapper::Resource
property :id, Serial
property :date, DateTime
property :number, Integer
property :site_id, Integer

#belongs_to :Site
end
DataMapper.finalize

time=Time.now.strftime("%Y-%m-%d %H:%M")

File.open(“Stats.txt”,“a+”) do |vStats|
Site.all.each do |site|
agent = Mechanize.new
page = agent.get(site.url)
hits = agent.page.search(“span.num_hits”).map(&:text).map(&:strip)
int=hits[0].to_i
vStats.puts “#{time} #{site.id} #{int}\n”

Statistic.create(
:date =>  time,
:number =>  int,
:site_id =>  site.id
)

end
end

currambero · February 26, 2014, 4:52pm

On Wed, Feb 26, 2014 at 4:30 PM, cristian cristian
[email protected] wrote:

The error that I get is this:
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:
607:in `resolve’: absolute URL needed (not “”) (ArgumentError)

This line means that the uri you have passed to “get” is the empty
string.
You can test this by printing the Site object before calling the url:

property :url, String
property :site_id, Integer

#belongs_to :Site
end
DataMapper.finalize

time=Time.now.strftime("%Y-%m-%d %H:%M")

File.open(“Stats.txt”,“a+”) do |vStats|
Site.all.each do |site|
p site # <-------------------------------------------------- add
this
)
end
end

Jesus.

currambero · February 27, 2014, 9:29am

Thank you Jesus, but thats strange because I use the same value from the
web site and write it to the text file and this works fine. Further down
in the code I do this:

vStats.puts “#{time} #{site.id} #{int}\n”

Where I use the variable “int”. Also, when I test this in irb I get
this:

irb(main):025:0> Site.all.each do |site|
irb(main):026:1* agent = Mechanize.new
irb(main):027:1> puts site.url
irb(main):028:1> end
http://www.pintamono.com
http://www.pintamono.com
=> [#<Site @id=1 @description=“My web” @url=“http://www.pintamono.com”>,
#<Site @id=2 @description=“My web2” @url=“http://www.pintamono.com”>]

currambero · February 27, 2014, 10:56am

On Thu, Feb 27, 2014 at 9:29 AM, cristian cristian
[email protected] wrote:

Thank you Jesus, but thats strange because I use the same value from the
web site and write it to the text file and this works fine. Further down
in the code I do this:

vStats.puts “#{time} #{site.id} #{int}\n”

Where I use the variable “int”.

Yes, but int is not the URL, and also you are not reaching that line,
because you get an exception first.
Can you try running with the statement I added in the real environment
and check what you get?

#<Site @id=2 @description=“My web2” @url=“http://www.pintamono.com”>]
Yes, looks good, it’s strange that it works in irb and not in the
script, but as a sanity check, try running printing the site and the
url before calling agent.get to be sure about what’s going on.

Jesus.