Forum: Ruby Mechanize problem "..absolute URL needed (not "").."

4e8d6556819733bd071089f37fb38f99?d=identicon&s=25 cristian cristian (currambero)
on 2014-02-26 16:30
Hello All!
This little (crappy :) ) code worked fine for a while and stopped
working. I hope somebody out there can give me a hint of what could be
wrong.

The Site table in the db has url in format "http://www.ruby-forum.com"
(without quotes). When I try to puts the url in irb I see the correct
url and the code still writes values to my txt file but doesnt insert
new values to the db.

The error that I get is this:
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:
607:in `resolve': absolute URL needed (not "") (ArgumentError)
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/h
ttp/agent.rb:223:in `fetch'
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize.r
b:440:in `get'
        from Stats_testDB2.rb:41:in `block (2 levels) in <main>'
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/colle
ction.rb:508:in `block in each'
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `block in each'
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `each'
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `each'
        from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/colle
ction.rb:505:in `each'
        from Stats_testDB2.rb:39:in `block in <main>'
        from Stats_testDB2.rb:38:in `open'
        from Stats_testDB2.rb:38:in `<main>'
---


This is my code:


require 'rubygems'
require 'mechanize'
require 'data_mapper'

DataMapper.setup(:default,'sqlite3:\Dropbox\Ruby\Stats\StatsDB.sqlite')

class Site
  include DataMapper::Resource
  property  :id,      Serial
  property  :description,  String
  property  :url,      String

  #has n,    :Statistics
end


class Statistic
  include DataMapper::Resource
  property  :id,      Serial
  property  :date,      DateTime
  property  :number,    Integer
  property  :site_id,    Integer

  #belongs_to  :Site
end
DataMapper.finalize

time=Time.now.strftime("%Y-%m-%d %H:%M")

File.open("Stats.txt","a+") do |vStats|
  Site.all.each do |site|
    agent = Mechanize.new
    page = agent.get(site.url)
    hits = agent.page.search("span.num_hits").map(&:text).map(&:strip)
    int=hits[0].to_i
    vStats.puts "#{time}  #{site.id}  #{int}\n"

    Statistic.create(
    :date =>  time,
    :number =>  int,
    :site_id =>  site.id
    )
  end
end
E088bb5c80fd3c4fd02c2020cdacbaf0?d=identicon&s=25 Jesús Gabriel y Galán (Guest)
on 2014-02-26 16:52
(Received via mailing list)
On Wed, Feb 26, 2014 at 4:30 PM, cristian cristian
<lists@ruby-forum.com> wrote:
> The error that I get is this:
> C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:
> 607:in `resolve': absolute URL needed (not "") (ArgumentError)

This line means that the uri you have passed to "get" is the empty
string.
You can test this by printing the Site object before calling the url:

>   property  :url,      String
>   property  :site_id,    Integer
>
>   #belongs_to  :Site
> end
> DataMapper.finalize
>
> time=Time.now.strftime("%Y-%m-%d %H:%M")
>
> File.open("Stats.txt","a+") do |vStats|
>   Site.all.each do |site|
       p site # <-------------------------------------------------- add
this
>     )
>   end
> end


Jesus.
4e8d6556819733bd071089f37fb38f99?d=identicon&s=25 cristian cristian (currambero)
on 2014-02-27 09:29
Thank you Jesus, but thats strange because I use the same value from the
web site and write it to the text file and this works fine. Further down
in the code I do this:

vStats.puts "#{time}  #{site.id}  #{int}\n"


Where I use the variable "int". Also, when I test this in irb I get
this:

irb(main):025:0> Site.all.each do |site|
irb(main):026:1* agent = Mechanize.new
irb(main):027:1> puts site.url
irb(main):028:1> end
http://www.pintamono.com
http://www.pintamono.com
=> [#<Site @id=1 @description="My web" @url="http://www.pintamono.com">,
#<Site @id=2 @description="My web2" @url="http://www.pintamono.com">]
E088bb5c80fd3c4fd02c2020cdacbaf0?d=identicon&s=25 Jesús Gabriel y Galán (Guest)
on 2014-02-27 10:56
(Received via mailing list)
On Thu, Feb 27, 2014 at 9:29 AM, cristian cristian
<lists@ruby-forum.com> wrote:
> Thank you Jesus, but thats strange because I use the same value from the
> web site and write it to the text file and this works fine. Further down
> in the code I do this:
>
> vStats.puts "#{time}  #{site.id}  #{int}\n"
>
>
> Where I use the variable "int".

Yes, but int is not the URL, and also you are not reaching that line,
because you get an exception first.
Can you try running with the statement I added in the real environment
and check what you get?


> #<Site @id=2 @description="My web2" @url="http://www.pintamono.com">]
Yes, looks good, it's strange that it works in irb and not in the
script, but as a sanity check, try running printing the site and the
url before calling agent.get to be sure about what's going on.

Jesus.
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.