Forum: Ruby open-uri 414 Request-URI Too Large

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Jay 9. (Guest)
on 2009-04-04 05:24
I am hitting a wall. This is my first attempt at ruby programming and I
am sure my code leaves a lot to be desired...be gentle.

I have a csv file I am loading into an array, it contains static pieces
of a url that i build other urls from.

It will loop through the first 10 in the array and always works. Number
11 always fails with:

C:/Ruby/lib/ruby/1.8/open-uri.rb:277:in `open_http': 414 Request-URI Too
Large (
OpenURI::HTTPError)
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:162:in `catch'
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:162:in `open_loop'
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:518:in `open'
        from C:/Ruby/lib/ruby/1.8/open-uri.rb:30:in `open'

So I shift my loop to start at 11 to the end and after the 10th time in
the loop it fails. Shift it again by 10, same thing. A total of 58 array
elements, csv file size is 44 KB. Rules out the data right?

Leads me to believe that I have a memory problem or maybe I need to
clear out the variable. The array data contains passwords and usernames
so I will sanitize them and upload if needed.

Here is the basic code.

require 'rubygems'
require 'hpricot'
require 'open-uri'
require 'csv'

fluffers = Array.new

urlA =
"http://mywebpage.com/fight.php?investor_id=6781410...
urlB = "http://mywebpage.com/bank.php?amount="
urlB2 = "&action=Deposit&do=Deposit"
urlH = "http://mywebpage.com/holy.php?cost=&action=Get%20p...
urlS = "http://mywebpage.com/index.php?"

# Populate the array with the static information
  CSV.open('z:/ruby/invest2.csv', 'r') do |row|
    fluffers.push row
  end

i = 0
until i == fluffers.length
  doc = Hpricot(open(urlStatus << fluffers[i].to_s))
  inMoney =
doc.search("//strong[@class='money']").inner_html.gsub(/\D/,'')
  inHoly = (doc/"#current_holy").inner_html
  inEnergy = (doc/"#current_energy").inner_html
  inStock = (doc/"#current_stock").inner_html

  if inMoney.to_i > 0
    open(urlB + inMoney + urlB2 + fluffers[i].to_s)
    print "deposit\n"
  end

  inStock.to_i.times do
    if inHoly.to_i < 20
      open(urlH + fluffers[i].to_s)
    end
    open(urlA + fluffers[i].to_s)
    print "a\n"
    doc = Hpricot(open(urlS << fluffers[i].to_s))
    inHoly = (doc/"#current_holy").inner_html
    print inHoly
  end
  i += 1
end

Thank You!
Ben L. (Guest)
on 2009-04-04 15:24
(Received via mailing list)
Look at these lines:

>
>  doc = Hpricot(open(urlStatus << fluffers[i].to_s))
>

And


>    doc = Hpricot(open(urlS << fluffers[i].to_s))
>


the urlStatus and urlS are getting larger each time this is called. You
want
to check which URI is being used as from the error message you get, it
is
clearly too large.

Ben
Jay 9. (Guest)
on 2009-04-04 16:11
Ben L. wrote:
> the urlStatus and urlS are getting larger each time this is called. You want
> to check which URI is being used as from the error message you get, it is
> clearly too large.
>
> Ben

Thanks!!! I changed it to a constant and it worked as needed.
This topic is locked and can not be replied to.