Forum: Ruby handle http request timed out

7bb2a94f867841b58214be09992831d3?d=identicon&s=25 Fosiul Alam (fosiul)
on 2013-04-15 22:56
Hi
i have written a script which connects to a website and read from their
but some time that website is soo slow that it gets time out .
so i have done this


http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 500
request = Net::HTTP::Get.new(uri.request_uri)

begin
response = http.request(request)
code=response.code
log_info("Response Code - #{code}")

end until response.code == "200"


But it gets time out .. now i can increase the http time out but thats
not a solution

is not there any way that i can do ..

As long as response.code == 200, it will keep try to get request without
producing time out or any error...
i thought the code i wrote it will do that job , but it does not ...
where i am doing wrong ??
or how can i get what i am looking for ??

Thanks
Aa082c8b00a50928e5860dcd70bf2368?d=identicon&s=25 tamouse mailing lists (Guest)
on 2013-04-16 04:55
(Received via mailing list)
On Mon, Apr 15, 2013 at 3:56 PM, Fosiul Alam <lists@ruby-forum.com>
wrote:
> begin
> is not there any way that i can do ..
>
> As long as response.code == 200, it will keep try to get request without
> producing time out or any error...
> i thought the code i wrote it will do that job , but it does not ...
> where i am doing wrong ??
> or how can i get what i am looking for ??

With this loop, you are making a new request each time, which starts
the whole retrieval process over again. AFAIK, the only way to make
this work is by setting the timeout long enough to retrieve all the
data. What you can do is set the get request inside a begin-rescue-end
block and use a progressive timeout backoff and a retry. Limit the
number of retries so you don't end up with an infinite loop. Robert
Klemme showed me how to do this with another sort of request in an
earlier message by passing a block to a method. I implemented it
something like this:


def retry_http_call(http,limit=3,backoff=500,&block)

  tries = 0

  begin
    yield http
  rescue Exception => e
    tries += 1
    http.read_timeout = backoff * tries
    retry if tries < limit
  end

end

Then you can use it like so:

Net::HTTP.start(uri.host, uri.port) do |http|

  request = Net::HTTP::Get.new(uri.request_uri)
  response = retry_http_call(http, 5) { |http| http.request(request) }

end

You may want to fiddle with the number of retries and the progressive
back off increment to see what is optimal for your implementation.


(Robert, if you would kindly check me on this?)
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.