Handle http request timed out

Ruby
1

Hi
i have written a script which connects to a website and read from their
but some time that website is soo slow that it gets time out .
so i have done this

http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 500
request = Net::HTTP::Get.new(uri.request_uri)

begin
response = http.request(request)
code=response.code
log_info(“Response Code - #{code}”)

end until response.code == “200”

But it gets time out … now i can increase the http time out but thats
not a solution

is not there any way that i can do …

As long as response.code == 200, it will keep try to get request without
producing time out or any error…
i thought the code i wrote it will do that job , but it does not …
where i am doing wrong ??
or how can i get what i am looking for ??

Thanks

2

On Mon, Apr 15, 2013 at 3:56 PM, Fosiul A. [email protected]
wrote:

begin
is not there any way that i can do …

As long as response.code == 200, it will keep try to get request without
producing time out or any error…
i thought the code i wrote it will do that job , but it does not …
where i am doing wrong ??
or how can i get what i am looking for ??

With this loop, you are making a new request each time, which starts
the whole retrieval process over again. AFAIK, the only way to make
this work is by setting the timeout long enough to retrieve all the
data. What you can do is set the get request inside a begin-rescue-end
block and use a progressive timeout backoff and a retry. Limit the
number of retries so you don’t end up with an infinite loop. Robert
Klemme showed me how to do this with another sort of request in an
earlier message by passing a block to a method. I implemented it
something like this:

def retry_http_call(http,limit=3,backoff=500,&block)

tries = 0

begin
yield http
rescue Exception => e
tries += 1
http.read_timeout = backoff * tries
retry if tries < limit
end

end

Then you can use it like so:

Net::HTTP.start(uri.host, uri.port) do |http|

request = Net::HTTP::Get.new(uri.request_uri)
response = retry_http_call(http, 5) { |http| http.request(request) }

end

You may want to fiddle with the number of retries and the progressive
back off increment to see what is optimal for your implementation.

(Robert, if you would kindly check me on this?)