# Parallel::ForkManager, Net::HTTP and catching Timeout::Error

**URL:** <https://www.ruby-forum.com/t/parallel-forkmanager-net-http-and-catching-timeout-error/180890>\
**Category:** Ruby\
**Created:** [December 31, 2009, 12:51am UTC](https://www.ruby-forum.com/t/parallel-forkmanager-net-http-and-catching-timeout-error/180890 "2009-12-31T00:51:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/f1d935/32.png) [@nvp](https://www.ruby-forum.com/u/nvp)\
**Post date:** [December 31, 2009, 12:51am UTC](https://www.ruby-forum.com/t/parallel-forkmanager-net-http-and-catching-timeout-error/180890/1 "2009-12-31T00:51:26Z")

</div>

Hello,

I’m about to release Parallel::ForkManager 1.1 and while generating  
new examples (that use Net::HTTP) for PFM 1.1 features, I catch the  
following error _sometimes_ (maybe 10-15% of the time) when I try to  
connect to a URL where the host part doesn’t exist or isn’t reachable:

/usr/lib/ruby/1.8/timeout.rb:60:in `rbuf_fill': execution expired (Timeout::Error) from /usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf\_fill’  
from /usr/lib/ruby/1.8/net/protocol.rb:86:in `read' from /usr/lib/ruby/1.8/net/http.rb:2212:in `read\_body\_0’  
from /usr/lib/ruby/1.8/net/http.rb:2173:in `read_body' from /usr/lib/ruby/1.8/net/http.rb:773:in `get’  
from /usr/lib/ruby/1.8/net/http.rb:1053:in `request' from /usr/lib/ruby/1.8/net/http.rb:2136:in `reading\_body’  
from /usr/lib/ruby/1.8/net/http.rb:1052:in `request' from /usr/lib/ruby/1.8/net/http.rb:1037:in `request’  
from /usr/lib/ruby/1.8/net/http.rb:543:in `start' from /usr/lib/ruby/1.8/net/http.rb:1035:in `request’  
from /usr/lib/ruby/1.8/net/http.rb:772:in `get' from ./parallel_http_get2.rb:36 from ./lib/parallel/forkmanager.rb:232:in `call’  
from ./lib/parallel/forkmanager.rb:232:in `start' from ./lib/parallel/forkmanager.rb:232:in `fork’  
from ./lib/parallel/forkmanager.rb:232:in `start' from ./parallel_http_get2.rb:30 from ./parallel_http_get2.rb:26:in `each’  
from ./parallel\_http\_get2.rb:26

Based on my review of the error message, it would seem to be a simple  
case that I should catch (rescue) ‘Exception’ because Timeout::Error  
is sending to stderr or such, but when I try to catch ‘Exception’ in  
my test program, nothing happens! I never catch the error!

Here’s my test code:

#!/usr/bin/env ruby

require ‘net/http’  
require ‘lib/parallel/forkmanager’

save\_dir = ‘/tmp’

my\_urls = [  
‘[http://www.cnn.com/index.html](http://www.cnn.com/index.html)’,  
‘[O'Reilly Media - Technology and Business Training](http://oreilly.com/index.html)’,  
‘[Cakewalk](http://www.cakewalk.com/index.html)’,  
‘[http://www.asdfsemicolonl.kj/index.htm](http://www.asdfsemicolonl.kj/index.htm)’  
]  
my\_timeout = 5 # seconds

max\_proc = 20  
pfm = Parallel::ForkManager.new(max\_proc)

pfm.run\_on\_finish(  
lambda {  
|pid,exit\_code,ident|  
print “\*\* PID (#{pid}) for #{ident} exited with code  
#{exit\_code}!\n”  
}  
)

my\_urls.each {  
|my\_url|

```
begin
    pfm.start(my_url) {
        url = URI.parse(my_url)
        out_file = save_dir + '/' + url.host + '.txt';

        http = Net::HTTP.new(url.host, url.port)
        http.open_timeout = http.read_timeout = my_timeout
        res = http.get(url.path)
        status = res.code

        if status.to_i == 200
            f = File.open(out_file, 'w')
            f.print res.body
            f.close()
            exit 0
        else
            exit 255
        end

        http = Net::HTTP.new(url.host, url.port)
        http.open_timeout = http.read_timeout = my_timeout
        res = http.get(url.path)
        status = res.code

        if status.to_i == 200
            f = File.open(out_file, 'w')
            f.print res.body
            f.close()
            exit 0 # start() with a block means that we exit with

```

status or else it’s 0 all the time.  
else  
exit 255  
end  
} # end pfm.start { … }  
rescue Exception =\> e  
print "Arggh, exception: ", e, “\n”  
exit 255  
end  
}

pfm.wait\_all\_children()

# end

Would somebody be able to shed some light on why I’m unable to handle  
the exceptions that Net::HTTP is throwing? Am I just trying to catch  
the wrong exception or is there something else? Is it because I’m  
trying to handle the exception in the child?

Note that this is: ruby 1.8.7 (2009-06-12 patchlevel 174) [i486-linux]  
under Ubuntu 9.10 (Karmic).

---

<div class="post-metadata">

**Author:** ![nvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/f1d935/32.png) [@nvp](https://www.ruby-forum.com/u/nvp)\
**Post date:** [December 31, 2009, 1:44am UTC](https://www.ruby-forum.com/t/parallel-forkmanager-net-http-and-catching-timeout-error/180890/2 "2009-12-31T01:44:43Z")

</div>

nvp [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Hello,
> 
> I’m about to release Parallel::ForkManager 1.1 and while generating  
> new examples (that use Net::HTTP) for PFM 1.1 features, I catch the  
> following error _sometimes_ (maybe 10-15% of the time) when I try to  
> connect to a URL where the host part doesn’t exist or isn’t reachable:
> 
> /usr/lib/ruby/1.8/timeout.rb:60:in `rbuf\_fill’: execution expired  
> (Timeout::Error)

\> Based on my review of the error message, it would seem to be a simple \> case that I should catch (rescue) 'Exception' because Timeout::Error \> is sending to stderr or such, but when I try to catch 'Exception' in \> my test program, nothing happens! I never catch the error! \> \> Here's my test code:

> ```
> begin
> pfm.start(my_url) {
> url = URI.parse(my_url)
> out_file = save_dir + '/' + url.host + '.txt';
> 
> http = Net::HTTP.new(url.host, url.port)
> http.open_timeout = http.read_timeout = my_timeout
> res = http.get(url.path)
> 
> ```

> ```
> } # end pfm.start { ... }
> rescue Exception => e
> print "Arggh, exception: ", e, "\n"
> exit 255
> end
> 
> ```
> 
> }

> Would somebody be able to shed some light on why I’m unable to handle  
> the exceptions that Net::HTTP is throwing? Am I just trying to catch  
> the wrong exception or is there something else? Is it because I’m  
> trying to handle the exception in the child?

It looks like the parent is trying to rescue exception in a child, which  
isn’t possible. Exceptions are Ruby-level objects that normally don’t  
get transferred/communicated between Unix processes. The parent can  
only normally get the exit status of a child process, otherwise it needs  
to resort to explicit IPC mechanisms (pipes/sockets/files/shared  
memory/…).

---

<div class="post-metadata">

**Author:** ![nvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/f1d935/32.png) [@nvp](https://www.ruby-forum.com/u/nvp)\
**Post date:** [December 31, 2009, 11:51pm UTC](https://www.ruby-forum.com/t/parallel-forkmanager-net-http-and-catching-timeout-error/180890/3 "2009-12-31T23:51:29Z")

</div>

On 31 déc. 2009, at 00:50, nvp wrote:

> /usr/lib/ruby/1.8/timeout.rb:60:in `rbuf\_fill’: execution expired  
> (Timeout::Error)

FWIW, I’m getting the exact same error - at roughly the same rate - in a  
regular mongrel-based application which talks to a separate HTTP backend  
and I would _love_ to get rid of it too 🙂