Threaded Socket call blocks main thread

Hi all. A blocking IO issue is puzzling me, maybe someone can offer
any insights.

I have written a plugin for Rails that does dns requests (to real-
time blackhole lists). It works quite well. I am doing the dns
requests (using Socket.gethostbyname) in separate threads.
Unfortunately, when the call to Socket blocks, the main thread is
blocking as well. And even though I have specified a parameter to
thread.join, the script won’t kill the blocked thread.

This is only a problem when no dns servers can be found or when
they’re terribly slow, but I must harden the plugin for these cases.

I have read posts on this list and it seems that any blocked IO such
as what I am doing will block the main thread in Ruby. Is there a
different way for me to kill the blocked thread and just move on? As
this plugin is meant for web applications, a timely response is quite
important. One or two seconds delay is fine, but no more.

Any help greatly appreciated.

require ‘socket’

module DNSBL_Check
$dnsbl_passed ||= []
DNSBLS = %w{list.dsbl.org bl.spamcop.net sbl-xbl.spamhaus.org}

private

Filter to check if the client is listed. This will be run before

all requests.
def dnsbl_check
return true if $dnsbl_passed.include? request.remote_addr

 passed = true
 threads = []
 request.remote_addr =~ /(\d+).(\d+).(\d+).(\d+)/

 # Check the remote address against each dnsbl in a separate thread
 DNSBLS.each do |dnsbl|
   threads << Thread.new("#$4.#$3.#$2.#$1.#{dnsbl}") do |host|
     logger.warn("Checking DNSBL #{host}")
     if Socket.gethostbyname("#{host}")[3][0,2]=="\177\000"
       logger.info("#{request.remote_addr} found using DNSBL #

{host}")
passed = false
end
end
end
threads.each {|thread| thread.join(2)} # join threads, but
use timeout to kill blocked ones

 # Add client ip to global passed cache if no dnsbls objected.

else deny service.
if passed
$dnsbl_passed = $dnsbl_passed[0,49].unshift request.remote_addr
logger.warn(“#{request.remote_addr} added to DNSBL passed cache”)
else
render :text => ‘Access denied’, :status => 403
return false
end
end
end

joost baaij wrote:

Hi all. A blocking IO issue is puzzling me, maybe someone can offer
any insights.

I have written a plugin for Rails that does dns requests (to real-
time blackhole lists). It works quite well. I am doing the dns
requests (using Socket.gethostbyname) in separate threads.
Unfortunately, when the call to Socket blocks, the main thread is
blocking as well. And even though I have specified a parameter to
thread.join, the script won’t kill the blocked thread.

Obviously when you join the thread, it can block the main thread. But in
my
experience with Ruby threads, if you don’t join the thread, it dies.
Which
leads a reasonable person to ask “what’s the point of the threads?”

Op 21-okt-2006, om 8:10 heeft Paul L. het volgende geschreven:

Obviously when you join the thread, it can block the main thread.
But in my
experience with Ruby threads, if you don’t join the thread, it
dies. Which
leads a reasonable person to ask “what’s the point of the threads?”

It seems very limited then. Just an easy way to parallelize (sp?)
some tasks.

Foregoing threads, is there a way to continue a script that’s waiting
for a blocked Socket.gethostbyname call? Afaik only actual files can
be opened non-blocking, right?

Thanks.

http://shnoo.gr/articles/2005/12/13/resolv-replace

Truly a hidden ruby gem. I would have never found it on my own. Many
thanks!!

Op 21-okt-2006, om 9:30 heeft snacktime het volgende geschreven: