Hi, I am working on a crawler, and I am encountering a large number of getaddrinfo: Temporary failure in name resolution exceptions when I am using open-uri. I am using a hybrid approach with a lot of forks and lot of ruby threads running inside each process. Each of these is making a open-uri calls. Is there something I should be doing to ensure the proper DNS resolution is occuring. When I dig the sites manually, all of them return a proper DNS entry. Secondly, I have tried the ruby resolver, and that gave poor resolution as well. I have tried this on both my linux machine and OS X, both running ruby 1.8.4. Any ideas?
on 2006-06-09 23:29