Forum: Ruby Ruby from source on Leopard excruciatingly slow internet talk

Ce2682c1ab17339a7ddd50a5e97c9dc2?d=identicon&s=25 Chris Shea (Guest)
on 2008-01-13 01:39
(Received via mailing list)
Just a couple of days ago I "upgraded" from Tiger to Leopard, and the
most horrible thing about it is that anything Ruby does over the
internet is so very very slow now. `which ruby` tells me I'm still
using my from-source install in /usr/local/bin, as does `which gem`.

I have a script that scrapes some stuff from craigslist that now takes
way too long (I might as well look myself). And my heart sinks when I
see "Updating metadata for 176 gems from http://gems.rubyforge.org".
That seriously takes forever (a little hyperbole). Nothing else is
suffering huge network slowness. Even a simple "require 'open-uri';
open('http://www.google.com')" takes 15 seconds (but half a second
tops in a browser). This all happened at a reasonable pace just a few
days ago. Other ruby installations on other machines (with other
operating systems) talk to the internet with appropriate quickness.

I know one other person who's experienced this, but with no solution.
Anyone else? I haven't found similar stories, but maybe that's because
no one else is having this problem, or the solution is so obvious no
one bothers to mention it.

Any pointers or ideas would be hugely appreciated.

Thanks,
Chris
Ef0db53920b243d6758c2f6b1306df0d?d=identicon&s=25 Steve Ross (cwd)
on 2008-01-13 01:45
(Received via mailing list)
This may not help you but I was having a slow-as-molasses DNS lookup
issue when I upgraded to Leopard. I disabled IPv6 and it sped things
up considerably. Anyhow, it's a change that might help and is easy to
undo.

HTH
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2008-01-13 01:48
(Received via mailing list)
On Jan 12, 2008, at 4:39 PM, Chris Shea wrote:

> open('http://www.google.com')" takes 15 seconds (but half a second
>
> Thanks,
> Chris
>


  Try adding this to your script:

# for OSX compatibility
Socket.do_not_reverse_lookup = true


Cheers-
- Ezra Zygmuntowicz
-- Founder & Software Architect
-- ezra@engineyard.com
-- EngineYard.com
E38cd950767e62fd3250795bbd208cf6?d=identicon&s=25 Pierre-Yves Gérardy (Guest)
on 2008-01-14 22:52
(Received via mailing list)
>         Try adding this to your script:
>
> # for OSX compatibility
> Socket.do_not_reverse_lookup = true

Any idea where to patch rubygems?
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2008-01-14 23:45
could try the macport ruby

Chris Shea wrote:
> Just a couple of days ago I "upgraded" from Tiger to Leopard, and the
> most horrible thing about it is that anything Ruby does over the
> internet is so very very slow now. `which ruby` tells me I'm still
> using my from-source install in /usr/local/bin, as does `which gem`.
>
Ef0db53920b243d6758c2f6b1306df0d?d=identicon&s=25 Steve Ross (cwd)
on 2008-01-15 05:14
(Received via mailing list)
I've had almost exactly the same experience. Really slow internet
connectivity in general. I think I have it narrowed down to wireless
using WEP. I hooked up an old fashioned ethernet cable (remember
those?) and everything is lickety split fast. HTH.
1bac2e65d64faf472cf2ebc94f0f5ee0?d=identicon&s=25 ara.t.howard (Guest)
on 2008-01-15 05:23
(Received via mailing list)
On Jan 14, 2008, at 9:14 PM, s.ross wrote:

> I've had almost exactly the same experience. Really slow internet
> connectivity in general. I think I have it narrowed down to
> wireless using WEP. I hooked up an old fashioned ethernet cable
> (remember those?) and everything is lickety split fast. HTH.

me too.  this a leoTard issue.  in general i'm finding it to be
sucking the big one for networking - you can see the same behavior
with curl as you'll find in ruby.

i also disabled ipv6 and am using opendns for lookups.  still slower
than tiger though.

a @ http://codeforpeople.com/
744b6438d1457d332d1a8d431cc2b641?d=identicon&s=25 Andrew Ohnstad (livitup)
on 2008-01-15 07:12
ara.t.howard wrote:
> On Jan 14, 2008, at 9:14 PM, s.ross wrote:
>
>> I've had almost exactly the same experience. Really slow internet
>> connectivity in general. I think I have it narrowed down to
>> wireless using WEP. I hooked up an old fashioned ethernet cable
>> (remember those?) and everything is lickety split fast. HTH.
>
> me too.  this a leoTard issue.  in general i'm finding it to be
> sucking the big one for networking - you can see the same behavior
> with curl as you'll find in ruby.
>
> i also disabled ipv6 and am using opendns for lookups.  still slower
> than tiger though.
>
> a @ http://codeforpeople.com/

I had this problem too and found some bogus DNS servers had creeped into
Network Setup, even though everything had been set to DCHP before (and I
assumed after) the upgrade.  My guess was that network was waiting for
them to timeout before contacting my real DNS server.  Key symptom is
eons to start loading a page/download a file/whatever, but normal
performance once the activity actually starts.

I know it sounds noob, but check to make sure there aren't any
unexpected DNS Servers in System Prefs -> Network -> Advanced (for the
correct interface) -> DNS.
Ef0db53920b243d6758c2f6b1306df0d?d=identicon&s=25 Steve Ross (cwd)
on 2008-01-15 07:29
(Received via mailing list)
On Jan 14, 2008, at 8:22 PM, ara.t.howard wrote:

> am using opendns for lookups

I tried opendns until the first bad lookup (address not in dns) when
opendns hijacked the request and redirected my browser to an ad. It
just seems creepy to have your dns lookups spoofed like that.
81b61875e41eaa58887543635d556fca?d=identicon&s=25 Frederick Cheung (Guest)
on 2008-01-15 11:53
(Received via mailing list)
On 15 Jan 2008, at 04:22, ara.t.howard wrote:

> with curl as you'll find in ruby.
>
> i also disabled ipv6 and am using opendns for lookups.  still slower
> than tiger though.
>
I had a problem like that. For some reason removing all locations from
the network prefs and adding them back in fixed it for some reason.

Fred
6df9db0ace4cfd315296c344dc41ab1d?d=identicon&s=25 Bryce Roney (bdude)
on 2008-02-04 01:28
I am having this problem too, I've tried disabling IPv6 and I've put the
Socket. thing in my configuration and it's still slow.
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2008-02-04 06:57
Bryce Roney wrote:
> I am having this problem too, I've tried disabling IPv6 and I've put the
> Socket. thing in my configuration and it's still slow.

could try
require 'resolv-replace'
6df9db0ace4cfd315296c344dc41ab1d?d=identicon&s=25 Bryce Roney (bdude)
on 2008-02-04 07:23
Roger Pack wrote:
> Bryce Roney wrote:
>> I am having this problem too, I've tried disabling IPv6 and I've put the
>> Socket. thing in my configuration and it's still slow.
>
> could try
> require 'resolv-replace'

Then it comes up with this:
LoadError: no such file to load -- resolve-replace
  from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'
  from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in
`require'
  from (irb):1
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2008-02-04 18:03
require 'resolv-replace'
017e05d1a49ffa59ea03e149e7af720b?d=identicon&s=25 Chris Shea (chrisshea)
on 2008-02-04 21:05
(Received via mailing list)
On Jan 12, 5:39 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
> Any pointers or ideas would be hugely appreciated.

After some pointers and ideas, things are running much faster. What
I've done:

1. Disabled IPv6 (thanks s.ross)
2. OpenDNS for lookup (thanks Ara, though those redirects on failed
lookups suck)
3. Started requiring 'resolv-replace' (thanks Roger Pack)

I'm using Ruby and the Internet together at near Tiger speeds. I added
"require 'resolv-replace'" to /usr/local/bin/gem as well, and I'm no
longer scared of "Updating metadata for 132 gems from
http://gems.rubyforge.org"

Thanks everyone. Hopes this thread helps anyone else having similar
issues.

Chris
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2008-02-06 04:31
Roger Pack wrote:
> require 'resolv-replace'

somehow my editing /etc/hosts once (took some hosts out), worked faster,
then adding them back in and it still worked fast. Weird.
Somebody elsewhere also said that doing the 'assist me' in networking
preferences fixed their problem.  Who knows what it really is :)
-Roger
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2008-02-06 05:00
laughably, the following script (which overwrites your /etc/hosts file
periodically with one or two entries)
seems to help.
I probably wouldn't really recommend this unless you really know what
you're doing, but here it is

count = 0
loop do
 a = File.open '/etc/hosts', 'w'
 a.write '127.0.0.1  localhost' + "\n"
 a.write '127.0.0.1  localhost2' + "\n" if (count%2) == 1
 a.close
 sleep 0.5
 count += 1
end

(save this as a file like 'renew_etc.rb', save a copy of your /etc/hosts
file somewhere, then run
sudo ruby renew_etc.rb)
You could try changing the sleep value.
Note that running this may have unforeseen effects.
I wouldn't believe it if it didn't seem to work.

Note that the default /etc/hosts file is
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1  localhost
255.255.255.255  broadcasthost
::1             localhost
fe80::1%lo0  localhost


if you need to recreate it.
GL.
-Roger
6df9db0ace4cfd315296c344dc41ab1d?d=identicon&s=25 Bryce Roney (bdude)
on 2008-02-08 08:29
I had success with
require 'resolv-replace'
# for OSX compatibility
Socket.do_not_reverse_lookup = true

is there a way I can have this enabled by default accross all scripts?
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2008-02-08 18:34
Bryce Roney wrote:
> I had success with
> require 'resolv-replace'
> # for OSX compatibility
> Socket.do_not_reverse_lookup = true
>
> is there a way I can have this enabled by default accross all scripts?

you could add that code to the lib included when you run
require 'rubygems'
as that is always loaded for app
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.