Problem with open-uri

Hello,

I'm trying to use open-uri on yahoo mail's login page.  If I do the

following:

#!/usr/bin/env ruby

require ‘open-uri’
require ‘uri’

f = open(“http://mail.yahoo.com”)


I get the following error:

/usr/lib/ruby/1.8/open-uri.rb:174:in open_loop': redirection forbidden: http://mail.yahoo.com/ -> https://login.yahoo.com/config/login_verify2?&.src=ym (RuntimeError) from /usr/lib/ruby/1.8/open-uri.rb:132:in open_uri’
from /usr/lib/ruby/1.8/open-uri.rb:528:in open' from /usr/lib/ruby/1.8/open-uri.rb:30:in open’
from ./readForms.rb:11


I've tried to use mechanize instead, but I can't seem to figure out

how to emulate the base_uri functionality in open-uri. Any ideas?

Regards,

jon

On 6/3/07, Jonathan N. [email protected] wrote:

f = open(“http://mail.yahoo.com”)
from /usr/lib/ruby/1.8/open-uri.rb:30:in `open’
from ./readForms.rb:11


I've tried to use mechanize instead, but I can't seem to figure out

how to emulate the base_uri functionality in open-uri. Any ideas?

Regards,

jon

You’re being redirected. Try the redirect url (and you can leave the
?&.src=ym off depending on what you want to do)

require ‘open-uri’
document = open( “Yahoo” ).read

Hi Todd,

Thanks for the quick response.  Unfortunately, I need to have the 

ability to take a simple uri and follow the redirections. open-uri
works for some uris, but not for yahoo for some reason. If you do
open(“https://mail.google.com”), it will follow the redirects and
successfully download the html at the redirected uri, which is
https://www.google.com/accounts/ServiceLogin. this looks like a bug in
open-uri to me, as mechanize (and firefox) is able to follow the
redirects for yahoo.

I can't find a way to report bugs for open-uri.  Anybody know how to 

do this?

Regards,

jon

Todd B. wrote:

On 6/3/07, Jonathan N. [email protected] wrote:

f = open(“http://mail.yahoo.com”)
from /usr/lib/ruby/1.8/open-uri.rb:30:in `open’
from ./readForms.rb:11


I've tried to use mechanize instead, but I can't seem to figure out

how to emulate the base_uri functionality in open-uri. Any ideas?

Regards,

jon

You’re being redirected. Try the redirect url (and you can leave the
?&.src=ym off depending on what you want to do)

require ‘open-uri’
document = open( “Yahoo” ).read

I think I figured this out. The following is the code that generates
the exception:


def OpenURI.redirectable?(uri1, uri2) # :nodoc:
# This test is intended to forbid a redirection from http://… to
# file:///etc/passwd.
# However this is ad hoc. It should be extensible/configurable.
uri1.scheme.downcase == uri2.scheme.downcase ||
(/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~
uri2.scheme)
end

This code is telling us that if there’s a redirection, it will only work
if the protocols are the same (http → http, or https → https), or if
both the first and the second uris are either http or ftp. This logic
prohibits a redirect from http to https, which is what the yahoo mail
uri does. Changing the last line to

(/\A(?:http|ftp|https)\z/i =~ uri1.scheme && 

/\A(?:http|ftp|https)\z/i =~ uri2.scheme)

will do the trick. Seems like this fix should get incorporated into
open-uri. Anyone out there on the open-uri team listening?

-jon

Jonathan N. wrote:

Hi Todd,

Thanks for the quick response.  Unfortunately, I need to have the 

ability to take a simple uri and follow the redirections. open-uri
works for some uris, but not for yahoo for some reason. If you do
open(“https://mail.google.com”), it will follow the redirects and
successfully download the html at the redirected uri, which is
https://www.google.com/accounts/ServiceLogin. this looks like a bug in
open-uri to me, as mechanize (and firefox) is able to follow the
redirects for yahoo.

I can't find a way to report bugs for open-uri.  Anybody know how to 

do this?

Regards,

jon

Todd B. wrote:

On 6/3/07, Jonathan N. [email protected] wrote:

f = open(“http://mail.yahoo.com”)
from /usr/lib/ruby/1.8/open-uri.rb:30:in `open’
from ./readForms.rb:11


I've tried to use mechanize instead, but I can't seem to figure out

how to emulate the base_uri functionality in open-uri. Any ideas?

Regards,

jon

You’re being redirected. Try the redirect url (and you can leave the
?&.src=ym off depending on what you want to do)

require ‘open-uri’
document = open( “Yahoo” ).read