Open URI Redirect Loop

Ruby
1

Hey guys,

I am using Nokogiri along with Open URI to obtain contents of websites.
I have ran into a problem with windowsphone.com.

Googling around doesn’t give too much insight into the problem. :\

irb(main):023:0>
Nokogiri::HTML(open(“http://www.windowsphone.com/search?q=angry+birds&start=1”))

RuntimeError: redirection forbidden:
http://www.windowsphone.com/search?q=angry+birds&start=1 →
https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&checkda=1&ct=1321939416&rver=6.1.6195.0&wp=MBI&wreply=http:%2F%2Fwww.windowsphone.com%2Fsearch%3Fq%3Dangry%2Bbirds%26start%3D1&lc=1033&id=268289
from /usr/local/lib/ruby/1.9.1/open-uri.rb:216:in open_loop' from /usr/local/lib/ruby/1.9.1/open-uri.rb:146:in open_uri’
from /usr/local/lib/ruby/1.9.1/open-uri.rb:671:in open' from /usr/local/lib/ruby/1.9.1/open-uri.rb:33:in open’
from (irb):23
from /usr/local/bin/irb:12:in `’

irb(main):024:0>
Nokogiri::HTML(open(“https://www.windowsphone.com/search?q=angry+birds&start=1”))

RuntimeError: HTTP redirection loop:
https://www.windowsphone.com/search?q=angry+birds&start=1
from /usr/local/lib/ruby/1.9.1/open-uri.rb:224:in open_loop' from /usr/local/lib/ruby/1.9.1/open-uri.rb:146:in open_uri’
from /usr/local/lib/ruby/1.9.1/open-uri.rb:671:in open' from /usr/local/lib/ruby/1.9.1/open-uri.rb:33:in open’
from (irb):24
from /usr/local/bin/irb:12:in `’

Thanks, help much appreciated! :slight_smile:

2

The HTTPS link you gave doesn’t work for me. The HTTP one redirects to
https://login.live.com/ - note the different protocol - and that’s
probably why open-uri bails out.

– Matma R.

3

Bartosz Dziewoński wrote in post #1033163:

The HTTPS link you gave doesn’t work for me. The HTTP one redirects to
https://login.live.com/ - note the different protocol - and that’s
probably why open-uri bails out.

– Matma R.

Hmm, I just copied / pasted them and true enough, neither one of them
did what it was supposed to (i.e. take me to the Windows marketplace
search with results of “Angry Birds”).

I Googled for Windows marketplace, clicked on the Windows marketplace
and performed the search directly from the website. It gave me the same
exact link.

The link now works. Looks like Microsoft must be doing something funky
with cookies, sessions, or whatever. Not sure yet.