Open of web page fails

Ruby
1

here is the URL (spans 2 lines) and it fails with a bad uri message

http://quote.yahoo.com/d/quotes.csv?s=MS+^DJI+^GSPC+^IXIC+IWB+IWM+IWV+TMW+^VIX+^TV.N+&f=d1t1sl1c1&e=csv

The message I get is URI::InvalidURIError: bad URI(is not URI?):

The same URI in a web page returns quote.csv, as expected

Any ideas? I am stumped

thanks in advance!

Joe

2

Whenever I do web stuff I prefer using an external library. ie Mechanize

Here is what it’d look like with mechanize :

require ‘Mechanize’
agent = Mechanize.new
file = agent.get(‘uri’) #Returns a Mechanize::File object
file.save(‘filename’)

You could also use blocks if you wanted.
That will save the file in the same directory as the script.

3

You need to escape the carets (^) for Ruby to understand - URI library
is pretty strict about what it accepts. You will probably need to
escape all other funky chars, too.

– Matma R.

4

I tried escaping the ^ and other symbols also but no luck

Joe

5

2011/10/7 Joe C. [email protected]:

I tried escaping the ^ and other symbols also but no luck

“Escape” as in “URL-escape”. For example, ^ becomes %5E. You can use
the CGI.escape() method for this (you’ll need to require ‘cgi’ first).

– Matma R.

6

open(“http://quote.yahoo.com/d/quotes.csv\?s=MS+\^DJI+\^GSPC+\^IXIC+IWB+IWM+IWV+TMW+\^VIX+\^TV.N+\&f=d1t1sl1c1\&e=csv”)

Here it is escaped and it still fails…any ideas?

Joe

7

Not backslash-escape, URL-escape - with the percent signs. Read my
previous post again - you can use CGI.escape() for this. Also read
this: URL encoding - Wikipedia

– Matma R.

2011/10/7 Joe C. [email protected]:

8

Thanks Matma - that put me on the correct track. This works fine:
“http://quote.yahoo.com/d/quotes.csv\?s=“+$symbol+”+^DJI+^GSPC+^IXIC+IWB+IWM+IWV+TMW+^VIX+^TV.N+&f=d1t1sl1c1\%26e=csv”

thanks again

Joe