# Question about Mechanize and Google API **URL:** \ **Category:** Ruby\ **Created:** [October 8, 2007, 5:49pm UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673 "2007-10-08T17:49:19Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/4da419/32.png) [@JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/u/JesSSSSs_Gabriel_y_S)\ **Post date:** [October 8, 2007, 5:49pm UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673/1 "2007-10-08T17:49:19Z")
Hi, I’m trying to use Mechanize to retrieve spreadsheets from Google Docs using the Google API. So far I got the ClientLogin to work, but I can’t seem to be able to use the Auth token they give me back. This is what I have: require ‘logger’ require ‘mechanize’ agent = WWW::Mechanize.new {|a| a.log = Logger.new(“test.log”)} response = agent.post(“[https://www.google.com/accounts/ClientLogin](https://www.google.com/accounts/ClientLogin)”, “Email” =\> “removed\_email\_address@domain.invalid”, “Passwd” =\> “xxxxx”, “service” =\> “wise”, “source” =\> “personal-test-1.0”, “accountType” =\> “GOOGLE”) h = {} response.body.each {|line| a = line.split(“=”); h[a[0]] = a[1]} auth = h[“Auth”] agent.basic\_auth(“GoogleLogin”, auth) # this didn’t work either #agent.auth(“GoogleLogin”, auth) page = agent.get(“[http://spreadsheets.google.com/feeds/spreadsheets/private/full](http://spreadsheets.google.com/feeds/spreadsheets/private/full)”) The problem is, according to google I have to: "After a successful authentication request, use the Auth value to create an Authorization header for each request: Authorization: GoogleLogin auth=yourAuthValue" I’ve checked and the ClientLogin is working and returning me an Auth token, but I can’t get Mechanize to use it. In the Mechanize logs I see this: I, [2007-10-08T17:01:52.211778 #24929] INFO – : Net::HTTP::Post: /accounts/ClientLogin D, [2007-10-08T17:01:52.394047 #24929] DEBUG – : request-header: accept-language =\> en-us,en;q0.5 D, [2007-10-08T17:01:52.394214 #24929] DEBUG – : request-header: connection =\> keep-alive D, [2007-10-08T17:01:52.394305 #24929] DEBUG – : request-header: accept =\> _/_ D, [2007-10-08T17:01:52.394394 #24929] DEBUG – : request-header: accept-encoding =\> gzip,identity D, [2007-10-08T17:01:52.394482 #24929] DEBUG – : request-header: user-agent =\> WWW-Mechanize/0.6.10 ([http://rubyforge.org/projects/mechanize/](http://rubyforge.org/projects/mechanize/)) D, [2007-10-08T17:01:52.394570 #24929] DEBUG – : request-header: content-type =\> application/x-www-form-urlencoded D, [2007-10-08T17:01:52.394656 #24929] DEBUG – : request-header: accept-charset =\> ISO-8859-1,utf-8;q=0.7,_;q=0.7 D, [2007-10-08T17:01:52.394742 #24929] DEBUG – : request-header: content-length =\> 104 D, [2007-10-08T17:01:52.394828 #24929] DEBUG – : request-header: keep-alive =\> 300 D, [2007-10-08T17:01:52.613138 #24929] DEBUG – : Read 563 bytes D, [2007-10-08T17:01:52.613358 #24929] DEBUG – : response-header: cache-control =\> no-cache D, [2007-10-08T17:01:52.613483 #24929] DEBUG – : response-header: date =\> Mon, 08 Oct 2007 15:00:17 GMT D, [2007-10-08T17:01:52.613603 #24929] DEBUG – : response-header: content-type =\> text/plain D, [2007-10-08T17:01:52.613721 #24929] DEBUG – : response-header: server =\> GFE/1.3 D, [2007-10-08T17:01:52.613837 #24929] DEBUG – : response-header: content-length =\> 563 D, [2007-10-08T17:01:52.613969 #24929] DEBUG – : response-header: pragma =\> no-cache I, [2007-10-08T17:01:52.614363 #24929] INFO – : status: 200 I, [2007-10-08T17:13:15.370472 #24929] INFO – : Net::HTTP::Get: /feeds/spreadsheets/private/full D, [2007-10-08T17:13:15.482357 #24929] DEBUG – : request-header: accept-language =\> en-us,en;q0.5 D, [2007-10-08T17:13:15.482481 #24929] DEBUG – : request-header: connection =\> keep-alive D, [2007-10-08T17:13:15.482534 #24929] DEBUG – : request-header: accept =\> / D, [2007-10-08T17:13:15.482587 #24929] DEBUG – : request-header: accept-encoding =\> gzip,identity D, [2007-10-08T17:13:15.482639 #24929] DEBUG – : request-header: user-agent =\> WWW-Mechanize/0.6.10 ([http://rubyforge.org/projects/mechanize/](http://rubyforge.org/projects/mechanize/)) D, [2007-10-08T17:13:15.482692 #24929] DEBUG – : request-header: referer =\> [https://www.google.com/accounts/ClientLogin](https://www.google.com/accounts/ClientLogin) D, [2007-10-08T17:13:15.482745 #24929] DEBUG – : request-header: accept-charset =\> ISO-8859-1,utf-8;q=0.7,_;q=0.7 D, [2007-10-08T17:13:15.482796 #24929] DEBUG – : request-header: keep-alive =\> 300 D, [2007-10-08T17:13:15.652098 #24929] DEBUG – : Read 141 bytes D, [2007-10-08T17:13:15.652313 #24929] DEBUG – : response-header: cache-control =\> no-cache, no-store, max-age=0, must-revalidate D, [2007-10-08T17:13:15.652438 #24929] DEBUG – : response-header: expires =\> Fri, 01 Jan 1990 00:00:00 GMT D, [2007-10-08T17:13:15.652604 #24929] DEBUG – : response-header: date =\> Mon, 08 Oct 2007 15:11:40 GMT D, [2007-10-08T17:13:15.652725 #24929] DEBUG – : response-header: content-type =\> text/html; charset=UTF-8 D, [2007-10-08T17:13:15.652859 #24929] DEBUG – : response-header: www-authenticate =\> No credentials were included in your request. D, [2007-10-08T17:13:15.652977 #24929] DEBUG – : response-header: server =\> GFE/1.3 D, [2007-10-08T17:13:15.653094 #24929] DEBUG – : response-header: set-cookie =\> S=trix=Xc7oom0LVJ8; Path=/ D, [2007-10-08T17:13:15.653211 #24929] DEBUG – : response-header: content-length =\> 141 D, [2007-10-08T17:13:15.653326 #24929] DEBUG – : response-header: pragma =\> no-cache D, [2007-10-08T17:13:15.659559 #24929] DEBUG – : saved cookie: S=trix=Xc7oom0LVJ8 I, [2007-10-08T17:13:15.659765 #24929] INFO – : status: 404 So, it seems that the credentials are not correctly set. Anybody knows how to do this? Thanks, Jesus. ---
**Author:** ![JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/4da419/32.png) [@JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/u/JesSSSSs_Gabriel_y_S)\ **Post date:** [October 8, 2007, 6:25pm UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673/2 "2007-10-08T18:25:09Z")
On 10/8/07, Jesús Gabriel y Galán [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Hi, > > I’m trying to use Mechanize to retrieve spreadsheets from Google Docs > using the Google API. So far I got the ClientLogin to work, but I > can’t seem to be able to use the Auth token they give me back. This is > what I have: From my experience Mechanize is a bit low level, especially in this area. You are expected to write the code yourself to handle, redirects, authentication, SSL etc. You might want to try open-uri or Hpricot as well. But I think there is no right-sized solution - you will still need to do a bit of code writing yourself. BTW - you should check out the new RESTful Web Services book (O’Reilly). In addition to just being an excellent book, it has a lot of practical advice on HTTP clients and directly manipulating HTTP headers, which is I suspect what you will ultimately need to do. ---
**Author:** ![JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/4da419/32.png) [@JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/u/JesSSSSs_Gabriel_y_S)\ **Post date:** [October 8, 2007, 7:38pm UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673/3 "2007-10-08T19:38:35Z")
Jesus, the Gdata gem works with the google Gdata API , I’m currently implementing the gcalendar API for that gem , but luckely for you, I think the 0.4version has some spreadsheet implementation. ---
**Author:** ![JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/4da419/32.png) [@JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/u/JesSSSSs_Gabriel_y_S)\ **Post date:** [October 8, 2007, 9:21pm UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673/4 "2007-10-08T21:21:53Z")
On 10/8/07, Matias Pablo B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > the Gdata gem works with the google Gdata API , I’m currently implementing > the gcalendar API for that gem , but luckely for you, I think the > 0.4version has some spreadsheet implementation. Hi, yes, I had checked the Gdata gem, but saw it was pre alpha. In any case I installed it and checked the spreadsheet class but I just saw a couple of methods to demo how to evaluate a cell, and not the complete functionality of the Google API. Am I right? I’m thinking of stop using Mechanize for this and use just net/http for the simple post and get I need to do, so I can customize the headers myself, as I’ve seen you do in the gdata classes. Thanks, Jesus. ---
**Author:** ![JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/4da419/32.png) [@JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/u/JesSSSSs_Gabriel_y_S)\ **Post date:** [October 9, 2007, 12:53am UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673/5 "2007-10-09T00:53:51Z")
Richard C. wrote: > From my experience Mechanize is a bit low level, especially in this area. > You are expected to write the code yourself to handle, redirects, > authentication, > SSL etc. > > You might want to try open-uri or Hpricot as well. But I think there is no > right-sized solution - you will still need to do a bit of code writing yourself. Just for the record, last time I checked, Mechanize _did_ use Hpricot to parse pages. AFAIK, you can even provide Mechainze with your own page parser class before fetching the stuff. mortee ---
**Author:** ![JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/4da419/32.png) [@JesSSSSs\_Gabriel\_y\_S](https://www.ruby-forum.com/u/JesSSSSs_Gabriel_y_S)\ **Post date:** [October 9, 2007, 9:33am UTC](https://www.ruby-forum.com/t/question-about-mechanize-and-google-api/116673/6 "2007-10-09T09:33:10Z")
On 10/9/07, mortee [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > parse pages. AFAIK, you can even provide Mechainze with your own page > parser class before fetching the stuff. Yes, Mechanize parses pages with Hpricot, and also allows you to use an Hpricot-like API on them for search: agent.get(‘[http://rubyforge.org](http://rubyforge.org)’).search(“//strong”) The problem I was having with it is the fact that I didn’t know how to specify arbitrary headers for my request, or make the Google authentication stuff to work. Working directly with net/https worked for me (code “inspired” by the Gdata gem :-): require ‘net/http’ require ‘net/https’ require ‘uri’ require ‘hpricot’ module Net class HTTPS \< HTTP def initialize(address, port = nil) super(address, port) self.use\_ssl = true end end end h = {“Email” =\> “removed\_email\_address@domain.invalid”, “Passwd” =\> “xxxxx”, “service” =\> “wise”, “source” =\> “personal-test-1.0”, “accountType” =\> “GOOGLE” } url = URI.parse(‘[https://www.google.com/accounts/ClientLogin](https://www.google.com/accounts/ClientLogin)’) response = Net::HTTPS.post\_form(url, h) auth = “” response.body.each do |line| if line =~ /Auth=/ auth = line.split(“=”)[1].chomp end end headers = {“Authorization” =\> “GoogleLogin auth=#{auth}”} http = Net::HTTP.new(“[spreadsheets.google.com](http://spreadsheets.google.com)”, 80) response, data = http.get(“/feeds/spreadsheets/private/full”, headers) document = Hpricot(data) document.search(“//entry”).each do |entry| p entry end This gives you a document with the list of private spreadsheets of the user. Then you can use Hpricot or whatever you want to parse the atom feed. Jesus.