Ruby HTTP GET Headers

ben · April 10, 2008, 4:46pm

I’ve looked here: http://snippets.dzone.com/posts/show/788 and I’m not
really understanding it. I am creating a client for a website, and I
want to set the User-Agent field of the header I am sending. Is there a
simple way to do this or is this along the lines of creating custom HTTP
objects? I’ve also looked into the open-uri class, but I get this error:
Errno::EHOSTDOWN: Host is down - connect(2)

method initialize in http.rb at line 560
method open in http.rb at line 560
method connect in http.rb at line 560
method timeout in timeout.rb at line 48
method timeout in timeout.rb at line 76
method connect in http.rb at line 560
method do_start in http.rb at line 553
method start in http.rb at line 542
method open_http in open-uri.rb at line 242
method buffer_open in open-uri.rb at line 626
method open_loop in open-uri.rb at line 164
method catch in open-uri.rb at line 162
method open_loop in open-uri.rb at line 162
method open_uri in open-uri.rb at line 132
method open in open-uri.rb at line 528
method open in open-uri.rb at line 30
at top level in rubyheader.rb at line 8

I attached the test file I am working on. Any help would be great.
Thanks.

ben · April 10, 2008, 9:03pm

Hi Ben,

I’m not sure if your code would work like you expect it to, but I’ve
written a client library for my Halcyon project you might find
interesting.

github.com

mtodd/halcyon/blob/master/lib/halcyon/client.rb

%w(net/http uri json).each {|dep|require dep}

module Halcyon
  
  # = Building Custom Clients
  # 
  # Once your Halcyon JSON Server App starts to take shape, it may be useful
  # to begin to write tests on expected functionality, and then to implement
  # API calls with a designated Client lib for your Ruby or Rails apps, etc.
  # The Base class provides a standard implementation and several options for
  # wrapping up functionality of your app from the server side into the
  # client side so that you may begin to use response data.
  # 
  # == Creating Your Client
  # 
  # Creating a simple client can be as simple as this:
  # 
  #   class Simple < Halcyon::Client
  #     def greet(name)
  #       get("/hello/#{name}")

This file has been truncated. show original

Note that you won’t need to use JSON.parse for the response data
(that’s something peculiar to Halcyon) and that you may get errors if
you don’t remove #to_mash calls because that’s a dependency fulfilled
by Merb’s Core Extensions. There may be other dependencies that I
haven’t thought about that will cause problems, but you should be able
to determine that from the errors when run.

You can get it working with:

c = Halcon::Client.new(‘http://google.com:80/’)
puts c.get(‘/’)

Matt T.

ben · April 11, 2008, 12:04pm

Ben A. wrote:

I’ve looked here: http://snippets.dzone.com/posts/show/788 and I’m not
really understanding it. I am creating a client for a website, and I
want to set the User-Agent field of the header I am sending. Is there a
simple way to do this or is this along the lines of creating custom HTTP
objects? I’ve also looked into the open-uri class,

The following both work for me:

In net/http, the request_get() method allows you to specify headers
for a GET request:

require ‘net/http’
require ‘uri’

pieces = URI.parse(“http://localhost/cgi-bin/ruby_test.rb”)
headers = {“User-Agent” => “A really cool browser”}

resp = Net::HTTP::start(pieces.host, pieces.port) do |connection|
connection.request_get(pieces.path, headers)
end

puts resp.body

In open-uri, the open() method allows you to specify headers:

require ‘open-uri’

url = ‘http://localhost/cgi-bin/ruby_test.rb’
headers = {“User-Agent” => “Bad Browser”}

open(url, headers) do |f|
puts f.read
end

I tested those scripts by putting the following simple cgi script on my
server. The cgi script just sends back whatever the User-Agent string
was in the request:

#!/usr/bin/env ruby

require ‘cgi’

c = CGI.new

puts “Content-type: text/html”
puts
puts c.user_agent