\
**Category:** Ruby\
**Created:** [March 26, 2009, 8:17am UTC](https://www.ruby-forum.com/t/https-html-parsing/163930 "2009-03-26T08:17:27Z")\
**Posts on this page:** 6\
**Page:** 1
**Author:**  [@arunvoip](https://www.ruby-forum.com/u/arunvoip)\
**Post date:** [March 26, 2009, 8:17am UTC](https://www.ruby-forum.com/t/https-html-parsing/163930/1 "2009-03-26T08:17:27Z")
Hi,
Can anybody please give me the details of how to parse html from a
‘https’ site using ‘net/http’. I came to learn that we can use basic
authentication techniques to do that. But I want to know whether there
is a way other than basic authentication to parse html content from a
https site. I dont want to use any external libraries and i only want to
use ‘net/http’ and not any parsing libraries like ‘Hpricot’ etc… Can
anybody please help me. I will be really greatfull.
N. B.
/usr/lib/ruby/1.8/net/http.rb:560:in `initialize': getaddrinfo: Name or service not known (SocketError) from /usr/lib/ruby/1.8/net/http.rb:560:in`open’
from /usr/lib/ruby/1.8/net/http.rb:560:in `connect' from /usr/lib/ruby/1.8/timeout.rb:53:in`timeout’
from /usr/lib/ruby/1.8/timeout.rb:93:in `timeout' from /usr/lib/ruby/1.8/net/http.rb:560:in`connect’
from /usr/lib/ruby/1.8/net/http.rb:553:in `do_start' from /usr/lib/ruby/1.8/net/http.rb:542:in`start’
from example.rb:5
This is the error which i get now when i use ‘net/http’.
Regards
Arun K.
---
**Author:**  [@arunvoip](https://www.ruby-forum.com/u/arunvoip)\
**Post date:** [March 27, 2009, 3:54pm UTC](https://www.ruby-forum.com/t/https-html-parsing/163930/2 "2009-03-27T15:54:02Z")
Arun K. wrote:
> /usr/lib/ruby/1.8/net/http.rb:560:in `initialize’: getaddrinfo: Name or
> service not known (SocketError)
> …
> from example.rb:5
> This is the error which i get now when i use ‘net/http’.
This error isn’t useful unless you also post your example.rb
Note that in order to talk to a https server, you will need net/https
not net/http.
In some distributions (e.g. Ubuntu) you won’t have net/https.rb until
you install another package. For Ubuntu, apt-get install libopenssl-ruby
---
**Author:**  [@arunvoip](https://www.ruby-forum.com/u/arunvoip)\
**Post date:** [March 28, 2009, 5:16am UTC](https://www.ruby-forum.com/t/https-html-parsing/163930/3 "2009-03-28T05:16:27Z")
Brian C. wrote:
> Arun K. wrote:
>
> > /usr/lib/ruby/1.8/net/http.rb:560:in `initialize’: getaddrinfo: Name or
> > service not known (SocketError)
> > …
> > from example.rb:5
> > This is the error which i get now when i use ‘net/http’.
>
> This error isn’t useful unless you also post your example.rb
>
> Note that in order to talk to a https server, you will need net/https
> not net/http.
>
> In some distributions (e.g. Ubuntu) you won’t have net/https.rb until
> you install another package. For Ubuntu, apt-get install libopenssl-ruby
Hi,
Thanks for ur reply.
require ‘net/https’
require ‘uri’
uri = “[https://www.cia.gov](https://www.cia.gov)”
url = URI.parse(uri)
req = Net::HTTP::Get.new(‘/’, “User-Agent”=\>“Mozilla/4.0 (compatible
MSIE 5.5; Windows NT 5.0)”)
https = Net::HTTP.new(url.host, url.port)
https.use\_ssl = true
res = https.start {|http|
http.request(req)
}
puts res.body
This is my entire code of example.rb. Now i used https. I’m now getting
a warning like this when i try to access https sites :
warning: peer certificate won’t be verified in this SSL session
and i’m not able to access sites where the url is redirecting.
Please help me
Regards
Arun K.
---
**Author:**  [@arunvoip](https://www.ruby-forum.com/u/arunvoip)\
**Post date:** [March 28, 2009, 9:59am UTC](https://www.ruby-forum.com/t/https-html-parsing/163930/4 "2009-03-28T09:59:06Z")
Eric H. wrote:
> On Mar 27, 2009, at 21:13, Arun K. wrote:
>
> > >
> >
> > require ‘net/https’
> > }
> > puts res.body
> >
> > This is my entire code of example.rb. Now i used https. I’m now
> > getting
> > a warning like this when i try to access https sites :
> >
> > warning: peer certificate won’t be verified in this SSL session
> >
> > and i’m not able to access sites where the url is redirecting.
>
> This is explained in `ri Net::HTTP` under “Following Redirection”
Hi,
I solved the redirecting problem. But still the https problem remains.
Please help me
Regards
Arun K.
---
**Author:**  [@arunvoip](https://www.ruby-forum.com/u/arunvoip)\
**Post date:** [March 28, 2009, 6:51pm UTC](https://www.ruby-forum.com/t/https-html-parsing/163930/5 "2009-03-28T18:51:06Z")
Arun K. wrote:
> I solved the redirecting problem. But still the https problem remains.
The error is just telling you that it doesn’t have any local CA root
certificates with which to verify the certificate presented by the peer.
If you want to enable certificate verification, and set the path to the
directory containing your CA root certificates, there is example code
here:
[http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/branches/ruby\_1\_8/sample/openssl/wget.rb?view=markup](http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/branches/ruby_1_8/sample/openssl/wget.rb?view=markup)
There is more documentation at the top of net/https.rb, which will be
installed on your system somewhere, e.g. /usr/lib/ruby/1.8/net/https.rb
You may already have root CA certificates installed. On my Ubuntu Hardy
box they are in /etc/ssl/certs/
---
**Author:**  [@arunvoip](https://www.ruby-forum.com/u/arunvoip)\
**Post date:** [March 28, 2009, 5:38am UTC](https://www.ruby-forum.com/t/https-html-parsing/163930/6 "2009-03-28T05:38:36Z")
On Mar 27, 2009, at 21:13, Arun K. wrote:
> >
>
> require ‘net/https’
> }
> puts res.body
>
> This is my entire code of example.rb. Now i used https. I’m now
> getting
> a warning like this when i try to access https sites :
>
> warning: peer certificate won’t be verified in this SSL session
>
> and i’m not able to access sites where the url is redirecting.
This is explained in `ri Net::HTTP` under “Following Redirection”