# Posting over https

**URL:** <https://www.ruby-forum.com/t/posting-over-https/71515>\
**Category:** Ruby\
**Created:** [October 9, 2006, 5:05pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515 "2006-10-09T17:05:51Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 9, 2006, 5:05pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/1 "2006-10-09T17:05:51Z")

</div>

Hello Rubyists,

I am struggling to post data over https and would be grateful for any  
help.

I know the server works because I can hit it with curl and get the  
response I expect:

curl -d url

When I try to do the same from my Ruby code the server doesn’t like it.

The URL to hit looks like:

[https://some.host.com/foo/bar.asp?service=someservice](https://some.host.com/foo/bar.asp?service=someservice)

My code looks like:

require ‘net/https’  
require ‘uri’

url =   
uri = URI.parse(url)  
request = Net::HTTP.new(uri.host, uri.port)  
request.use\_ssl = true  
request.verify\_mode = OpenSSL::SSL::VERIFY\_NONE  
data = { ‘key1’ =\> ‘value1’,  
‘key2’ =\> ‘value2’,  
‘keyn’ =\> ‘valuen’ }.map {|k,v| “#{URI.escape  
(k.to\_s)}=#{URI.escape(v.to\_s)}”}.join(‘&’)  
response = request.post(“#{uri.path}?#{uri.query}”, data)

When I tried this with curl, I used the value of data as my url-  
encoded form data and everything was fine.

Any ideas?

Thanks in advance,  
Andy S.

---

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 9, 2006, 5:54pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/2 "2006-10-09T17:54:00Z")

</div>

On Tue, Oct 10, 2006 at 12:04:26AM +0900, Andrew S. wrote:

> When I try to do the same from my Ruby code the server doesn’t like it.  
> url =   
> When I tried this with curl, I used the value of data as my url-  
> encoded form data and everything was fine.
> 
> Any ideas?

Try using Mechanize. You can post forms over https with it, and you  
won’t have to deal with encoding query parameters or setting up  
Net::HTTP.

[http://mechanize.rubyforge.org/](http://mechanize.rubyforge.org/)

–Aaron

---

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 9, 2006, 6:54pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/3 "2006-10-09T18:54:12Z")

</div>

Hi,

> > I am struggling to post data over https and would be grateful for any  
> > help.

> Try using Mechanize. You can post forms over https with it, and you  
> won’t have to deal with encoding query parameters or setting up  
> Net::HTTP.

OK, but I would still like to understand what’s wrong with my code so  
I can learn and improve.

Thanks anyway for the pointer,

Andy S.

---

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 9, 2006, 7:02pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/4 "2006-10-09T19:02:14Z")

</div>

On 10/9/06, Andrew S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> OK, but I would still like to understand what’s wrong with my code so  
> I can learn and improve.

Its admirable, but I think Net::HTTP is a bit too low level, and  
requires a  
bit too much HTTP knowledge for many applications. Mechanize seems  
to be the business, and possibly Hpricot might also have similar  
functionality.

I found Net::HTTP a bit too frustrating to use and the lack of good  
examples  
bothered me a lot. It seems that the real work along these lines  
(programmatic  
HTTP/HTML processing) is done with Mechanize & Hpricot.

---

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 9, 2006, 7:52pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/5 "2006-10-09T19:52:23Z")

</div>

On Tue, 10 Oct 2006, Andrew S. wrote:

> uri = URI.parse(url)  
> data and everything was fine.
> 
> Any ideas?
> 
> Thanks in advance,  
> Andy S.

fortytwo :/var/www/html \> cat form.cgi  
#! /usr/local/bin/ruby  
require ‘cgi’  
require ‘yaml’  
(c=CGI.new).out(“type” =\> “text/plain”){ c.params.to\_yaml }

fortytwo :/var/www/html \> cat post.rb  
require ‘cgi’  
require ‘net/https’  
require ‘uri’

url = ARGV.shift || “[https://fortytwo.merseine.nu/form.cgi](https://fortytwo.merseine.nu/form.cgi)”  
uri = URI.parse url

data = {  
‘key1’ =\> ‘value1’,  
‘key2’ =\> ‘value2’,  
‘keyn’ =\> ‘valuen’,  
}

e = lambda{|x| CGI.escape x}  
q = lambda{|h| h.map{|k,v| [e[k], e[v]].join ‘=’}.join(‘&’)}  
form = q[data]

class HTTPS \< Net::HTTP  
def initialize \*a, &b  
super  
self.use\_ssl = true  
self.verify\_mode = OpenSSL::SSL::VERIFY\_NONE  
end  
end

HTTPS.start(uri.host, uri.port) do |https|  
res = https.post uri.path, form  
puts res.body  
end

## fortytwo :/var/www/html \> ruby post.rb [https://fortytwo.merseine.nu/form.cgi](https://fortytwo.merseine.nu/form.cgi)

key1:

- value1  
key2:
- value2  
keyn:
- valuen

-a

---

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 10, 2006, 3:16pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/6 "2006-10-10T15:16:51Z")

</div>

Hello again,

> ## fortytwo :/var/www/html \> ruby post.rb [https://fortytwo.merseine.nu/](https://fortytwo.merseine.nu/) form.cgi
> 
> key1:
> 
> - value1  
> key2:
> - value2  
> keyn:
> - valuen

Many thanks for the pointers to Mechanize, Hpricot and this fully-  
fledged code sample. I do appreciate your taking the time to  
respond, and so quickly.

Adapting my code in the light of Ara’s code above, with  
‘self.set\_debug\_output $stderr’ added to the HTTPS initialize method,  
I compared the client-server conversation with curl’s, which I knew  
worked. This led me to discover that the server expected the content-  
type header to be set to ‘application/x-www-form-urlencoded’.

For the record, here’s the code which works with the server I am  
talking to (Protx’s payment gateway). Now that I have it working,  
I’ll take Aaron and Richard’s advice and look into replacing it with  
higher-level code using Mechanize.

require ‘net/https’  
require ‘uri’

class HTTPS \< Net::HTTP  
def initialize \*a, &b  
super  
self.use\_ssl = true  
self.verify\_mode = OpenSSL::SSL::VERIFY\_NONE  
#self.set\_debug\_output $stderr  
end  
end

url = ‘[https://some.host.com/foo/bar.asp?service=someservice](https://some.host.com/foo/bar.asp?service=someservice)’  
uri = URI.parse url  
data = {  
‘key1’ =\> ‘value1’,  
‘key2’ =\> ‘value2’,  
‘keyn’ =\> ‘valuen’  
}  
e = lambda {|x| URI.escape x}  
q = lambda {|h| h.map {|k,v| [e[k], e[v]].join ‘=’}.join(‘&’)}  
form = q[data]  
HTTPS.start(uri.host, uri.port) do |https|  
response = https.post(“#{uri.path}?#{uri.query}”, form, {‘content-  
type’ =\> ‘application/x-www-form-urlencoded’})  
puts response.body  
end

Regards,  
Andy S.

---

<div class="post-metadata">

**Author:** ![starmaerker](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/e9bcb4/32.png) [@starmaerker](https://www.ruby-forum.com/u/starmaerker)\
**Post date:** [October 10, 2006, 5:15pm UTC](https://www.ruby-forum.com/t/posting-over-https/71515/7 "2006-10-10T17:15:33Z")

</div>

On 10/10/06, Andrew S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > keyn:  
> > type header to be set to ‘application/x-www-form-urlencoded’.  
> > def initialize \*a, &b  
> > ‘key1’ =\> ‘value1’,  
> > end
> 
> Regards,  
> Andy S.
> 
> Thanks for sharing this, I could not help you out as I lost my old POST  
> code, now I’ll just adapt your’s (thx Ara) 🙂  
> Robert

–  
Deux choses sont infinies : l’univers et la bêtise humaine ; en ce qui  
concerne l’univers, je n’en ai pas acquis la certitude absolue.

- Albert Einstein