Problem with Net/HTTP

Ruby
1

Hi all,

I am running into a serious problem with a custom library I am building
in Ruby. Basically I am writing a Ruby interface to a custom web
service that we have here at our company. The problem I am running into
is that I am posting the XML document to the service and it is
responding, but net/http is telling me there is no body
(response.body.empty? => true). If I post to the service from a web
form or using PHP I get a response. This only seems to happen when
there is an error with the request, for example the request is missing
an postal code. The service returns an XML doc with the errors but
net/http won’t let me see it. A 200 status code is returned and I can
not find no obvious difference between the headers of a successful and
failed request. I can’t even find a way to see the raw response the
net/http received so I can check to see if it is just parsing it
incorrectly.

I am open to any suggestions at this point as this is a big problem if I
can’t get this working. Thanks

Peer

2

On Fri, May 25, 2007 at 01:21:04AM +0900, Peer A. wrote:

I am running into a serious problem with a custom library I am building
in Ruby. Basically I am writing a Ruby interface to a custom web
service that we have here at our company. The problem I am running into
is that I am posting the XML document to the service and it is
responding, but net/http is telling me there is no body
(response.body.empty? => true).

Can you prove the response is there? The best way to do this is

tcpdump -i eth0 -n -s0 -A tcp port 80

and look at the raw IP packets being sent by Ruby, and the response
coming
back.

If you are on Windoze, then a sniffer like wireshark will do the trick.

I can’t even find a way to see the raw response the
net/http received so I can check to see if it is just parsing it
incorrectly.

There are two possible cases: Ruby is sending the request differently to
PHP, or Ruby is parsing the response differently. Using tcpdump you
should
be able to find out (and compare what PHP is sending with what Ruby is
sending)

HTH,

Brian.

3

On Fri, May 25, 2007 at 02:21:43AM +0900, Peer A. wrote:

That gave me somewhere to do, but I failed to mention that the web
service requires an SSL connection so everything in the dump is
encrypted. Nuts! Not sure what to do now. I may try to generate the
request on my own.

Are you sure you used net/https and enabled ssl? This question seems to
have
arisen several times in the last day or two.

4

Thanks Brian,

That gave me somewhere to do, but I failed to mention that the web
service requires an SSL connection so everything in the dump is
encrypted. Nuts! Not sure what to do now. I may try to generate the
request on my own.

Thanks

Peer

Brian C. wrote:

On Fri, May 25, 2007 at 01:21:04AM +0900, Peer A. wrote:

I am running into a serious problem with a custom library I am building
in Ruby. Basically I am writing a Ruby interface to a custom web
service that we have here at our company. The problem I am running into
is that I am posting the XML document to the service and it is
responding, but net/http is telling me there is no body
(response.body.empty? => true).

Can you prove the response is there? The best way to do this is

tcpdump -i eth0 -n -s0 -A tcp port 80

and look at the raw IP packets being sent by Ruby, and the response
coming
back.

If you are on Windoze, then a sniffer like wireshark will do the trick.

I can’t even find a way to see the raw response the
net/http received so I can check to see if it is just parsing it
incorrectly.

There are two possible cases: Ruby is sending the request differently to
PHP, or Ruby is parsing the response differently. Using tcpdump you
should
be able to find out (and compare what PHP is sending with what Ruby is
sending)

HTH,

Brian.

5

Unfortunately, I am sure. Most of the time there is no problem with the
request. Just the occasional error response is not showing a body.

However, after playing with cURL to get a good look at the whole request
I think I may have found the problem. The failing requests are
returning headers that look like this:

=start
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Fri, 25 May 2007 11:11:28 GMT
Server: Apache
WWW-Authenticate: Basic realm=“none”
Content-Length: 692
Cache-Control: private
Content-Type: text/xml

=end

I think that the HTTP 100 response code is confusing the response
somehow. I can’t find anyone else who has had to deal with this as it
isn’t exactly a redirection. I will keep looking and let you know what
I find.

Thanks again for your help

Peer

Brian C. wrote:

On Fri, May 25, 2007 at 02:21:43AM +0900, Peer A. wrote:

That gave me somewhere to do, but I failed to mention that the web
service requires an SSL connection so everything in the dump is
encrypted. Nuts! Not sure what to do now. I may try to generate the
request on my own.

Are you sure you used net/https and enabled ssl? This question seems to
have
arisen several times in the last day or two.

6

On Fri, May 25, 2007 at 08:22:20PM +0900, Peer A. wrote:

WWW-Authenticate: Basic realm=“none”
Content-Length: 692
Cache-Control: private
Content-Type: text/xml

=end

See RFC 2616 section 8.2.3. The server will only send a 100 Continue if
the
client has explicitly asked for it, and therefore the client should be
able
to handle it.

Another approach you can do is to build a plain TCP to SSL proxy
(e.g. stunnel), and point your client at stunnel with plain HTTP. This
will
allow you to tcpdump the plain connection.

Alternatively, start modifying net/http to show you the raw data sent
and
received into openssl, or make an IO proxy object which does that.

Regards,

Brian.