# How to properly format data when using TCPSocket.send()

**URL:** <https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661>\
**Category:** Ruby\
**Created:** [December 29, 2006, 6:44pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661 "2006-12-29T18:44:08Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 6:44pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/1 "2006-12-29T18:44:08Z")

</div>

It would be greatly appreciated if someone would clue me in to what I am  
doing wrong here. I am trying to send this data below to httpd(Apache)  
however I am receiving a http response code of 400 (bad request).  
Apache is logging “request failed: error reading the headers” in  
error\_log, however if I replace the \r\n with \x0d\x0a and send this  
through netcat to Apache I get the proper response code I am looking  
for.

http\_socket = TCPSocket.new("#{i}", 80); http\_socket.send(“PUT  
/puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:  
close\r\nContent-type: text/html\r\nContent-Length:  
309\r\nSee Spot Run!See Spot  
Run!See Spot Run!\r\n\r\n”, 0)

Thank you,

Nathan

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 6:53pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/2 "2006-12-29T18:53:07Z")

</div>

On 12/29/06, Nathan T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> http\_socket = TCPSocket.new(“#{i}”, 80); http\_socket.send("PUT  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

Why are you putting HTML content in the header block? Try putting it  
after  
the \r\n\r\n that terminates the header. Also, you want to get the  
content  
length value correct. If that still doesn’t work, you could try POST  
instead  
of PUT (except you already implied that PUT works correctly on this  
Apache  
server with netcat).

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 6:57pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/3 "2006-12-29T18:57:09Z")

</div>

On 29.12.2006 18:44, Nathan T. wrote:

> /puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:  
> close\r\nContent-type: text/html\r\nContent-Length:  
> 309\r\nSee Spot Run!See Spot  
> Run!See Spot Run!\r\n\r\n", 0)

You need “\r\n\r\n” to separate the header from the body. Your code  
shows just a single pair.

Btw, why don’t you use Net::HTTP?

```
robert
```

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 7:57pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/4 "2006-12-29T19:57:25Z")

</div>

On 12/29/06, Nathan T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > Btw, why don’t you use Net::HTTP?  
> > METHOD = ‘PUT’

I think you would do well to do some basic reading on HTTP. To  
understand how headers work, you might even read RFC-822. HTTP doesn’t  
use multiple TCP connections (not in the sense of relating them to  
each other, at any rate), and I don’t know what you mean when you say  
that it uses multiple TCP “sessions.”

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 7:29pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/5 "2006-12-29T19:29:15Z")

</div>

Robert K. wrote:

> On 29.12.2006 18:44, Nathan T. wrote:
> 
> > /puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:  
> > close\r\nContent-type: text/html\r\nContent-Length:  
> > 309\r\nSee Spot Run!See Spot  
> > Run!See Spot Run!\r\n\r\n", 0)
> 
> You need “\r\n\r\n” to separate the header from the body. Your code  
> shows just a single pair.
> 
> Btw, why don’t you use Net::HTTP?
> 
> robert

I tried using “\r\n\r\n”, but I’m still having some problems due to the  
way the HTTP protocol uses multiple TCP sessions and my lack of  
understanding on how to handle that with Net::TCPSocket. Looks I will  
have to use Net:HTTP. I am new to OO programming, and I don’t quite  
understand this portion of the Net::HTTP::Put documentation.

Constants  
METHOD = ‘PUT’  
REQUEST\_HAS\_BODY = true  
RESPONSE\_HAS\_BODY = true

I know that I should be passing a URI and some content to include in  
the “PUT” request but I do not see any explanation of how to do that.  
How do I know what parameters this class accepts?

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 8:28pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/6 "2006-12-29T20:28:06Z")

</div>

Francis C. wrote:

> On 12/29/06, Nathan T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > > Btw, why don’t you use Net::HTTP?  
> > > METHOD = ‘PUT’
> 
> I think you would do well to do some basic reading on HTTP. To  
> understand how headers work, you might even read RFC-822. HTTP doesn’t  
> use multiple TCP connections (not in the sense of relating them to  
> each other, at any rate), and I don’t know what you mean when you say  
> that it uses multiple TCP “sessions.”

I mean that a new TCP session is created for every HTTP request/response  
pair. For example if you were to send two different GET requests you  
would use two TCP sessions(3-way hand shake, and graceful close).  
However, rethinking the situation this should not be a problem for what  
I am trying to do. Even if the server responds with multiple response  
codes it should be ok since they would be transported via the same TCP  
session as the initial request sent from the client. Unfortunately the  
packets generated and sent out on the wire from my code does not seem to  
be waiting for the HTTP response data before it sends out a request for  
a graceful close of the TCP session.

03:27:40.315919 IP 127.0.0.1.51200 \> 127.0.0.1.80: P 1:257(256) ack 1  
win 257 \<nop,nop,timestamp 37976322 37976315\>  
0x0000: 4500 0134 529f 4000 4006 e922 7f00 0001  
E…4R.@.@…"…  
0x0010: 7f00 0001 c800 0050 5f61 2a67 5fe0 3c72  
…P\_a_g\_.\<r  
0x0020: 8018 0101 ff28 0000 0101 080a 0243 7902  
…(…Cy.  
0x0030: 0243 78fb 5055 5420 2f70 7574 7465 7374  
.Cx.PUT./puttest  
0x0040: 2e68 746d 6c20 4854 5450 2f31 2e30 0d0a  
.html.HTTP/1.0…  
0x0050: 486f 7374 3a20 3132 372e 302e 302e 310d  
Host:.127.0.0.1.  
0x0060: 0a41 6363 6570 743a 2069 6d61 6765 2f67  
.Accept:.image/g  
0x0070: 6966 2c20 696d 6167 652f 782d 7862 6974  
if,.image/x-xbit  
0x0080: 6d61 702c 2069 6d61 6765 2f6a 7065 672c  
map,.image/jpeg,  
0x0090: 2069 6d61 6765 2f70 6a70 6567 2c20 2a2f  
.image/pjpeg,._/  
0x00a0: 2a0d 0a43 6f6e 7465 6e74 2d74 7970 653a  
\*…Content-type:  
0x00b0: 2074 6578 742f 6874 6d6c 0d0a 436f 6e74  
.text/html…Cont  
0x00c0: 656e 742d 4c65 6e67 7468 3a20 3330 390d  
ent-Length:.309.  
0x00d0: 0a0d 0a3c 6874 6d6c 3e3c 6865 6164 3e53  
…S  
0x00e0: 6565 2053 706f 7420 5275 6e21 3c2f 6865  
ee.Spot.Run!\</he  
0x00f0: 6164 3e3c 7469 746c 653e 5365 6520 5370  
ad\>See.Sp  
0x0100: 6f74 2052 756e 213c 2f74 6974 6c65 3e3c  
ot.Run!\<  
0x0110: 626f 6479 3e53 6565 2053 706f 7420 5275  
body\>[See.Spot.Ru](http://See.Spot.Ru)  
0x0120: 6e21 3c2f 626f 6479 3e3c 2f68 746d 6c3e  
n!  
0x0130: 0d0a 0d0a …  
03:27:40.316702 IP 127.0.0.1.80 \> 127.0.0.1.51200: . ack 257 win 265  
\<nop,nop,timestamp 37976322 37976322\>  
0x0000: 4500 0034 4570 4000 4006 f751 7f00 0001  
E…4Ep@.@…Q…  
0x0010: 7f00 0001 0050 c800 5fe0 3c72 5f61 2b67  
…P…_.\<r\_a+g  
0x0020: 8010 0109 91bb 0000 0101 080a 0243 7902  
…Cy.  
0x0030: 0243 7902 .Cy.  
03:28:17.925697 IP 127.0.0.1.51200 \> 127.0.0.1.80: F 257:257(0) ack 1  
win 257 \<nop,nop,timestamp 38013926 37976322\>  
0x0000: 4500 0034 52a0 4000 4006 ea21 7f00 0001  
E…4R.@.@…!..  
0x0010: 7f00 0001 c800 0050 5f61 2b67 5fe0 3c72  
…P\_a+g_.\<r  
0x0020: 8011 0101 fedd 0000 0101 080a 0244 0be6  
…D…  
0x0030: 0243 7902 .Cy.  
03:28:17.932379 IP 127.0.0.1.80 \> 127.0.0.1.51200: P 1:468(467) ack 258  
win 265 \<nop,nop,timestamp 38013932 38013926\>  
0x0000: 4500 0207 4571 4000 4006 f57d 7f00 0001  
E…Eq@.@…}…  
0x0010: 7f00 0001 0050 c800 5fe0 3c72 5f61 2b68  
…P…\_.\<r\_a+h  
0x0020: 8018 0109 fffb 0000 0101 080a 0244 0bec  
…D…  
0x0030: 0244 0be6 4854 5450 2f31 2e31 2034 3033  
.D…HTTP/1.1.403  
0x0040: 2046 6f72 6269 6464 656e 0d0a 4461 7465  
.Forbidden…Date  
0x0050: 3a20 4672 692c 2032 3920 4465 6320 3230  
:.Fri,.29.Dec.20  
0x0060: 3036 2030 393a 3237 3a34 3020 474d 540d  
06.09:27:40.GMT.  
0x0070: 0a53 6572 7665 723a 2041 7061 6368 652f  
.Server:.Apache/

See above where the client sends out a segment with the FIN bit set  
before the response from the Apache server arrives. When I run this  
code ruby just hangs. I suppose awaiting the HTTP response that will  
never be sent up the stack.

This is the code I was attempting to read the response with.

puts http\_socket.readlines; http\_socket.close

This code works with two other requests I’ve setup, but not with my  
attempt to send the PUT request. Perhaps there is something I am not  
understanding here?

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 8:40pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/7 "2006-12-29T20:40:38Z")

</div>

> See above where the client sends out a segment with the FIN bit set  
> before the response from the Apache server arrives. When I run this  
> code ruby just hangs. I suppose awaiting the HTTP response that will  
> never be sent up the stack.

I guess a good question would be “does ruby’s TCPSocket take advantage  
of a TCP half close and still received data after it sends a FIN  
advising the other side that it will not be sending any more data?” I  
have consulted the “TCP/IP Illustrated”, but it is apparently a decision  
that is up to the application developer that does the implementation.  
Perhaps my understanding is skewed and this is even a deeper question.  
Does Ruby’s Net::TCPSocket class use the underlying OS’s TCP/IP stack?  
I think that it would, but I am not yet skilled in the Ruby Way, or it’s  
underlying code. I am still consulting RFC 2822 so my next post should  
include more educated questions.

- Nathan

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 30, 2006, 2:10am UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/8 "2006-12-30T02:10:38Z")

</div>

On Dec 29, 2006, at 09:44, Nathan T. wrote:

> http\_socket = TCPSocket.new(“#{i}”, 80); http\_socket.send(“PUT  
> /puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:  
> close\r\nContent-type: text/html\r\nContent-Length:  
> 309\r\nSee Spot Run!See Spot  
> Run!See Spot Run!\r\n\r\n”, 0)

the net/http library handles this much better:

require ‘net/http’

data = “See Spot Run!See Spot Run!\</  
title\>See Spot Run!”

uri = URI.parse ‘[http://example.com/puttest.html](http://example.com/puttest.html)’

Net::HTTP.start uri.host, uri.port do |http|  
http.post uri.request\_uri, data, ‘Content-Type’ =\> ‘text/html’  
end

–  
Eric H. - removed\_email\_address@domain.invalid - [http://blog.segment7.net](http://blog.segment7.net)

I LIT YOUR GEM ON FIRE!

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [December 29, 2006, 10:52pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/9 "2006-12-29T22:52:02Z")

</div>

Nathan T. wrote:

> /puttest.html HTTP/1.0\r\nHost: #{i}\r\nConnection:  
> close\r\nContent-type: text/html\r\nContent-Length:  
> 309\r\nSee Spot Run!See Spot  
> Run!See Spot Run!\r\n\r\n", 0)

Looks like you found your answer, but one tool for sockets programming,  
if you aren’t already using it, is ‘nc’. It can be used as a ‘telnet’  
replacement but more relevant to your case, you can make it into a web  
(or any) server and see every byte your client is sending:

nc -l -k 3001

…will start your ‘server’. Then you set your client to connect to  
localhost 3001, you can use nc as a client to test that as well:

echo hello | nc localhost 3001

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [January 3, 2007, 8:09pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/10 "2007-01-03T20:09:31Z")

</div>

> I tried using “\r\n\r\n”, but I’m still having some problems due to the  
> way the HTTP protocol uses multiple TCP sessions and my lack of  
> understanding on how to handle that with Net::TCPSocket.

I just wanted to add some comments about using TCPSocket that might help  
others as myself.

I found out that the problem I was having with TCPSocket was a problem  
with the closing of the session. I was sending some data to craft an  
HTTP SEARCH request using the format client = TCPSocket.new(host, 80);  
data = "SEARCH /vroot/folder/ HTTP/1.1  
Content-Type: text/xml  
Content-Length: 253  
Host: server.tld

\<?xml version=\"1.0\"?\>

\<D:searchrequest xmlns:D = “DAV:”\>  
\<D:sql\>  
SELECT “DAV:contentclass”, “DAV:displayname”  
FROM “/vroot/folder/”  
WHERE “DAV:ishidden” = false  
AND “DAV:isfolder” = false  
\</D:sql\>  
\</D:searchrequest\>"

client.send(data, 0)  
response = client.recvfrom(1024)  
puts response  
client.close

However this would just cause the script to hang at the command line and  
the web server would never send it’s response until I sent an interrupt  
to the running script. I realized through some tcpdump magic that the  
web server would send the response after I killed the script so I  
modified the end of the code to;

client.send(data, 0)  
client.shutdown( how=1 )  
response = client.recvfrom(1024)  
puts response  
client.close

It works… Everyone’s happy. This is just an FYI for anyone running in  
to a similar problem implementing a network client, sorry for the  
verboseness.

- Nathan

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [January 3, 2007, 11:56pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/11 "2007-01-03T23:56:58Z")

</div>

This is so much easier with http-access2. It handles keep alive  
connections, ssl, cookies, headers, pretty much everything you would  
want without having to do all the hard work yourself.

Chris

---

<div class="post-metadata">

**Author:** ![shadowcipher](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@shadowcipher](https://www.ruby-forum.com/u/shadowcipher)\
**Post date:** [January 2, 2007, 8:02pm UTC](https://www.ruby-forum.com/t/how-to-properly-format-data-when-using-tcpsocket-send/80661/12 "2007-01-02T20:02:24Z")

</div>

Thank you for your example Eric. I was having problems figuring out  
how to add the header values until I saw how you did it. I am up and  
running with my script now. Thanks to all.