Request.env and accessing raw header requests

Rails
1

I’d like to get the raw http request from a client so I can capture it

  • warts and all.

I know it’s possible to access the attributes from within a request
using request.env (e.g. request.env[“HTTP_ACCEPT”] to get the accept
header). However, I want the actual raw header sent to the server so I
can accurately spoof the client. Building a request from the various
request.env attributes may miss things (like whether the client
incorrently sends “user_agent” or “user-agent”) as I assume it does
some housekeeping (plus there could be linefeed issues etc that i’d
like to capture). In essence I want the raw!

I believe it is possible to access what is sent to WebBrick through
the doGet function, but is it possible to abstract it so it’ll work on
any server environment and can I get direct access to the header
directly from my application code?

Any ideas?

Thanks,

-Paul

2

I have an app that captures this and you are right, clients send some
weird headers.

I have a logs and log_details where I store everything single header
for every single request.

I gather the data with a simple helper method that doubles as a hit
counter (yes, the client wanted a hit counter, how lame)

…

controller.request.env.each do |header|
log_detail = LogDetail.new
log_detail.log_id = log.id
log_detail.detail_key = header[0]
log_detail.detail_value = header[1]
log_detail.save
end
…

And here’s some of the headers I get…

mysql> describe log_details;
±-------------±-------------±-----±----±--------±---------------+
| Field | Type | Null | Key | Default | Extra |
±-------------±-------------±-----±----±--------±---------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| log_id | int(11) | NO | | | |
| detail_key | varchar(255) | YES | MUL | NULL | |
| detail_value | varchar(255) | YES | | NULL | |
±-------------±-------------±-----±----±--------±---------------+
4 rows in set (0.01 sec)

mysql> select detail_key from log_details group by detail_key;
±------------------------------+
| detail_key |
±------------------------------+
| CONTENT_LENGTH |
| CONTENT_TYPE |
| DOCUMENT_ROOT |
| FCGI_ROLE |
| GATEWAY_INTERFACE |
| HTTP_AAAAAAAAAAAAAAA | <-- crazy
| HTTP_ACCEPT |
| HTTP_ACCEPT_CHARSET |
| HTTP_ACCEPT_ENCODING |
| HTTP_ACCEPT_ENCODXNG |
| HTTP_ACCEPT_LANGUAGE |
| HTTP_CACHE_CONTROL |
| HTTP_CLIENT_IP |
| HTTP_COMPAQ_HTTPSERVERVERSION |
| HTTP_COMPAQ_WBEM_USERNAME |
| HTTP_CONNECTION |
| HTTP_COOKIE |
| HTTP_COOKIE2 |
| HTTP_DATE |
| HTTP_EXPECT |
| HTTP_EXTENSION |
| HTTP_FILE_REDIRECT |
| HTTP_FORWARDED |
| HTTP_FOUR11REMOTEIP |
| HTTP_FROM |
| HTTP_HELPAGENT |
| HTTP_HOST |
| HTTP_IF_MODIFIED_SINCE |
| HTTP_KEEP_ALIVE |
| HTTP_MAX_FORWARDS |
| HTTP_MT_PROXY_ID |
| HTTP_NOVINET |
| HTTP_NPFREFR |
| HTTP_OAS_IP |
| HTTP_OXPRY_CONNECTION |
| HTTP_PRAGMA |
| HTTP_PROFILE |
| HTTP_PROXY_CONNECTION |
| HTTP_RANGE |
| HTTP_REFERER |
| HTTP_SWF_HDR_MSG |
| HTTP_TE |
| HTTP_UA_COLOR |
| HTTP_UA_CPU |
| HTTP_UA_OS |
| HTTP_UA_PIXELS |
| HTTP_UA_VOICE |
| HTTP_USER_AGENT |
| HTTP_VIA |
| HTTP_WEFERER |
| HTTP_WSHOST |
| HTTP_WSIP |
| HTTP_XXXXXX |
| HTTP_XXXXXXX |
| HTTP_XXXXXXXXXXXXXXX |
| HTTP_X_AIM |
| HTTP_X_AOL_AUTH |
| HTTP_X_BLUECOAT_VIA |
| HTTP_X_BROWSER_TIMESTAMP |
| HTTP_X_CACHEBUSTER |
| HTTP_X_COMPRESSION |
| HTTP_X_EGZ |
| HTTP_X_FILTERGATE_REQUEST |
| HTTP_X_FORWARDED |
| HTTP_X_FORWARDED_FOR |
| HTTP_X_ICAP_VERSION |
| HTTP_X_IMFORWARDS |
| HTTP_X_IWPROXY_NESTING |
| HTTP_X_IWX |
| HTTP_X_MCPROXYFILTER |
| HTTP_X_MOZ |
| HTTP_X_NAVID |
| HTTP_X_NOVINET |
| HTTP_X_PAGEVIEW |
| HTTP_X_PROXY_ID |
| HTTP_X_PS3_BROWSER |
| HTTP_X_PTAG |
| HTTP_X_TEAMSITE_PREREMAP |
| HTTP_X_UP_DEVCAP_CHARSET |
| HTTP_X_UP_DEVCAP_IMMED_ALERT |
| HTTP_X_UP_DEVCAP_ISCOLOR |
| HTTP_X_UP_DEVCAP_MAX_PDU |
| HTTP_X_UP_DEVCAP_MSIZE |
| HTTP_X_UP_DEVCAP_NUMSOFTKEYS |
| HTTP_X_UP_DEVCAP_SCREENCHARS |
| HTTP_X_UP_DEVCAP_SCREENDEPTH |
| HTTP_X_UP_DEVCAP_SCREENPIXELS |
| HTTP_X_UP_DEVCAP_SOFTKEYSIZE |
| HTTP_X_UP_FAX_ACCEPTS |
| HTTP_X_UP_FAX_LIMIT |
| HTTP_X_UP_SUBNO |
| HTTP_X_UP_UPLINK |
| HTTP_X_UP_WAPPUSH_SECURE |
| HTTP_X_UP_WAPPUSH_UNSECURE |
| HTTP_X_UP_WTLS_INFO |
| HTTP_X_WAP_CLIENT_IP |
| HTTP_X_WAP_PROFILE |
| HTTP_YAHOOREMOTEIP |
| HTTP________ |
| HTTP___________ |
| HTTP________________ |
| HTTP_________________ |
| nokeepalive |
| PATH |
| QUERY_STRING |
| RAW_POST_DATA |
| REDIRECT_nokeepalive |
| REDIRECT_QUERY_STRING |
| REDIRECT_STATUS |
| REDIRECT_URL |
| REMOTE_ADDR |
| REMOTE_PORT |
| REQUEST_METHOD |
| REQUEST_URI |
| SCRIPT_FILENAME |
| SCRIPT_NAME |
| SERVER_ADDR |
| SERVER_ADMIN |
| SERVER_NAME |
| SERVER_PORT |
| SERVER_PROTOCOL |
| SERVER_SIGNATURE |
| SERVER_SOFTWARE |
±------------------------------+
123 rows in set (0.45 sec)

–
James M.

3

This is really useful. Many thanks James - I’ll let you know how I get
on…

-Paul

4

Try request.env[‘RAW_POST_DATA’].

bijah

5

Hi James,

As I said, your help was really appreciated, but I think I still have
an issue.

After some further investigation it looks like request.env accesses
the CGI environment variables (a la
http://hoohoo.ncsa.uiuc.edu/cgi/env.html).
I need access to the originating HTTP request sent to the server… I
wonder if anyone out there knows a way to get access to this? I know
that in php it is possible using apache_request_headers();

-Paul

6

@request.raw_post is a wrapper for request.env[‘RAW_POST_DATA’].

Bijah