Newbie Question: Can I setup mongrel as a proxy between a browser and the web?

I am trying to create a proxy which I can route my browser requests
through
in order to gather information on the web page loading. At this time I
am
particularly interested in the many GET requests associated with loading
a
single web page. Note: This does not have anything to do with rails, my
goal
is to run a ruby app in the background, set firefox to proxy all
requests
through it, log those requests, then return the fulfilled request back
to
the browser (i.e. the web page still loads in the browser).

I posted this on a ruby forum, and it was suggested to do something
along
the lines of:

require ‘rubygems’
require ‘mongrel’
require ‘logger’

class HeaderHandler < Mongrel::HttpHandler

@@logger = Logger.new(‘headers.log’)

def process(request,response)
response.start(200) do |head, out|
head[“Content-Type”] = “text/plain”
@@logger.info request.params
end
end
end

server = Mongrel::HttpServer.new(“127.0.0.1”, “2222”)
server.register("/", HeaderHandler.new)
server.register("/favicon.ico", Mongrel::Error404Handler.new(""))
server.run.join

The problem is that this only records the first request (i.e.
google.com,
and not any other requests that are associated with loading that page),
and
the page is never loaded in the browser as the requests is never
actually
proxied.

I have played around with this a bit, and tried changing some things up,
but
I haven’t really come up with anything. If anyone could point me in the
right direction, I would be grateful. Thanks!

Hello there. I’m the one that made that suggestion to you earlier, I
had a feeling it was more than just capturing the headers.

Do you happen to be running apache on the webserver? You can log all
of it and not worry about a proxy. Here are two ways.

  1. mod_dumpio

Put this in your apache config

DumpIOInput On
LogLevel debug

  1. mod_log_forensic

Put this in your apache config

ForensicLog /some/path/to/a/logfile.log

Try MouseHole: http://code.whytheluckystiff.net/mouseHole/

Evan

At 12:39 PM 2/6/2008, [email protected] wrote:

through
the browser (i.e. the web page still loads in the browser).
Hi Arash,

You may have interest in coding a solution to this, for your own
reasons. I totally support those inclinations! :slight_smile: I’m going to go a
little off-topic with an idea though:

Personally, I use “Tamper Data”
(https://addons.mozilla.org/en-US/firefox/addon/966) within FireFox to
handle this type of thing. It’s a most excellent resource: it’s like a
wire sniffer at the HTTP level only. It supports decoding request data
and tracking POST, GET and Ajax stuff.

It’s been invaluable for me in some circumstances - I hope the
reference is helpful.

Steve

p.s. It’s been particularly helpful in trapping precisely certain live
GET & Ajax requests which I then convert into testing requests to
re-create the errors.

Hi Guys,

Thanks for your responses.

Dusty: Im not sure what the server does, but what I am developing is
intended to be a client side testing utility. i.e. all that will be
installed on the machine is a browser, and some programming environment
(i.e.
ruby)

Evan: I saw mousehole yesterday, but I didn’t quite understand how I
could
edit a script to do what I want (for now I am starting off small, and I
just
want to collect header information, but eventually I assume it will be a
whole variety of information I would like to log.). I will try and take
another look at it.

Thanks for your help.

Hi Steve,

I wish I could use something like that (in fact, I often use firebug to
see
what is actually going on), but I am trying to build something that is
browser independent.

Ideally, what I am looking for is something which just listens to all
the
traffic going into and out of a browser (using whatever protocol) and do
some analysis on that traffic (even if that means I need to duplicate
all
the data elsewhere on my disk). but alas, I come from a desktop
background
and I am not that well versed in web related issues. So I suppose I will
just take baby steps (with your guy’s help, if you dont mind.)

–arash

On 7 Feb 2008, at 00:20, Arash N. wrote:

come from a desktop background and I am not that well versed in web
related issues. So I suppose I will just take baby steps (with your
guy’s help, if you dont mind.)

http://www.xk72.com/charles/

“Charles is an HTTP proxy / HTTP monitor / Reverse Proxy that enables
a developer to view all of the HTTP traffic between their machine and
the Internet. This includes requests, responses and the HTTP headers
(which contain the cookies and caching information).”

Browser independent, it intercepts, decodes and displays all browser
<–> server communications, including decoding Flash AMF data. It’s
invaluable for Flash --> server and “ajax” development; sounds like
it’ll do some of what you want, too.

cheers
i

Message-ID:
single web page. Note: This does not have anything to do with rails,
You may have interest in coding a solution to this, for your own
reference is helpful.
http://rubyforge.org/mailman/listinfo/mongrel-users


Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users


Igor C. // POKE // 10 Redchurch Street // E2 7DD // +44 (0)20 7749
5355 // www.pokelondon.com

It might be better if you use a packet sniffer like WireShark or
tcpdump. Mongrel wouldn’t be able to handle non-http traffic.

http://www.wireshark.org/
http://www.tcpdump.org/tcpdump_man.html

Arash, mongrel-esi is a fairly simple proxy server… You could
possibly
modify or extract the bits of it you needed. The bulk of the proxy code
is
in just one file:
http://code.google.com/p/mongrel-esi/source/browse/trunk/lib/esi/handler.rb

It uses Net::HTTP library, forwarding POST and GET requests… might be
helpful…

-Todd

Hi Igor,

sounds like its exactly what I want, provided I can plug into it
programmatically. I’ll take a look today. thanks.

If you are willing to test with a Windows client, there are two very
useful
utilities available: http://www.pocketsoap.com/tcptrace/ and
http://www.pocketsoap.com/tcptrace/pt.aspx (Proxy trace is probably what
you
are looking for. You run it on your local machine, and just configure
your
browser to use localhost as your proxy. It works with any browser, as
they
all support forward proxies.)

Cheers,
Brian

On Feb 6, 2008 7:20 PM, Arash N. [email protected] wrote:

and I am not that well versed in web related issues. So I suppose I will

Subject: [Mongrel] Newbie Question: Can I setup mongrel as a proxy
particularly interested in the many GET requests associated with

It’s been invaluable for me in some circumstances - I hope the
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users


Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

  • Brian G.

http://opensolaris.org/os/project/nycosug/

http://www.genunix.org/wiki/index.php/OpenSolaris_New_User_FAQ