Using an HTTP proxy to record web traffic

As part of debugging WWW::Mechanize, I realized that the best way to
compare what my browser is doing to what mechanize is doing would be to
compare their HTTP transactions incoming and outgoing. What I want is
an HTTP proxy that can record the transactions and also supports SSL.

The Ruby project doesn’t have any code
http://rubyforge.org/projects/rpp/

This free program doesn’t support SSL, and costs $600.

This doesn’t support SSL but is supposed to soon.
http://kevinlangdon.com/serviceCapture/

Any other suggestions?

Dan K. wrote:

As part of debugging WWW::Mechanize, I realized that the best way to
compare what my browser is doing to what mechanize is doing would be to
compare their HTTP transactions incoming and outgoing. What I want is
an HTTP proxy that can record the transactions and also supports SSL.

MouseHole is a scriptable proxy. http://mousehole.rubyforge.org/

A script to record traffic would a cinch. Basically,

  1. Download MouseHole from the above link. Instructions for running it
    are there as well.

  2. Add a script named `proxyrecord.user.rb’ to your
    ~/.mouseHole/userScripts/ directory.
    Contains:

    MouseHole.script do
        name "Proxy Record"
        namespace "http://yourdomain.com/"
        description "Records all traffic, very snoopy."
        include_match %r!http://.*!
        version "1.0"
    
        rewrite do |req, res|
            # store contents of `req' and `res' in the database or a
            # log file or something
        end
    end
    
  3. In your browser, setup MouseHole as your proxy
    (http://127.0.0.1:37004/)

  4. Visit http://127.0.0.1:37004/ and activate the Proxy Record script by
    checking its box.

As for SSL, I don’t think proxies can actually intercept unencrypted
SSL. It uses the CONNECT method.

_why

Very helpful, why, thanks. I presume I would need to use something
like WATIR to see the unencrypted contents of an SSL connection by
getting it directly from IE.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs