Bind NICs to Mechanize

Ruby
1

Hi,

I may shooting a wrong question, but I saw the same question here that
left unanswered and gone 60 days more, so I want to raise it once again.

I have 3 NIFs :

1.　PPP1 (111.111.111.111)：PPTP to VPN
　2.　eth2 (192.168.0.2)　　：LAN
　3.　ppp3 (333.333.333.333)：PPPoE to ASDL

and want to bind a specific interface to Mechanize agent somehow like :

agent.bind(Socket.sockaddr_in(80, “111.111.111.111”))
　agent.bind(Socket.sockaddr_in(80, “192.168.0.2”))
　agent.bind(Socket.sockaddr_in(80, “333.333.333.333”))

like the way Socket does. But I haven’t have any luck yet.

Did anyone try yet or shouldn’t expect Mechanize package working in this
way?

any suggestion would be appreciated !!

Thanks

John

2

On 3/29/2010 9:26 PM, John Y. wrote:

any suggestion would be appreciated !!

Thanks

John

It doesn’t work that way. Mechanize is a client, not a server. The NIC
used is decided by the destination IP and the routes in your Operating
System. Only server sockets can be bound to one or more IPs.

3

Walton H. wrote:

It doesn’t work that way. Mechanize is a client, not a server. The NIC
used is decided by the destination IP and the routes in your Operating
System. Only server sockets can be bound to one or more IPs.

Thank you Walton,

I am afraid that your comment would be the most the answer after
spending quite sometime searching for solution. You helped me to put the
period to my approach.

It may not be Ruby specific problem, but let me explain the situation.
My client has 3 accounts on B2B site and all 3 accounts need to accessed
with specific IP address (3 different offices, 3 different IPs). I was
asked to write script to show consolidated inventory list over those 3
accounts.

So I sat up a Debian with 3 NICs, VPN/LAN/PPP to 3 different offices
network and “proxying” 3 different IPs to gather 3 account inventory
level and consolidate them “simultaneously”. Right now to change
interface, I manually change routing to NIFs when switching account, but
the process is slow and it’s accessing account one by one, even though
the connections are simultaneously available, causing slight difference
of the inventory level by the time 3rd account was checked out.

My goal is to launch 3 threads of Mechanize agents to access the same
URI (inventory.b2b.com/login) through 3 different NIFs (ppp1/eth2/ppp3)
.

As you suggested, if the client could not control the choice of NIFs,
then I thought it should be done with NAT, Proxy or some sort of
routing, but I am running out of idea.

Could you give me a hint where should I look up to ?

Thank you again in advance.

John

4

Good Evening,

On Tue, Mar 30, 2010 at 9:58 PM, John Y. [email protected]
wrote:

My goal is to launch 3 threads of Mechanize agents to access the same
URI (inventory.b2b.com/login) through 3 different NIFs (ppp1/eth2/ppp3)
.

Your best bet might be to set up 3 proxies and have each of the
Mechanize
clients hit a different proxy (using set_proxy for each instance) which
would push each request to the proxy’s assigned server.

Something nice and simple like this should work (
em-proxy/examples/line_interceptor.rb at master · igrigorik/em-proxy · GitHub)

  • you would need 3 changes to this file
  1. Obviously line 3 needs to have a different port for each of the
    proxies
  2. Line 4 would need to point to a different ip corresponding to each of
    the
    three instances of inventory.b2b.com and the port would be 80 obviously
  3. You don’t want any substitution here so line 12 would simply be resp

In theory that should push the requests to the appropriate server for
you
without a huge amount of work. I think you’ll find em-proxy extremely
flexible in terms of getting this task done for you. As with anything
that
messes with network traffic - there is some overhead - but you seem much
more concerned with concurrency than milliseconds of delay.

John

5

Evening,

If you wish to run the example as is you need to run all three lines

ruby examples/appserver.rb 81 - this runs a webserver which is
referenced in
line 4 of the line_interceptor example as the final destination. Think
of it
as a tiny tiny version of the webserver you are trying to connect to.

ruby examples/line_interceptor.rb - this fires up the proxy - the error
you
are getting is that it is trying to connect to the server started just
above
which you are I’m guessing not running

curl localhost - this uses curl to make a request that hits the proxy
that
then sends the request off to the appserver.rb to get the response -
modifies the result and returns it to you.

You will need rack installed to have appserver.rb work.

Hope this helps move you along…

John

6

Thank you, Mr. John W Higgins.

I am not familiar with Ruby’s server/proxy stuff, so I simply follow
the example (presented in the page you recommended below) before
applying my actual case.

John W Higgins wrote:

Something nice and simple like this should work (

em-proxy/examples/line_interceptor.rb at master · igrigorik/em-proxy · GitHub)

I am not fully understand em-proxy but when executing the example ;

ruby examples/appserver.rb 81

ruby examples/line_interceptor.rb

And I encountered the error when running line_interceptor ;

ruby line_interceptor.rb

/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:572:in
`start_tcp_server’: no acceptor (RuntimeError)

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:572:in
`start_server’

from
/usr/lib/ruby/gems/1.8/gems/em-proxy-0.1.2/lib/…/lib/em-proxy/proxy.rb:10:in
`start’

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`call’

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run_machine’

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run’

from
/usr/lib/ruby/gems/1.8/gems/em-proxy-0.1.2/lib/…/lib/em-proxy/proxy.rb:5:in
`start’

from line_interceptor.rb:6

Only modification I made was :

original> require ‘lib/emproxy’

modified>require ‘rubygems’

modified>require ‘em-proxy’

I think the problem is in either “appserver.rb” or eventmachine’s port
specification and googled to find :

http://whatdoesthiserrormean.com/errors/93

which currently under maintenance. I am sure it’s simple installation
matter, so I would wait for while until the site back and find reported
errors, but if you could suggest my em-proxy setting-up problem, I will
appreciate very much.

John

7

Thank you, Thank you, Mr. John W Higgins for very quick response.

About Curl, I could not proceed there yet, but as far as curl and rack
gem installation, it was smooth and I did not see any problem.

ruby examples/line_interceptor.rb - this fires up the proxy - the error you
are getting is that it is trying to connect to the server started just above
which you are I’m guessing not running

I tested with FireFox ;

request: http://localhost:81/
respose: hello world:

So I assumed the server was working, but also I saw other comments
explaining the cause of error:

eventmachine-0.12.10/lib/eventmachine.rb:572:in `start_tcp_server’:
no acceptor (RuntimeError)

might be that the port (81) could be used. It does not make sense to
me, but somehow I might mess my port use or some connection trouble with
em-proxy. Can you think of any other possibility ?

8

Thank you Mr.Alexey B.

I think you are correct as some people suggested me the same.

you can use proxy solution for specific sites or
you can write paths for selected sites in gateway table on your computer.

I guess you are suggesting to deal with iptables (and/or proxy server)
and route commands to fire the particular packets to targeted site.

At this point, I prefer 100% ruby solution, but until I can make
em-proxy working, I will try debian’s command line and independent proxy
server and report back here.

John

9

Hi. You cannot bind the client socket. You must bind only server
socket which recieves client connections. In your case you can use
proxy solution for specific sites or you can write paths for selected
sites in gateway table on your computer.

On 4/1/10, John W Higgins [email protected] wrote:

which you are I’m guessing not running

–
Sent from my mobile device

With regards,
Alexei Bovanenko

10

Dear Mr.Martin DeMello.

I was happened to be root so I did not have that problem, but I was not
aware of the point you posted. I would be easily mistaken. I will pay
more attention for handling&choosing the port.

Thank you for your remark.

John

Martin DeMello wrote:

Also note that if you do not run the proxy as root, it will not be
able to listen on any port lower than 1024

martin

11

On Thu, Apr 1, 2010 at 9:52 PM, John W Higgins [email protected]
wrote:

Are you running a web server on this system? Does firefox return anything
for http://localhost:80 ? If so then that is the problem - you will need to
change the port in the example from 80 to something else. I might have
misread your issue last night. So if you changed the proxy’s port from 80 to
lets say 82 then look at http://localhost:82 you should be getting the
response from the proxy.

Also note that if you do not run the proxy as root, it will not be
able to listen on any port lower than 1024

martin

12

Dear John W Higgins,

Are you running a web server on this system?

Yes, you are right. I was running Apache2 and after stopping Apache2,
em-proxy example works fine.

Now I do not know how to route the packet from this proxy to specified
NIF ( say, ppp1 ). I expected to be able to set em-proxy something like
Squid proxy server;

Squid setting
http_port 0.0.0.0:81
acl localhost81 src 127.0.0.1/255.255.255.255:81
tcp_outgoing_address ppp1_ip localhost81

I’m not sure if the above is correct for Squid or need additional route
setting, but what I need is that em-proxy specify outgoing ip to route
the packet to specified NIF.

Would you tell me if it’s the correct direction and if so, how can I
set these settings on em-proxy ?

Thank you very much.

John

13

Morning,

On Thu, Apr 1, 2010 at 12:19 AM, John Y. [email protected]
wrote:

I tested with FireFox ;

request: http://localhost:81/
respose: hello world:

Are you running a web server on this system? Does firefox return
anything
for http://localhost:80 ? If so then that is the problem - you will need
to
change the port in the example from 80 to something else. I might have
misread your issue last night. So if you changed the proxy’s port from
80 to
lets say 82 then look at http://localhost:82 you should be getting the
response from the proxy.

John