Eventmachine http

dubstep · January 4, 2012, 3:25pm

Hi,

I am working on a ruby script that download some web page (from a list
of url stored in an array) but it takes some time to do it and when the
script use openuri, the script still waiting until it gets response from
remote server
I found about EventMachine and em-http on internet but I don’t find a
good tutorial for that
what I want is launching simultaneously N web requests and treat the
result in parallel:

launch by 5 url
{
download page
parse page
}

regards

rubix01 · January 4, 2012, 8:09pm

On Wed, Jan 4, 2012 at 6:25 AM, rubix Rubix [email protected]
wrote:

Here’s an example script to do exactly that:

gist.github.com

https://gist.github.com/phiggins/507196

em_300_gets.rb

require 'eventmachine'
require 'em-http'

c = 4
urls = DATA.readlines.map {|s| s.strip }
pending = urls.size
before = Time.now

puts "urls: #{pending}"
puts "concurrency: #{c}"

This file has been truncated. show original

pete

rubix01 · January 4, 2012, 9:07pm

thx,

Is there a way to use EventMachine to access database too (executing sql
requests waiting for http response)?
regards,