Help connecting to a web page, doing a search and filtering out the returned data!


#1

Hello team,
I need recommendations on what to use to accomplish the following with a
Ruby program.
Today, I do the following by hand:

  1. http://www.bmwusa.com/
  2. Click bottom “Certified Pre_Owned” which takes me to:
    http://www.bmwusa.com/Standard/Content/CPO/Default.aspx
  3. Next I click “Model Search” taking me to:
    http://www.bmwusa.com/Standard/Content/CPO/ModelSearch.aspx
  4. I then select “Body Styles”, “Series” and fill in some number, as
    shown below:

Price Limit No Limit $15,000-$25,000 $20,000-$30,000
$25,000-$35,000
$30,000-$40,000 $35,000-$45,000 $40,000-$50,000 $45,000-$100,000
$100,000+
Transmission Either Automatic Manual
Zip Code
Distance No Limit 10 Miles 25 Miles 50 Miles 100 Miles 150 Miles
200
Miles 500 Miles

I then get back a list which looks like the following partial list
below.
Please notice that the search returned 350 vehicles. Sometime it can
return
double that amount.
I am only interested in a model 328i or 335i 2007. I would like to
purchase
one of these vehicles, but I am watching for the “right” price. I just
don’w
want to be browsing everyday and filtering through the list by hand.
Please also notice that I would like to enter the parameters listed
above.

Also, I could actually do a “global” search without entering any
filtering
data. In other words, I could click search without selecting body type
or
anything like that, but the returned data could be large. Then again a
Ruby
program would go though the returned data rather quickly. So I guess I
can
do it that way if the programming part is easier than entering
parameters. I
would like to learn both ways, though!

For instance, I did it before writing this to test it and I got;
13508 vehicles found. To view details about a specific vehicle, click
the
model name.

What tool should I use to accomplish this task?

Thank you

Search Results
350 vehicles found. To view details about a specific vehicle, click the
model name.
330Ci
Convertiblehttp://www.bmwusa.com/Standard/Content/CPO/CPOModelDetail.aspx?VID=11532&searchtype=cpo&bodystyle=b_convertible&series=s_3&transmission=&pricelimit=4&searchzip=11201&distance=9999&page=1&sortby=DistCalc(40.6944,%2073.9902)&sortasc=True&pagerecs=4
2006 Automatic 20,845 Titanium Silver Metallic $32,995.00 Hassel BMW
Freeport, NY 20.95 Miles 330Ci
Convertiblehttp://www.bmwusa.com/Standard/Content/CPO/CPOModelDetail.aspx?VID=10030&searchtype=cpo&bodystyle=b_convertible&series=s_3&transmission=&pricelimit=4&searchzip=11201&distance=9999&page=1&sortby=DistCalc(40.6944,%2073.9902)&sortasc=True&pagerecs=4
2006 Automatic 26,208 Sparkling Graphite Metallic $32,995.00 Hassel BMW
Freeport, NY 20.95 Miles 328i
Convertiblehttp://www.bmwusa.com/Standard/Content/CPO/CPOModelDetail.aspx?VID=11135&searchtype=cpo&bodystyle=b_convertible&series=s_3&transmission=&pricelimit=4&searchzip=11201&distance=9999&page=1&sortby=DistCalc(40.6944,%2073.9902)&sortasc=True&pagerecs=4
2007 Automatic 16,360 Monaco Blue Metallic $37,995.00 Hassel BMW
Freeport, NY 20.95 Miles


#2

Being no expert in crawling through ajax this is the best I know:
http://www.scribd.com/doc/17764/Crawling-AJAX-SShah

2009/4/10 Ruby S. removed_email_address@domain.invalid


#3

On Fri, 2009-04-10 at 01:34 +0900, Ruby S. wrote:

Hello team,

What tool should I use to accomplish this task?

take a look at mechanize.