Is there is a way to get page as text (like curl in php) and parse it to
get needed info?
Also, I don’t really know, is there is a way when getting url through
something like curl to enter user data in forms or activate javascript
on page (if there is authentication before showing page etc)?
If you want complex interactions, particularly Javascript, then
Watir-Webdriver is a powerful tool for driving browsers.
If you can get what you want without using Javascript and you just want
to submit forms or directly extract data from a server by firing URLs at
it; try Mechanize.
I was really happy with Nokogiri for parsing DOM content. For actually
activating Javascript and interacting with a page, Watir is nice and
easy
to use. But it really depends on what exactly you’re trying to achieve.
Thank you very much for your answers, guys! You helped me much
Nitay A. wrote in post #1109936:
I was really happy with Nokogiri for parsing DOM content. For actually
activating Javascript and interacting with a page, Watir is nice and
easy
to use. But it really depends on what exactly you’re trying to achieve.
There is site with list of some objects (houses, flats etc).
I just want to request pages with entered info in form fields like
“number of rooms, floor” etc
So I can have list of objects that I interested in every hour.