# Scrape data from Javascript inside HTML source

**URL:** <https://www.ruby-forum.com/t/scrape-data-from-javascript-inside-html-source/165277>\
**Category:** Ruby\
**Created:** [April 11, 2009, 7:09am UTC](https://www.ruby-forum.com/t/scrape-data-from-javascript-inside-html-source/165277 "2009-04-11T07:09:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![yaxm](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/54ee81/32.png) [@yaxm](https://www.ruby-forum.com/u/yaxm)\
**Post date:** [April 11, 2009, 7:09am UTC](https://www.ruby-forum.com/t/scrape-data-from-javascript-inside-html-source/165277/1 "2009-04-11T07:09:28Z")

</div>

Hi,  
is there any scraper that interact with Javascript function inside a  
HTML page? Sometimes, the data is returned from a Javascript function  
or a javascript variable. so I wonder if there’s a easy way to get data  
out by evaluating and evaluating the javascript based on the context of  
the page in Ruby?

Yaxm

---

<div class="post-metadata">

**Author:** ![yaxm](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/54ee81/32.png) [@yaxm](https://www.ruby-forum.com/u/yaxm)\
**Post date:** [April 11, 2009, 7:20am UTC](https://www.ruby-forum.com/t/scrape-data-from-javascript-inside-html-source/165277/2 "2009-04-11T07:20:39Z")

</div>

Yaxm Y. wrote:

> is there any scraper that interact with Javascript function inside a  
> HTML page? Sometimes, the data is returned from a Javascript function  
> or a javascript variable. so I wonder if there’s a easy way to get data  
> out by evaluating and evaluating the javascript based on the context of  
> the page in Ruby?

In test? Or “scraping” a target website to see what it’s got?

Either way, I would use Nokogiri to rip the HTML and find tags,  
then  
use racc and rkelly to interpret the JavaScript and find its variables.

By “would” I mean I already _do_ that. Here’s the rkelly calls required:

```
   RKelly.parse(js).pointcut('TargetMethod()'). # with the ()
       matches.each do |updater|
     updater.grep(RKelly::Nodes::ArgumentsNode).each do |thang|
       p thang
     end
   end

```

However, if you are attacking other peoples’ websites to scrape out  
their data,  
you might instead try Watir. It just runs a web browser and evaluates  
its JS  
directly.
