# Help with Screen Scraper!

**URL:** <https://www.ruby-forum.com/t/help-with-screen-scraper/153651>\
**Category:** Ruby\
**Created:** [November 19, 2008, 8:52pm UTC](https://www.ruby-forum.com/t/help-with-screen-scraper/153651 "2008-11-19T20:52:44Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![stockwet](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/cc9497/32.png) [@stockwet](https://www.ruby-forum.com/u/stockwet)\
**Post date:** [November 19, 2008, 8:52pm UTC](https://www.ruby-forum.com/t/help-with-screen-scraper/153651/1 "2008-11-19T20:52:44Z")

</div>

Hi,

I need some help with a simple screen scraper app. Here’s how it works:

1. I open a connection to a url
2. I look for a specific string
3. When I get the string, I do something with it
4. Important: I’m polling a url looking for data changes.

Problem:  
Everything works fine, until the website is updated with new content. At  
that point, my code throws an “private method `scan’ called for  
nil:NilClass (NoMethodError)”

I can get the code to work using  
begin  
…  
rescue  
retry  
end

But, the problem is that it causes the connection to close and re-open.  
This causes a further delay in grabbing the data.

Is there a way to handle this? Again, the polling works fine until the  
server updates the new page that I’m scanning. At that time, the error  
is thrown and I either have to re-open a connection or fail on error. My  
Java application seems to be handling this more gracefully, but, Ruby is  
faster on the polling. Thanks for any insight.

Regards.

* * *

require ‘open-uri’  
pval = “14”  
url=“[http://www.someurl.com](http://www.someurl.com)”  
stop = false

```
while stop == false
    #Create timestamp
    sts="S " << Time.now.strftime("%m/%d/%y %H %M

```

%S.“)\<\<Time.now.usec.to\_s  
puts sts  
lines=open(url).read.split(”\n")  
#Get Lines of data and find the SearchValue  
tdate=lines.detect {|line| line =~ /SearchValue/}.scan(/\d+/)  
#Compare SearchValue to pval  
if tdate[1] != pdate

```
    puts tdate[0]
    ts="F "<<Time.now.strftime("%m/%d/%y %H %M

```

%S.")\<\<Time.now.usec.to\_s  
else  
puts “…”  
end  
end