# Regex problem, probably simple **URL:** \ **Category:** Ruby\ **Created:** [May 14, 2007, 8:23pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150 "2007-05-14T20:23:42Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 14, 2007, 8:23pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/1 "2007-05-14T20:23:42Z")
Hello! I’m trying to do the following: I want to scrape something of a webpage. It has a massive content and I want to find the thing that comes after the first occourance of “a href=” after the occourance of “id=xxx”. So: … \ … … How can I do this? Thank you! ---
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 14, 2007, 9:47pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/2 "2007-05-14T21:47:56Z")
Dear Jim, assuming that you have the following in webpage “a0.html”: … \ … … , you can run the following script: my\_page=IO.readlines(“a0.html”).to\_s r1=/\/ r2=/(?=]+\>)/ r3=/]+)\>/ text=my\_page.split(r1) text2=text[1…-1].join.split(r2)[1] ref=r3.match(text2) p 'the first link was : ’ + ref[1] I read in the entire page into a string my\_page, split that into an Array at the first occurrence of regexp r1, join it back again into a string, then split that into an array using regexp r2, which keeps the delimiter (of form ]+\> …that’s what the (?= … ) syntax is for) , rather than dropping it, as in the first split. If there is text before the first occurrence of r3, you’ll find it in the first element of the splitted string: text[1…-1].join.split(r2)[0], and the first occurrence of r3 is in the second element text2. If you want more information about Regexps, you’ll might find this helpful: > **[Ruby Regexp Class - Regular Expressions in Ruby](https://www.regular-expressions.info/ruby.html)** > > Learn how to use regular expressions with Ruby and the Ruby Regexp class Best regards, Axel ---
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 14, 2007, 10:04pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/3 "2007-05-14T22:04:04Z")
Jim Kronhamn wrote: > … > > … > > How can I do this? > > Thank you! Jim, Is the “???” the thing you want to capture? If so, the following should do the trick: if page.body =~ /\.+?\]\*)/m capture = $1 end If this seems somewhat perlish, it’s because a perlmonger taught me this line. Dan ---
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 15, 2007, 9:57pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/4 "2007-05-15T21:57:45Z")
> if page.body =~ /\.+?\]\*)/m > capture = $1 > end > > If this seems somewhat perlish, it’s because a perlmonger taught me this > line. I’m a recovering Perlmonger, and that’s exactly what I’d do in that situation. – Giles B. I’m running a time management experiment: I’m only checking e-mail twice per day, at 11am and 5pm. If you need to get in touch quicker than that, call me on my cell. Blog: [http://gilesbowkett.blogspot.com](http://gilesbowkett.blogspot.com) Portfolio: [http://www.gilesgoatboy.org](http://www.gilesgoatboy.org) ---
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 14, 2007, 10:23pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/5 "2007-05-14T22:23:02Z")
Thank you for the answers! Worked like a charm. ---
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 15, 2007, 11:10pm UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/6 "2007-05-15T23:10:53Z")
If you going to scrape anything more complex than what can be handled by a few regular expressions, then you might wan’t to take a look a whytheluckystiff’s Hpricot library: [http://code.whytheluckystiff.net/hpricot/](http://code.whytheluckystiff.net/hpricot/) It’s excellent for scraping web pages… Best regards, Christian blog: [http://inferencing.blogspot.com](http://inferencing.blogspot.com) ---
**Author:** ![bigjimmyk3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/6de8d8/32.png) [@bigjimmyk3](https://www.ruby-forum.com/u/bigjimmyk3)\ **Post date:** [May 17, 2007, 1:41am UTC](https://www.ruby-forum.com/t/regex-problem-probably-simple/104150/7 "2007-05-17T01:41:36Z")
On 5/15/07, Christian Theil H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > If you going to scrape anything more complex than what can be handled by a > few regular expressions, then you might wan’t to take a look a > whytheluckystiff’s Hpricot library: > [http://code.whytheluckystiff.net/hpricot/](http://code.whytheluckystiff.net/hpricot/) > > It’s excellent for scraping web pages… There’s also scrubyt: [http://scrubyt.org/](http://scrubyt.org/) It incorporates Hpricot and gives you both a higher-level approach and a way to drop down to Hpricot if needed. I think it’s also going to incorporate FireWatir in the nearish future, or use it somehow (forgot details). – Giles B. I’m running a time management experiment: I’m only checking e-mail twice per day, at 11am and 5pm. If you need to get in touch quicker than that, call me on my cell. Blog: [http://gilesbowkett.blogspot.com](http://gilesbowkett.blogspot.com) Portfolio: [http://www.gilesgoatboy.org](http://www.gilesgoatboy.org)