Forum: Ruby removing Whitespace using regexp

19eb75164135659a8fae98101b1c250e?d=identicon&s=25 Arun Kumar (arun_nss)
on 2009-05-06 07:54
Hi,
   Previously I posted a topic on how to strip all html tags and getting
the remaining text using regexp. Luckily I got one. This is the regexp:

/([^>]*)(?=<[^>]*?>)/im

In this case I'm able to get all the data between the html tags. But one
small problem. I'm getting output like this :


  Example Web Page


You have reached this web page by typing &quot;example.com&quot;,
&quot;example.net&quot;,
  or &quot;example.org&quot; into your web browser.
These domain names are reserved for use in documentation and are not
available
  for registration. See RFC
  2606, Section 3.

This is the output which I get when I parse the html content of
example.com using the above regexp. Here you can see some white space
between the data(ie. between 'Example web page' and 'You have
reached...'. These whitespaces are generated in place of the html tags
which I avoided using the above regexp. I want to remove those
whitespaces. I think that modifying the above regexp will give me the
right output without white spaces. Can somebody please help me.

Thanks
Arun
1d53b088a989e069b94597c282eebbbc?d=identicon&s=25 Simon Krahnke (Guest)
on 2009-05-06 09:05
(Received via mailing list)
* Arun Kumar <arunkumar@innovaturelabs.com> (07:54) schrieb:

> Hi,
>    Previously I posted a topic on how to strip all html tags and getting
> the remaining text using regexp. Luckily I got one. This is the regexp:
>
> /([^>]*)(?=<[^>]*?>)/im

And what do you do with this regexp?

> In this case I'm able to get all the data between the html tags. But one
> small problem.

Hasn't everybody told you, there are problems with parsing HTML with
regexps?

> This is the output which I get when I parse the html content of
> example.com using the above regexp. Here you can see some white space
> between the data(ie. between 'Example web page' and 'You have
> reached...'. These whitespaces are generated in place of the html tags
> which I avoided using the above regexp.

Really? Aren't they just from all the meaningless whitespace that's in
a typical HTML document?

> I want to remove those
> whitespaces. I think that modifying the above regexp will give me the
> right output without white spaces. Can somebody please help me.

There are easy ways to strip all the whitespace, which is certainly not
what you want, and there is a simple way to reduce all runs of
whitespace
by just one space (gsub(/\s+/, ' '), which probably also not what you
want.

Selectively removing some of the whitespace isn't easy at all, but it is
probably a lot easier with a real HTML parser.

mfg,                        simon .... l
F16e67cf8c0d59e1b98f5ba46d84b31d?d=identicon&s=25 Sriram Varahan (sriram)
on 2009-05-06 09:06
Hey Arun,


How about doing a gsub on the output to remove white spaces.

For example:

"Example Web Page".gsub(" ","")

This would remove the white spaces.


Hope this helps.

Regards
Sriram.
1e736afdcdfe0753d67a81d449aca590?d=identicon&s=25 Srijayanth Sridhar (Guest)
on 2009-05-06 12:19
(Received via mailing list)
I know your boss and whoever it is who is dangling your carrots won't
let
you use Hpricot, but tell him you will use Hpricot to get properly
formatted
html and then write a parser to parse the properly formatted html. Even
he
can't be opposed to that(seeing as how he wants you to reinvent wheels).
That way you can get rid of your whitespace problem and deal with the
cosmos
at large.

Jayanth
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-05-06 13:02
(Received via mailing list)
2009/5/6 Sriram Varahan <sriram.varahan@gmail.com>:

> How about doing a gsub on the output to remove white spaces.
>
> For example:
>
> "Example Web Page".gsub(" ","")
>
> This would remove the white spaces.

I would rather do

s.gsub /\s+/, ' '

Because your statement removes *all* whitespace:

irb(main):002:0> "Example Web Page".gsub(" ","")
=> "ExampleWebPage"

This is usually not what you want.

> Hope this helps.

Dito.

Cheers

robert
134ea397777886d6f0aa992672a50eaa?d=identicon&s=25 Mark Thomas (Guest)
on 2009-05-07 13:40
(Received via mailing list)
> I know your boss and whoever it is who is dangling your carrots won't let
> you use Hpricot, but tell him you will use Hpricot to get properly formatted
> html and then write a parser to parse the properly formatted html. Even he
> can't be opposed to that(seeing as how he wants you to reinvent wheels).
> That way you can get rid of your whitespace problem and deal with the cosmos
> at large.

Here's what we know about Arun from his previous posts...
* he is a "trainee" doing assignments.
* he is learning ruby
* "nobody else" around him knows ruby
* his "boss"/teacher is giving him specific assignments that seem to
be purely academic exercises, because the constraints (e.g. don't use
gsub, parse "example.com") would otherwise be completely ridiculous.
* He doesn't have the authority to re-scope the assignment or offer
alternate solutions.

I believe he is asking us to do his homework.
1e736afdcdfe0753d67a81d449aca590?d=identicon&s=25 Srijayanth Sridhar (Guest)
on 2009-05-07 14:21
(Received via mailing list)
How many places do you know that have extensive Ruby training programs
that
expect you to write HTML parsers armed with nothing but regular
expressions?
;)

I live in Bangalore, and I don't know one. Either he truly has a
sadistic
boss, or his truth is stranger than fiction. I don't doubt that it is
homework of some sort.

Jayanth
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.