Forum: Ruby html method on watir don't return the real html

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
97334b75c93574ed3e514f33849f0953?d=identicon&s=25 Mario Ruiz (tcblues)
on 2009-05-18 19:17
Hi,
I'm using watir to get the html of a page in order to verify every
single page on w3c.org but what I get with watir is something like:
<HTML lang=is xml:lang="is"
xmlns="http://www.w3.org/1999/xhtml"><HEAD><TITLE>Certus... Games</TITLE>
<META http-equiv=Cache-Control content=no-cache>
<META http-equiv=Pragma content=no-cache>
<META http-equiv=Expires content=-1>
<META http-equiv=Content-Type content="text/html; charset=utf-8"><LINK
media=screen href="/CF/css/screen.css" type=text/css
rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css
rel=stylesheet><LINK media=screen href="/CF/css/lib/iestyles.css"
type=text/css rel=stylesheet>
<SCRIPT src="/CF/js/lib/jquery.js" type=text/javascript></SCRIPT>
....

And the real content is:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" lang="is" xml:lang="is">
  <head>
    <meta http-equiv="Cache-Control" content="no-cache"/>
    <meta http-equiv="Pragma" content="no-cache"/>
    <meta http-equiv="Expires" content="-1"/>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"
/>
....


Any idea how to get the real html content?

thank you.
97334b75c93574ed3e514f33849f0953?d=identicon&s=25 Mario Ruiz (tcblues)
on 2009-05-20 15:55
Since I think it's impossible to get the real html I'm trying to do it
using net/http library.

In my case I need  first to login the user onto the webpage and then go
to the pages I want to get the html.
This is what I'm doing... but it seems to me the user can't login:

    require 'net/http'
    require 'activesupport'
    server="myserver.com"
    port=80

    http = Net::HTTP.new(server, port)
    http.start()

    data={":loginName=>"myLoginName",
      :password=>"myPassord"
    }

    resp,data=http.post("/login.do",data.to_query)
    if resp.code=="200" then
      headers={
        "Cookie"=>resp["set-cookie"].to_s()
      }
    end
    resp,data=http.get('/MyAccount.do',headers)
    html=data
    http.finish()

Any idea???

Thanks.

Mario Ruiz wrote:
> Hi,
> I'm using watir to get the html of a page in order to verify every
> single page on w3c.org but what I get with watir is something like:
> <HTML lang=is xml:lang="is"
> xmlns="http://www.w3.org/1999/xhtml"><HEAD><TITLE>Certus... Games</TITLE>
666b4e17b4bb0e2d999037a25f65a7cb?d=identicon&s=25 Heesob Park (phasis)
on 2009-05-21 09:36
(Received via mailing list)
Hi,

2009/5/19 Mario Ruiz <tcblues@gmail.com>:
> rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css
>
> Any idea how to get the real html content?
>
If you are using Ruby 1.8.6 on Windows, try this code:

require 'win32/api'
require 'watir'

IID_IPersistStreamInit =
[0x7FD52380,0x4E07,0x101B,0xAE,0x2D,0x08,0x00,0x2B,0x2E,0xC7,0x13].pack('LSSC8')

CreateStreamOnHGlobal = Win32::API.new('CreateStreamOnHGlobal', 'LLP',
'L','ole32')
OleSaveToStream = Win32::API.new('OleSaveToStream', 'LL', 'L','ole32')
GetHGlobalFromStream = Win32::API.new('GetHGlobalFromStream', 'LP',
'L','ole32')
GlobalSize = Win32::API.new('GlobalSize', 'L', 'L')
GlobalLock = Win32::API.new('GlobalLock', 'L', 'L')
CopyMemory = Win32::API.new('RtlMoveMemory', 'PLL', 'V')
GlobalUnlock = Win32::API.new('GlobalUnlock', 'L', 'L')

browser = Watir::Browser.new
browser.goto("http://www.google.com")

ptr = browser.document.inspect.scan(/:(0x[\da-f]+)?>/).to_s.hex
p = 0.chr * 4
CopyMemory.call(p,ptr+16,4)
data = p.unpack('L').first
p = 0.chr * 4
CopyMemory.call(p,data,4)
dispatch = p.unpack('L').first

lpVtbl = 0.chr * 4
table = 0.chr * 28
CopyMemory.call(lpVtbl,dispatch,4)
CopyMemory.call(table,lpVtbl.unpack('L').first,28)
table = table.unpack('L*')
queryInterface = Win32::API::Function.new(table[0],'PPP','L')
p = 0.chr * 4
hr = queryInterface.call(dispatch,IID_IPersistStreamInit,p)
persiststream = p.unpack('L').first

p = 0.chr * 4
CreateStreamOnHGlobal.call(0, 1, p)
stream = p.unpack('L').first
if OleSaveToStream.call(persiststream,stream) == 0
  p = 0.chr * 4
  GetHGlobalFromStream.call(stream,p)
  h = p.unpack('L').first
  size = GlobalSize.call(h)
  buf = 0.chr * size
  ptr = GlobalLock.call(h)
  if ptr != 0
    CopyMemory.call(buf,ptr,size)
    GlobalUnlock.call(ptr)
    html = buf[16..-1]
  else
    html = nil
  end
end

puts html


Regards,
Park Heesob
97334b75c93574ed3e514f33849f0953?d=identicon&s=25 Mario Ruiz (tcblues)
on 2009-05-25 13:05
It's working!!!!! Thanks a lot.

Since I'm not using the last Watir version I changed Watir::Browser.new
for Watir::IE.new and it's working fine.

Again... thanks a lot.


Heesob Park wrote:
> Hi,
>
> 2009/5/19 Mario Ruiz <tcblues@gmail.com>:
>> rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css
>>
>> Any idea how to get the real html content?
>>
This topic is locked and can not be replied to.