\
**Category:** Ruby\
**Created:** [October 27, 2008, 4:46pm UTC](https://www.ruby-forum.com/t/hpricot-not-returning-the-right-html/151897 "2008-10-27T16:46:23Z")\
**Posts on this page:** 2\
**Page:** 1
**Author:**  [@monstars99](https://www.ruby-forum.com/u/monstars99)\
**Post date:** [October 27, 2008, 4:46pm UTC](https://www.ruby-forum.com/t/hpricot-not-returning-the-right-html/151897/1 "2008-10-27T16:46:23Z")
Hi when i get this url
[http://www.basketball-bund.net/index.jsp?Action=100&Verband=100](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100)
it shows a table with basketball info at page 1
if i want to go to page 2
this is the working url
[http://www.basketball-bund.net/index.jsp?Action=100&Verband=100&startrow=10](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100&startrow=10)
like it says it starts with the 10th result…
wellif i enter the url into the browser address bar it works fine… but
when i look for the html in Hpricot it just returns the first page…
ive found out that if the startrow bit is wrong then it always shows the
first page… but itseems to be right as its working in the browser…
i got the same problem using
URI.parse
here is my code
q =
‘[Bundesligen](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100&viewid=&startrow=10)’
f = open(q)
f.rewind
doc = Hpricot(Iconv.conv(‘utf-8’, f.charset, f.readlines.join(“\n”)))
form = doc.search(“//form[@name=ligaliste]”)
can anyone help me pls
thx
---
**Author:**  [@monstars99](https://www.ruby-forum.com/u/monstars99)\
**Post date:** [November 3, 2008, 12:34pm UTC](https://www.ruby-forum.com/t/hpricot-not-returning-the-right-html/151897/2 "2008-11-03T12:34:19Z")
hmmm noone replied… well in case someone hase the same problem… i
have found the solution
q = “[Deutscher Basketball Bund](http://www.basketball-bund.net/index.jsp?#%7Bsearch_string%7D)”
agent = WWW::Mechanize.new
doc =
agent.get(“[Bundesligen](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100)”)
doc = agent.get(q)
```
doc = doc.search('body').to_html
#convert iso15 to utf8
doc = Iconv.iconv("UTF-8", "ISO-8859-15", doc).to_s
#make it hpricot
doc = Hpricot(doc)
##end crawling
@q = q
form = doc.search("//form")
```
it seems that its because of cookcies or something… that i needed to
reload the page once before i try to do my own search… thats why i call
the agent.get twice
hope this helps anyone
Hannes Rammer wrote:
> Hi when i get this url
>
> [Bundesligen](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100)
>
> it shows a table with basketball info at page 1
>
> if i want to go to page 2
>
> this is the working url
>
> [Bundesligen](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100&startrow=10)
>
> like it says it starts with the 10th result…
>
> wellif i enter the url into the browser address bar it works fine… but
> when i look for the html in Hpricot it just returns the first page…
>
> ive found out that if the startrow bit is wrong then it always shows the
> first page… but itseems to be right as its working in the browser…
>
> i got the same problem using
>
> URI.parse
>
> here is my code
>
> q =
> ‘[Bundesligen](http://www.basketball-bund.net/index.jsp?Action=100&Verband=100&viewid=&startrow=10)’
>
> f = open(q)
> f.rewind
> doc = Hpricot(Iconv.conv(‘utf-8’, f.charset, f.readlines.join(“\n”)))
> form = doc.search(“//form[@name=ligaliste]”)
>
> can anyone help me pls
>
> thx