Forum: Ruby Get Source Code of a http://... site?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Fc582698581884352e745d1d4c64699d?d=identicon&s=25 Joshua Muheim (josh)
on 2009-04-20 10:35
Hi all

I want to download the source code of a website using Ruby? How can I
achieve this? ...when searching with Google, I only get download links
for the Ruby source etc. ;-)

Thanks
Josh
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-04-20 10:43
(Received via mailing list)
2009/4/20 Joshua Muheim <forum@josh.ch>:
> I want to download the source code of a website using Ruby? How can I
> achieve this? ...when searching with Google, I only get download links
> for the Ruby source etc. ;-)

The source code of a web site is usually not available apart from the
JavaScript you see in pages - for good reasons (security,
copyright...).

Cheers

robert
Fc582698581884352e745d1d4c64699d?d=identicon&s=25 Joshua Muheim (josh)
on 2009-04-20 10:46
Robert Klemme wrote:
> 2009/4/20 Joshua Muheim <forum@josh.ch>:
>> I want to download the source code of a website using Ruby? How can I
>> achieve this? ...when searching with Google, I only get download links
>> for the Ruby source etc. ;-)
>
> The source code of a web site is usually not available apart from the
> JavaScript you see in pages - for good reasons (security,
> copyright...).
>
> Cheers
>
> robert

Oh, I wasn't clear enough, I just need the XHTML code, no
behind-the-scenes Ruby or PHP or stuff. ;-)

It's because I want to use captchator.com and I need to check the result
(0 or 1) using an URL like this:

http://captchator.com/captcha/check_answer/#{captc...
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-04-20 10:50
(Received via mailing list)
2009/4/20 Joshua Muheim <forum@josh.ch>:
> Oh, I wasn't clear enough, I just need the XHTML code, no
> behind-the-scenes Ruby or PHP or stuff. ;-)

There are mechanize, hpricot and a few other alternatives that can
help you there.

Cheers

robert
Fc582698581884352e745d1d4c64699d?d=identicon&s=25 Joshua Muheim (josh)
on 2009-04-20 11:04
Robert Klemme wrote:
> 2009/4/20 Joshua Muheim <forum@josh.ch>:
>> Oh, I wasn't clear enough, I just need the XHTML code, no
>> behind-the-scenes Ruby or PHP or stuff. ;-)
>
> There are mechanize, hpricot and a few other alternatives that can
> help you there.
>
> Cheers
>
> robert

Thanks, but that would be a big overkill I guess. I found the following
site which explains how to do it with just a few original methods:

http://blog.thembid.com/2007/08/06/using-ruby-to-s...
48409ce1953c290351fcb875b20eccbb?d=identicon&s=25 Daniel Huckstep (Guest)
on 2009-04-21 00:54
(Received via mailing list)
If you just need to get it, for archive or something, and just chose
ruby, have a look at httrack. Not ruby, just its own app, but it
mirrors pages.

- Daniel
Fc582698581884352e745d1d4c64699d?d=identicon&s=25 Joshua Muheim (josh)
on 2009-04-21 01:04
Daniel Huckstep wrote:
> If you just need to get it, for archive or something, and just chose
> ruby, have a look at httrack. Not ruby, just its own app, but it
> mirrors pages.
>
> - Daniel

Thanks, I know httrack... had some problems with it lately, though...
;-)
This topic is locked and can not be replied to.