# LWP like library for Ruby?

**URL:** <https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843>\
**Category:** Ruby\
**Created:** [May 28, 2006, 7:53pm UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843 "2006-05-28T19:53:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![gman327](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/cc9497/32.png) [@gman327](https://www.ruby-forum.com/u/gman327)\
**Post date:** [May 28, 2006, 7:53pm UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843/1 "2006-05-28T19:53:50Z")

</div>

Can anybody tell me does Ruby support perl like LWP programming?

ie. scraping content of websites with support for Cookies

---

<div class="post-metadata">

**Author:** ![gman327](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/cc9497/32.png) [@gman327](https://www.ruby-forum.com/u/gman327)\
**Post date:** [May 28, 2006, 8:09pm UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843/2 "2006-05-28T20:09:20Z")

</div>

Robert N. wrote:

> Can anybody tell me does Ruby support perl like LWP programming?
> 
> ie. scraping content of websites with support for Cookies

I believe www-mechanize will do what you want.

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![gman327](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/cc9497/32.png) [@gman327](https://www.ruby-forum.com/u/gman327)\
**Post date:** [May 29, 2006, 5:29am UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843/3 "2006-05-29T05:29:30Z")

</div>

I cannot seem to install www-mechanize using gem install:

> ```
> Could not find www-mechanize (> 0) in the repository
> 
> ```
> 
> root@choochoo:~# gem search mechanize
> 
> \*\*\* LOCAL GEMS \*\*\*  
> root@choochoo:~#

Is it called something different in gems?

---

<div class="post-metadata">

**Author:** ![gman327](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/cc9497/32.png) [@gman327](https://www.ruby-forum.com/u/gman327)\
**Post date:** [May 29, 2006, 5:32am UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843/4 "2006-05-29T05:32:50Z")

</div>

Actually, here is a more interesting failure:

root@choochoo:~# gem install mechanize  
Attempting local installation of ‘mechanize’  
Local gem file not found: mechanize\*.gem  
Attempting remote installation of ‘mechanize’  
Install required dependency ruby-web? [Yn]  
Successfully installed mechanize-0.4.5  
Successfully installed ruby-web-1.1.1  
/usr/local/lib/site\_ruby/1.8/rubygems/custom\_require.rb:21:in  
`require__': no such file to load -- rdoc/rdoc (LoadError) from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in`require’  
from /usr/local/lib/site\_ruby/1.8/rubygems/doc\_manager.rb:43:in  
`generate_rdoc' from /usr/local/lib/site_ruby/1.8/rubygems/gem_commands.rb:215:in`execute’  
from  
/usr/local/lib/site\_ruby/1.8/rubygems/gem\_commands.rb:214:in  
`each' from /usr/local/lib/site_ruby/1.8/rubygems/gem_commands.rb:214:in`execute’  
from  
/usr/local/lib/site\_ruby/1.8/rubygems/gem\_commands.rb:153:in  
`each' from /usr/local/lib/site_ruby/1.8/rubygems/gem_commands.rb:153:in`execute’  
from /usr/local/lib/site\_ruby/1.8/rubygems/command.rb:49:in  
`invoke' from /usr/local/lib/site_ruby/1.8/rubygems/cmd_manager.rb:94:in`process\_args’  
from /usr/local/lib/site\_ruby/1.8/rubygems/cmd\_manager.rb:67:in  
`run' from /usr/local/lib/site_ruby/1.8/rubygems/gem_runner.rb:13:in`run’  
from /usr/bin/gem:17  
root@choochoo:~# which mechanize  
root@choochoo:~#

---

<div class="post-metadata">

**Author:** ![gman327](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/cc9497/32.png) [@gman327](https://www.ruby-forum.com/u/gman327)\
**Post date:** [May 29, 2006, 8:49am UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843/5 "2006-05-29T08:49:16Z")

</div>

On May 28, 2006, at 11:29 PM, Xeno C. wrote:

> Actually, here is a more interesting failure:

[snip]

Looks like it failed with the installation of the documentation. It’s  
probably still ok. Note that WWW::Mechanize does not install a binary  
like lwp does.

try doing

require ‘rubygems’  
require ‘mechanize’

www = WWW::Mechanize.new

---

<div class="post-metadata">

**Author:** ![gman327](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/cc9497/32.png) [@gman327](https://www.ruby-forum.com/u/gman327)\
**Post date:** [May 29, 2006, 9:10am UTC](https://www.ruby-forum.com/t/lwp-like-library-for-ruby/61843/6 "2006-05-29T09:10:32Z")

</div>

Try HTTPAccess2, it works for me…

http-access2 gives something like the functionality of libwww-perl (LWP)  
in  
Ruby.

[http://raa.ruby-lang.org/project/http-access2/](http://raa.ruby-lang.org/project/http-access2/)

Horacio

Monday 29 May 2006 15:47ã?Logan C. ã?ã??はæ?¸ãã¾ã?ã?: