Forum: Ferret Crawler for Ferret

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
6ce70ae187bc066c24e0fffd74521305?d=identicon&s=25 Huang, Zijian(Victor) (Guest)
on 2009-03-19 23:12
(Received via mailing list)
Hi, guys:
    Can you please recommend a good crawler for Ferret? Nutch is pretty
powerful in the Java side, do we have some thing is similar in Ruby? It
will be great if the crawler also handlers incremental index update
easily.

Thanks

Victor
C9dd93aa135988cabf9183d3210665ca?d=identicon&s=25 Jens Krämer (Guest)
on 2009-03-19 23:29
(Received via mailing list)
On 19.03.2009, at 22:32, Huang, Zijian(Victor) wrote:

> Hi, guys:
>     Can you please recommend a good crawler for Ferret? Nutch is
> pretty powerful in the Java side, do we have some thing is similar
> in Ruby? It will be great if the crawler also handlers incremental
> index update easily.
>

RDig can do http crawling, but cannot really be compared with Nutch
feature- and performance wise as it was designed for intranet use, say
indexing the web pages of a few hosts.


Cheers,
Jens


--
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49351467660 | Telefax +493514676666
kraemer@webit.de | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold
Bd2d9c0de2fe0e4caecb5e302f405647?d=identicon&s=25 Timothy Goddard (Guest)
on 2009-03-20 00:27
(Received via mailing list)
I wrote one called Suckr.

http://goddard.net.nz/projects/suckr/

It does the crawling, including incremental update and provides a
command line
search interface. I've had some periodic stability issues with this on
the old
Debian box I've been using it on myself - please test thoroughly.

It has some documentation in the README file. Please let me know if you
have
any questions.

Cheers,

Tim
457cf540784a12ba2f30e06565a2c189?d=identicon&s=25 Hugh Sasse (Guest)
on 2009-03-20 13:54
(Received via mailing list)
On Thu, 19 Mar 2009, Huang, Zijian(Victor) wrote:

> Hi, guys:
>     Can you please recommend a good crawler for Ferret? Nutch is pretty
> powerful in the Java side, do we have some thing is similar in Ruby? It
> will be great if the crawler also handlers incremental index update
> easily.

And then this shows up in my news feeds:

http://www.rubyinside.com/building-a-search-engine...

I've not followed the links off it, though, so YMMV.
>
> Thanks
>
> Victor
>
>
        Hugh
This topic is locked and can not be replied to.