# FeedTools **URL:** \ **Category:** Ruby\ **Created:** [July 27, 2006, 10:12pm UTC](https://www.ruby-forum.com/t/feedtools/100238 "2006-07-27T22:12:35Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 27, 2006, 10:12pm UTC](https://www.ruby-forum.com/t/feedtools/100238/1 "2006-07-27T22:12:35Z")
Can someone verify for me that feedtools is incorrectly parsing [http://torrentspy.com/rss.asp](http://torrentspy.com/rss.asp) ? In my tests, feedtools can only find 1 entry for that rss. Other sites seem to work, so far. TIA ---
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 28, 2006, 1:02am UTC](https://www.ruby-forum.com/t/feedtools/100238/2 "2006-07-28T01:02:54Z")
Ray C. wrote: > Can someone verify for me that feedtools is incorrectly parsing > [http://torrentspy.com/rss.asp](http://torrentspy.com/rss.asp) ? In my tests, feedtools can only find 1 > entry for that rss. Other sites seem to work, so far. > > TIA Ray- it seems to be working for me, though if you picked up a new version in the last few days, it was updated this afternoon to fix a couple issues. > > FeedTools::FEED\_TOOLS\_VERSION::STRING > > =\> “0.2.27” > > f = FeedTools::Feed.open(‘[http://torrentspy.com/rss.asp](http://torrentspy.com/rss.asp)’) > > =\> #\ > > f.items.size > > =\> 25 > > f.entries.size > > =\> 25 ---
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 28, 2006, 4:06am UTC](https://www.ruby-forum.com/t/feedtools/100238/3 "2006-07-28T04:06:02Z")
Thanks a lot you guys. It seems that that specific problem was solved when I updated feedtools through gems. I was using 0.2.25 and now I’m on 0.2.26 I notice that the previous poster is on 0.2.27, but I am trying to stick to gem installs. For a follow-up question, anyone know of a good ruby feed validator? Currently I crawl what I think are feeds using my own custom crawler: ``` feed = FeedTools::Feed.new feed.feed_data = _my_data_returned_from_crawler ``` I would like to 1) validate the feed before submitting feed contents to feedtools 2) try to extract a feed link out of \_my\_data\_returned\_from\_crawler in the case that it is not a valid feed. I have seen the ruby extension for [feedvalidator.org](http://feedvalidator.org), but unfortunately I it won’t work for me since I want to keep the validation on my local machine. Thanks a lot again. ---
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 28, 2006, 9:01am UTC](https://www.ruby-forum.com/t/feedtools/100238/4 "2006-07-28T09:01:36Z")
Hi, - Ray C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): > Thanks a lot you guys. It seems that that specific problem was solved > when I updated feedtools through gems. I was using 0.2.25 and now I’m > on 0.2.26 I notice that the previous poster is on 0.2.27, but I am > trying to stick to gem installs. This is not a problem of version 0.2.25: irb(main):001:0\> require ‘rubygems’ =\> true irb(main):002:0\> require\_gem ‘feedtools’ =\> true irb(main):003:0\> FeedTools::FEED\_TOOLS\_VERSION::STRING =\> “0.2.25” irb(main):004:0\> f = FeedTools::Feed.open(“[http://torrentspy.com/rss.asp](http://torrentspy.com/rss.asp)”) =\> #\ irb(main):005:0\> f.items.size =\> 25 irb(main):006:0\> f.entries.size =\> 25 Regards Lutz ---
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 28, 2006, 5:06am UTC](https://www.ruby-forum.com/t/feedtools/100238/5 "2006-07-28T05:06:06Z")
> Thanks a lot you guys. It seems that that specific problem was solved > when I updated feedtools through gems. I was using 0.2.25 and now I’m > on 0.2.26 I notice that the previous poster is on 0.2.27, but I am > trying to stick to gem installs. > > For a follow-up question, anyone know of a good ruby feed validator? There isn’t one. You need Python for that. > Currently I crawl what I think are feeds using my own custom crawler: > > ``` > feed = FeedTools::Feed.new > feed.feed_data = _my_data_returned_from_crawler > > ``` > > I would like to 1) validate the feed before submitting feed contents to > feedtools 2) try to extract a feed link out of > \_my\_data\_returned\_from\_crawler in the case that it is not a valid feed. If you’re thinking about auto-discovery, FeedTools already has that built-in. It just doesn’t work in some cases (for example, on Engadget). This should be fixed in the next release, hopefully. > I have seen the ruby extension for [feedvalidator.org](http://feedvalidator.org), but unfortunately > I it won’t work for me since I want to keep the validation on my local > machine. What ruby extension? Please send me a link. Cheers, Bob A. ---
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 28, 2006, 10:09am UTC](https://www.ruby-forum.com/t/feedtools/100238/6 "2006-07-28T10:09:46Z")
Hi Bob, I was referring to this: [http://feedvalidator.rubyforge.org/](http://feedvalidator.rubyforge.org/) I’m not sure exactly how to use auto-discovery since I use my own crawler for http retrieval. Can you point me at the method that does the auto-discovery? Perhaps the [http://torrentspy.com/rss.asp](http://torrentspy.com/rss.asp) case was broken for me since my crawler returned a different output than what would have been gotten using feedtools internals. Ray ---
**Author:** ![varac](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@varac](https://www.ruby-forum.com/u/varac)\ **Post date:** [July 28, 2006, 4:01pm UTC](https://www.ruby-forum.com/t/feedtools/100238/7 "2006-07-28T16:01:58Z")
> Hi Bob, > > I was referring to this: > [http://feedvalidator.rubyforge.org/](http://feedvalidator.rubyforge.org/) > > I’m not sure exactly how to use auto-discovery since I use my own > crawler for http retrieval. Can you point me at the method that does > the auto-discovery? If FeedTools fails to parse it checks the mime-type of the thing it retrieved and if it was an (x)html mime-type it looks for the auto-discovery stuff. This logic mostly happens in the update! method, but also partly in the HtmlHelper module. That said, I’d recommend against using your own crawler for dealing with feeds unless you’re only pulling the feed once. FeedTools has a lot of functionality built in that’s designed to prevent feeds from being unnecessarily polled, thus wasting people’s bandwidth. I suppose you might have actually implemented all that functionality yourself, but I’m going to go out on a limb and guess that you haven’t added ETag/Last-Modified/If-None-Match/feed time-to-live support into your crawler. Just be aware that failing to implement that stuff can and will cost other people hundreds of dollars in bandwidth if you poll too often without having that code in place. It’s not exactly polite. > Perhaps the [http://torrentspy.com/rss.asp](http://torrentspy.com/rss.asp) case was broken for me since > my crawler returned a different output than what would have been gotten > using feedtools internals. Very possible. Cheers, Bob A.