Problems with test cases and other things with Ruby-WordNet

Ruby
1

I tried to post this as a ticket to Ruby-WordNet trac as a ticket but
it kept rejecting it as “Possible Spam” so here it. (For those of you
having problems running convertdb.rb and core dumps - jump to the end)

Ubuntu 7.04
ruby 1.8.5 (2006-08-25) [i486-linux]
WordNet-3.0

After much heartache I was able to successfully install Ruby WordNet
but some of the tests fail at the end when running ‘ruby test.rb’. I
am able to load up the library from Linguistics as well as separately
using require ‘wordnet’, but all commands bring back nil values. The
pre-packaged download does not let me run the test.rb at all as it
returns:

./tests/wntestcase.rb:57:in require': no such file to load -- test/ unit/mock (LoadError) from ./tests/wntestcase.rb:57 from test.rb:15:in require’
from test.rb:15
from ./utils.rb:665:in `verboseOff’
from test.rb:14

I tried installing the Test::Unit::Mock library but didn’t get very
far. I checked out the latest source code from SVN using:

svn checkout svn://deveiate.org/Ruby-WordNet/trunk

I was able to run the tests using this revision. After successfully
converting the WordNet data files - I ran ruby test.rb and received
the following errors:

0 patterns given on the command line
Required 3 files.
Loaded suite WordNet
Started
…FF…FF.E.F…FFFFFFFFF
Finished in 0.787878 seconds.

  1. Failure:
    test_familiarity_for_testwords_should_all_return_a_fixnum(LexiconTests)
    [./tests/lexicon.tests.rb:92:in
    test_familiarity_for_testwords_should_all_return_a_fixnum' ./tests/lexicon.tests.rb:88:in each’
    ./tests/lexicon.tests.rb:88:in
    test_familiarity_for_testwords_should_all_return_a_fixnum' ./tests/wntestcase.rb:141:in run’]:
    expected to be an instance of
    but was
    .

  2. Failure:
    test_grep_finds_compound_words(LexiconTests)
    [./tests/lexicon.tests.rb:137:in test_grep_finds_compound_words' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method []' for nil:NilClass"> ---Backtrace--- ./lib/wordnet/lexicon.rb:278:in grep’
    ./tests/lexicon.tests.rb:138:in test_grep_finds_compound_words' ./tests/lexicon.tests.rb:137:in test_grep_finds_compound_words’
    ./tests/wntestcase.rb:141:in `run’

  1. Failure:
    test_lookup_synsets(LexiconTests)
    [./tests/lexicon.tests.rb:156:in test_lookup_synsets' ./tests/lexicon.tests.rb:151:in each’
    ./tests/lexicon.tests.rb:151:in test_lookup_synsets' ./tests/wntestcase.rb:141:in run’]:
    expected to be an instance of
    WordNet::Synset but was
    .

  2. Failure:
    test_morphology_of_dictionary_word_should_return_root_word(LexiconTests)
    [./tests/lexicon.tests.rb:104:in
    test_morphology_of_dictionary_word_should_return_root_word' ./tests/wntestcase.rb:141:in run’]:
    <“angry”> expected but was
    .

  3. Error:
    test_perl_tests(LexiconTests):
    WordNet::LookupError: Failed lookup of synset ‘00466621%n’:No such
    synset
    ./lib/wordnet/lexicon.rb:239:in lookup_synsets_by_key' ./lib/wordnet/lexicon.rb:238:in each’
    ./lib/wordnet/lexicon.rb:238:in lookup_synsets_by_key' ./tests/linguawordnet.tests.rb:17:in test_perl_tests’
    ./tests/wntestcase.rb:141:in `run’

  4. Failure:
    test_reverse_morphology_should_return_inverse(LexiconTests)
    [./tests/lexicon.tests.rb:129:in
    test_reverse_morphology_should_return_inverse' ./tests/wntestcase.rb:141:in run’]:
    expected to be =~
    </^angr/>.

  5. Failure:
    test_hyponym_traversal_with_no_block_and_false_second_arg_should_return_holonyms_but_not_the_origin(SynsetTests)
    [./tests/synset.tests.rb:208:in
    test_hyponym_traversal_with_no_block_and_false_second_arg_should_return_holonyms_but_not_the_origin' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method traverse' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:209:in test_hyponym_traversal_with_no_block_and_false_second_arg_should_return_holonyms_but_not_the_origin’
    ./tests/synset.tests.rb:208:in
    test_hyponym_traversal_with_no_block_and_false_second_arg_should_return_holonyms_but_not_the_origin' ./tests/wntestcase.rb:141:in run’

  1. Failure:
    test_hyponym_traversal_with_no_block_should_return_appropriate_hyponyms(SynsetTests)
    [./tests/synset.tests.rb:192:in
    test_hyponym_traversal_with_no_block_should_return_appropriate_hyponyms' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method traverse' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:193:in test_hyponym_traversal_with_no_block_should_return_appropriate_hyponyms’
    ./tests/synset.tests.rb:192:in
    test_hyponym_traversal_with_no_block_should_return_appropriate_hyponyms' ./tests/wntestcase.rb:141:in run’
  1. Failure:
    test_part_of_speech_should_return_the_symbol_part_of_speech(SynsetTests)
    [./tests/synset.tests.rb:243:in
    test_part_of_speech_should_return_the_symbol_part_of_speech' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method part_of_speech' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:243:in test_part_of_speech_should_return_the_symbol_part_of_speech’
    ./tests/synset.tests.rb:243:in
    test_part_of_speech_should_return_the_symbol_part_of_speech' ./tests/wntestcase.rb:141:in run’
  1. Failure:
    test_pos_should_return_the_synsets_singlechar_part_of_speech(SynsetTests)
    [./tests/synset.tests.rb:253:in
    test_pos_should_return_the_synsets_singlechar_part_of_speech' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method pos' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:253:in test_pos_should_return_the_synsets_singlechar_part_of_speech’
    ./tests/synset.tests.rb:253:in
    test_pos_should_return_the_synsets_singlechar_part_of_speech' ./tests/wntestcase.rb:141:in run’

  1. Failure:
    test_relations(SynsetTests)
    [./tests/synset.tests.rb:108:in test_relations' ./tests/synset.tests.rb:105:in each’
    ./tests/synset.tests.rb:105:in test_relations' ./tests/wntestcase.rb:141:in run’]:
    <#<WordNet::Synset:0xfb7945db0/1 (noun): “” ()>>
    of type WordNet::Synset
    expected to respond_to?<:instance_hyponyms>.

  2. Failure:
    test_synset_should_respond_to_traverse_method(SynsetTests)
    [./tests/synset.tests.rb:138:in
    test_synset_should_respond_to_traverse_method' ./tests/wntestcase.rb:141:in run’]:

    of type
    expected to respond_to?<:traverse>.

  3. Failure:
    test_traversal_break_after_3_should_include_three_sets_plus_origin(SynsetTests)
    [./tests/synset.tests.rb:225:in
    test_traversal_break_after_3_should_include_three_sets_plus_origin' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method traverse' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:226:in test_traversal_break_after_3_should_include_three_sets_plus_origin’
    ./tests/synset.tests.rb:225:in
    test_traversal_break_after_3_should_include_three_sets_plus_origin' ./tests/wntestcase.rb:141:in run’

  1. Failure:
    test_traversal_with_false_second_arg_should_not_include_origin(SynsetTests)
    [./tests/synset.tests.rb:172:in
    test_traversal_with_false_second_arg_should_not_include_origin' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method traverse' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:173:in test_traversal_with_false_second_arg_should_not_include_origin’
    ./tests/synset.tests.rb:172:in
    test_traversal_with_false_second_arg_should_not_include_origin' ./tests/wntestcase.rb:141:in run’
  1. Failure:
    test_traversal_with_true_second_arg_should_include_origin(SynsetTests)
    [./tests/synset.tests.rb:149:in
    test_traversal_with_true_second_arg_should_include_origin' ./tests/wntestcase.rb:141:in run’]:
    Exception raised:
    Class:
    Message: <"undefined method traverse' for nil:NilClass"> ---Backtrace--- ./tests/synset.tests.rb:150:in test_traversal_with_true_second_arg_should_include_origin’
    ./tests/synset.tests.rb:149:in
    test_traversal_with_true_second_arg_should_include_origin' ./tests/wntestcase.rb:141:in run’

25 tests, 116 assertions, 14 failures, 1 errors

sudo ruby install.rb appears to work - but when using the library in a
ruby program nil values are returned.

sudo irb

require ‘wordnet’
lex = WordNet::Lexicon::new
lex.lookup_synsets(“shoe”, WordNet::Noun)
=> nil

I’m not sure what I’ve done wrong up to this point. Hopefully the
information I’ve provided can show it’s something wrong on my end.

For those of you looking for the instructions to get you to the point
where you can run the installation tests, I’ve provided the KEY steps
here for installation on Ubuntu 7.04:

#Download Berkeley DB from Oracle (MUST BE 4.4 or earlier!)
wget http://download.oracle.com/berkeley-db/db-4.4.20.tar.gz #untar
and install per INSTALL file instructions

#Get the ruby bindings for BDB (MUST BE VERSION 0.6.0 - I had core
dumps with using any other binding during the convertdb command)
wget ftp://moulon.inra.fr/pub/ruby/bdb-0.6.0.tar.gz

#when compiling the bindings point to the proper headers
ruby extconf.rb --with-db-include=/home/johndoe/db-4.4.20/build_unix
make
sudo make install

svn checkout svn://deveiate.org/Ruby-WordNet/trunk
ruby convertdb.rb /home/johndoe/WordNet-3.0/dict/

Any information or help in getting the test and the wordnet library to
work - would be greatly appreciated. Perhaps a brave soul can make a
gem out of it…

Henry

2

On 7/21/07, H H [email protected] wrote:

am able to load up the library from Linguistics as well as separately
from test.rb:14
0 patterns given on the command line
./tests/lexicon.tests.rb:88:in each' ./tests/wntestcase.rb:141:in run’]:
3) Failure:
test_morphology_of_dictionary_word_should_return_root_word(LexiconTests)
./lib/wordnet/lexicon.rb:239:in lookup_synsets_by_key' <nil> expected to be =~ ---Backtrace--- test_hyponym_traversal_with_no_block_should_return_appropriate_hyponyms’

./tests/synset.tests.rb:243:in
./tests/wntestcase.rb:141:in `run’]:

  1. Failure:
    [./tests/synset.tests.rb:225:in
    ./tests/wntestcase.rb:141:in run' ---Backtrace--- test_traversal_with_true_second_arg_should_include_origin’

lex = WordNet::Lexicon::new
#Download Berkeley DB from Oracle (MUST BE 4.4 or earlier!)
sudo make install

svn checkout svn://deveiate.org/Ruby-WordNet/trunk
ruby convertdb.rb /home/johndoe/WordNet-3.0/dict/

Any information or help in getting the test and the wordnet library to
work - would be greatly appreciated. Perhaps a brave soul can make a
gem out of it…

Henry,

So far my experience with Ruby bindings for wordnet is, its stinking
piece of shit. Its about time someone took ownership of this project
and do something about it. I know I am not helping you much, but NLP
is the one thing that makes me jealous of pythonists.

3

On Jul 21, 2007, at 8:15 AM, H H wrote:

I tried to post this as a ticket to Ruby-WordNet trac as a ticket but
it kept rejecting it as “Possible Spam” so here it. (For those of you
having problems running convertdb.rb and core dumps - jump to the end)

First of all, I apologize for the “Possible Spam” thing. I was getting
hammered with people rewriting all my trac wiki pages with links to spam
sites, so I’m trying out the new trac anti-spam plugin. Apparently it
needs
some tuning.

And also: thanks for taking the time to not only bring this up, but
in a way
that’s useful and can be used to improve the library for others.

After much heartache I was able to successfully install Ruby WordNet
but some of the tests fail at the end when running ‘ruby test.rb’.

I’m sorry you had to spend so much time getting things to run. I made
the
initial choice to use a BDB database to make access to the dictionaries
faster, to allow on-the-fly modification of terms, and also because
there
wasn’t a usable C interface to the WordNet dictionaries at the time.
I’ve
since wondered if it would have been less fragile, albeit much
slower, to
store the parsed dictionaries in PStores or something.

I am able to load up the library from Linguistics as well as
separately using require ‘wordnet’, but all commands bring back nil
values. The pre-packaged download does not let me run the test.rb at
all as it returns:

I’ve removed the packaged-up source, as it’s undergone quite a lot of
bitrot. I’ll be posting a new tarball from SVN today or tomorrow.

I checked out the latest source code from SVN using:

svn checkout svn://deveiate.org/Ruby-WordNet/trunk

I was able to run the tests using this revision. After successfully
converting the WordNet data files - I ran ruby test.rb and received
the following errors:
[snip]

I’m not sure where those errors are coming from. Using bdb 6.0 and
BerkeleyDB 4.5, I get:

$ ./convertdb.rb
WordNet Lexicon Converter
This program will convert WordNet data files into databases
used by Ruby-WordNet. This will not affect existing WordNet files,
but will require up to 40Mb of disk space.
Continue? [y]:

Warning: Existing data in the Ruby-WordNet databases
will be overwritten.
Continue? [n]: y
Where can I find the WordNet data files?
Data directory [/usr/local/WordNet-2.1/dict]: /usr/local/WordNet-3.0/
dict
Converting index files…

index.noun…
…committing…done (117798 entries, 0 errors).
index.verb…committing…done (11529 entries, 0 errors).
index.adj…committing…done (21479 entries, 0 errors).
index.adv…committing…done (4481 entries, 0 errors).
Checkpointing DB and cleaning logs…done.
Converting morph files…
verb.exc…committing…done (2401 entries, 0 errors).
adv.exc…committing…done (7 entries, 0 errors).
adj.exc…committing…done (1490 entries, 0 errors).
cousin.exc…missing: skipped
noun.exc…committing…done (2054 entries, 0 errors).
Checkpointing DB and cleaning logs…done.
Converting data files…
data.adj…committing…done (18156 entries, 0 errors).
data.adv…committing…done (3621 entries, 0 errors).
data.verb…committing…done (13767 entries, 0 errors).
data.noun…committing
…done (82115 entries, 0 errors).
Checkpointing DB and cleaning logs…done.
done.

$ ./test.rb
0 patterns given on the command line
Required 3 files.
Loaded suite WordNet
Started
…
Finished in 6.089183 seconds.

25 tests, 267 assertions, 0 failures, 0 errors

Obviously this doesn’t help with your problem, but it does seem to
indicate
that it’s related to some difference in our environments rather than
something endemic to the code.

Any information or help in getting the test and the wordnet library to
work - would be greatly appreciated.

I’d be happy to help you figure out how to make it work for you,
which will
help me write better troubleshooting and instructions.

Perhaps a brave soul can make a gem out of it…

Since it relies rather heavily on the bdb library, a gem is somewhat
useless
without a corresponding BDB gem on which it can depend. I suppose I
could
make one that just had ‘bdb’ in the ‘requirements’, but I’d think
that would
be misleading and prone to causing more frustration.

Alternatively, it could be written differently, removing its
dependency on
BDB, but I’m afraid I’m not up to that task at the moment, as it’s
working
fine for what I need it to do. I’d welcome patches, though.

–
Michael G. [email protected]
Rubymage, Architect, Believer
The FaerieMUD Consortium http://www.FaerieMUD.org/
ruby -e “p 12383406064495388618631689469409153107.to_s(36).tr(‘z’,’ ')”

4

On Jul 21, 2007, at 8:37 PM, hemant wrote:

So far my experience with Ruby bindings for wordnet is, its stinking
piece of shit. Its about time someone took ownership of this project
and do something about it. I know I am not helping you much, but NLP
is the one thing that makes me jealous of pythonists.

Wow, okay. Care to elaborate on your experience?

As for taking ownership of the project – you’re welcome to write
your own
library which is not a “stinking piece of shit” and take ownership of
that.
My giving the source I wrote for myself for my own projects away does
not
give you some kind of guarantee that it’ll exactly suit your needs. I
don’t
have any record of your suggestions, patches, or problem reports,
which is
the only way I’d know that it wasn’t working for you. I’m happy to
help try
to diagnose problems, but if you’re just sitting quietly fuming to
yourself,
there isn’t much I can do for you.

I’m sorry you so strongly disapprove of my code. I, too, am jealous
of all
the great NLP tools that Python has (especially nltk), but I’m at least
trying to change that instead of sulking in a corner hurling insults at
people that are giving you stuff for nothing.

Not to mention link parser support in linguistic library,which doesn’t
work at all.

It certainly works for me, and I again haven’t had any reports from
you to
the contrary. My email address is in the README of every one of my
projects.

$ irb
Requiring ‘linguistics’ module…
Calling Linguistics::use( :en )
irb(main):001:0>
Linguistics::EN.has_link_parser?
=> true
irb(main):002:0> “Get the leftmost ball from the rack.”.sentence.verb
=> “get”
irb(main):003:0> “Get the leftmost ball from the rack.”.sentence.subject
=> nil
irb(main):004:0> “Get the leftmost ball from the rack.”.sentence.object
=> “ball”
irb(main):005:0> puts “Get the leftmost ball from the
rack.”.sentence.diagram

 +-------------------------Xp-------------------------+
 |       +--------------MVp-------------+             |
 |       +-----------Os-----------+     |             |
 |       |    +---------Ds--------+     +----Js---+   |
 +---Wi--+    |        +-----A----+     |   +--Ds-+   |
 |       |    |        |          |     |   |     |   |

LEFT-WALL get.v the leftmost[?].a ball.n from the rack.n .

=> nil

–
Michael G. [email protected]
Rubymage, Architect, Believer
The FaerieMUD Consortium http://www.FaerieMUD.org/
ruby -e “p 12383406064495388618631689469409153107.to_s(36).tr(‘z’,’ ')”

5

On 7/25/07, Michael G. [email protected] wrote:

your own
to diagnose problems, but if you’re just sitting quietly fuming to

work at all.
Linguistics::EN.has_link_parser?
±------------------------Xp-------------------------+
–
My apologies Michael, I honestly thought project went dormant, but i
am happy to be proved otherwise.

Yes, I got lots of segfaults when i tried to install the library and
in fact, I was able to install it on my machine eventually with
wordnet, link parser and everything, but it didn’t work in many places
as advertised. Couple of guys on #ruby-lang had similar experience.

Anyway, i hope my comments were well intended, I am not shy from
contributing back to your project, but only so much time we have.

6

Hi Michael,

Glad you still read the mailing lists. I am going to try reinstalling
with bdb 4.6 and see if that makes a difference. I may have received
the core dumps because I used the ruby bdb driver 0.5.9. I’ll see if
that solves the issues I’m having.

Cheers,
Henry

7

Well I recompiled using bdb version 4.6.18 and it did not core dump.
That means the key is getting the correct version of the bdb ruby
bindings (6.0). I’m still getting the same errors when running the
test cases.

Not sure what else to try from here.