Need for speed -> a C extension?

On Wed, Apr 20, 2011 at 8:55 AM, Martin H. [email protected] wrote:

$ ruby -d j.rb ->

http://pastie.org/1815275

"Exception `ArgumentError’ at
/Users/maasha/maasha_install/lib/ruby/1.9.1/x86_64-darwin10.6.0/digest/md5.bundle

  • undefined class/module Digest::Base"

Also, can you just try “require ‘digest/md5’” in irb?

irb(main):001:0> require ‘digest/md5’
ArgumentError: undefined class/module Digest::Base
from internal:lib/rubygems/custom_require:29:in require' from <internal:lib/rubygems/custom_require>:29:inrequire’
from (irb):1
from /Users/maasha/maasha_install/bin/irb:12:in `’

Okay, so the issue is in your ruby not RubyInline.

I updated my machine to 10.6.7 (uname: 10.7.0 :slight_smile: and my previously
compiled 1.9.2 still works correctly. Can you try to install a fresh
build in another prefix (by hand, RVM, or homebrew) and see if the
same behavior is still present?

Thanks.

Martin H. [email protected] wrote:

Linux bixeonws 2.6.32-30-server #59-Ubuntu SMP Tue Mar 1 22:46:09 UTC
2011 x86_64 GNU/Linux

What is this Digest::Base anyway?

“This abstract class provides a common interface to message digest
implementation classes written in C.” Maybe both machines are missing
something so it didn’t compile properly?

Can we have it shot and killed?

I don’t think so :slight_smile:

On Wed, Apr 20, 2011 at 11:34 AM, [email protected] wrote:

What is this Digest::Base anyway?

“This abstract class provides a common interface to message digest
implementation classes written in C.” Maybe both machines are missing
something so it didn’t compile properly?

$ cd ext/digest/
$ make
make: Nothing to be done for `all’.

$ make clean
$ make all
gcc -I. -I…/…/.ext/include/x86_64-linux -I…/…/./include
-I…/…/./ext/digest -DRUBY_EXTCONF_H=“extconf.h” -fPIC -O3 -ggdb
-Wextra -Wno-unused-parameter -Wno-parentheses -Wpointer-arith
-Wwrite-strings -Wno-missing-field-initializers -Wno-long-long -fPIC
-o digest.o -c digest.c
gcc -shared -o …/…/.ext/x86_64-linux/digest.so digest.o -L. -L…/…
-L. -rdynamic -Wl,-export-dynamic -Wl,-R
-Wl,/nas/fs/users/cameron/unix/.rvm/rubies/ruby-1.9.2-p180/lib
-L/nas/fs/users/cameron/unix/.rvm/rubies/ruby-1.9.2-p180/lib -lruby
-lpthread -lrt -ldl -lcrypt -lm -lc
cp …/…/./ext/digest/lib/digest/hmac.rb …/…/.ext/common/digest
cp …/…/./ext/digest/lib/digest.rb …/…/.ext/common
cp …/…/./ext/digest/digest.h …/…/.ext/include/ruby

(Linux 2.6.32-71.24.1.el6.x86_64)

On Wed, Apr 20, 2011 at 5:18 PM, Martin H. [email protected] wrote:

What is this Digest::Base anyway? Can we have it shot and killed?

http://www.ruby-doc.org/stdlib/libdoc/digest/rdoc/classes/Digest/Base.html

If you still have the log for your Ruby compile lying around (and the
config file ./configure has generated), you could grep for OpenSSL
errors (that’s what Ruby uses for SSL and, I think, crypto in
general).


Phillip G.

Though the folk I have met,
(Ah, how soon!) they forget
When I’ve moved on to some other place,
There may be one or two,
When I’ve played and passed through,
Who’ll remember my song or my face.

On Wed, Apr 20, 2011 at 10:24 AM, Phillip G.
[email protected] wrote:

On Wed, Apr 20, 2011 at 5:18 PM, Martin H. [email protected] wrote:

What is this Digest::Base anyway? Can we have it shot and killed?

http://www.ruby-doc.org/stdlib/libdoc/digest/rdoc/classes/Digest/Base.html

If you still have the log for your Ruby compile lying around (and the
config file ./configure has generated), you could grep for OpenSSL
errors (that’s what Ruby uses for SSL and, I think, crypto in
general).

I just remembered something I heard on IRC the other night, that Ruby
1.9 sometimes has a problem with MacPorts’s OpenSSL. I don’t know the
details, but that might be something to consider, if you Martin is
using MacPorts.

MacPorts ticket: https://trac.macports.org/ticket/28582

Sorry guys, I have been busy with a few other things before continuing
on this one (also solving the pesky issue with my RubyInline install).

@Robert.

I have been thinking hard about your comments - these contain a lot of
programming insight of the kind you dont get from a “learnings
book”. However, I struggle with grasping the
wisdom:

"Frankly, I find your code has a design issue: it seems you mix data
and iteration in a single class. This is visible from how #match
works

def match(pattern, pos = 0, max_edit_distance = 0)
@pattern = pattern
@pos = pos
@max_edit_distance = max_edit_distance
@vector = vector_init

IMHO it would be better to separate representation of the sequence and
the matching process. The matcher then would only carry a reference
to the sequence and all the data it needs to do matching.
"

What is this “mixing data with iteration in a single class”? To me you
have data and then you iterate - what exactly is the problem? What
should I do for separating these?

&&

“Maybe on the interface, but you create side effects on the String (Seq
in your case). This is neither a clean separation (makes classes
bigger and harder to understand) nor is it thread safe (e.g. if you
want to try to concurrently match several patterns against the same
sequence).”

Again, “separation”, but what is the problem with big classes? When is a
class too big? And how to divide your code in the best way?

Also, I managed to get down to a single vector, but I think I may have
more .dup’s than needed - though removing any causes erroneous output:

http://pastie.org/1902844

Cheers,

Martin

On 15.05.2011 11:16, Martin H. wrote:

"Frankly, I find your code has a design issue: it seems you mix data

IMHO it would be better to separate representation of the sequence and
the matching process. The matcher then would only carry a reference
to the sequence and all the data it needs to do matching.
"

What is this “mixing data with iteration in a single class”? To me you
have data and then you iterate - what exactly is the problem? What
should I do for separating these?

I am referring to http://pastie.org/1808127 : The issue with the code
presented lies in the state: an instance of Seq represents a sequence
of items (in your case amino acids, I guess). You need state to
represent this sequence. This state is stored in instance variables of
an instance of class Seq.

The first thing that your method #match (see above) does, is to set some
other instance variables of Seq. This poses a problem if

  • the Seq instance is frozen, i.e. immutable,

  • more than one matching processes are under way concurrently (i.e.
    #match is invoked from more than one thread at a time).

The reason is that you mixed state needed to represent your sequence
with state needed to execute the matching process. Apart from the
problems listed above this also makes code harder to read and more error
prone. For example, you might modify the Seq implementation and
accidentally reuse the name of an instance variable which then can make
your code break in unpredictable ways.

“Maybe on the interface, but you create side effects on the String (Seq
in your case). This is neither a clean separation (makes classes
bigger and harder to understand) nor is it thread safe (e.g. if you
want to try to concurrently match several patterns against the same
sequence).”

Again, “separation”, but what is the problem with big classes? When is a
class too big? And how to divide your code in the best way?

There is no easy answer to that. In this case you violate modularity by
lumping too many different functionalities together in a single class.
Apart from the advantage to avoid mischief laid out above by decoupling
the matching process from the sequence representation you also gain
modularity if you let the matching process only rely on the public
interface of Seq. That way you can even have different implementations
of sequences which can all be scanned by the same representation. I am
not saying you should necessarily do this as efficient matching might
also need some knowledge of Seq internals, but this is to illustrate
what kind of things you should consider when designing classes.

Also, I managed to get down to a single vector, but I think I may have
more .dup’s than needed - though removing any causes erroneous output:

http://pastie.org/1902844

That code still stores matching state in the Seq instance. And you have
quite a few #dups which have object creation overhead. I think you
should better work with two Array of Score instances and swap them at
each sequence position. IMHO you do not even need to reinitialize Score
instances because of your dynamic programming approach it is guaranteed
that you access instances sequentially from index 0 on and need to
access at most last[i - 1], last[i] and current[i - 1].

Kind regards

robert

On Tue, Apr 19, 2011 at 3:12 PM, Martin H. [email protected]alid wrote:

Ruby I compiled and installed myself. Inline was installed: gem install
RubyInline

I hesitate to suggest this, given that if you’ve compiled and
installed Ruby yourself you’re probably well aware of the various
possibilities. But: does the extension have to be in C and does it
have to be the Ruby you’ve compiled and installed? If not, and you’re
having problems getting the C extensions you want to work with Ruby,
have you considered using JRuby and Java? I’ve found the integration
of Java with JRuby to be fairly easy: I have had some problems, but
that’s due to my ignorance, not JRuby or Java, and I’ve usually found
a way round my problem quickly. And using JRuby with Java for the bits
that need to be fast has had a similar speed to compiled FreePascal.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs