Range#=== operator

Ruby IronRuby
1

Forwarding this onto the list for Peter who’s having mail problems …

From: Peter Bacon D. [mailto:[email protected]]
Sent: Thursday, November 08, 2007 2:35 AM
To: John L. (DLR)
Subject: FW: Range#=== operator

I have noticed an interesting feature in the current snapshot. If you
test a numeric range against a non-numeric value, such as a string you
get a System.ArgumentException whereas other Ruby implentations return
false.
E.g.

IronRuby:
(1…5) === ‘x’
gives
System.ArgumentException: wrong number or type of arguments for `<=>’

CRuby:
(1…5) === ‘x’
gives
false

There are two interesting points here:

The first is that debugging code with lightweight code generation is
really difficult; tracking down what actually causes the exception to be
thrown is hidden by the DynamicSite layer. If you break on the
exception being thrown you have already lost the trace back to the
original problem. This is because the exception is not thrown directly
but via binding to an invocation site. Only by trawling through the
code and putting break points all over the place was I able to ascertain
that the problem is that when you apply the === operator to the Range
the begin and end attributes of Range have the <=> operator applied to
the argument (in this case you have something like FixNum <=>
MutableString) and the current system is unable to find a suitable
method to invoke. See RubyMethodGroupInfo.SetInvocationRuleInternal()
and RubyMethodGroupInfo.GetMethodCandidate(). I.E. There is no implicit
conversion between FixNum and MutableString. How do you guys debug
situations like this?

The second point is to do with this result itself. Are you aiming to
produce identical results to other Ruby implementations or are there
other criteria for correctness. Arguably the expression given above is
invalid as there is no correct type conversion. Equally false would
seem a reasonable result; certainly it is false that ‘x’ is in the range
of (1…5). If false is what you want, what is the accepted way of
dealing with the situation? Do you want to fiddle with the protocols
for type conversions? I imagine not. Is Range a specific case, in
which case you would code a specific version of <=> operator for it?

Interestingly the following code returns false as expected:
1 == ‘x’
gives
false
The difference here is that the == operator is being used directly to
test equality whereas early the <=> operator was being used to compare
the values previously.

Of interest, both IronRuby and CRuby throw argument exceptions/errors
for the following comparisons.
1 > ‘x’
and
1 < ‘x’

What are the correct semantics here? What do you think should be done
to deal with this discrepancy?

Regards,
Pet

2

Here’s a couple of debugging tips:
-X:SaveAssemblies -X:StaticMethods command line options,
use these when you need to see into a DynamicMethod. Instead of
DynamicMethod’s we’ll create methods via Reflection.Emit and we’ll save
them to disk at process exit as snippets???.dll (where ??? is a number,
usually 1). You can then open the assembly w/ reflector or ildasm to
see what’s going on in the generated code. This is especially useful if
you have a small repro snippet like the one here as snippets will be
pretty easy to dig through and find the particular piece of generated
code you want.

            When debugging rules in general the best place to put a

break point is either the language’s RubyActionBinder.MakeRule - where
you’ll see all language specific binding kick in (and boy does Ruby have
a lot of language specific bindings currently :)!) or you can look at
ActionBinder.GetRule to see the whole process take place. Unfortunately
I don’t know the ruby code base well enough to give you better hints
than that (but if you want know how to debug IronPython comparisons…)

            A final useful one is -X:ExceptionDetail - this will

cause the CLR exception to get printed instead of a more filtered
language exception. Also if the exception comes from a DynamicSite I’ll
usually try and find the exceptions message somewhere in the source code
which usually is in some rule generation code.

Hopefully that’s helpful on the debugging side of things - and maybe
some of this you’ve figured out but it might be useful for other people
as well.

From: [email protected]
[mailto:[email protected]] On Behalf Of John L. (DLR)
Sent: Thursday, November 08, 2007 7:27 AM
To: [email protected]
Subject: [Ironruby-core] Range#=== operator

Forwarding this onto the list for Peter who’s having mail problems …

From: Peter Bacon D. [mailto:[email protected]]
Sent: Thursday, November 08, 2007 2:35 AM
To: John L. (DLR)
Subject: FW: Range#=== operator

I have noticed an interesting feature in the current snapshot. If you
test a numeric range against a non-numeric value, such as a string you
get a System.ArgumentException whereas other Ruby implentations return
false.
E.g.

IronRuby:
(1…5) === ‘x’
gives
System.ArgumentException: wrong number or type of arguments for `<=>’

CRuby:
(1…5) === ‘x’
gives
false

There are two interesting points here:

The first is that debugging code with lightweight code generation is
really difficult; tracking down what actually causes the exception to be
thrown is hidden by the DynamicSite layer. If you break on the
exception being thrown you have already lost the trace back to the
original problem. This is because the exception is not thrown directly
but via binding to an invocation site. Only by trawling through the
code and putting break points all over the place was I able to ascertain
that the problem is that when you apply the === operator to the Range
the begin and end attributes of Range have the <=> operator applied to
the argument (in this case you have something like FixNum <=>
MutableString) and the current system is unable to find a suitable
method to invoke. See RubyMethodGroupInfo.SetInvocationRuleInternal()
and RubyMethodGroupInfo.GetMethodCandidate(). I.E. There is no implicit
conversion between FixNum and MutableString. How do you guys debug
situations like this?

The second point is to do with this result itself. Are you aiming to
produce identical results to other Ruby implementations or are there
other criteria for correctness. Arguably the expression given above is
invalid as there is no correct type conversion. Equally false would
seem a reasonable result; certainly it is false that ‘x’ is in the range
of (1…5). If false is what you want, what is the accepted way of
dealing with the situation? Do you want to fiddle with the protocols
for type conversions? I imagine not. Is Range a specific case, in
which case you would code a specific version of <=> operator for it?

Interestingly the following code returns false as expected:
1 == ‘x’
gives
false
The difference here is that the == operator is being used directly to
test equality whereas early the <=> operator was being used to compare
the values previously.

Of interest, both IronRuby and CRuby throw argument exceptions/errors
for the following comparisons.
1 > ‘x’
and
1 < ‘x’

What are the correct semantics here? What do you think should be done
to deal with this discrepancy?

Regards,
Pet

3

Thanks Dino,

I did use the SaveAssemblies and StaticMethods options – cribbed from
one of
John’s blog postings, I think. From the look of it (the exception is
pretty
comprehensive) there is no filtered language exception going on in
IronRuby
at this stage. Or is this just when in debug? I haven’t tried release.

The trouble with the generated code was that it appeared that the
exception
had already been flagged for execution at generation time; One of the
snippets was an action to create and throw the ArgumentException [See
RubyOps.MakeInvalidArgumentTypesError].

The way I debugged it, as you mention toward the bottom of your mail,
was to
search the code that creates the specific ArgumentException dynamic
action
(searching for the error string) and then find which bits of code use
it. I
also experimented with placing break points at various places around the
DLR
in the AST generation and action binding code.

I guess it just requires a bit of getting used to and knowing where to
look.
Trawling through the RubyOnRails code generation (e.g. ActiveRecord) is
also
really hard work too.

One other question though: How do you decide whether to use dynamic
sites
to trigger code rather than calling code directly when in coding in C#.
For
instance, why does Protocols.Compare invoke _CompareSharedSite rather
than
call a C# function directly? Sorry if I am being dumb.

By the way, a simple way to get the desired results for the case
equality
issue mentioned before is to put a try/catch around the Compare and
return
false if an ArgumentException is thrown. See the attached patch. Is
this
too much of a hack or acceptable?

Pete

4

On 11/8/07, Peter Bacon D. [email protected] wrote:

One other question though: How do you decide whether to use dynamic
sites to trigger code rather than calling code directly when in coding in
C#. For instance, why does Protocols.Compare invoke _CompareSharedSite
rather than call a C# function directly? Sorry if I am being dumb.

Even builtins like the String or Range classes can have their comparison
functions overridden by user code. So you really need to perform the
comparison “in Ruby”.

5

Of course, you are quite right. Thanks for that.

6

Curt H. wrote:

perform the comparison “in Ruby”.
Exactly. The trick with library methods is to figure out what pattern
MRI uses. Typically, they do a method lookup on the type, which in our
world translates into a DynamicSite. However there are some places where
they just call some library method directly and bypass any
overridden/monkeypatched method. So it requires some experimentation :).

Protocols.Compare is one of the stranger things I’ve discovered. MRI
will call <=>, and then call < and > on the result object… which of
course can be overridden to do whatever you want. Because Compare is
kind of crazy I’m not surprised we get it slightly wrong.

Peter Bacon D. wrote:

By the way, a simple way to get the desired results for the case
equality issue mentioned before is to put a try/catch around the
Compare and return false if an ArgumentException is thrown. See the
attached patch. Is this too much of a hack or acceptable?

Peter, I think the right fix here is for Fixnum <=> to return nil if it
gets passed a non-comparable object. That seems to be the standard MRI
behavior for <=>:

irb(main):013:0> (1…5) === ‘x’
=> false

And then I think our Protocols.Compare needs to check if <=> returned
null, and not call > or < if that was the case. Indeed, if you define
‘<’ and ‘>’ on NilClass it doesn’t change the compare behavior:

irb(main):014:0> class NilClass
irb(main):015:1> def < x; puts ‘<’; 0; end
irb(main):016:1> def > x; puts ‘>’; 0; end
irb(main):017:1> end
=> nil
irb(main):018:0> (1…5) === ‘x’
=> false

Of course, it might require more experiments to get it exactly right for
all cases.

  • John
7

Would it be fair to follow this as a general rule: when coding Ruby
methods
in C#, if you are calling code that is exposed as a method or operator
in a
Ruby class or module then invoke it via a DynamicSite, if it is purely
private code to the C# method then call it directly?
Pete

8

Peter Bacon D.:

Would it be fair to follow this as a general rule: when coding Ruby
methods
in C#, if you are calling code that is exposed as a method or operator
in a
Ruby class or module then invoke it via a DynamicSite, if it is purely
private code to the C# method then call it directly?

Yes. But since we alias types it’s sometimes not blindingly obvious that
you’ll need a site. The good news is that the specs do a very good job
at catching cases where there’s some observable behavior.

-John

9

John M. wrote:

Peter, I think the right fix here is for Fixnum <=> to return nil if it
gets passed a non-comparable object. That seems to be the standard MRI
behavior for <=>:

irb(main):013:0> (1…5) === ‘x’
=> false

Oops, my example is wrong, it should be:

irb(main):001:0> 1 <=> ‘x’
=> nil

IronRuby:

1 <=> ‘x’
System.ArgumentException: wrong number or type of arguments for `<=>’

So we definitely need another overload for Fixnum#<=>, and possibly
other places as well that implement <=>.

  • John
10

Peter Bacon D. wrote:

Would it be fair to follow this as a general rule: when coding Ruby methods
in C#, if you are calling code that is exposed as a method or operator in a
Ruby class or module then invoke it via a DynamicSite, if it is purely
private code to the C# method then call it directly?
Pete

There is no hard rule for Ruby; sometimes you are supposed to dispatch
for functionality, sometimes you call directly. There’s no one true way
unfortunately.

Of course, as you figure out how different methods work, document that
:slight_smile:

  • Charlie
11

Peter Bacon D. wrote:

Would it be fair to follow this as a general rule: when coding Ruby methods
in C#, if you are calling code that is exposed as a method or operator in a
Ruby class or module then invoke it via a DynamicSite, if it is purely
private code to the C# method then call it directly?
Pete

This rule won’t hold for all cases, since frequently Ruby’s of two minds
on whether you can override some method X used by method Y such that Y
will see it. Sometimes Y will do a full dynamic dispatch and sometimes
it will do a direct call, even to the same piece of code that’s bound to
external (i.e. in-script) calls to X.

  • Charlie
12

Charles Oliver N. wrote:

it will do a direct call, even to the same piece of code that’s bound to
external (i.e. in-script) calls to X.

Pardon the duplication; the other mail must have gotten delayed…

  • Charlie
13

Hello,

Where is help needed? Reading through the list archive, it looks like
people are working on the zlib and yaml implementation. Is there help
needed there still? Where would my time spent working be most
beneficial?

Thanks,
Mike

14

As I’ve seen there’s plenty of things to do!

By the way, I started to work on StringIO but never asked if someone
else is already on that… is there anyone?

Eduardo Flores

15

Peter Bacon D.:

It is a shame that the code is not being developed directly against the
RubyForge repository as this definitely gives a feeling of them and us
and
makes it difficult to see what is being done, what progress is being
made
and it creates extra work for the maintainers ensuring that the
internal and
external repositories are synched up - there have been a number of
issues
recently about dodgy file paths and project references.

Unfortunately, as much as we would love to make the “two repository”
problem go away, the reality is that too much of our internal testing
infrastructure (our SNAP queue, which is a lab of 20+ computers which
run our check-in, sign-off, performance tests among others) is tightly
coupled to TFS.

There’s also another problem that isn’t visible to external folks: the
DLR is undergoing a massive refactoring. Internally, all of the projects
that depend on DLR (including some unannounced projects) live inside of
the same TFS repository. What this means is that if a DLR change breaks
something, we have to fix it otherwise SNAP won’t allow the check-in.
This forcing function lets us keep all of our code sync’d with DLR
changes which is goodness. It also helps us understand in real time the
impact that DLR changes have on consumers. Think about this as “real
time continuous integration”.

The latter case explains why we haven’t sync’d the SVN repository in two
weeks. We had a large change (the stuff in the demos we had for
RubyConf) that included some DLR changes that broke all sorts of stuff
in, um, interesting ways. It’s taken us this long to get the check-in
through the queue (it went in yesterday), so we’re preparing to push a
new set of sources out today.

Some kind of work tracking tool would be beneficial, even if it was
just a
priority list of things that needed to be done so that people out here
could
pick up something. I suppose the problem is also that, for instance,
no one
knows who I am and what quality of code I can produce and so it would
be
risky to allocate work to me. How could this be solved.

I think what we can do is get together at the beginning of each week
with a quick ‘scrum’ via email (probably better since we work across all
sorts of different time zones). This way we can announce the things that
we’re working on, so that we’re sure that we’re not stepping on other
folks’ toes. Sound reasonable? I’ll kick off a scrum mail on Monday.

b) The
quality/style/efficiency is not accepted by the maintainers and my code
goes
to waste.

If this were to happen, I don’t think that your code goes to waste.
You’ve learned something valuable and the best way of improving is to
get continuous, critical feedback on the progress of your work.

Like so many other things, it’s about confidence - and there’s no better
way of building confidence (in both directions) by submitting code that
you’ve thought about, receiving / incorporating feedback and repeating.

Thanks,
-John

16

I was thinking about this issue last night. I am interested in working
on
the number stuff (Numeric, BigNum, FixedNum, Float, etc) but I am
concerned
that a) Someone internally is already doing this and just as I am about
to
send in a patch a new splat of code gets popped across from the internal
source control to SVN and all has been wasted; b) The
quality/style/efficiency is not accepted by the maintainers and my code
goes
to waste.

It is a shame that the code is not being developed directly against the
RubyForge repository as this definitely gives a feeling of them and us
and
makes it difficult to see what is being done, what progress is being
made
and it creates extra work for the maintainers ensuring that the internal
and
external repositories are synched up - there have been a number of
issues
recently about dodgy file paths and project references.

Some kind of work tracking tool would be beneficial, even if it was just
a
priority list of things that needed to be done so that people out here
could
pick up something. I suppose the problem is also that, for instance, no
one
knows who I am and what quality of code I can produce and so it would be
risky to allocate work to me. How could this be solved.

My personal aim is to send in a few patches and hopefully, get enough
feedback so that I am capable of producing code that is acceptable and
fits
in with the way the internal developers are working. Then may be the
risk
of me putting time in would be reduced for me and the risk of the
maintainers accepting that I am going to take on a task is reduced.

My 2 pence

Pete

17

Eduardo A. Flores Verduzco:

As I’ve seen there’s plenty of things to do!

By the way, I started to work on StringIO but never asked if someone
else is already on that… is there anyone?

Nobody is working on StringIO to my knowledge. If you want to stake a
claim to it, please add your name and the class in the wiki page:

http://ironruby.rubyforge.org/wiki/wiki.pl?Core

Thanks!
-John

18

Peter Bacon D.:

I was thinking about this issue last night. I am interested in working
on the number stuff (Numeric, BigNum, FixedNum, Float, etc) but I am
concerned that a) Someone internally is already doing this and just as
I am about to send in a patch a new splat of code gets popped across
from the internal source control to SVN and all has been wasted; b) The
quality/style/efficiency is not accepted by the maintainers and my code
goes to waste.

If you want to look at numeric stuff, go for it. Our numeric classes
could
really use some love. I know Tomas & I aren’t working on those.

Regarding quality/style/efficiency–hey, my code gets rejected too
sometimes :slight_smile:

It is a shame that the code is not being developed directly against the
RubyForge repository as this definitely gives a feeling of them and us
and makes it difficult to see what is being done, what progress is
being made and it creates extra work for the maintainers ensuring that
the internal and external repositories are synched up - there have been
a number of issues recently about dodgy file paths and project
references.

For what it’s worth, I do a lot of my work against the SVN layout, so I
feel that pain too. The rakefile is still not perfect yet, but John is
working on it. Well, at least I complain to him when it’s not working :slight_smile:

Some kind of work tracking tool would be beneficial, even if it was
just a priority list of things that needed to be done so that people
out here could pick up something. I suppose the problem is also that,
for instance, no one knows who I am and what quality of code I can
produce and so it would be risky to allocate work to me. How could
this be solved.

I really liked what Evan P. said at RubyConf about “free flowing
commit
bit”. We can’t implement it quite like Rubinius but I think the idea is
anyone that does a good patch is trusted as someone who can do good
quality
work.

My personal aim is to send in a few patches and hopefully, get enough
feedback so that I am capable of producing code that is acceptable and
fits in with the way the internal developers are working. Then may be
the risk of me putting time in would be reduced for me and the risk of
the maintainers accepting that I am going to take on a task is reduced.

I look forward to reviewing your patches!

Also, keep in mind some of the feedback is just because there’s a lot of
details to learn about, that no one could be expected to know without
some
experience with the system. I had a decent ramp-up period myself to get
familiar with all of the stuff that goes in IronRuby.

Cheers,
John

19

Mike Jarosch:

Where is help needed? Reading through the list archive, it looks like
people are working on the zlib and yaml implementation. Is there help
needed there still? Where would my time spent working be most
beneficial?

I believe that Michael Moore started to look at yaml, but then got busy
with other things. Feel free to take a look, Mike - I believe that folks
who have looked at yaml have been converging on porting JvYaml as a good
starting point.

Thanks,
-John