Compliant_on(:ruby) in RubySpecs

Ruby IronRuby
1

I noticed that besides not_compliant_on to disable tests on specific
Ruby implementations, there is also compliant_on to enable tests only on
selected implementations. There are ~120 such instances. To make
IronRuby compliant, we not only have to reduce the number of *_tags.txt
files, we also have to add “:ir” to all uses of compliant_on.

However, this seems backwards. Shouldn’t we try to remove most uses of
compliant_on, and instead add tags for the implementations where the
test does not work? There may be a few valid uses of compliant_on for
tests which only work on, say, JRuby. However, only a handful of the
occurrences are of this kind. The vast majoriy include :ruby which means
it should be the standard.

Should it be a goal to change compliant_on so that it does not allow
:ruby?

Thanks,
Shri

2

I can check with Brian, but my understanding is as follows. The spec’s
are meant to be a complete packaged spec. Tag files are a convenience
for using the spec’s as a regression test. I feel, and I would guess
this to be Brian’s feelings, that when an implementation is complete (as
in everything is implemented), there should not be a set of tag files
for that implementation.

The other goal of the spec’s is to be a spec. To that goal, you
shouldn’t have to go outside of the spec’s to find out compliance
information. Including the information into the file makes it easier to
see what a specific implementation of Ruby supports. Keeping the code
DRY in tests isn’t as important as conveying information in those tests.

JD

From: Shri B.
Sent: Wednesday, January 07, 2009 12:26 AM
To: Jim D.
Cc: [email protected]
Subject: compliant_on(:ruby) in RubySpecs

I noticed that besides not_compliant_on to disable tests on specific
Ruby implementations, there is also compliant_on to enable tests only on
selected implementations. There are ~120 such instances. To make
IronRuby compliant, we not only have to reduce the number of *_tags.txt
files, we also have to add “:ir” to all uses of compliant_on.

However, this seems backwards. Shouldn’t we try to remove most uses of
compliant_on, and instead add tags for the implementations where the
test does not work? There may be a few valid uses of compliant_on for
tests which only work on, say, JRuby. However, only a handful of the
occurrences are of this kind. The vast majoriy include :ruby which means
it should be the standard.

Should it be a goal to change compliant_on so that it does not allow
:ruby?

Thanks,
Shri

3

I see the point of having all the information within the specs. It is
useful to read a spec and to be able to know how other implementations
behave (assuming the implementation is solid enough as to not cause a
huge number of usages of not_compliant_on in the test code base).

However, I think compliant_on(:ruby, :ir) for example should be replaced
with not_compliant_on(:jruby, :rubinius). That is, only opt-out should
be supported, not opt-in. Otherwise, other implementations may not run
the test even if they support that functionality.

From: Jim D.
Sent: Wednesday, January 07, 2009 10:57 AM
To: Shri B.
Cc: [email protected]
Subject: RE: compliant_on(:ruby) in RubySpecs

I can check with Brian, but my understanding is as follows. The spec’s
are meant to be a complete packaged spec. Tag files are a convenience
for using the spec’s as a regression test. I feel, and I would guess
this to be Brian’s feelings, that when an implementation is complete (as
in everything is implemented), there should not be a set of tag files
for that implementation.

The other goal of the spec’s is to be a spec. To that goal, you
shouldn’t have to go outside of the spec’s to find out compliance
information. Including the information into the file makes it easier to
see what a specific implementation of Ruby supports. Keeping the code
DRY in tests isn’t as important as conveying information in those tests.

JD

From: Shri B.
Sent: Wednesday, January 07, 2009 12:26 AM
To: Jim D.
Cc: [email protected]
Subject: compliant_on(:ruby) in RubySpecs

I noticed that besides not_compliant_on to disable tests on specific
Ruby implementations, there is also compliant_on to enable tests only on
selected implementations. There are ~120 such instances. To make
IronRuby compliant, we not only have to reduce the number of *_tags.txt
files, we also have to add “:ir” to all uses of compliant_on.

However, this seems backwards. Shouldn’t we try to remove most uses of
compliant_on, and instead add tags for the implementations where the
test does not work? There may be a few valid uses of compliant_on for
tests which only work on, say, JRuby. However, only a handful of the
occurrences are of this kind. The vast majoriy include :ruby which means
it should be the standard.

Should it be a goal to change compliant_on so that it does not allow
:ruby?

Thanks,
Shri

4

http://rubyspec.org/wiki/rubyspec/Guards

If you scroll to the compliant portion of the page, you’ll see that they
actually have different meanings, and they are based on the reference
implementation (CRuby). I had forgotten this until just now.

JD

From: Shri B.
Sent: Wednesday, January 07, 2009 11:39 AM
To: Jim D.
Cc: [email protected]
Subject: RE: compliant_on(:ruby) in RubySpecs

I see the point of having all the information within the specs. It is
useful to read a spec and to be able to know how other implementations
behave (assuming the implementation is solid enough as to not cause a
huge number of usages of not_compliant_on in the test code base).

However, I think compliant_on(:ruby, :ir) for example should be replaced
with not_compliant_on(:jruby, :rubinius). That is, only opt-out should
be supported, not opt-in. Otherwise, other implementations may not run
the test even if they support that functionality.

From: Jim D.
Sent: Wednesday, January 07, 2009 10:57 AM
To: Shri B.
Cc: [email protected]
Subject: RE: compliant_on(:ruby) in RubySpecs

I can check with Brian, but my understanding is as follows. The spec’s
are meant to be a complete packaged spec. Tag files are a convenience
for using the spec’s as a regression test. I feel, and I would guess
this to be Brian’s feelings, that when an implementation is complete (as
in everything is implemented), there should not be a set of tag files
for that implementation.

The other goal of the spec’s is to be a spec. To that goal, you
shouldn’t have to go outside of the spec’s to find out compliance
information. Including the information into the file makes it easier to
see what a specific implementation of Ruby supports. Keeping the code
DRY in tests isn’t as important as conveying information in those tests.

JD

From: Shri B.
Sent: Wednesday, January 07, 2009 12:26 AM
To: Jim D.
Cc: [email protected]
Subject: compliant_on(:ruby) in RubySpecs

I noticed that besides not_compliant_on to disable tests on specific
Ruby implementations, there is also compliant_on to enable tests only on
selected implementations. There are ~120 such instances. To make
IronRuby compliant, we not only have to reduce the number of *_tags.txt
files, we also have to add “:ir” to all uses of compliant_on.

However, this seems backwards. Shouldn’t we try to remove most uses of
compliant_on, and instead add tags for the implementations where the
test does not work? There may be a few valid uses of compliant_on for
tests which only work on, say, JRuby. However, only a handful of the
occurrences are of this kind. The vast majoriy include :ruby which means
it should be the standard.

Should it be a goal to change compliant_on so that it does not allow
:ruby?

Thanks,
Shri