Kirbybase Hacks NilClass

Ruby
1

I’m examining Kirbybase for use it a project and was surprised to
find this bit of code in it:

#-----------------------------------------------------------------------

NilClass

#-----------------------------------------------------------------------

class NilClass

#-----------------------------------------------------------------------
# method_missing

#-----------------------------------------------------------------------
#
# This code is necessary because if, inside a select condition code
# block, there is a case where you are trying to do an expression
# against a table field that is equal to nil, I don’t want a method
# missing exception to occur. I just want the expression to be
nil. I
# initially had this method returning false, but then I had an
issue
# where I had a YAML field that was supposed to hold an Array.
If the
# field was empty (i.e. nil) it was actually returning false
when it
# should be returning nil. Since nil evaluates to false, it
works if I
# return nil.
# Here’s an example:
# #select { |r| r.speed > 300 }
# What happens if speed is nil (basically NULL in DBMS terms)?
Without
# this code, an exception is going to be raised, which is not
what we
# really want. We really want this expression to return nil.
def method_missing(method_id, *stuff)
return nil
end
end

This has been a popular discussion lately, but I just don’t think
loading a database library should fundamentally change the language.
ActiveRecord doesn’t hack NilClass and we seem to do okay with that.
Why can’t the above example just be coded as:

select { |r| r.speed and r.speed > 300 }

or:

select { |r| r.speed > 300 rescue false }

Are Kirbybase users really liking this behavior?

James Edward G. II

2

Hi –

On Sun, 5 Mar 2006, James Edward G. II wrote:

end

select { |r| r.speed > 300 rescue false }

Or, depending on circumstances: r.speed.to_i > 300. I would
definitely steer clear of adding method_missing to nil. Tons of code,
including all the Ruby code in the standard library, is written
without it. I don’t know how often it would matter, but the main
thing is that nil behaves in an unambiguous, official way, and
changing that is risky.

David

–
David A. Black ([email protected])
Ruby Power and Light (http://www.rubypowerandlight.com)

“Ruby for Rails” chapters now available
from Manning Early Access Program! Ruby for Rails

3

On Sun, 2006-03-05 at 01:37 +0900, James Edward G. II wrote:

#-----------------------------------------------------------------------
issue
Without
ActiveRecord doesn’t hack NilClass and we seem to do okay with that.
James Edward G. II
I’ve never used KirbyBase, but from this it looks like it could just
catch NoMethodError in #select…? Introducing this new behaviour to nil
could seriously suck in at least one case I can think of: while coding
stuff up I often get a nomethoderror for nil telling me I’m off by one
or have passed silly arguments…

4

[email protected] wrote:

#---------------------------------------------------------------------------
ActiveRecord doesn’t hack NilClass and we seem to do okay with that.
Why can’t the above example just be coded as:

select { |r| r.speed and r.speed > 300 }

or:

select { |r| r.speed > 300 rescue false }

Or, depending on circumstances: r.speed.to_i > 300. I would …

Thanks very much for the feedback on my overriding
NilClass#medthod_missing in KirbyBase. I don’t think the three examples
shown above would really be acceptable, because, could you imagine
having to do this for every query where there was a chance that a
#method_missing could be thrown? I think that would get old very fast.

However, the last example does give me an idea. I’m going to take a
look at the KBTable#select method and see if I can wrap the query
processing logic in a begin/rescue block. It may be as simple as what
David shows here, have the rescue return false. The problem I can see
is when there is an OR condition in the block, i.e.:

#select { |r| r.speed > 300 or r.range < 900 }

Now, here’s a very simplified version of KBTable#select, with a
proposed adding of a begin/rescue block around the query instead of the
current redefining of NilClass#method_missing:

def select(&query_block)
valid_matches = []
table_records.each do |table_record|
begin
if query_block(table_record)
valid_matches << table_record
end
rescue NoMethodError
end
end
end

Now, lets say we are looping through the table records and we get to a
record where :speed is nil and :range is 800. According to the query
block, this record should qualify as a valid match. But, if I don’t
override NilClass#method_missing to return fasle, but, instead, try to
catch it with a begin/rescue block like I show in the code above, the
record will not be added to valid matches. The problem with the code
above is that I can’t get granular enough. There is no way to get
inside the query block and put a begin/rescue around each condition
within the block.

If I could get enough info about the calling method, when I’m inside
NilClass#method_missing, then maybe I could conditionally return a false
only when it was apparent that the #method_missing exception was raised
in KBTable#select. But, the arguments passed into #method_missing are
the symbol of the method, in this case :> and an array containing the
arguments passed to the missing method, in this case [300]; not enough
info for me to determine if this came from KBTable#select.

I would be very interested in any suggestions, ideas, or feedback on
this issue.

Jamey

5

Jamey C. wrote:

This has been a popular discussion lately, but I just don’t think

the current redefining of NilClass#method_missing:
end
condition within the block.
this issue.
Hey, I think I will just respond to myself with a suggestion. :slight_smile:

What has been kicked around when this question has come up before is for
me to create something like a NULL class and use this to designate a
null value in the database, instead of using nil. This would take care
of having to override NilClass#method_missing. It would bring up a few
other problems, namely, what if someone wants to store the string value
NULL in a field. But I imagine these could be overcome.

If this is indeed a really big issue (meaning KirbyBase’s overriding of
NilClass#method_missing), then I could get to work on this. Thoughts?

Jamey

6

On 3/4/06, Jamey C. [email protected] wrote:

What has been kicked around when this question has come up before is for
me to create something like a NULL class and use this to designate a
null value in the database, instead of using nil. This would take care
of having to override NilClass#method_missing. It would bring up a few
other problems, namely, what if someone wants to store the string value
NULL in a field. But I imagine these could be overcome.

I was discussing this with one of the guys from new_haven.rb, Matthew
Desmarais
when we were heading down to NYC.rb

A NULL Class was the best thing we came up with too… even though
this might create some issues within KirbyBase that will need to be
worked around, it’d make this great piece of software much less scary
to people who might like to use it.

7

On Mar 4, 2006, at 11:56 AM, Jamey C. wrote:

Thanks very much for the feedback on my overriding
NilClass#medthod_missing in KirbyBase. I don’t think the three
examples shown above would really be acceptable, because, could you
imagine having to do this for every query where there was a chance
that a #method_missing could be thrown? I think that would get
old very fast.

There are a few problems with this line of thinking, in my opinion:

  1. You change the semantics of the language to get around this.
    Overlooking the dangers of this for now, how is someone even suppose
    to know to write non-standard Ruby when using the Kirbybase library?

  2. If the normal Ruby behavior is bothering a user, they can add the
    one-line hack with the knowledge of the trade-off they are choosing.

  3. It’s just too dangerous. It’s very possible you break other Ruby
    libraries with a hack like this. We don’t want to not be able to use
    Kirbybase for a project because it breaks library XYZ.

I’m really not trying to be mean. I am very much liking what I have
seen about Kirbybase, which is why I’m telling you about the barrier
that is keeping me from using it. :wink:

James Edward G. II

8

On Mar 4, 2006, at 12:04 PM, Jamey C. wrote:

What has been kicked around when this question has come up before
is for me to create something like a NULL class and use this to
designate a null value in the database, instead of using nil.

I know I would feel a lot safer with that. The only minus there is
that NULL will be a true value, again with surprising behavior for
people who are expecting normal Ruby. If we have to have one though,
I would prefer this one by far.

It would bring up a few other problems, namely, what if someone
wants to store the string value NULL in a field. But I imagine
these could be overcome.

Could it be stored using Kirbybase’s escapes (&NULL;)?

James Edward G. II

9

[Jamey C. [email protected], 2006-03-04 18.56 CET]

However, the last example does give me an idea. I’m going to take a
look at the KBTable#select method and see if I can wrap the query
processing logic in a begin/rescue block. It may be as simple as what
David shows here, have the rescue return false. The problem I can see
is when there is an OR condition in the block, i.e.:

#select { |r| r.speed > 300 or r.range < 900 }
[…]
Now, lets say we are looping through the table records and we get to a
record where :speed is nil and :range is 800. According to the query
block, this record should qualify as a valid match. But, if I don’t
override NilClass#method_missing to return fasle, but, instead, try to
catch it with a begin/rescue block like I show in the code above, the
record will not be added to valid matches. The problem with the code
above is that I can’t get granular enough. There is no way to get
inside the query block and put a begin/rescue around each condition
within the block.

But doesn’t NULL always propagate in “database semantics”? I mean, if
any
subexpression is NULL, the whole expression becomes NULL. It’s the
expected
behaviour (but maybe not to the users of KirbyBase). In SQL, normally,
one
uses NVL(a, b) (with the meaning ‘a.nil? ? b : a’) to guard
subexpressions
against NULL.

10

James Edward G. II wrote:

It would bring up a few other problems, namely, what if someone wants
to store the string value NULL in a field. But I imagine these could
be overcome.

Could it be stored using Kirbybase’s escapes (&NULL;)?

James Edward G. II

Without thinking too deeply about this, I came up with using a marker
value to designate non-NULL values. When a field is written, use ‘*’ as
the first character and then write the field’s value. NULL is written
as the empty string. When reading a field, test to see if it’s the
empty string. If it is then your value is NULL, otherwise toss the
first character and read your non-null value after that.

This is probably pretty naive and it would introduce some extra
performance overhead though. Just a thought.

11

Jamey C. wrote:

What has been kicked around when this question has come up before is for
me to create something like a NULL class and use this to designate a null
value in the database, instead of using nil. This would take care of
having to override NilClass#method_missing. It would bring up a few other
problems, namely, what if someone wants to store the string value NULL in
a field. But I imagine these could be overcome.

If you define a new NULL class, you have the problem of it being true in
conditionals. I think it would be difficult to make it work intuitively,
and
impossible to make it as well as the nil.method_missing hack works.

James Edward G. II wrote:

This has been a popular discussion lately, but I just don’t think loading
a database library should fundamentally change the language.

As good a point as that is, I think this is KirbyBase’s best option. I
think
the risk of nil.method_missing returning nil breaking something is low,
as
has been argued in the debates on this list (Guy Decoux?). If it did,
wouldn’t we have heard about it by now?

It does probably demand a prominent warning in the README.

Cheers,
Dave

12

angus wrote:

above is that I can’t get granular enough. There is no way to get

I’m not so sure about that. Here’s a link to a page talking about how
the developer of SQLite decided how to treat NULL:

http://www.sqlite.org/nulls.html

If I read it correctly, he says that all of the major SQL dbms’ treat
“null OR true” as true. Maybe I’m reading this page wrong, but that
seems to go against what you are saying.

I’m definitely not a database expert, so anyone with knowledge on this,
please chime in.

In the meantime, I’m going to start taking a look at adding a Null class
(or some such construct) to KirbyBase, to see if I can do away with the
NilClass#method_missing issue.

Jamey

13

Dave B. wrote:

As good a point as that is, I think this is KirbyBase’s best option. I think
the risk of nil.method_missing returning nil breaking something is low, as
has been argued in the debates on this list (Guy Decoux?). If it did,
wouldn’t we have heard about it by now?

It does probably demand a prominent warning in the README.

Hey, Dave. Here’s what I have smack dab in the middle of the README:

== Warning

KirbyBase defines #method_missing for NilClass. This might bite you in
the
butt if you override NilClass.method_missing yourself.

The more I think about this, though, could it be as simple as
subclassing NilClass, call it KBNil or whatever? Then I could override
KBNil#method_missing and that wouldn’t mess up NilClass. It would be
fairly easy for me to change KirbyBase to return a KBNil for an empty
field value instead of nil.

Thoughts on this?

Jamey

14

Jamey C. wrote:

against NULL.

I’m definitely not a database expert, so anyone with knowledge on
this, please chime in.

In the meantime, I’m going to start taking a look at adding a Null
class (or some such construct) to KirbyBase, to see if I can do away
with the NilClass#method_missing issue.

Here’s another follow-up to Angus comment about NULL values in OR
expressions that, to me, is evidence that SQL treats the expression
“null OR true” as true. This excerpt is from:

“The OR operand treats NULL values in a similar fashion. If the other
operand is TRUE, the result of the OR is TRUE (because the real value of
the NULL operand doesn?t matter.) On the other hand, if the other
operand is either FALSE or NULL, the result of the OR operation is
NULL.”

Here’s a link to a Microsoft page talking about NULL.

It says, in the second table, that the result of applying an OR operator
to two Boolean operands, one being NULL, and the other being TRUE, is
TRUE.

Although, to be honest, I don’t really care how SQL does it. One reason
I wrote KirbyBase is because I don’t like using SQL in my programs and
the thought of being able to express my query in Ruby code was very
attractive.

I want KirbyBase to follow Matz Principle of Least Surprise motto. To
me, if I have a query that says, “nil > 300 or 800 < 900”, I think that
query is true. That, to me follows POLS, but of course, everyone has
different ideas of what is Surprising. :slight_smile:

I know James said in an earlier email that the proper Ruby expression,
in order to make sure the query didn’t throw an exception is:

#select { |r| r.speed and r.speed > 300 }

But, even if that is “proper” Ruby code, I don’t want the user to have
to do that for every freakin query just because one of the field
values might be nil. What if you were testing for three or four
conditions? To cover your ass, you would have to write a query like
this:

#select { |r| (r.speed and r.speed > 300) or (r.range and r.range < 900)
or (r.service_date and r.service_date < Date.today) }

That’s just ugly.

You should be able to just write:

#select { |r| r.speed > 300 or r.range < 900 or r.service_date <
Date.today }

To me, that is POLS. Maybe you don’t agree with me. I guess that means
that this is my contribution to “Opinionated Software”. :slight_smile:

Anyway, I hear ya, James. You want me to quit messin’ around with
NilClass. Let me see what I can whip up with a Null class or maybe I can
subclass Nil (I’ll call it KBNil, or something) and override the
#method_missing method in the subclass.

Jamey

15

On Mar 4, 2006, at 8:42 PM, Jamey C. wrote:

Hey, Dave. Here’s what I have smack dab in the middle of the README:

This will probably require some deep magic:
% irb
irb(main):001:0> class KBNil < NilClass
irb(main):002:1> end
=> nil
irb(main):003:0> k = KBNil.new
NoMethodError: undefined method new' for KBNil:Class from (irb):3 irb(main):004:0> k = KBNil.allocate TypeError: allocator undefined for KBNil from (irb):4:inallocate’
from (irb):4

Undaunted I tried to trick it some more:
irb(main):005:0> class KBNil < NilClass
irb(main):006:1> def self.new
irb(main):007:2> Class.instance_method(:new).bind(self).call
irb(main):008:2> end
irb(main):009:1> end
=> nil
irb(main):010:0> k = KBNil.new
TypeError: allocator undefined for KBNil
from (irb):7:in new' from (irb):7:innew’
from (irb):10

Ok, I said let’s try the same trick with allocate:

irb(main):011:0> class KBNil < NilClass
irb(main):012:1> def self.allocate
irb(main):013:2> Class.instance_method(:allocate).bind
(self).call()
irb(main):014:2> end
irb(main):015:1> end
=> nil
irb(main):016:0> k = KBNil.new
TypeError: allocator undefined for KBNil
from (irb):7:in new' from (irb):7:innew’
from (irb):16

And lo, it still did not work. At this point I think it will require
a more advanced rubyist than myself. Either that or a C extension.

I ran into pretty much the same problem with subclassing FalseClass.

nil.dup doesn’t really work either. So this comes to an issue of
database semantics vs. ruby semantics. Which is probably part of the
reason that most DBs do use a custom query language or a variant of SQL.

16

Jamey C. wrote:

fairly easy for me to change KirbyBase to return a KBNil for an empty
field value instead of nil.

You can subclass NilClass, but it has no allocator, so you’d have to
define
your own, and such an object still wouldn’t be considered false in
conditionals. There is only one nil, and only one false. So there is no
point in subclassing NilClass.

It seems to me that without changing nil.method_missing, you can’t have
your
“select do |rec| rec.a < 0 or rec.b < 0 end”. You may be able to
(re-)define
nil.method_missing only for the duration it’s required, but that may end
up
be more confusing to users.

Cheers,
Dave

17

On Mar 4, 2006, at 9:00 PM, Logan C. wrote:

Boolean ops involving only TRUE and FALSE work as usual
Ooops, before someone hits me, I made a mistake in the above NULL or
FALSE is NULL not FALSE.

18

On Mar 4, 2006, at 6:07 PM, angus wrote:

But doesn’t NULL always propagate in “database semantics”? I mean,
if any
subexpression is NULL, the whole expression becomes NULL. It’s the
expected
behaviour (but maybe not to the users of KirbyBase). In SQL,
normally, one
uses NVL(a, b) (with the meaning ‘a.nil? ? b : a’) to guard
subexpressions
against NULL.

Well based on my intro to db class SQL uses a three-way boolean logic

NULL or TRUE = TRUE
NULL or FALSE = FALSE

NULL and TRUE = NULL
NULL and FALSE = FALSE

not NULL = NULL

Boolean ops involving only TRUE and FALSE work as usual

19

Dave B. wrote:

You can subclass NilClass, but it has no allocator, so you’d have to define
your own, and such an object still wouldn’t be considered false in
conditionals. There is only one nil, and only one false. So there is no
point in subclassing NilClass.

Yeah, I tried some of the same stuff Logan did in his email and finally
realized that NilClass won’t let you subclass it.

But, what if I did something like this:

class KBNil
def method_missing(a,b)
nil
end

def nil?
    true
end

end

Then, since, nil and false are the only things that don’t evaluate to
true, if you do:

k = KBNil.new

puts ‘True!’ if k > 300 or 800 < 900 # True!

puts ‘False!’ unless k > 300 or 800 < 900 # False!

puts ‘False!’ unless k =~ /bob/ # False!

puts ‘False!’ unless k < Date.today # False!

k.nil? # true

I’m just thinking off the top of my head, but what do you think about
this?

Jamey

20

Jamey C. wrote:

finally realized that NilClass won’t let you subclass it.
end
end

Then, since, nil and false are the only things that don’t evaluate to
true, if you do:

k = KBNil.new

puts ‘True!’ if k > 300 or 800 < 900 # True!

puts ‘False!’ unless k > 300 or 800 < 900 # False!

Oops! That should read:

puts ‘False!’ unless k > 300 and 800 < 900 False!

Jamey