Validation enhancements -- reflection, overriding, and schem

I’ve been looking at getting more and more information from the
underlying database and automatically inserting that into my models.
I’ve read many of the arguments, thoughts, and ideas about this. I
understand the different points of view. This sort of thing is more
for an integration database and not an application database, and I’m
using an integration database.

Also, I don’t entirely know the etiquette of the list and I apologize
if pasting large blocks of code is gauche, but this is more of a
proof-of-concept request-for-comments than a finalized plugin or patch.

So this, below, is a stab at taking validation information from the
DDL. I was part-way through this before I learned about the
enforce_schema_rules and schema_validations plugins, but those weren’t
exactly for me. I learned some from them, but there were some things I
needed to do differently.

############################################################
module ActiveRecordExtensions
module Validation
module FromDDL

  def self.included(base)
    base.extend(ClassMethods)
  end


  module ClassMethods

    def validations_from_ddl
      validations = []

      validations << column_type_validations
      validations << constraint_validations
      validations << index_validations
      validations.flatten!

      class_eval validations.join("\n")
    end


    def column_type_validations
      validations = []

      self.content_columns.each do |col|
        case col.type
          when :string
            validations << "validates_length_of :#{col.name},

:maximum => #{col.limit}"
when :integer
validations << “validates_numericality_of :#{col.name},
:only_integer => true”
when :float
validations << “validates_numericality_of :#{col.name}”
when :datetime
if col.name.ends_with?(’_date’)
validations << “validates_date :#{col.name}”
elsif col.name.ends_with?(’_time’)
validations << “validates_date_time :#{col.name}”
end
when :time
validations << “validates_time :#{col.name}”
end

        validations << "validates_presence_of :#{col.name}" unless

col.null
end

      validations
    end

    def constraint_validations
      validations = []

      self.core_content_columns.each do |col|
        constraints_on(col.name).each do |constraint|
          case constraint[:type]
            when 'inclusion', 'exclusion'
              validations << "validates_#{constraint[:type]}_of

:#{col.name}, :in => [#{constraint[:in]}]"
end
end
end

      validations
    end

    def index_validations
      validations = []

      self.connection.indexes(self.table_name).each do |index|
        next unless index.unique

        # I'm not sure about this, but there needs to be a way to

ensure the uniqueness validation
# is formatted correctly no matter the order the columns
appear in the index specification
scope_columns = index.columns
unique_col = scope_columns.detect { |c|
!c.ends_with?(’_id’) }

        next unless unique_col

        scope_columns.delete(unique_col)

        validation  = "validates_uniqueness_of :#{unique_col}"
        validation += ", :scope => [#{ scope_columns.collect { |c|

“:#{c}” }.join(’, ') }]" unless scope_columns.blank?
validations << validation
end

      validations
    end

    # This is *slightly* Oracle-specific
    def constraints_on(column_name)
      (owner, table_name) =

self.connection.instance_variable_get(’@connection’).describe(self.table_name)

      column_constraints_sql = <<-END_SQL
        select
          uc.search_condition
        from
          user_constraints uc,
          user_cons_columns ucc
        where
          uc.owner = '#{owner}' and
          uc.table_name = '#{table_name}' and
          uc.constraint_type = 'C' and
          uc.status = 'ENABLED' and
          uc.constraint_name = ucc.constraint_name and
          ucc.owner = '#{owner}' and
          ucc.table_name = '#{table_name}' and
          ucc.column_name = '#{column_name.upcase}'
      END_SQL

      constraints = []

      self.connection.select_all(column_constraints_sql).each do

|row|
condition = row[‘search_condition’] || ‘’
next unless condition.starts_with?(column_name)

        if    tmp = condition.match(/^#{column_name} (not )?in

((.+))$/)
values = tmp[2]
constraints << { :type => tmp[1] ? ‘exclusion’ :
‘inclusion’, :in => values }
end
end

      constraints
    end

  end

end

end
end

module ActiveRecord
class Base
include ActiveRecordExtensions::Validation::FromDDL
end
end
############################################################

That automatically produces a set of validations in any model that
includes a call to ‘validations_from_ddl’. It’s not everything, but it
keeps you from having to repeat yourself when putting constraints in
the database and the model. As for the argument about how you can’t
put minimum length information in the database, I don’t think anyone
has mentioned how you can put a check constraint on that as well, at
least in Oracle. And before anyone talks about the differences between
Oracle and Postgres and MySQL and where the constraints should go,
there are always constraints in the database. Even “lowly MySQL”
which “doesn’t use constraints” has varchar fields with limits. And I
think we all know what good ol’ MySQL does when you hand it a value too
long to fit in your varchar column.

But there’s more! I thought about inheritance and subclassing and
maybe wanting something more restrictive than the database. What if
I have multiple classes all residing in one table? (STI, anyone?) In
that case, the table has to support the longest possible value. Now,
if you have a parent class that gives a length maximum of 30 and a
subclass that gives a length maximum of 20, what happens? For values
<= 20, no errors. For values between 21 and 30, one errors. For
values >= 31, two errors. TWO ERRORS! This is unacceptable.

Well, maybe not “unacceptable”, but at least unexpected and
undesirable.

So I looked into how to give validation overrides. I learned about the
good work done on reflections by Michael S… Once again, the
plugin didn’t exactly suit my needs, so I grabbed it and made it my
own.

############################################################
module ActiveRecord
module Reflection # :nodoc:

# Holds all the meta-data about a validation as it was specified in

the Active Record class.
class ValidationReflection < MacroReflection
attr_accessor :block

  def klass
    @active_record
  end

  def attr_name
    @name
  end

  def validation
    @macro
  end

  def configuration
    @options
  end
end

end
end

module ActiveRecordExtensions
module Validation
module Reflection

  VALIDATION_METHODS = ActiveRecord::Base.methods.select { |m|

m.starts_with?(‘validates_’) && m != ‘validates_each’ &&
!m.match(/with(out)?/) }.freeze

  def self.included(base)

    VALIDATION_METHODS.each do |validation|
      base.class_eval <<-eval_string

        class << self
          alias :#{validation}_without_reflection :#{validation}

          def #{validation}_with_reflection(*attr_names)
            attrs = attr_names.dup
            configuration = attrs.last.is_a?(Hash) ? attrs.pop : {}
            val_type = validation_method(configuration[:on] ||

:save)
val_blocks = { :before => [], :after => [], :new => []
}

            val_blocks[:before] =

read_inheritable_attribute(val_type) || []

            #{validation}_without_reflection(*attr_names)

            val_blocks[:after] =

read_inheritable_attribute(val_type) || []
val_blocks[:new] = val_blocks[:after] -
val_blocks[:before]

            # I don't think this will work when giving multiple

attr_names.
# We just won’t do that. Only one attribute per
validation line!

            attrs.zip(val_blocks[:new]).each do |attr_name, block|
              val =

ActiveRecord::Reflection::ValidationReflection.new(:#{validation},
attr_name, configuration, self)
val.block = block
add_validation_for(attr_name, val)
end
end
alias :#{validation} :#{validation}_with_reflection
end

      eval_string
    end

    base.extend(ClassMethods)
  end


  module ClassMethods

    # Returns a hash of ValidationReflection objects for all

validations in the class
def validations
read_inheritable_attribute(‘validations’) || {}
end

    # Returns a hash of ValidationReflection objects for all

validations defined for the field +attr_name+
def validations_for(attr_name)
validations[attr_name.to_s]
end

    # Returns an array of ValidationReflection objects for all

+validation+ type validations defined for the field +attr_name+
def validation_for(attr_name, validation)
(validations[attr_name.to_s] || {})[validation.to_s]
end

    def add_validation_for(attr_name, validation)
      vals = validations
      ((vals[attr_name.to_s] ||= {})[validation.validation.to_s]

||= []) << validation
write_inheritable_attribute(‘validations’, vals)
end

  end

end


module Unique

  def self.included(base)
    base.extend(ClassMethods)
  end


  module ClassMethods

    def add_validation_for(attr_name, validation)
      vals = validations
      (vals[attr_name.to_s] ||= {})[validation.macro.to_s] =

validation
write_inheritable_attribute(‘validations’, vals)

write_unique_validations(validation_method(validation.configuration[:on]
|| :save))
end

    private

    def write_unique_validations(key)
      new_methods = []
      validations.each do |attr_name, vals|
        vals.each do |method, val|
          new_methods << val.block if

validation_method(val.configuration[:on] || :save) == key
end
end
write_inheritable_attribute(key, new_methods)
end

  end

end

end
end

module ActiveRecord
class Base
include ActiveRecordExtensions::Validation::Reflection
include ActiveRecordExtensions::Validation::Unique
end
end
############################################################

All the Procs I can’t get any information about, they didn’t make me
happy. The way the validations are stored as simple arrays of Procs,
not so good. I fully accept that this approach is hackery, but it
works (as long as you pay attention to the ‘one attribute per
validation’ comment).

I know this breaks some things. For the lovers of validates_length_of,
that’s not a problem for me. I don’t use it in its full overloaded
glory. I use it only for exact lengths and have
validates_maximum_length_of and validates_minimum_length_of to use when
appropriate. This also breaks some of the simplicity that can come
from multiple calls to validates_format_of. For instance, it’s simpler
to check for presence of both a letter and a digit by having two
separate regexes than one. Like I said, proof of concept, request for
comments, not final. The uniqueness can be made to check more than
just the validation command name and attribute name. But this is a
start. And it keeps you from getting multiple nonsensical errors if
you happen to do something like

validates_presence_of :name
validates_presence_of :name
validates_presence_of :name

or

validates_length_of :name, :is => 8
validates_length_of :name, :is => 9
validates_length_of :name, :is => 10

I’m not saying you would or even should do something like that, but
it could come up. Maybe. Somehow. And the validation errors would
not be pretty.

Of course, the next step would be to tie this in to client-side
JavaScript validations (also Michael S.). If done well (and
that’s a bit of an if), the DRY principle can be upheld quite nicely by
using the database to store constraints and have them be pulled into
the model (where they can be overriden and more can be defined) and
from there into the view. It’s DRY and it’s multi-level. It’s the
best of both worlds?

(Once again, I apologize for the length. And I’m looking forward to
discussion.)


-yossef

On Oct 12, 2006, at 3:05 PM, Yossef M. wrote:

I’ve been looking at getting more and more information from the
underlying database and automatically inserting that into my models.
I’ve read many of the arguments, thoughts, and ideas about this. I
understand the different points of view. This sort of thing is more
for an integration database and not an application database, and I’m
using an integration database.

(Once again, I apologize for the length. And I’m looking forward to
discussion.)


-yossef

Hey Yossef-

Looks like you have done some very nice work here. Can I make a

suggestion that you make this into a plugin and put it in svn
somewhere like rubyforge or another free svn host? Then I think you
would get a lot more feedback on this if people can just drop it into
their plugins dir and try it out. Let me know if you need a hand
making it work as a plugin.

Cheers-
-Ezra

Thanks, Ezra.

I figured that would be the case but didn’t go the pre-packaged plugin
route because of some of the problems mentioned (breaking multiple
calls to validates_length_of and validates_format_of). But if it’ll
get more discussion going as a plugin, a plugin (with caveats) it’ll
be. I should have time for that sometime soon, maybe this weekend.


-yossef

Ezra Z. wrote:

Cheers-
-Ezra

Ezra,

Thanks again for your comment. I was finally able to get some time to
make this a plugin and put it on Rubyforge. It’s available at
http://rubyforge.org/projects/validate-ddl/.


-yossef