Dynamically generating models and/or migrations at runtime

I’m working on a project at the moment that has a rather unusual
requirement and I’m hoping to get some advice on the best way to handle
it
or even some pointers to info that can help me build a solution.

Ok, so this is what I need to do. The application stores and manages
various types of media files but each deployment of the application has
completely different metadata requirements for the media files.

This metadata can contain an arbitrary number of fields of different
types
(single line text, multi-line text, checkboxes, selected values, etc.)
and
also often requires validation particularly presence and uniqueness
validations.

The application needs to be able to easily retrieve values and most
importantly
has to be able to handle full searching capabilities on
these
fields.

One option I considered was using a property list arrangement where the
database table simply contained a property name and value for each
metadata
field of each media file. However, when prototyping this solution it
quickly became apparent that it simply wasn’t going to be efficient
enough
for the searching and retrieval of records particularly when the
database
can be reasonably large e.g. a recent deployment had 3000 media files
and
there were over 20 metadata fields. Also, the queries to do a search and
retrieve the relevant records quickly became very complex.

Another option that the system is currently using is that the metadata
config is defined upfront and a migration is run during deployment to
create a the table and model with a standard name so that the media
model
can be associated with it which the system then uses. This generally
works
pretty fine but it does cause some significant deployment and testing
issues.

For example, writing unit tests becomes much more challenging when you
don’t know the config until deployment. Although I could write a sample
config and test the code that way, it won’t allow me to test the
specific
requirements of a particular deployment.

Similarly, in development, it currently requires me to copy a migration
from the config into the main folder, run it, do all of my testing and
development and then I have to remember to rollback and remove that
migration from the main folder so that the application is in a standard
state. This particularly becomes challenging when I’m bug fixing and I
need
to have the application in a specific configuration for testing and
debugging purposes. Trying to switch between the various configurations
becomes a real nightmare.

Ideally, what I would like is to be able to dynamically create the table
and model including validations, etc. from a config file when the server
is
started. Even better would be if I could maintain multiple metadata
setups
in the one database with each one having its own table so that all I
need
to do to switch between them is change which config file the application
is
currently using.

I’m sure this can be done with Rails but there is very little
information
that I’ve been able to find that can point me in the right direction of
how
to build it during my research over the past few days so any help or
suggestions would be much appreciated!

On Monday, April 15, 2013 2:24:43 AM UTC+1, remov[email protected] wrote:

I’m working on a project at the moment that has a rather unusual requirement and
I’m hoping to get some advice on the best way to handle it or even some pointers
to info that can help me build a solution.
Ok, so this is what I need to do. The application stores and manages various
types of media files but each deployment of the application has completely
different metadata requirements for the media files.
This metadata can contain an arbitrary number of fields of different types
(single line text, multi-line text, checkboxes, selected values, etc.) and also
often requires validation particularly presence and uniqueness validations.
The application needs to be able to easily retrieve values andmost
importantlyhas to be able to handle full searching capabilities on these fields.

I’d use something like elasticsearch to handle the searching side of
this, rather than having some monstrous join to join your metadata
key/value table. With this you store your data as you like it - as long
as you can produce a json document representing the record with all
these metadata fields that should be searchable it won’t care how your
relational database stored them. You might also want to try a document
oriented database like mongo rather than forcing a relational model on
this (you can actually use elasticsearch like this too)

Fred

Did you ask this question twice?