ODBC app in Ruby - I don't believe it

[ed_ruder]

On Oct 13, 2010, at 14:46 , David M. wrote:

The deal-breaker came when I
found it was apparently impossible to emit HAML from the controller and
have it processed.

I’m pretty sure that’s not true, though it’s probably a bad idea and the code
would likely be ugly.

It might be possible, but I spent three or four hours trying, and
failed. More about this in a following paragraph.

I certainly don’t see how abandoning the idea of
separating logic from presentation wholesale is better.

Neither did the people on the Sequel list. But I should say that I
didn’t abandon separating logic from presentation. It was my MODEL that
ended up having to handle most of the presentation that kept climbing
out of Haml.

It’s a program for a pharmaceutical company to track their products,
processing, and inventory. My PostgreSQL schema uses a custom data type
(‘percentage’ which is a numeric(5,2) that cannot be set above 100.00 or
below 0.00),

Huh. DataMapper does support custom, user-defined types, though I’m not sure
how easy it is to do that with an existing adapter. (To give you an idea,
there is currently a working DataMapper adapter for Google App Engine.)

I found all kinds of indications that even ActiveRecord could do what
I wanted. But in the end, whether or not something is possible is
completely irrelevant in the face of “can I make it do that in a
reasonable amount of time?”

IIRC, DataMapper and Sequel both seemed to have fairly similar
capabilities based on various people’s descriptions, and I think both of
them were recommended to me a couple of times. Sequel’s documentation
kicks titanic ass, though: far and away the best I’ve ever gotten with
any Ruby-related thingamabob. Well written and well organized RDoc-y
parts, AND extensive walk-throughs, examples, and discussions, all
installed right on my system, plus a developer whom I’ve seen easily
answer questions that made my eyes cross.

HAML’s documentation looked really good at first, but that’s because
it’s good at telling you what it thinks you should know. It was
completely silent on how to make it process the contents of a variable
as HAML, or to otherwise get it inserted before interpretation. Nor
could whichever list I asked on provide a workable answer. (feature I
cannot find or use) == !(feature)

You’d think so. I need it because I am using a reverse proxy. However,
the redirect attaches the port number,

Yeah, it shouldn’t do that. Either Rails doesn’t, or nginx does strip that
(unlikely), because I’ve hosted production Rails apps in exactly this
environment – nginx listening on example.com, forwarding to multiple Mongrels
on various ports, not necessarily on the same machine.

So, probably somewhere in Ramaze.

I think it’s probably in Innate, but I’m not really clear on how Innate
and Ramaze are related. Alas, Ramaze’s docs are rather scattered and not
entirely up to date.

[ed_ruder]

On Oct 14, 2010, at 23:18 , David M. wrote:

So, I hate to say it, but…

These things are open source. The source is open. More than that, Haml has a
decent test suite, which is always a good place to look when you want to know
how to use some code.

I don’t know why you hate to say that.

But the point is, while good docs are a good thing, I think working and
readable code is at least as important.

I would agree with that.

But reading the code fundamentally tells you WHAT the code does. I look
to the documentation to find out WHY?

There was nothing in the Sequel documentation that told me how I could
get it to start identifying UUID codes as “type uuid” instead of
pretending they were strings. But the code was fairly readable, and
after following a few class definitions back up into the code, I
eventually found a section that had a case statement regarding db types
and Ruby types. and I figured out what to add to get it to give me
UUIDs. There were still a few parts where I was scratching my head and
going “What does this part do?”

It’s quite possible that the answer on how to get Haml to interpret
stuff was staring me in the face at some point, and I missed it. I’m
afraid that doesn’t change my point at all.

“I couldn’t make the Frobulator fricassee.”
“Oh, the Frobulator can definitely fricassee.”
“I couldn’t find any information about it in the documentation.”
“Well, it’s not in the documentation as such, unless you already know
that fricassee is just a special case of fricolating a tri-fold array.
But it’s easy to find in the source code.”
“I haven’t been able to figure out how this code works.”

Whether I’m too ignorant to figure out from the source code how to
fricassee, or the source code’s so poorly written that most people can’t
figure it out, or the docs fail to explain it, or it just can’t do that
in the first place, the end result is the same: no fricassee-ing.

I could tell pretty early on that my biggest problem with finding a good
ORM was not that there were very few that were capable of doing what I
wanted, but that very few of the people who’d written the documentation
were describing their ORM’s capabilities from the point of view that I
had: from the database, instead of from the middleware. I kept finding
tutorials that would say “or you can edit the blahblah file to work with
an existing legacy database.” However, I couldn’t even get a blahblah
file to edit from either Rails or Ramaze, because all the tutorials and
documentation took it for granted that I was interested in making a new
web site from scratch, and didn’t really care all that much about the
details of data storage.
Sinatra (or rather, the ORM that came bundled with Sinatra) had a tool
that was supposed to build me a class/object ORM file from my schema,
but it couldn’t handle UUID primary keys, so it only created about 33%
of the material I needed.
I then had to decide if it would be more efficient to just walk
through the whole tutorial, learning about scaffolding and migrations,
knowing that I would not have any use whatsoever for knowledge related
to migrations, until I finally knew enough about the whole system that I
would have the background needed to understand the code well enough to
be able to figure out how to make it do what I wanted, OR would it be a
better use of my time to start over with some other ORM or library or
whatever, where I would be able to get started sooner by being able to
take better advantage of my existing knowledge? I chose the latter
route, although I would not be able to prove to somebody if it was the
better choice or not.

The better I get at reading other people’s Ruby code, the less I’ll need
to rely on good documentation.
There is a nebulous point of diminishing returns, where “better” (aka
"more) documentation becomes a waste of time; there is a certain amount
of knowledge that the user/programmer can be expected to know. For
example, the Sequel documentation takes it for granted that I am totally
familiar with SQL; most of the docs explain what the code does in terms
of the equivalent SQL code. Should there be even more docs for people
trying to use Sequel who’ve never worked directly with a SQL database
before? It would make Sequel even more accessible to more programmers,
but it’s probably not worth the effort it would take for somebody to
create that much more documentation.

Certainly my inability to get Haml to work for me does not mean that its
documentation sucks, or that it sucks. It just didn’t work for me.

whatever you’re trying to render. Note that this will likely be much more
efficient if you can keep the Engine object around, and that’s true for any
template system, which is probably why it makes more sense for you to take
whatever you were going to do here and put it in a partial…

Honestly, I have no idea what you just said. I certainly don’t see a
clear path between a code fragment in a test suite to, I guess,
embedding “Haml::Engine.new(@varWithMyHamlCodeInIt)” inside a Haml
template. If you do, please do NOT tell me. I’m not using Haml, and at
this point can’t imagine any benefit I could derive that would pay back
the cost of switching my existing code base over to it.

But this, in the end, is all nit-picky detail. My original discussion
was all about how the current cacophony of Ruby-based middleware was
actually quite cool and powerful, but that the original poster’s to make
something that is so often described as “really easy” to work at all was
not some giant Ruby conspiracy or a massive delusion on our part, but
reflected the fact that, like myself, his background and knowledge base
were not the assumed default, and I believe that this was one of the
reasons why he found the whole process so unexpectedly unproductive.

[ed_ruder]

On Wednesday, October 13, 2010 07:03:04 pm Dave H. wrote:

On Oct 13, 2010, at 14:46 , David M. wrote:

I certainly don’t see how abandoning the idea of

separating logic from presentation wholesale is better.

Neither did the people on the Sequel list. But I should say that I didn’t
abandon separating logic from presentation. It was my MODEL that ended up
having to handle most of the presentation that kept climbing out of Haml.

To be honest, that sounds even worse. I push most of the logic back into
the
model and most of the presentation into the view, leaving controllers
mostly
for routing requests and authentication.

I found all kinds of indications that even ActiveRecord could do what I
wanted. But in the end, whether or not something is possible is
completely irrelevant in the face of “can I make it do that in a
reasonable amount of time?”

I absolutely agree. However…

Sequel’s documentation kicks titanic
ass, though: far and away the best I’ve ever gotten with any Ruby-related
thingamabob. Well written and well organized RDoc-y parts, AND extensive
walk-throughs, examples, and discussions, all installed right on my
system, plus a developer whom I’ve seen easily answer questions that made
my eyes cross.

In terms of developers, I’ve seen the same with DataMapper…

Anyway, getting to the point:

HAML’s documentation looked really good at first, but that’s because it’s
good at telling you what it thinks you should know. It was completely
silent on how to make it process the contents of a variable as HAML, or to
otherwise get it inserted before interpretation.

So, I hate to say it, but…

These things are open source. The source is open. More than that, Haml
has a
decent test suite, which is always a good place to look when you want to
know
how to use some code. It took me about three minutes to find this:

http://github.com/nex3/haml/blob/master/test/haml/template_test.rb

In particular, this line:

Haml::Engine.new(File.read(File.join(TemplateTest::TEMPLATE_PATH,
“_#{name}.haml”))).render(self, locals)

Seems to me that you’d just replace the “File.read” part with the source
of
whatever you’re trying to render. Note that this will likely be much
more
efficient if you can keep the Engine object around, and that’s true for
any
template system, which is probably why it makes more sense for you to
take
whatever you were going to do here and put it in a partial…

Now, granted, that should only ever be a stopgap solution. After getting
that
working, the immediate next thing would be to bother the list with,
“Hey, I
got this working, is this likely to continue to work? If not, how can I
get it
working?”

But the point is, while good docs are a good thing, I think working and
readable code is at least as important. If the docs are almost
completely
comprehensive, but the code is shitty, I’m screwed as soon as I manage
to ask
a question the docs can’t answer. If the docs are a good starting point,
but
the code is solid, I have confidence that I can answer any question.
If the
docs are absolutely worthless, I can’t even get started.

So it seems to me like the ideal amount of documentation is enough to
get
started, and more only if people have time, or where the code is
particularly
unclear.

By the way, I generally only allow this excuse for libraries. If I’m
programming in Ruby, I’d better be able to read Ruby code. The same is
NOT
true if I’m a user – I should NEVER have to dig into the code of nginx.

[ed_ruder]

On Fri, Oct 15, 2010 at 8:14 PM, Xavier N. [email protected] wrote:

If you pick AWDwR you have a very balanced perspective on everything
Rails in a few hours of reading. You have a teacher that has prepared
the material properly. If you want to get the same knowledge by
yourself reverse engineering Rails, good luck with that.

Though, if you’re new to web development, then your background and
knowledge
base are not the assumed defaults, and AWDWR will be, as Dave H.
said,
an unexpectedly unproductive experience.

If you got anything out of “a few hours of reading”, than I am envious.
I
spent months reading it, and got almost nothing out of it. To be fair, I
had
no prior web experience, I had never seen MVC before, had never tested
before, it was my first time using a database, and my Ruby code looked a
lot
like my C code.

For someone interested in ActiveRecord, I’d suggest watching the 15
minute
blog (http://media.rubyonrails.org/video/rails_blog_2.mov), finding a
bare
bones working AR implementation like this one (
themomorohoax.com - This website is for sale! - themomorohoax Resources and Information.),
and then reading the guides.rubyonrails.org section on models, while
iteratively building the bare bones file up to do what you want.

[ed_ruder]

On Sat, Oct 16, 2010 at 1:57 AM, Dave H.
[email protected] wrote:

The better I get at reading other people’s Ruby code, the less I’ll need to rely
on good documentation.

Reading source code is good. But in order to use something you
shouldn’t need to reverse engineer an implementation, that’s a waste
of time in my view. It might be equivalent for small libs, but
generally speaking it is better that you get the stuff digested, and
presented as only the experience and motivations of the author can do.

If you pick AWDwR you have a very balanced perspective on everything
Rails in a few hours of reading. You have a teacher that has prepared
the material properly. If you want to get the same knowledge by
yourself reverse engineering Rails, good luck with that.

[ed_ruder]

On Friday, October 15, 2010 06:57:42 pm Dave H. wrote:

On Oct 14, 2010, at 23:18 , David M. wrote:

So, I hate to say it, but…

These things are open source. The source is open. More than that, Haml
has a decent test suite, which is always a good place to look when you
want to know how to use some code.

I don’t know why you hate to say that.

Because in a way, it’s admitting that the documentation isn’t
sufficient, and
it’s also implying that you should dig into the source code, that this
would
be a perfectly normal thing to do.

That’s workable, but certainly not ideal. I hate to say it because I
don’t
think people should have to dig into source code to get stuff done –
but it
is nice in that you can do that.

But the point is, while good docs are a good thing, I think working and
readable code is at least as important.

I would agree with that.

But reading the code fundamentally tells you WHAT the code does. I look to
the documentation to find out WHY?

I don’t see how that’s a problem in the Haml case.

Let me put it this way: I’m not a Haml developer, never have been, but
that
took me almost no time to find, and it does make sense to me why it’d be
that
way. Specifically, it makes sense that you’d have a template object that
represents some sort of “compiled” form of a template (or at least
parsed
form) so that you can efficiently combine that with a given environment
(the
self and locals arguments), rather than re-parsing the template on every
request.

This is far from just a Ruby thing – in particular, I know that Java’s
JSP
files actually get compiled to .class files, just like any other Java
source.

There was nothing in the Sequel documentation that told me how I could get
it to start identifying UUID codes as “type uuid” instead of pretending
they were strings. But the code was fairly readable, and after following a
few class definitions back up into the code, I eventually found a section
that had a case statement regarding db types and Ruby types. and I figured
out what to add to get it to give me UUIDs. There were still a few parts
where I was scratching my head and going “What does this part do?”

Right – and that makes sense. Documentation generally covers the public
API.
If you’re hacking on the source, you get to keep both pieces when it
breaks.
It would be nice if there was better documentation as to where to get
started,
for people who do want to play with the source, but ultimately, if
you’re
playing with the source, the goal should either be a fork or a patch,
probably
a patch.

It’s quite possible that the answer on how to get Haml to interpret stuff
was staring me in the face at some point, and I missed it. I’m afraid that
doesn’t change my point at all.

Actually, it does entirely. It changes your point from “I can’t believe
Ruby
won’t let me do this,” or “I can’t believe Ruby makes it so difficult to
do
this,” to “I might have missed how to do this.” One is a statement about
the
tool itself – the language, the framework, or the community behind it
– and
the other is a statement about your ability to interpret it.

It’s still a good point, but it’s a much more humble one.

Whether I’m too ignorant to figure out from the source code how to
fricassee, or the source code’s so poorly written that most people can’t
figure it out, or the docs fail to explain it, or it just can’t do that in
the first place, the end result is the same: no fricassee-ing.

However, those are each very different problems with very different
solutions.

If it’s just you, then I should do what I can to give you the tools you
need
to be able to figure it out yourself, should you need to. I’ve been
trying to
do that here.

If it’s really that poorly written, that absolutely is a good reason to
use
something else.

If the docs don’t explain it, and it’s something that’s supposed to be a
feature, the docs should be fixed. However, if the docs don’t explain it
because it was never designed as a feature in the first place, but is
just a
sort of accidental result of how it’s designed, that suggests either
it’s fine
the way it is, or it should be added as an official, public-API feature
with
quality documentation.

In particular, documenting something generally makes it part of the
public
API, which means people will be cross if you change it. (If it was
documented
but not public, it wouldn’t be ideal for your purposes, anyway.)

In the last case, the feature should be added if it makes sense.

Now, the ideal steps for you in each of these cases is roughly the same:
“Is
there a way to fricassee? I don’t see it in the documentation, and I
can’t
figure it out…” or “It looks like I can fricassee this way. Is that
likely
to change in the future? If so, can I get an official public API for
it?”

I don’t know if you tried these. If you did and got no response, that’s
certainly a failing of the community, and I apologize.

I kept finding
tutorials that would say “or you can edit the blahblah file to work with
an existing legacy database.” However, I couldn’t even get a blahblah
file to edit from either Rails or Ramaze,

Well, but which file? If it was schema.rb, I don’t know that it would’ve
helped…

Sinatra (or rather, the ORM that came bundled
with Sinatra)

Which is that? I wasn’t aware Sinatra came bundled with an ORM. I wasn’t
aware
that it made sense for there to be – seems like you’d just use an ORM
directly, without Sinatra knowing or caring about it.

Should there be even
more docs for people trying to use Sequel who’ve never worked directly
with a SQL database before? It would make Sequel even more accessible to
more programmers, but it’s probably not worth the effort it would take for
somebody to create that much more documentation.

Well, in that case, the idea would be to go elsewhere to learn SQL.

I certainly don’t see a clear
path between a code fragment in a test suite to, I guess, embedding
“Haml::Engine.new(@varWithMyHamlCodeInIt)” inside a Haml template. If you
do, please do NOT tell me. I’m not using Haml, and at this point can’t
imagine any benefit I could derive that would pay back the cost of
switching my existing code base over to it.

Sorry, I have to, because it’s still a one-liner – the result of that
‘render’ call seems like it’d be a string, like any other you might
generate,
and Haml doesn’t escape strings unless you tell it to. The naive usage
would
look like this:

= Haml::Engine.new(@var).render(self)

If I had doubts it’d be a string, I’d test it in IRB.

But this, in the end, is all nit-picky detail.

Maybe, and I apologize if that’s all it is. My point for getting into
this was
to find out where the gap between my experience of things truly being
easy
(even when starting out), and your experience of things being difficult
or
impossible.

If it’s something I can clear up here, that’s helpful. If it’s something
that’s missing from the community as a whole, that’s also good to know,
too.

like myself, his background and knowledge base were not the
assumed default, and I believe that this was one of the reasons why he
found the whole process so unexpectedly unproductive.

If that’s really all it is, then I hope, if nothing else, I’ve helped
bring
you closer to that.

From where I’m standing, it looks like the pain point was going against
the
grain while also being a newbie. That’s part of why I’ve beaten this
Haml
horse to death – the answer to the question of “How do I embed Haml
code into
my controller?” is generally “Don’t do that. Use partials instead.” A
ton of
newbie questions reduce to something similar.

Unfortunately, we can’t do that with all of your problems. For example,
my
kneejerk reaction when I see people ask “How do I tie this to a legacy
schema?” is “Don’t do that, migrate to something sane.” But that’s
obviously
wrong.

For what it’s worth, as badly as we might need better documentation and
tutorials for working with legacy databases, I found it even more
frustrating
in the world of Oracle ADF, where an existing, legacy Oracle database
(complete with DBA) was assumed, and there was no easy, step-by-step
guide on
getting from zero to a Hello World app.

[ed_ruder]

On Oct 16, 2010, at 8:45 , David M. wrote:

It’s quite possible that the answer on how to get Haml to interpret stuff
was staring me in the face at some point, and I missed it. I’m afraid that
doesn’t change my point at all.

Actually, it does entirely.

Only if you ascribe the wrong point in the first place.

It changes your point from “I can’t believe Ruby
won’t let me do this,” or “I can’t believe Ruby makes it so difficult to do
this,” to “I might have missed how to do this.” One is a statement about the
tool itself – the language, the framework, or the community behind it – and
the other is a statement about your ability to interpret it.

It’s still a good point, but it’s a much more humble one.

array_of_points_in_previous_paragraph.match(mypoint) => nil

Whether I’m too ignorant to figure out from the source code how to
fricassee, or the source code’s so poorly written that most people can’t
figure it out, or the docs fail to explain it, or it just can’t do that in
the first place, the end result is the same: no fricassee-ing.

However, those are each very different problems with very different solutions.

If it’s just you, then I should do what I can to give you the tools you need
to be able to figure it out yourself, should you need to. I’ve been trying to
do that here.

And thank you, but really; please don’t try and solve the problems that
I used as examples. They are no longer blocking items, and I do not need
to fix them.

I started with Haml in the first place because the documentation looked
good. And it is good, but I eventually realized that I was having some
pretty fundamental problems with the paradigm on which it’s based.
Because even if “it’s just you,” the best solution is not necessarily
“do[ing] what I can to give you the tools you need
to be able to figure it out yourself, should you need to.” In this case,
I think a much better solution was to find a tool that worked the way I
expected it to. Maybe my ultimate choice was less capable than the one
I abandoned. But if its easier for me to use, then for me, the
replacement will do more, and do it more easily, than the “better” tool
that does not match well with my existing experience and knowledge.
Quality of code notwithstanding. Documentation notwithstanding.

Which is that? I wasn’t aware Sinatra came bundled with an ORM. I wasn’t aware
that it made sense for there to be – seems like you’d just use an ORM
directly, without Sinatra knowing or caring about it.

I don’t remember. Nor does it matter (to me at least), since I
eventually selected Ramaze and Sequel.

Maybe, and I apologize if that’s all it is. My point for getting into this was
to find out where the gap between my experience of things truly being easy
(even when starting out), and your experience of things being difficult or
impossible.

An admirable goal.

Unfortunately, we can’t do that with all of your problems. For example, my
kneejerk reaction when I see people ask “How do I tie this to a legacy
schema?” is “Don’t do that, migrate to something sane.” But that’s obviously
wrong.

Oh, good. See below.

For what it’s worth, as badly as we might need better documentation and
tutorials for working with legacy databases, I found it even more frustrating
in the world of Oracle ADF, where an existing, legacy Oracle database
(complete with DBA) was assumed, and there was no easy, step-by-step guide on
getting from zero to a Hello World app.

Wheeee. {grimace}

I would suggest, however, that the best place to start is to understand
that there’s a serious problem with the very idea of “legacy” databases.
My database is NOT a legacy database. It’s a brand new installation,
it’s entirely under my control. I am beholden to nobody. To use the term
“legacy” to describe it is extremely misleading. However, I happen to
feel that the database itself is the most well suited component to
ensure the integrity of the data, and the more I work with the various
Ruby-based widgets, tools, and libraries, the more I believe this. I
will NEVER give my Ruby code or an ORM the ability to alter my DB
schema. I also considered and rejected MySQL as the datastore, because
it does not meet my minimum standards for data safety.

You said above that your kneejerk reaction was to “migrate to something
sane.” We agree that that’s not really the right response, but we might
not agree on why. I read the Rails/ActiveRecord documentation with what
eventually became horror. MySQL’s little foibles (like silent truncation
of over-long strings) were nothing compared to the gaping holes and
systemic inadequacies of ActiveRecord. I assumed at first I was just
missing an entire chunk of documentation, the one where it talked about
how you could create foreign key relationships in the database, among
other things. Instead I found pages of documentation about ‘migration,’
where the docs bragged about how easy it was to just swap out MySQL
for SQLite, or vice versa, because it was supporting only the least
common denominator of functionality from the DB engines.

I make my database engine work for a living. While it would, in theory,
be possible to “migrate” my data from PostgreSQL to Oracle, SQLServer,
or (I suspect) DB2, I believe it would be extremely difficult to
replicate the current functionality with MySQL, and utterly impossible
with SQLite. In practice, moving to a new DB store would require a lot
of handwork, rewriting various triggers, creating new ones to replace
the custom data types, adding new linking tables to replace the foreign
key arrays, and so on. There is no way that it could be done by generic
migration code.

I eventually found some other blogs wherein people had commented on
similar experiences, which was why I wanted to reassure the original
poster that “it wasn’t just him not getting it” or some such thing. I
knew when I started working on this project that I would find myself
holding some assumptions that were going to trip me up, and I’ve tried
to keep an open mind and learn new ways of doing stuff and of thinking
about stuff. However, building my web apps from a data-centric, and
DB-centric, perspective is not going to change, at least not until some
kind of reason for doing so that’s a few orders of magnitude more
compelling than anything I’ve seen to date comes along.

Templating foundered on more or less the same problem. I first had to
‘template’ web pages using Microsoft’s IDC/HTX system. This would be
what they had before they invented Active Server Pages. It was pretty
awful. (For one thing, no cookies. Period.) A year or two later, I went
looking for an alternative. I evaluated ASP, WebObjects, SilverStream,
and Tango, and Tango blew my socks off. It did not use templating as
such. Each ‘page’ was an interpreted file, the file contained subunits
that would have markup, and then you could also embed code within the
markup material. Not unlike Haml’s partials, but it all happened within
a single file.

That was in 1997. I was expecting, upon my return to DB<->web
programming, to find things fabulously more advanced. I didn’t expect to
find them almost unrecognizable. Unfortunately, the lack of recognition
isn’t because everything’s fabulously more advanced. There’s been a lot
of sideways motion.

The tutorial that I kept hoping to find, and that I didn’t see
anywhere, was the one that would start out like “Let’s build a
blogging site. Here’s the tables we’re going to use. Here’s the SQL code
you would run on your database to create them. Now, run this magical
command and Railamanatra will construct models and web pages and views
and stuff to help you get started . . . .” or maybe “Here’s how to
convert your current PHP-based blogging site to Railamanatra, without
having to alter the existing data or database.”

So, in a nutshell, there are two paradigms that are pervasive in the
current Ruby-tool-for-web community that will tend to make people who
don’t share them incredibly miserable. The one I’ve mostly covered above
is the middleware-centric perspective, and it seems fairly obvious to me
this is because most of the current tools were created by people who
started with Rails. Rails, as the designers clearly explain, is
“opinionated software.” I like that. I agree with their idea that
there’s a lot of merit in making something that does NOT try to be
everything-agnostic, that provides suggestions, guide rails, defaults,
and implicit assumptions. However, in this particular case, MY
perspective is that the data is god-like, the DB engine is the emperor,
and the middleware are its lackeys and peons, and Rails really hates
that. Alas, all the viable alternatives I found still believe that to
one degree or another. I picked Ramaze in the end because it felt less
strongly about that than my other options.

In fact, I think that this ‘opinion’ is actually quite mild in some of
the other platforms; the authors were, in fact, trying to go for a more
generalized approach. But the documentation still reflects that
original Rails point of view more strongly.

Just figuring out that I had to go shopping for multiple different parts
in order to replace what Tango did for me was one unexpected stumble.
When I did, then I realized that, for me, starting with the ORM was
absolutely the right thing to do. I had a seriously complex schema and
if my ORM couldn’t keep up, it was pretty irrelevant what else I had.
And Sequel has outstanding, phenomenal documentation, and appeared to
have a notably data-centric perspective. Once I had that, then I picked
out things that went well with it.

Next paradigm, see next message.

[ed_ruder]

The other paradigm that I’ve tripped over is that I want my models to
know how to present themselves. Tango did this to a limited extent, and
I really liked it. I was expecting a beautiful world of
self-constructing data when I started working with Rails. Something like
this:

blog.author
=> “Huffy Angelaton”

puts blog.author
=> “Huffy Angelaton”

puts Form.new(:action=>"/savepost"){
blog.author
}
=> “<form action=”/savepost">"

I found nothing that even remotely approached that. So I wrote my own.

There was a pretty long discussion on the Ramaze list about this,
because I’m well aware that this is a big-time violation of the
Model-View-Controller idea. Unfortunately, MVC just can’t handle this
without a huge amount of extra lumps of code. Well, it can handle the
previous example. But it really can’t handle the following one. Or (to
inject at least a smidge of humility), I was quite unable to find a way
to do it.

This is a slightly modified example from my working code. A “Batch” is a
particular batch of manufactured product, specifically an herbal
medicine. I’ve obfuscated the UUIDs.

b = Batch[‘3705bf0e-4264-6654-ff6c-00065b3f562c’]
=>#<Batch @values={:wv=>“1:1.7”, :notes=>nil, :active=>true,
:herbpercent=>#BigDecimal:101f28328,‘0.0’,4(12), :pressed=>nil,
:batchcode=>“06082030.1”, :herbweight=>3741,
:lot_id=>“e62b56d2-a741-xxxx-xxxx-xxxxxxxxxxxx”, :initvol=>6360,
:created=>Fri Aug 04 00:00:00 -0700 2006, :finalvol=>8812,
:extract_id=>“f2580cc8-2354-xxxx-xxxx-xxxxxxxxxxxx”,
:batch_id=>“3451a2a6-5fc1-xxxx-xxxx-xxxxxxxxxxxx”, :id=>1534,
:decocted=>false,
:cultivation_id=>“3705ff6c-2355-xxxx-xxxx-xxxxxxxxxxxx”}>

However, a batch also contains multiple non-active ingredients, called
components.

b.components
=> [
#<Component
@values={:percentage=>#BigDecimal:101f220b8,‘0.1E2’,4(8),
:batch_id=>“3705ff6c-5fc1-xxxx-xxxx-xxxxxxxxxxxx”,
:componentcategory_id=>“4ace3c04-2357-xxxx-xxxx-xxxxxxxxxxxx”}>,
#<Component
@values={:percentage=>#BigDecimal:101f21a50,‘0.5E2’,4(8),
:batch_id=>“3705ff6c-5fc1-xxxx-xxxx-xxxxxxxxxxxx”,
:componentcategory_id=>“4ace3f92-2357-xxxx-xxxx-xxxxxxxxxxxx”}>,
#<Component
@values={:percentage=>#BigDecimal:101f21410,‘0.4E2’,4(8),
:batch_id=>“3705ff6c-5fc1-xxxx-xxxx-xxxxxxxxxxxx”,
:componentcategory_id=>“4ace430c-xxxx-xxxx-xxxx-xxxxxxxxxxxx”}>]

And each component, in turn, is linked to a category:

b.components[0].componentcategory
=> #<Componentcategory @values={:name=>“Glycerine”, :sequence=>3,
:componentcategory_id=>“4ace3c04-2357-xxxx-xxxx-xxxxxxxxxxxx”}>

If I want to create a form for a user to edit the batch data, what I put
in my code is this:

e.editform.to_form

and what I get is this:

<input name='Component:-7531801290:componentcategory_id'

value=‘4ace3f92-xxxx-xxxx-xxxx-xxxxxxxxxxxx’ type=‘hidden’/>

<option value='4acb07a0-xxxx-xxxx-xxxx-xxxxxxxxxxxx'>Olive Oil</option>

<textarea name='notes' rows='8'

cols=‘80’>

"

A batch has more than one component, and each component can be a
particular substance. In order to present that HTML code, my batch
object had to know to take its own fields, like “batchcode” and turn
them in to text input fields. The size of the field is based on how much
space is reserved in the database itself for that data. Each Component
of a Batch has to do the same thing. “Percentage” is a text input field
long enough for the custom data type (a numeric(5,2) that only accepts
numbers in the range 0.00 to 100.00). But since there are multiple
components, and HTML forms don’t have support for nesting, each form
field has to get a unique name so that when the data comes back, it’s
possible to figure out which percentage belongs to which component.
Finally, each component is linked to a 'Componentcategory." In order
to allow the user to select a different category, the form has to know
what EVERY row in the table is, in order to create the list of
.

This last bit is really critical. If you just ask the object itself, it
cannot tell you what you need to know to create the form.

b.components[0].componentcategory
=> #<Componentcategory @values={:name=>“Glycerine”, :sequence=>3,
:componentcategory_id=>“4ace3c04-2357-xxxx-xxxx-xxxxxxxxxxxx”}>

To find out what other categories this component could be, you have to
back up from this component category to the class, then query for all
members of the class, then use that information to construct one form
field, using the identity of the current active instance of the class
to figure out which is marked as “selected”.

I tried to get to this point with templating, but pretty soon my
templates were hardly more than <@htmlfromcontroller>, so
what’s the point?

I must admit, I’m still not too sure what paradigm IS behind the current
flock of templating tools, except that it isn’t like this, and doing it
this way is totally rocking, so I have no incentive to change.

[ed_ruder]

My apologies if quoting is mangled, there’s only so much I can do
about broken email clients…

On Sun, Oct 17, 2010 at 2:09 AM, Dave H.
[email protected] wrote:

I would suggest, however, that the best place to start is to understand that
there’s a serious problem with the very idea of “legacy”
databases. My database is NOT a legacy database. It’s a brand new installation,
it’s entirely under my control. I am beholden to
nobody. To use the term “legacy” to describe it is extremely misleading.
However, I happen to feel that the database itself is the
most well suited component to ensure the integrity of the data, and the more I
work with the various Ruby-based widgets, tools, and
libraries, the more I believe this. I will NEVER give my Ruby code or an ORM the
ability to alter my DB schema. I also considered
and rejected MySQL as the datastore, because it does not meet my minimum
standards for data safety.

Legacy system:

Wikipedia:
“A legacy system is an old technology, computer system, or application
program that continues to be used, typically because it still
functions for the users’ needs, even though newer technology or more
efficient methods of performing a task are now available.”

ichnet.org:
“Those systems in existence and either deployed or under development
at the start of a modernization program. All legacy systems will be
affected by modernization to a greater or lesser extent. Some systems
will become transition systems before they are retired.”

bbn.com:
“A customer’s existing system, often a database system.”

So, yes, your database is a legacy system, as far as switching horses
(to a very different programming style and methodology for web
development) mid-race is concerned.

You said above that your kneejerk reaction was to “migrate to something sane.”
We agree that that’s not really the right response,
but we might not agree on why. I read the Rails/ActiveRecord documentation with
what eventually became horror. MySQL’s little
foibles (like silent truncation of over-long strings) were nothing compared to
the gaping holes and systemic inadequacies of
ActiveRecord. I assumed at first I was just missing an entire chunk of
documentation, the one where it talked about how you could
create foreign key relationships in the database, among other things. Instead
I found pages of documentation about ‘migration,’
where the docs bragged about how easy it was to just swap out MySQL for
SQLite, or vice versa, because it was supporting only
the least common denominator of functionality from the DB engines.

The least common denominator being, allegedly, the SQL standard. Which
pretty much no existing SQL database supports in a portable fashion.
So, what shall be done give that, a) time, b) money, c) knowledge are
in finite supply? ActiveRecord/Rails opted for portability as much as
is possible. Frankly, you can be happy that it is trivial now to
change ActiveRecord for something different and have Rails still work.

I make my database engine work for a living. While it would, in theory, be
possible to “migrate” my data from PostgreSQL to Oracle,
SQLServer, or (I suspect) DB2, I believe it would be extremely difficult to
replicate the current functionality with MySQL, and utterly
impossible with SQLite. In practice, moving to a new DB store would require a
lot of handwork, rewriting various triggers, creating new
ones to replace the custom data types, adding new linking tables to replace the
foreign key arrays, and so on. There is no way that it
could be done by generic migration code.

And given that you know how much work you face, how much DB engine
specific work at that, you seriously expect a tool to cover every
possible case and usage scenario? That’s a rather arrogant attitude at
best.

I eventually found some other blogs wherein people had commented on similar
experiences, which was why I wanted to reassure the
original poster that “it wasn’t just him not getting it” or some such thing. I
knew when I started working on this project that I would find
myself holding some assumptions that were going to trip me up, and I’ve tried to
keep an open mind and learn new ways of doing stuff
and of thinking about stuff. However, building my web apps from a data-centric,
and DB-centric, perspective is not going to change, at
least not until some kind of reason for doing so that’s a few orders of
magnitude more compelling than anything I’ve seen to date
comes along.

Rails is a result of work done 37signals, derived from the experiences
these people had when creating brand spanking new web applications.
That’s the space Rails occupies and, for better or for worse,
dominates.

Rails never was good at being put on top of existing databases or to
fit into existing systems, since that is not the development goal for
Rails.

Templating foundered on more or less the same problem. I first had to ‘template’
web pages using Microsoft’s IDC/HTX system. This
would be what they had before they invented Active Server Pages. It was pretty
awful. (For one thing, no cookies. Period.) A year or
two later, I went looking for an alternative. I evaluated ASP, WebObjects,
SilverStream, and Tango, and Tango blew my socks off. It
did not use templating as such. Each ‘page’ was an interpreted file, the file
contained subunits that would have markup, and then you
could also embed code within the markup material. Not unlike Haml’s partials,
but it all happened within a single file.

I have the distinct feeling that you are strongly opposed to Java’s
“One File, One Class” approach, too.

That was in 1997. I was expecting, upon my return to DB<->web programming, to
find things fabulously more advanced. I didn’t
expect to find them almost unrecognizable. Unfortunately, the lack of
recognition isn’t because everything’s fabulously more
advanced. There’s been a lot of sideways motion.

I can’t comment on that, but I doubt that expecting that programming
changed since the last paradigm shift to OOP is an approach that lends
itself to safety (alas). The changes were in methodology, like the
Model-View-Controller pattern, which nicely separates area of
responsibility of code, minimizing dependencies of otherwise unrelated
code.

The tutorial that I kept hoping to find, and that I didn’t see anywhere, was
the one that would start out like “Let’s build a blogging
site. Here’s the tables we’re going to use. Here’s the SQL code you would run on
your database to create them. Now, run this
magical command and Railamanatra will construct models and web pages and views
and stuff to help you get started . . . .” or
maybe “Here’s how to convert your current PHP-based blogging site to
Railamanatra, without having to alter the existing data or
database.”

That’s because Rails aims to be all-encompassing. You write everything
you need in one language, instead of having to switch gears going from
Ruby to SQL to HTML. And nobody said that Rails fits your problem
space, either.

Rails is excellent if you can control everything in a project, from DB
schema, to server deployment, to the server’s software. It’s not a
good tool if you want to make a square peg fit a round hole, however.

So, in a nutshell, there are two paradigms that are pervasive in the current
Ruby-tool-for-web community that will tend to make people
who don’t share them incredibly miserable. The one I’ve mostly covered above is
the middleware-centric perspective, and it seems
fairly obvious to me this is because most of the current tools were created by
people who started with Rails. Rails, as the designers
clearly explain, is “opinionated software.” I like that. I agree with their
idea that there’s a lot of merit in making something that does
NOT try to be everything-agnostic, that provides suggestions, guide rails,
defaults, and implicit assumptions. However, in this
particular case, MY perspective is that the data is god-like, the DB engine is
the emperor, and the middleware are its lackeys and
peons, and Rails really hates that. Alas, all the viable alternatives I found
still believe that to one degree or another. I picked Ramaze
in the end because it felt less strongly about that than my other options.

Then don’t use Rails. It’s as easy as that.

In fact, I think that this ‘opinion’ is actually quite mild in some of the other
platforms; the authors were, in fact, trying to go for a more
generalized approach. But the documentation still reflects that original Rails
point of view more strongly.

That could be because the Merb and Rails teams (and code) merged
between Rails 2.0 and the current Rails 3.0 release. And as we all
know, the documentation ranks pretty low on teh “things developers do
for fun” scale (much to my own annoyance).

Just figuring out that I had to go shopping for multiple different parts in
order to replace what Tango did for me was one unexpected
stumble. When I did, then I realized that, for me, starting with the ORM was
absolutely the right thing to do. I had a seriously
complex schema and if my ORM couldn’t keep up, it was pretty irrelevant what
else I had. And Sequel has outstanding, phenomenal
documentation, and appeared to have a notably data-centric perspective. Once I
had that, then I picked out things that went well with
it.

Fortunately, thanks to the middle-ware centric approach, you can
actually exchange these things with relative ease.

Well, to cut a long story short: Don’t be surprised that Rails doesn’t
fit your problem. Instead of venting about it, move on, and look for
something that fits, or build your own (or fork Rails / the parts of
Rails you like, and build from that). It’s open source, nobody’ll stop
you.

–
Phillip G.

Though the folk I have met,
(Ah, how soon!) they forget
When I’ve moved on to some other place,
There may be one or two,
When I’ve played and passed through,
Who’ll remember my song or my face.