Radiant Plugins and Philosophy

Ruby Apps Radiant CMS
1

This is another email about plugins and philosophy. I’ve had a change of
heart.

I’ve gotten a lot of my plugin architecture to work. Changing the admin
interface through callbacks, adding callbacks to controllers for form
processing, behaviors and tags which already worked. I just want to take
a minute and beg the question, just because we can, should we?

Look at what is required:

Changing views (with our new forms)
Changing helpers (to display new forms)
Changing controllers (to process new form elements)
Changing models (to facilitate has_many of any new models we create)
Adding database tables through migrations
Adding new routes with routing

We want plugins that do that? What we’re creating is a /separate
application/ altogether. We’ve touched almost every piece of the
application. What we’ve done is spread additional logic to support the
plugins throughout the system, and then the logic inside the plugins
themselves. I can tell you from building this system that there is no
clean way to do it, and it will probably become unmaintainable. There is
no simple way to drop in a lot of complex functionality like comments
and have it work out.

I would put forth some combination of these options to choose from
instead of doing the plugin system like this.

  1. Slow development of radiant where it is at 0.5.0 and call it the
    radiant CMS base or core or framework or whatever. Then build very
    domain specific extension from it and have those be placed over the core
    as separate products. Extensions could be developed and added onto by
    different people that all started from radiant CMS base. (more on how
    this works afterward)

  2. Put things like comments, trackbacks, tags, etc. into the system but
    have them turned off by default and only turned on only through settings
    menu (this only extends to blogs and must all be done by the core team)

I’m going to talk about #1, it’s a weird and I’m sure unpopular idea. If
we left a certain core functionality and called it radiant core, then we
could build out plugins as generators which overwrote pieces of the core
system. We could just completely overwrite certain views, controllers,
helpers, models, etc.

It would almost be as if we were patching over the core system with a
domain specific version of the system and allowing developers to go from
there. It would be easy to add migrations to the normal migrations
directory, and just overwrite and add other pieces to the system as
needed. We can even use the plugins and generator scripts that are
already in the system right now to achieve this.

This only works if we stabilize a build that is called radiant core and
attempt to keep it as stable as possible. I strongly believe this is a
really great thing to do and I will probably begin writing these
generators anyway so that people can see how they are created and start
creating some of their own.

There is a downside, which is that only one extension could be added at
a time, so extensions would have to come in many flavors but that is
what the community is really good at. I know that this is a system that
will work for everyone here who is struggling to implement radiant in
what they are doing. The system might look like:

$ radiant/: script/plugin install http://svn.radiantcms.org/radiant_blog
$ radiant/: script/generate radiant:blog
$ radiant/: rake migrate

I’m interested in what everyone thinks.

Josh

2

Following this theme I have spent the night creating a script called
build_extension which I will be releasing later today that will allow
developers to easily create these extensions.

The idea is this

  1. Check out the core version of radiant that the extension will be
    built upon.
  2. Develop in this directory just like you were developing a normal app,
    changing things wherever.
  3. Export a fresh copy of the core version you are building upon into a
    directory inside your development root (somewhere like radiant_core)
  4. run 'script/build_extension PluginName PathToFreshRadiantCore (ex:
    ./script/build_extension radiant_blog radiant_core)
    This will check each of the files in your development directory against
    the fresh core version, create the plugin structure, copy those files
    which you have changed and create the generators for overwriting those
    files.
  5. Distribute the plugin

Your users then just do

$ script/plugin install path.to - a really cool domain parked on Park.io
$ script/generate your_plugin --force
$ rake migrate

which will then overwrite and/or add any models, views, controllers,
helpers, libraries, and tests that there were packaged in the plugin. It
also adds the proper numbered migrations to the db/migrate directory so
that ‘rake migrate’ works correctly.

This should make it very easy for developers to simply check out the
core version and then write and package very large pieces of
functionality as extensions. They could essentially create a system that
looked nothing like the original system if that’s what they wanted. All
the generators and plugin structure is written for them making it easy
to just continue developing and release newer versions on a regular
basis.

When new version of the radiant core are released developers will have
to migrate their extension over to the new version and run
create_extension again to build a plugin against the newer version. This
should only be a problem for larger versions.

The door will soon be wide open for developers to use all of their
skills without having to worry about learning a new plugin system, I
can’t wait to see what you guys come up with.

Josh

3

Josh F. wrote:

Changing views (with our new forms)
Changing helpers (to display new forms)
Changing controllers (to process new form elements)
Changing models (to facilitate has_many of any new models we create)
Adding database tables through migrations
Adding new routes with routing

We want plugins that do that?

Yup. We do. If this is ugly and hard to maintain against the current
Rails code base I’d like to see if we can get the core changes we need
pushed back into Rails. It seems like views and routing are the hard
spots right now. It should be much easier to create plugins with views
that use existing layouts, but Rails makes this hard because it only
allows a controller to use one directory structure as it’s base view
directory.

Please submit a diff of your plugin architecture. I would be very
interested to see the problems you are running into.

–
John L.
http://wiseheartdesign.com

4

Josh F. wrote:

Following this theme I have spent the night creating a script called
build_extension which I will be releasing later today that will allow
developers to easily create these extensions.

…

I’m interested in this route as well. Let us know how it turns out.

–
John L.
http://wiseheartdesign.com

5

John W. Long wrote:

Rails code base I’d like to see if we can get the core changes we need
pushed back into Rails. It seems like views and routing are the hard
spots right now. It should be much easier to create plugins with views
that use existing layouts, but Rails makes this hard because it only
allows a controller to use one directory structure as it’s base view
directory.

Aren’t these (mostly) exactly the same problems that Engines solve?

Jay L.

6

John W. Long wrote:

…It should be much easier to create plugins with views
that use existing layouts, but Rails makes this hard because it only
allows a controller to use one directory structure as it’s base view
directory.

Rails Engines handle the problem of view architecture pretty well (they
can install new views, which can then be over-ridden). They also handle
the related problem of public-directory architecture (new stylesheets,
etc.).

–Al Evans

7

Jay L. wrote:

Yup. We do. If this is ugly and hard to maintain against the current
Rails code base I’d like to see if we can get the core changes we need
pushed back into Rails. It seems like views and routing are the hard
spots right now. It should be much easier to create plugins with views
that use existing layouts, but Rails makes this hard because it only
allows a controller to use one directory structure as it’s base view
directory.

Aren’t these (mostly) exactly the same problems that Engines solve?

But that’s what I mean. The fact that when people start trying to write
applications in Rails that have plugins, they seem to go down the same
route. To me this indicates that something generic needs to be pushed
back into Rails core.

–
John L.
http://wiseheartdesign.com

8

John W. Long wrote:

But that’s what I mean. The fact that when people start trying to write
applications in Rails that have plugins, they seem to go down the same
route. To me this indicates that something generic needs to be pushed
back into Rails core.

I agree… I think DHH has some fundamental (though unstated) objection
to Engines that keep them out of the core. The impression I get is that
he thinks we ain’t gonna need it, even though we already do.

Jay

9

I’ve been using engines for a while, even submitted some patches to
make it easier to over-ride images etc in the public-directory. That’s
sort of how I see the philosophy of engines: make something that works
by itself that can selectively overridden. They don’t have
overriddable routes last I saw, but I was going to give them a patch
if they didn’t do that soon. Productize seemed to do that pretty well.
The only thing Engines didn’t handle very well besides routes are
models. Controllers could be opened and extended in your app, but
models would have to be completely overridden from scratch.

Anyway, not suggesting anything for Radiant plugins just yet. Just
chiming in on Engines…

-jeff

10

Core’s objection to engines is that the pieces are too big. Engines
encapsulate essentially an entire application, whole vertical slices of
functionality. The view of core on this is that if you want an
application then build an application, you most likely aren’t going to
drop in more than one engine at a time ever since the slices are so big
and most likely won’t work with one another without some major
modification on your part so what’s the point in having them pluggable?
It makes more sense to just download a complete application and go from
there. Adding mroe than one large vertical slices of functionality most
often leads to collisions where two plugins are overwriting the some
pieces of the same classes or views, that is why it’s useless to have a
system where you can install more than one.

That being said the other thing I dislike about engines is that engines
start out with the philosophy of ‘the engine is your application’ and
then you can override views in your application. This is completely
opposite of what makes sense in our situation, where your application is
your application and the views you already have get overridden by the
plugins in the engine.

I think the right idea is to actually start with a core set of
functionality and then build out applications from there. Because even
if we get some ‘plugin engine’ system working, you’re only going to be
able to install one of these ‘plugin engines’ simply because the
vertical slices would have to overlap one another otherwise. you
couldn’t have two plugin engines that override one view for instance, it
just doesn’t make sense.

And since you’re only going to be able to install one, you might as well
just make it overwrite the core application you had previously so that
it’s easier to develop and you don’t have to work inside two /app
directories which makes life hell.

Just my 10000 cents…:slight_smile: Sorry for the longs posts

josh @ besquared

11

Josh F. wrote:

Core’s objection to engines is that the pieces are too big. Engines
encapsulate essentially an entire application, whole vertical slices of
functionality. The view of core on this is that if you want an
application then build an application, you most likely aren’t going to
drop in more than one engine at a time ever since the slices are so big
and most likely won’t work with one another without some major
modification on your part so what’s the point in having them pluggable?
It makes more sense to just download a complete application and go from
there. Adding mroe than one large vertical slices of functionality most
often leads to collisions where two plugins are overwriting the some
pieces of the same classes or views, that is why it’s useless to have a
system where you can install more than one.

That being said the other thing I dislike about engines is that engines
start out with the philosophy of ‘the engine is your application’ and
then you can override views in your application. This is completely
opposite of what makes sense in our situation, where your application is
your application and the views you already have get overridden by the
plugins in the engine.

I don’t know that much about what other people are doing with engines,
but I offer my own psitsNOT as a counter-example. It simply adds
image-editing to any Rails app. To do this, it needs its own views, but
it doesn’t impinge on the other functionality of the app in any way. All
the app needs to do is override the PictureController, and all it has to
do there is provide a source of images.

I’m not trying to be difficult, but plug-in functionality of this sort
CAN be desirable. Image editing could be useful in a wide variety of
apps, even ones in which it is completely beside the point of the app
(blogs, catalogs, …). psitsNOT doesn’t interfere in any way with any
other functionality of the application; it doesn’t even use the
database.

(Demo: http://psitsnot.alevans.com
svn: http://svn.openprofile.net/plugins/p_sitsnot_engine )

I don’t really think this is a unique case, and I apologize for being a
bit off-topic:-)

–Al Evans

12

If it doesn’t modify anything then how do you route to it? Or administer
it from the interface that looks like the rest of the site? It adds
functionality that’s true, but only for developers who are willing to
take the time to integrate some type of management it into their admin
interface. If that’s the case (which I assume it would be) I have to go
in and modify your controllers, views, and models to get it to fit into
the system I’m already using.

What we’re really after is a strong integrated look, feel, and
philosophy between all pieces of the application. This is one thing that
helps makes applications usable, and what makes the other CMS systems
suck. By making things over modularized you doom yourself to a system
that becomes ugly and unusable. I don’t want to repeat the mistakes of
every CMS that has tried this approach for the past decade.

Imagine a world where there are several Radiant extension projects which
different people contribute to. The bloggers contribute to the radiant
blog extension, the community sites contribute to the radiant community
extension, etc. All the while radiant core continues to be stable and
grows feature rich, but its focus is on features that can be used by all
extensions to improve them in their specific problem domains. And all of
it would be easily packaged, shipped and dropped on top of the core.

I’m not saying we can’t use an engine style plugin for the packaging and
shipping, although we’d have to modify the engine behavior in some ways.
I’m just saying that the philosophy of what seems on the outside to be
nice like dropping in a gallery, comments, and maybe 2 or 3 other
vertical plugins on top of a system just turns it into a huge giant mess
with application logic spread out into 3 or 4 different app directories
not to mention the main one.

If we are to use an engine like system for this I think that what should
happen is that groups should get together and build massive pieces of
functionality instead of many small ones. Individuals can still build
small pieces, but it would be for a particular project/extension. A
group might build a blog with every blog feature they want and make it
look, feel, and work really well together, then package it as a radiant
engine. This means we could have several blog extensions that each take
advantage of the great behavior, tagging, and administrative strengths
of radiant, while each taking a different approach to how they integrate
their blogging features. This represents a change in how this community
would be structured, but I think it is an important one, and one that
could make radiant one of the best content management systems out there.

This will be my final post on this topic of this length. I hope my ideas
had some impact to someone somewhere. :slight_smile:

Josh @ besquared

13

Being new to Ruby and Rails I don’t know much about the pros/cons of
plugins
and engines but it seems to me that in order for Radiant to succeed it
must
have an easy way for users to extend the system to do what they want.
Tools
like Movable Type, Drupal, and ezPublish (and many others) because they
have
a good plugin mechanism. I work with a commercial CMS product that has
everything including the kitchen sink in it whether you need it or not.
I’d
prefer to be able to have a stable core product with a way to add things
like a blog, image gallery, or whatever other needs a client of mine
wants.

In my mind, adding a new “extension” to Radiant should be as simple as
exporting a zip or tar file into a single folder, much like Rails
plugins
work.

Again, I’m new to Rails but what about components? On the surface they
sound
like they are exactly what Radiant needs. What are the pros/cons to
going
that route? Are they limited to front-end functionality or can they also
handle model and controller changes?

Kyle H.
www.kyleheon.com
[email protected]

14

On 13-Jul-2006 21:11 -0500, Josh F. was heard to say:

I’m not saying we can’t use an engine style plugin for the packaging and
shipping, although we’d have to modify the engine behavior in some ways.
I’m just saying that the philosophy of what seems on the outside to be
nice like dropping in a gallery, comments, and maybe 2 or 3 other
vertical plugins on top of a system just turns it into a huge giant mess
with application logic spread out into 3 or 4 different app directories
not to mention the main one.

If you want plugins to be fine-grained you will not get around of having
application logic spread over all of them. Our task is to make it as
easy
as possible for people to write this application logic and more
importantly
keep it small in size.

would be structured, but I think it is an important one, and one that
could make radiant one of the best content management systems out there.

This puts Radiant more into the category of a CMS framework: Radiant as
the
core system and one large extension that implements all-you-need for a
particular scenario, like blog. I personally do not like this approach
and
rather go with fine-grained plugins that do small things here and there.
Eventually, the user does not care over how many directories the
application
logic is spread out and the plugin developer doesn’t either if the API
for
the plugin system is well done.

Oliver