Configuration Convention

This is probably one of this topics that will get little attention.
Nonetheless…

Have other developers started to feel like the configuration files for
all the various project tools they use are starting to overshadow the
rest
of the project files? For instance, In many of my projects the meat of
the
project consists of a few source directories lib/, test (or spec)
and
sometimes bin/, usually four general documentation files
README, HISTORY, LICENSE and MANIFEST and then of a few non
vcs-tracked things like pkg/, log/ and web/ directories. All the
rest
consist of various configuration tool files. A quick look at various
projects on github provides:

  • .git
  • .gitignore
  • .document
  • .rdoc_options
  • .yardopts
  • .yardoc
  • .autotest
  • .travis.yml
  • .rspec
  • .ruby
  • .test
  • .braids
  • .vclog
  • .rvmrc
  • .rbenv-version
  • .test-unit.yml
  • .gemtest
  • .gemspec -or- foo.gemspec
  • Guardfile
  • Rakefile
  • Gemfile
  • Gemfile.lock
  • Procfile
  • config.ru

And there are no doubt many more. Feel free to mention notable ones I’ve
missed.

Now, obviously not all of these will apply to every project. But I can
imagine that given enough time and a rather thorough developer, a
project could acquire configurations for a couple dozen tools. Think
code
coverage, code analysis, IDE/RAD configuration, etc. I suspect there is
a
saturation point --at some point it just becomes too much to remember.
Even
so, it could amount to quite a few files, well exceeding the number of
toplevel “meat” files of a project.

Another thing to notice is that there are almost universally two file
formats: Ruby or YAML, and three naming schemes: dot files, Foofile
files
and lowercase with special extension files.

While there are obviously some files that will always remain (e.g.
.git), I
wonder if it is possible for a convention to ever develop to mitigate
all
this. Most likely that would be in the form of a common directory to
hold
all these files, although conceivably, it could be in the form of a
couple
of shared files --one for Ruby code and one for YAML.

On Mon, Mar 5, 2012 at 8:14 PM, Intransition [email protected]
wrote:

This is probably one of this topics that will get little attention.
Nonetheless…

Have other developers started to feel like the configuration files for all
the various project tools they use are starting to overshadow the rest of
the project files?

My first gut reaction was of agreement, “yes, I do!” since I’d
recently pretty much thought the same thing recently about an
open-source ruby project hosted on GitHub.

However, after thinking about it more while reading through your list
(below), I realize that this was only a passing thought and that I
don’t really feel this way, at least not completely (more on this
later).

For instance, In many of my projects the meat of the
project consists of a few source directories lib/, test (or spec) and
sometimes bin/, usually four general documentation files
README, HISTORY, LICENSE and MANIFEST and then of a few non
vcs-tracked things like pkg/, log/ and web/ directories. All the rest
consist of various configuration tool files. A quick look at various
projects on github provides:

  • .git
  • .gitignore

Since these have nothing to do with ruby and are simply an artifact of
the VCS in use, at most they may add “dotfile fatigue” when viewing
the list of all files in a clone of the project. Even if an opt-in,
ruby-wide, project structure convention is agreed-upon, it can’t/won’t
be able to comment on these types of files, per se.

  • .vclog
  • .rvmrc
  • .rbenv-version
  • .test-unit.yml
  • .gemtest
  • .gemspec -or- foo.gemspec

After thinking about this list, I realize I already put up with loads
of “dotfiles” in my “home” directory on any Un*x system already, which
system works great since the default behavior of the ls command is
to “hide” dotfiles from me unless I ask to see them.

I recon I’m okay with projects having dotfiles in the project root
since it’s a well-established convention already. I’m even okay with
the fact that some projects commit some of these files into the
project’s repository (as opposed to individuals simply having their
own personal, local copies).

However, I “feel” that a project should consider adding dotfiles to
the project’s code repo as the exception to the more general rule of
having your [D]VCS ignore them by default. I also generally feel that
dotfiles should be reserved for the build, test suite, packaging, and
[D]VCS management tools, which is already pretty much the case for
most files. So I think we’ve already got a good convention going
already, IMHO.

  • Guardfile
  • Rakefile
  • Gemfile
  • Gemfile.lock
  • Procfile

Hmm, I feel a little differently about these “Configfiles”. I get the
lineage, these harkening back to the standard Makefile convention, the
capitalization making this file stand apart. Rakefile is the new, ruby
equivalent of the Makefile. Then came the Gemfile and several other
projects have their Awsomefile too.

I’d like any convention, however [un]official to steer developers from
taking up too much namespace with just Anyoldfile. I’m not sure what
gem/project uses Procfile or Guardfile so I can’t comment on these
(I’m too lazy too Google them right now). I just know that I’ll
generally be very conservative and won’t create a gem/lib that has a
Configfile unless it really, really makes sense. I’ll also steer away
from using “just any ol’ project” that (mis)uses any convention too
much in a manner with which I disagree, without offering enough good,
bug-free functionality to make it worth it.

Hmm. Not sure what I would’ve recommended the standard rack
application definition file to be had I been involved, but my first
thought is that I’m not a fan of this and I hope that other projects
don’t go too far with this convention of “config.my-extension”.

toplevel “meat” files of a project.

Another thing to notice is that there are almost universally two file
formats: Ruby or YAML, and three naming schemes: dot files, Foofile files
and lowercase with special extension files.

If your config file is ruby (and isn’t a Configfile), I’d prefer it
still have the standard “rb” extension. Rake cut out its own extension
of *.rake but I hope no one gets too crazy adding custom extensions.

Likewise, if your config format is YAML, please use a “yml” or
“yaml” extension to make it clear. Just my opinion – it might be your
conventionally named config file, but if it uses another
language/markup internally, please use that language/markup’s naming
convention (this would also apply to JSON, XML, etc, though these are
notably uncommon in ruby projects). It’s true that a quick glance can
almost always tell me the format, but still…

While there are obviously some files that will always remain (e.g. .git), I
wonder if it is possible for a convention to ever develop to mitigate all
this. Most likely that would be in the form of a common directory to hold
all these files, although conceivably, it could be in the form of a couple
of shared files --one for Ruby code and one for YAML.

Now, for application-level configuration, I like the convention used
by several existing libraries and frameworks of having a “config”
directory to hold all your configuration files. However, the kinds of
files listed above aren’t application-level config files, but
“application-development-level” tools. These kinds of files should
stay in the root of the application folder, IMHO, just like I expect
my system application’s to keep my personal configurations in the root
of my user directory using dotfiles or dot-directories. App-level
configs can continue to go in a “config” directory.

Just FYI, I have my own convention I’ve used twice now with two rails
apps, where my application level configs (YAML files inside the
“config” directory) make use of both the “yml” and “yaml” extension.
All my “config/.yml" files are ignored and are not added to my code
repository, being local deployment-specific configurations (such as
the rails standard “config/database.yml”). For all other
application-level config files that contain configuration that isn’t
deployment- or host-specific (and doesn’t ever contain any kind of
credential or token) uses the "config/
.yaml” namespace and does get
added to the repo. Finally, for each “config/.yml" file that my
application needs, I do commit a generic version with the "
.yml-dist”
extension. I’ve just picked this up from various other blogs/projects
and it generally works. However, that’s all a little off topic though.

In summary, I feel that basically/overall, a good, de facto convention
is already being followed with me favoring a default attitude of not
commiting application-development-level configs, using dotfiles while
avoiding Configfiles and config.custom-extension files.

That’s my $0.02

Thomas S. wrote in post #1050299:

Now, obviously not all of these will apply to every project. But I can
imagine that given enough time and a rather thorough developer, a
project could acquire configurations for a couple dozen tools. Think
code
coverage, code analysis, IDE/RAD configuration, etc. I suspect there is
a
saturation point --at some point it just becomes too much to remember.

Why remember? Files are in the file system or VCS repository where you
can find them. I am not sure I understand the issue you are having.

Even
so, it could amount to quite a few files, well exceeding the number of
toplevel “meat” files of a project.

I don’t see why this should be a problem. Even more so: why are these
configuration files not “meat”? That would mean you could have the
project without them and not lose anything. I doubt that. Reminds me
of people coming out of a meeting and saying “now off to some real
work” - that’s just nonsense as communication is a large part of our
profession - as much as hacking.

While there are obviously some files that will always remain (e.g.
.git), I
wonder if it is possible for a convention to ever develop to mitigate
all
this. Most likely that would be in the form of a common directory to
hold
all these files, although conceivably, it could be in the form of a
couple
of shared files --one for Ruby code and one for YAML.

That directory does exist already - as your posting proves: it’s the
project’s root directory. :slight_smile:

Kind regards

robert

Hmm. Not sure what I would’ve recommended the standard rack
application definition file to be had I been involved, but my first
thought is that I’m not a fan of this and I hope that other projects
don’t go too far with this convention of “config.my-extension”.

I don’t like config.ru either.

The first time I saw it, I had to think about russia.

config.rack would have been better for me. But then again
I dont know why it is not a .rb file residing in a rack/
subdir anyway.

Files are in the file system or VCS repository where
you can find them.

If you CAN find them at all. :slight_smile:

In this example it is not so bad because the dotfiles are
at least some sort of convention and hidden by a default
view.

why are these configuration files not “meat”?

Because they are not an integral part of the project of
course. The whole .git directory can just be removed
without breaking the project in question.

That would mean you could have the project without them
and not lose anything. I doubt that.

Sure you lose information if you remove the .git directory.

But the project will still work.

I find the Information that is stored in git often totally
irrelevant to the project in question. I rather keep a
good tutorial and describe what conventions are in use.
And document design decisions.

I do not feel any particular interest to really document
every change to a project. That sounds much more fitting
for a project which has a user base of like 500 million
computers, not your typical average ruby project of …
5 other computers (haha sorry!).

Reminds me of people coming out of a meeting and saying
“now off to some real work” - that’s just nonsense as
communication is a large part of our profession - as
much as hacking.

Meaningless communication is still useless communication.

It is as if you argue that projects without .git no longer
have any meaning at all.

I can relate to a lot of what 7rans wrote. I also have to
say that I myself never had any problem with directory
structures. I tend to delete things I have no need of.

I myself also don’t have a good standard I can follow.

I try to come up with something new every once in a while

So far it works ok.

Examples:

project consists of a few source directories lib/,
test (or spec)

I have decided to use both lib/ and test/.

spec/ I never use. Either it is part of a test, then
it will reside in test/, or it is a specification,
then it will go into a directory I call DOCUMENTATION/

(Yup, I upcase that one. But you can downcase it and
make it shorter, like doc or docu)

sometimes bin/

So far I found bin/ totally useless. It even disturbs
my directory layout. There needs to be a better
convention to call .rb scripts that are to be called
standalone than put them into bin/.

All my files in lib/ are usually always part of the
library and project, but also can be called and used
standalone. Now where should I put them? In bin/ or
in lib/?

My only exception to it was to provide ONE general
wrapper over all the other scripts in lib/ and from
this file in bin/, put it there so I can call the
other scripts. I still don’t like the bin/ directory
and if you ask me, the typical linux FHS is a total
mess.

usually four general documentation files README,
HISTORY, LICENSE and MANIFEST

I would really put these into their own directory,
or into documentation/

It feels much better to have all documentation
available in one directory, IMHO.

and then of a few non vcs-tracked things like
pkg/, log/ and web/ directories.

Well this kinda sucks. Especially web/.
And pkg/

log/ well, I suppose it is useful enough, more so
than web/ and pkg/ at least.

All the rest consist of various configuration tool
files. A quick look at various projects on github
provides:

You really have way too much of these. Good that I
am not in charge, I would be too tempted to remove
them all. :wink:

On Tuesday, March 6, 2012 11:02:42 AM UTC-5, [email protected]
wrote:

Why remember? Files are in the file system or VCS repository where you
can find them. I am not sure I understand the issue you are having.

I just mean remembering all the tools, how to use them, configure them
and
what they are for.

Think of it this way. The files that change the most are the lib/ and
test/ files. But they are not given the “prime real-estate” of
top-level
files. Rather they are categorized into an allotted subdirectory.
(Interestingly C projects actually do put source files at the top). So
why
are config files, which rarely change once properly configured, always
in
our face? And if that wasn’t the point why are they more often dotfiles
which are intended to be hidden? Maybe if they were actually hidden,
but
except for using ls every other tool I use pretty much shows them. So
they aren’t really.

In the end I’d prefer to see a dedicated directory that became the
convention, and baring that at least just stop with all the .dotfiles
and
Configfiles and just use regular old files. Let Gemfile be
gemfile.rb
and .yardopts be yardopts.txt, etc.

On Monday, March 5, 2012 11:14:24 PM UTC-5, Kendall G. wrote:

After thinking about this list, I realize I already put up with loads
of “dotfiles” in my “home” directory on any Un*x system already, which
system works great since the default behavior of the ls command is
to “hide” dotfiles from me unless I ask to see them.

I think “put up with” is the operative phrase. Yes, ls hides them but
often file selectors in apps do not and it is very annoying to weed
scroll
pass them all the time. For a long time I promoted the XDG base
directory
standard in hopes that, in time, it would significantly curtail this,
moving the file to ~/.config/ instead. But I’ve pretty much given up
on
this. So instead I have abandoned my ~/ directory moved all my files
to
~/desktop/, which really is a pretty convenient place for them anyway.

So, I do view reams of dotfiles in toplevel folders as a “great” system.
It’s a clutter. When viewing files on github for instance, dot files are
not hidden.

I recon I’m okay with projects having dotfiles in the project root

most files. So I think we’ve already got a good convention going
already, IMHO.

Most of those dotfile really need to be in the repo. I don’t think
there’s
much of a choice --expect that the tool maker could use something else
besides a dotfile. It would be unwise not to check in test configuration
for instance.

projects have their Awsomefile too.

Yea I basically concur. To me a Configfile is basically a dotfile that
the
tool designer has decided must be seen all the time. Perhaps it’s a
philosophical stance against hidden config files, or perhaps viewed as
helpful to the end-developer to immediately know it (such as a
Rakefile),
or maybe even a little ego. Nonetheless, it has the same potential of
clutter --though in part worse b/c at lease dotfiles can be hidden in a
ls. Of course, it’s not so bad if there’s no more than a two or three
of
them, but imagine if all those dotfiles used the Configfile pattern
instead! And I have to add, my least favorite thing about Configfiles is
that are capitalized, something that has generally been reserve for
documentation files.

Hmm. Not sure what I would’ve recommended the standard rack
application definition file to be had I been involved, but my first
thought is that I’m not a fan of this and I hope that other projects
don’t go too far with this convention of “config.my-extension”.

This doesn’t bother me as much, but I take your point. It’s not ideal to
be
creating a bunch of new extensions when it’s really just a ruby script.
Maybe rack-config.rb or just rack.rb would have been a better
choice.

If your config file is ruby (and isn’t a Configfile), I’d prefer it
still have the standard “rb” extension. Rake cut out its own extension
of *.rake but I hope no one gets too crazy adding custom extensions.

Again, would rake.rb and rake-foo.rb be a better pattern?

Likewise, if your config format is YAML, please use a “yml” or

“yaml” extension to make it clear. Just my opinion – it might be your
conventionally named config file, but if it uses another
language/markup internally, please use that language/markup’s naming
convention (this would also apply to JSON, XML, etc, though these are
notably uncommon in ruby projects). It’s true that a quick glance can
almost always tell me the format, but still…

Aside on this, sure with there where only one possible extension for
yaml
files. I know the official standard is .yaml but everyone seems to use
.yml in practice.

Now, for application-level configuration, I like the convention used
by several existing libraries and frameworks of having a “config”
directory to hold all your configuration files. However, the kinds of
files listed above aren’t application-level config files, but
“application-development-level” tools. These kinds of files should
stay in the root of the application folder, IMHO, just like I expect
my system application’s to keep my personal configurations in the root
of my user directory using dotfiles or dot-directories. App-level
configs can continue to go in a “config” directory.

Yea, I have thought about using config/ for my tools. But the
aesthetics
always bothered me as it sticks out against the typical three or four
letter directories. More recently I’ve actually considered dot/ as an
alternative.

In summary, I feel that basically/overall, a good, de facto convention

is already being followed with me favoring a default attitude of not
commiting application-development-level configs, using dotfiles while
avoiding Configfiles and config.custom-extension files.

I almost never I have personal config files that I wouldn’t check-in. I
think that must be more of app thing where-as I mostly work on libs.

I don’t think there is enough of a standard personally.

On Monday, March 5, 2012 10:14:44 PM UTC-5, Intransition wrote:

Have other developers started to feel like the configuration files for
all the various project tools they use are starting to overshadow the rest
of the project files?

I think this is a pretty good example of what I am talking about:

https://github.com/travis-ci/travis-ci

Strewn amidst the CHANGLOG.textile, README.textile, TODO.textile and
LICENSE files the travis-ci project has .gitignore, .rspec, .slugignore,
.travis.yml, Gemfile, Gemfile.lock, Procfile, Rakefile, Thorfile and
config.ru.

On Tue, Mar 6, 2012 at 2:14 PM, Intransition [email protected]
wrote:

While there are obviously some files that will always remain (e.g. .git),
I wonder if it is possible for a convention to ever develop to mitigate all
this. Most likely that would be in the form of a common directory to hold
all these files, although conceivably, it could be in the form of a couple
of shared files --one for Ruby code and one for YAML.

I agree. It’s insane not to have all configuration files in a “config”
directory. But you can understand how it’s evolved, and sadly it’s not
going to change.

Gavin

On Tue, Mar 6, 2012 at 9:55 AM, Intransition [email protected]
wrote:

I think “put up with” is the operative phrase. Yes, ls hides them but
often file selectors in apps do not and it is very annoying to weed scroll
pass them all the time. For a long time I promoted the XDG base directory
standard in hopes that, in time, it would significantly curtail this, moving
the file to ~/.config/ instead. But I’ve pretty much given up on this. So
instead I have abandoned my ~/ directory moved all my files to
~/desktop/, which really is a pretty convenient place for them anyway.

So, I do view reams of dotfiles in toplevel folders as a “great” system.
It’s a clutter. When viewing files on github for instance, dot files are not
hidden.

Well, if ruby projects start having as many dotfiles in their project
root folders as I have dotfiles/dotdirs in my Ubuntu Desktop install
I’ll probably tend to agree w/you more on the dotfile issue and wish
for more of a convention.

having your [D]VCS ignore them by default. I also generally feel that
dotfiles should be reserved for the build, test suite, packaging, and
[D]VCS management tools, which is already pretty much the case for
most files. So I think we’ve already got a good convention going
already, IMHO.

Most of those dotfile really need to be in the repo.

I agree that the typical project’s set of dotfiles do legitimately
belong in the repo, hence my phrase, “…which is already pretty much
the case for most files” above, this referring to the fact that
dotfiles are already pretty much only for “…the build, test suite,
packaging, and [D]VCS management tools” in the typical ruby project.
Which is why I say, yay, “… we’ve already got a good convention
going…” since I’m okay w/dotfiles.

I don’t think there’s
much of a choice --expect that the tool maker could use something else
besides a dotfile. It would be unwise not to check in test configuration for
instance.

Yep, I basically agree (sans the desire to use something besides a
dotfile). I only originally mentioned the whole attitude of “…ignore
them by default” as just that, a default attitude, not so much as a
commentary about existing projects including unnecessary files
(…although there have been some… :-).

But once again, w/dotfiles I don’t claim to necessarily have any
technological/organizational reason to hold my opinion. It’s just
that, my opinion, subject to change at any moment.

capitalization making this file stand apart. Rakefile is the new, ruby
much in a manner with which I disagree, without offering enough good,
imagine if all those dotfiles used the Configfile pattern instead! And I
don’t go too far with this convention of “config.my-extension”.

This doesn’t bother me as much, but I take your point. It’s not ideal to be
creating a bunch of new extensions when it’s really just a ruby script.
Maybe rack-config.rb or just rack.rb would have been a better choice.

I certainly just want to be clear that I’m certainly not for proposing
any change in the naming scheme for rake files (lest some other reader
thinks that). But yeah, I have no idea what I would’ve come up with,
possibly the above or who knows, after some thought the same “*.rake”
system that’s in use now. I’m not an absolutist. Oops, I mean, I’m
usually not an absolutist. So I’m open to some
tools/gems/libraries creating their own file extension – it would
just be a case-by-case basis. Again, however, I’m for promotion of a
general attitude of “don’t go around creating new, unnecessary file
naming conventions if existing ones fit the bill, unless it really
makes sense

conventionally named config file, but if it uses another
language/markup internally, please use that language/markup’s naming
convention (this would also apply to JSON, XML, etc, though these are
notably uncommon in ruby projects). It’s true that a quick glance can
almost always tell me the format, but still…

Aside on this, sure with there where only one possible extension for yaml
files. I know the official standard is .yaml but everyone seems to use
.yml in practice.

Yeah, I noticed the same thing.

of shared files --one for Ruby code and one for YAML.
of my user directory using dotfiles or dot-directories. App-level
configs can continue to go in a “config” directory.

Yea, I have thought about using config/ for my tools. But the aesthetics
always bothered me as it sticks out against the typical three or four letter
directories. More recently I’ve actually considered dot/ as an
alternative.

Again, I’d just like to clarify that I specifically don’t use the
“config” directory for my tools (or “application developemnt level”
as opposed to “application level” config files – sorry for my lack of
clarity). Meaning, I don’t currently and don’t propose that your test
suite, build, packaging, etc. files go in the “config” directory. Just
the application level ones.

Now, if you don’t like the aesthetics of a “config” dir, you could try
“etc” :-). Just sayin’

In summary, I feel that basically/overall, a good, de facto convention
is already being followed with me favoring a default attitude of not
commiting application-development-level configs, using dotfiles while
avoiding Configfiles and config.custom-extension files.

I almost never I have personal config files that I wouldn’t check-in. I
think that must be more of app thing where-as I mostly work on libs.

Yeah, I think you’re right, it is generally more of an app thing, and
even there, there aren’t many now. Although I’ve like the practice,
for instance, of not checking “.rvmrc” files into an app’s repo, for
instance. I’ve also with apps (not libraries) told git to ignore any
“.rspec” file too since I’d prefer people use their own ~/.rspec file
and not override my ~/.rspec file with willy-nilly preferences put in
a project-commited .rspec file. If you must define a configuration,
you can configure have your setup code use “RSpec.configure”. Anyhow,
I’m just pointing out that, at least for app as opposed to lib
development, there are cases and arguments for keeping some dotfiles
out of repos, including ones that I’ve noticed generally seem to just
be added to a project’s repo.

On Saturday, March 10, 2012 3:30:06 PM UTC-5, Intransition wrote:

Something like that, maybe even offer a couple of intelligent variations
(e.g. group the same extensions together or not). While not a true
de-cluttering solution. It would at least help a lot. Maybe if enough of up
put in support requests to Github they would incorporate something like
this. And even easier, maybe someone could write a better ls?

s/up/us

These load and high roads are not mutually exclusive. So both are worth
pursuit.

s/load/low/

On Friday, March 9, 2012 11:20:43 PM UTC-5, Gavin S. wrote:

I agree. It’s insane not to have all configuration files in a “config”
directory. But you can understand how it’s evolved, and sadly it’s not
going to change.

I think there is a chance that it still could. Basically, I see two
paths
–a low road and a high road. The low road is simply that “directory
listings” use a bit more intelligence in sorting order. Something like:

.quietdir1/
.quietdir2/
 nomaldir/
 Louddir/
 CAPSDIR/
.quietrc1
.quietrc2
normal1.ext1
normal3.ext1
normal2.ext2
Loudfiles1
Loudfiles2
CAPFILES1
CAPFILES2

Something like that, maybe even offer a couple of intelligent variations
(e.g. group the same extensions together or not). While not a true
de-cluttering solution. It would at least help a lot. Maybe if enough of
up
put in support requests to Github they would incorporate something like
this. And even easier, maybe someone could write a better ls?

The high road, on the other hand, is for a very sophisticted solution to
be
developed that has very compelling set of features, if then adopted by
just
one highly popular project, it stands a reasonable chance of wide-spread
adoption.

These load and high roads are not mutually exclusive. So both are worth
pursuit.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs