Documenting DSLs

Ruby
1

Hey, has anyone thought about or worked on a “next gen” documentation
system for handling Ruby DSLs?

I have some code that I’d really like to DSLify, but I don’t want to
loose the RDocs. Let me give a simple example.

What mysev does.

class MyService < Service

register :myserv

action :foo

# What foo does.
def foo
  ...
end

end

This is a (simplified but) common pattern is some of my work. I’d love
to write this as a DSL:

What mysev does.

service :myserve do

# What foo does.
action :foo do
  ...
end

end

Ideally, I suppose the doc tool could be taught that #service
translates into a subclass of Service, and #action translates into a
method. (Not sure how feasible that is though.)

No doubt I could roll my own special documentation tool for my
personal needs, but I already have too much to do. I’d much prefer a
general solution.

T.

2

On Aug 11, 10:20 am, Trans [email protected] wrote:

to write this as a DSL:

Ideally, I suppose the doc tool could be taught that #service
translates into a subclass of Service, and #action translates into a
method. (Not sure how feasible that is though.)

No doubt I could roll my own special documentation tool for my
personal needs, but I already have too much to do. I’d much prefer a
general solution.

Well, since no one responded here I can only assume there are no
solutions out there. So I was thinking how I might go about addressing
the issue.

It occurs to me that probably the easiest and potentially most
powerful means of documenting ruby methods and classes/modules is to
do it in Ruby itself.

doc ‘What mysev does.’

service :myserve do

doc 'What foo does.'

action :foo do
  ...
end

end

Associating the docs to their corresponding code could be implemented
in general via Object#method_added and inherited(?). One could also
add their own #capture_docs call in places of special importance.
Under normal operation of course #doc is just and noop.

This seems like such an interesting and potentially powerful way of
going about things, I’m thinking of creating an implementation.

Then it occurs to me, assuming of course there’s is no terrible flaw
in this idea that I’ve overlooked, wouldn’t it be crazy cool if in
Ruby ‘#’ was a special method call? These aren’t your father’s
comments! Oh no. They a living, breathing comments :wink:

Thoughts?
T.

3

Thomas S. wrote:

Hey, has anyone thought about or worked on a “next gen” documentation
system for handling Ruby DSLs?

How about just writing plain old narrative documentation instead of
using a specialized system?

If you want pretty HTML output, you might want to consider Mizuho [1] or
Asciidoc [2]. Some Ruby on Rails manuals are in the process of being
converted to Mizuho/Asciidoc format. See [3] for the output.

[1] https://github.com/FooBarWidget/mizuho/tree
[2] http://www.methods.co.nz/asciidoc/
[3] http://izumi.plan99.net/manuals/creating_plugins.html

4

On Aug 14, 7:42 am, Phlip [email protected] wrote:

  • insert the test code into the HTML
  • syntax-highlight the code
  • insert the test results into the HTML

That satisfies the goal “tests are documentation”.

That’s a tall order! I’m not sure about mixing the tests into the docs
though. That can be a lot a tests and could obscure the docs.
Nonetheless I’m sure there are ways.

But when I tried to submit a proposal, with an introductory patch, to the
maintainers of RDoc, I got flamed and I gave up. Great community here…

Hmmm… well I imagine that’s bound to happen at times. I would just
take to mean that your idea is too futuristic for RDoc, and really
needs a new project.

T.

5

On Aug 14, 2008, at 9:58 AM, Trans wrote:

That satisfies the goal “tests are documentation”.

That’s a tall order! I’m not sure about mixing the tests into the docs
though. That can be a lot a tests and could obscure the docs.
Nonetheless I’m sure there are ways.

I wrote a hack for RDoc a while back that executed code during the
documentation process, but pulled it when I thought through the
massive security implications…

Dave

6

Trans wrote:

Hey, has anyone thought about or worked on a “next gen” documentation
system for handling Ruby DSLs?

I always figured a literate documentor would…

  • read the code
  • convert the comments to HTML
  • run the tests
  • insert the test code into the HTML
  • syntax-highlight the code
  • insert the test results into the HTML

That satisfies the goal “tests are documentation”.

But when I tried to submit a proposal, with an introductory patch, to
the
maintainers of RDoc, I got flamed and I gave up. Great community here…

7

On Aug 14, 6:33 am, Hongli L. [email protected] wrote:

Thomas S. wrote:

Hey, has anyone thought about or worked on a “next gen” documentation
system for handling Ruby DSLs?

How about just writing plain old narrative documentation instead of
using a specialized system?

If you want pretty HTML output, you might want to consider Mizuho [1] or
Asciidoc [2]. Some Ruby on Rails manuals are in the process of being
converted to Mizuho/Asciidoc format. See [3] for the output.

Actually I don’t see anything very special about this. That’s not to
say it’s not well done. But it’s just a markup format (Asciidoc)
wrapped in an ERB based template tool. That’s fairly easy to whip
together. Plus, why use a Python-based markup tool when we have RD and
Markdown (among others) for Ruby?

RDoc ties code to docs, it’s an API documentor, not just a format
transformation system. But maybe your point is that coupling is a
waste of time, and we should just write our docs to be processed
indifferent to any relation to code?

T.

8

Really the same could be said about gem install hooks, etc. Someone
could write a rogue gem to blow away any arbitrary amount of data.

Though a bad example in your code and whoops! :stuck_out_tongue: Maybe when
sandboxing is more ubiquitous?

–Jeremy

On Thu, Aug 14, 2008 at 10:34 AM, Dave T. [email protected] wrote:

documentation process, but pulled it when I thought through the massive
security implications…

Dave

–
http://jeremymcanally.com/
http://entp.com/

My books:

http://humblelittlerubybook.com/ (FREE!)

9

On Aug 14, 12:18 pm, Hongli L. [email protected] wrote:

documentation. This is made possible by the fact that Perl documentation
It is for these reasons that I think the Asciidoc format is ideal for

  • Syntax highlighting. This makes example code easier to read, and more
    visually pleasing.
  • Multi-page output. If your document is long, then having everything on
    a single page can be quite annoying and can make the document hard to
    navigate.
  • Many formatting options, including definition lists, tables, images
    with captions, note/tip/warning/caution paragraphs, etc.

You make an excellent argument.

While I still think it would be nice if an API documentation tool for
Ruby could better handle DSLs, I agree that narrative documentation
should get more focus.

As for Python being used, I don’t really see the problem. Personally I
don’t care what language a tool is written in, as long as it gets the
job done. I don’t think rewriting Asciidoc in Ruby is worth the
investment.

Sure. But as a Rubyists, I prefer Ruby solutions when they are
available.

T.

10

Thomas S. wrote:

Actually I don’t see anything very special about this.
That’s not to
say it’s not well done. But it’s just a markup format (Asciidoc)
wrapped in an ERB based template tool. That’s fairly easy to whip
together. Plus, why use a Python-based markup tool when we have RD and
Markdown (among others) for Ruby?

RDoc ties code to docs, it’s an API documentor, not just a format
transformation system. But maybe your point is that coupling is a
waste of time, and we should just write our docs to be processed
indifferent to any relation to code?

T.

It is often said that Ruby projects lack narrative documentation.
Documenting individual DSL methods will not be very helpful to people
who are new to the DSL. New developers will want narrative, introductory
texts with lots of examples, especially when they don’t know which DSL
method to look at in the first place. RDoc and similar tools are good
for API reference-style documentation, but less suitable for narrative
documentation.

Consider for example the documentation for a typical Perl module. It
usually contains both narrative text, as well as reference-style
documentation. This is made possible by the fact that Perl documentation
format, POD, allows free-style documentation. There is no enforced
structure, so that the author can write the documentation in whatever
way he thinks is best.

Asciidoc allows this. It allows one to structure documentation in any
way he wants, in a syntax that’s still clearly readable and easily
writable. Asciidoc can be converted to many formats, including HTML,
Unix man pages and PDF. For example, the Git documentation is written in
Asciidoc.

It is for these reasons that I think the Asciidoc format is ideal for
documenting DSLs.

Furthermore, it is my opinion that not only the documentation content is
important, but also its presentation. A nicely formatted and visually
pleasing document invites people to read it. It makes the documentation
less boring, so people will be able to read the contents faster and be
able to remember the contents better. Indeed, Mizuho isn’t very special,
but its goal is to make good-looking HTML output for Asciidoc documents
and not to be special. Mizuho/Asciidoc provide the following things that
are not found in Markdown for example:

  • Syntax highlighting. This makes example code easier to read, and more
    visually pleasing.
  • Multi-page output. If your document is long, then having everything on
    a single page can be quite annoying and can make the document hard to
    navigate.
  • Many formatting options, including definition lists, tables, images
    with captions, note/tip/warning/caution paragraphs, etc.

As for Python being used, I don’t really see the problem. Personally I
don’t care what language a tool is written in, as long as it gets the
job done. I don’t think rewriting Asciidoc in Ruby is worth the
investment.