Pythonic indentation (or: beating a dead horse)

On May 27, 4:32 pm, Caleb C. [email protected] wrote:

On 5/27/09, Roger P. [email protected] wrote:

If you’re interested, please see:
endless.rb · GitHub

Could you post us a few examples of what code formatted this way looks

Oops, I should have done that when I posted before. Thanks for the reminder.

Fairly awesome. But I think we should consider keeping the colon, and
here’s why: blocks that are not delimited by a colon at the start
behave exactly as normal Ruby blocks do, requiring an “end” to close
them, and more importantly ignoring indentation. That way, existing
code can be run through the preprocessor when it’s “on”, and will
continue to work. And it leaves people to pursue ASCII art with their
indentation, if they are so inclined.

Matz says the colon won’t work as a delimiter, though. I’m curious to
know why, but even so, surely something would work.

J Haas [email protected] wrote:

few areas where I think Python beats Ruby, and that’s syntatically-
significant
indentation.
You think so, I’m not. :wink:

like this:

        end
      end
    end
  end
end

end
end

This is not DRY.
Objection! It is DRY: You have already mentioned that in Ruby indention
is not significant. The “ends” in your example are the only (not
repeated!) representation of the end of a block.

Regards,
Jan

On May 27, 2009, at 9:35 AM, J Haas wrote:

worked fine.)
Ok, but now you’re beginning to see the edge cases that make this sort
of transformation difficult on a global level. Specifically, how would
you differentiate between these two cases using Pythonic indentation:

cat conditional_case.rb
result = case ARGV[0]
when /^\d*$/
“That’s an integer”
when /^[A-Za-z]*$/
“That looks like a word…”
else
“That’s not an integer or a word”
end if ARGV[0]

puts result

cat case_and_conditional.rb
result = case ARGV[0]
when /^\d*$/
“That’s an integer”
when /^[A-Za-z]*$/
“That looks like a word…”
else
“That’s not an integer or a word”
end
if ARGV[0]
puts result
end

Attaching a conditional to the end of a case statement (or any block,
for that matter) is possible in Ruby because the block is just an
expression, and the statement doesn’t end until the newline. With your
preprocessor, adding "end " when the indentation level decreases would
yield the first form, but then how do I get the second form? I suppose
I could both decrease the indentation level and leave a blank space,
but then it seems like we’re adding an awful lot of formatting rules
just to have the code look nice and still function. On top of that, I
think this makes things much less clear! Consider that in “Pythonic
form” the above cases would become:

cat pythonic_conditional_case.rb
result = case ARGV[0]:
when /^\d*$/
“That’s an integer”
when /^[A-Za-z]*$/
“That looks like a word…”
else
“That’s not an integer or a word”
if ARGV[0]
puts result

cat pythonic_case_and_conditional.rb
result = case ARGV[0]:
when /^\d*$/
“That’s an integer”
when /^[A-Za-z]*$/
“That looks like a word…”
else
“That’s not an integer or a word”

if ARGV[0]:
puts result

Looking at those two pieces of code, is it clear that they do two
different things? At the very least, I think that this sort of change
to the syntax would have to be optional. This would, of course, mean
that you’re not getting rid of all of the "end"s in code, so then
what’s the point (other than to confuse matters of syntax even more)?

Let me also provide a counter-point. I’m currently using MacVim with
the ruby-vim files and a coloring scheme which colors “def…end”
pairs in yellow and other “case/if/do/etc…end” pairs in purple. This
gives me a good way to quickly look at some code, and if I see that
the “end” immediately preceding a “def” is purple, then I know I’ve
forgotten and “end” somewhere in that function definition.

Finally, let me suggest a bit of a “wisdom of crowds” argument: if the
advantages to significant whitespace were so unambiguously clear as
you’d have us believe, then why do most languages not use it (even
those developed before many of the advantages of advanced text editors/
IDEs existed)?

  • Josh

On Wed, May 27, 2009 at 10:41 AM, Reid T.
[email protected]wrote:

I find this code less readable than the above original code with the
'end’s included

I like the symmetry that end statements give to the code. It feels
more…
balanced or something.

On 27 May 2009, at 17:41, Reid T. wrote:

I find this code less readable than the above original code with the
'end’s included

+1 - the significant white space is one of several reasons I’ve never
managed to get comfy with python.

Personally I like ‘end’ because it makes it explicitly clear that the
programmer’s intent was to end a block at that point, and I’ve wasted
enough of my life tracking down nesting errors not to want to give up
that extra bit of insight. If other people feel differently, well it’s
there code and they’re free to do as they please with it so long as
I’m not expected to maintain it.

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

On Thu, 2009-05-28 at 14:40 +0900, Joshua B. wrote:

the ruby-vim files

could you provide an url for these?

Thanks,
reid

On May 27, 2009, at 12:35 PM, J Haas wrote:

worked fine.)
controllers.

transformable into indentation only blocks. Does that make sense?
github.com/binarylogic/authlogic_example/tree):
helper_method :current_user_session, :current_user
@current_user = current_user_session &&
end
def store_location

Likewise, all the methods added will be available for all

 @current_user_session = UserSession.find
   redirect_to new_user_session_url
 session[:return_to] = request.request_uri

hard to do. But the impossible examples seem to be defeated fairly

filter_parameter_logging :password, :password_confirmation
current_user_session.record

 session[:return_to] = request.request_uri

Nothing particularly special about this code, right? Pretty standard

def current_user:

def redirect_back_or_default(default):
redirect_to(session[:return_to] || default)
session[:return_to] = nil

As I’ve watched this debate unfold I’ve watched the stronger
criticisms fall
apart. The strongest type of criticism would be that it can’t be
done, or that
it’s too hard to be done. But the impossible examples seem to be
defeated
fairly easily. Moreover, the solutions are backward compatible to
existing
Ruby.

Now when I look at this latest example I see some ordinary code that’s
44 lines
long. With the pythonic scheme it looks like it’s only 35 lines
long. I find
it difficult to convince myself that it’s a good idea to make code 25%
larger
just to preserve some ends of dubious value.

I suppose I could try to come up with some nonsense argument that
‘end’ makes
everything more readable. But that would just be prejudice. The
pythonic
example is trivially easy to read. It can’t just be me. There seems
to be no
shortage of Python folk who have no problem. Objectively, the ends
just take
up a whole lot of space.

On 5/27/09, Roger P. [email protected] wrote:

#you can explicitly end just the last scope of something deeply nested.
class A
class B
class C
module D
foo
end #actually 4 ends

I assume that that end is optional, too?

Yes.

end also doubles as the equivalent for Python’s “pass”, as well?

Mostly. As I understand it, pass in python is a no-op. End ‘stand in’,
if you will, for the value of the entire statement it’s ending.

(In endless.rb, if you really want a no-op to end your statement, you
should use an (appropriately indented) semicolon.)

So you can have ends if you want them but they’re optional?

You got it. Your end now HAS to be intented right, tho.

Does it work with {}'s too?

No… seemed a little weird to me to have { without the matching }.
But there’s no technical reason why { couldn’t be autoended too.

So the basic rule it uses is that if something returns to the original
indentation level without being an end, it assumes it should have been
an end, is that right? (i.e. it requires blocks to be at a greater
indentation level).

Yep.

J Hass wrote:

Fairly awesome. But I think we should consider keeping the colon, and
here’s why: blocks that are not delimited by a colon at the start
behave exactly as normal Ruby blocks do, requiring an “end” to close
them, and more importantly ignoring indentation. That way, existing
code can be run through the preprocessor when it’s “on”, and will
continue to work. And it leaves people to pursue ASCII art with their
indentation, if they are so inclined.

In my preprocessor, end is still allowed, it’s just optional now. So
you can copy and paste from a normal ruby file and not worry. (As log
as the snippet you paste is properly indented.)

On 28 May 2009, at 15:06, Juan Z. wrote:

As I’ve watched this debate unfold I’ve watched the stronger
criticisms fall
apart. The strongest type of criticism would be that it can’t be
done, or that
it’s too hard to be done. But the impossible examples seem to be
defeated
fairly easily. Moreover, the solutions are backward compatible to
existing
Ruby.

Some cases have been presented elsewhere in this thread where the
pythonic indentation would fall flat: specifically for expressions of
the form

a = case x
when…
when…
when…
end if y

It’s not a common formulation in this form, but it highlights the
problem of using significant whitespace as an expression delimiter in
expression-based languages. Were there a way to solve this ambiguity
elegantly then there would be a case in favour of introducing pythonic
indentation, but until then this would either introduce an unnecessary
limitation on the things that can be expressed in ruby or lead to a
considerably more complicated lexer that might have other implications.

everything more readable. But that would just be prejudice. The
pythonic
example is trivially easy to read. It can’t just be me. There
seems to be no
shortage of Python folk who have no problem. Objectively, the ends
just take
up a whole lot of space.

Begin…end indicates explicitly that a series of expressions is
contained between these statements and that their return value is
available to use as part of a more complex expression which may have
modifiers, something of great value to some of us. Your experience
might well be very different and this might not be a use case you
commonly encounter, but please don’t make the mistake of believing
that your use case invalidates the utility of this language feature.
Most of the code I work on takes this form and has perhaps a 10%
density of ‘end’ statements, if that.

Admittedly when I’ve worked on Rails apps I’ve seen the 25%
demonstrated in your quoted example frequently: to my mind that’s a
code smell suggesting that much of that code is boilerplate ripe for
abstraction and meta-programming. Such an approach would bring the
‘end’ density back down to an acceptable and useful level.

The real answer to this issue is to have a pythonic indentation pre-
processor for those who find that of use, and to leave the core
language syntax alone.

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

Eleanor McHugh [email protected] writes:

It’s not a common formulation in this form, but it highlights the
problem of using significant whitespace as an expression delimiter in
expression-based languages. Were there a way to solve this ambiguity
elegantly then there would be a case in favour of introducing pythonic
indentation, but until then this would either introduce an unnecessary
limitation on the things that can be expressed in ruby or lead to a
considerably more complicated lexer that might have other implications.

There’s a simply way to solve it. Since everything is an expression,
we can easily wrap expressions and subexpressions in parentheses, and
by the way resolve any ambiguity:

(a = ((case x
when …
when …
when …
end) if y))

((a = (case x
when …
when …
when …
end))
if y)

Begin…end indicates explicitly that a series of expressions is
contained between these statements and that their return value is
available to use as part of a more complex expression which may have
modifiers, something of great value to some of us. Your experience
might well be very different and this might not be a use case you
commonly encounter, but please don’t make the mistake of believing
that your use case invalidates the utility of this language feature.
Most of the code I work on takes this form and has perhaps a 10%
density of ‘end’ statements, if that.

It could be reduced to 0%, if you notice that the close parenthesis is
all that is needed to end the subexpressions.

(a = ((case x
when …
when …
when …) if y))

((a = (case x
when …
when …
when …))
if y)

Also we could decide to always put the operator first, to simplify a
little:

(a = (if y
(case x
when …
when …
when …)))

(if y
(a = (case x
when …
when …
when …)))

so it would be clearer what is IF’ed.

The real answer to this issue is to have a pythonic indentation pre-
processor for those who find that of use, and to leave the core
language syntax alone.

Exactly. The concrete syntax should be purely a question of
presentation and user interface, that should be dealt with by the
editor. Source files could be kept in s-exp form, like it was
intended originally (50 years ago).

On May 28, 7:19 am, Caleb C. [email protected] wrote:

In my preprocessor, end is still allowed, it’s just optional now. So
you can copy and paste from a normal ruby file and not worry. (As log
as the snippet you paste is properly indented.)

This was my point. It would be good to still have the option of
freeform indentation. Much Ruby code isn’t presently canonically
formatted, and it’d break if put through your preprocessor. One
solution to this would be to only activate the preprocessor on a per-
file basis, but I think a better solution would be for the
preprocessor not to damage existing code. The ultimate goal in the
long term should be to make this style part of core Ruby syntax rather
than the result of a preprocessor, and the best way to do that would
be for the parser to be able to distinguish between indentation-aware
blocks and “old-style” blocks.

On 5/28/09, Eleanor McHugh [email protected] wrote:

Some cases have been presented elsewhere in this thread where the
pythonic indentation would fall flat: specifically for expressions of
the form

a = case x
when…
when…
when…
end if y

I feel that I did address this point adequately already, so it’s
annoying to see both you and Joshua B. bring it up again,
without even acknowledging that I had done so. Perhaps you don’t agree
with my points, but as you haven’t tried to rebut them, I think it
more likely that you did not read what I had to say at all.

To reiterate: Yes, you can’t really write that expression without an
end. Cases like this are why I decided, in my preprocessor, to allow
users to put in an end if they really want to. It’s sufficiently rare
to need more tokens on the same line after an end that it’s not
unreasonable to ask the user to type an extra (normally unnecessary)
end in those cases. 99% of ends can still be eliminated. If that
proportion were lower – 50% or 75% or even 90%, I would say that the
convenience of not having to type end is outweighed by the additional
burden of remembering that at least a tenth of the time you do anyway.
But when you get it down to the less than once a week level, a small
exception like this is ok. Very occasionally, you have to type end in
endless ruby. Very occasionally, you have to type pass in python. BFD.

It’s not a common formulation in this form, but it highlights the
problem of using significant whitespace as an expression delimiter in
expression-based languages. Were there a way to solve this ambiguity
elegantly then there would be a case in favour of introducing pythonic
indentation, but until then this would either introduce an unnecessary
limitation on the things that can be expressed in ruby or lead to a
considerably more complicated lexer that might have other implications.

I disagree that indentation sensitivity ‘considerably’ complicates the
lexer. endless.rb has about 180 lines of code, of which at least 25%
are concerned with the mechanics of pre-processing and wouldn’t be
necessary if it were fully integrated into the lexer. It was
moderately difficult, but not bad. Quite a few other ruby features
(here documents, utf-8 in identifiers, the ambiguity of some
operators) were what I would call complicated; by comparison
indentation was pretty easy.

The real answer to this issue is to have a pythonic indentation
pre-processor for those who find that of use, and to leave the
core language syntax alone.

I did, already.

On 5/28/09, J Haas [email protected] wrote:

On May 28, 7:19 am, Caleb C. [email protected] wrote:

In my preprocessor, end is still allowed, it’s just optional now. So
you can copy and paste from a normal ruby file and not worry. (As log
as the snippet you paste is properly indented.)

This was my point. It would be good to still have the option of
freeform indentation. Much Ruby code isn’t presently canonically
formatted, and it’d break if put through your preprocessor. One

If the end is misaligned, it will cause a warning on ruby 1.9. I think
the language is already moving in the direction of encouraging you to
properly indent everything anyway.
We’ve had changes to ruby before which break old code before, and I
think it’s ok for this to be one too. (If this is going to be
integrated into the core language at all.) Always maintaining
backwards compatibility is one of the things that makes systems with a
long history kludgy, and I’m glad ruby doesn’t have that hangup.

solution to this would be to only activate the preprocessor on a per-
file basis, but I think a better solution would be for the
preprocessor not to damage existing code. The ultimate goal in the
long term should be to make this style part of core Ruby syntax rather
than the result of a preprocessor, and the best way to do that would
be for the parser to be able to distinguish between indentation-aware
blocks and “old-style” blocks.

Let’s look at an example:

if foo: #new-style indentation block
while bar #old-style ended block
baz
end

Maybe you have something else in mind, idunno. To me, this just looks
totally weird, and I wouldn’t expect it to work at all. Parsing this
(in a program) probably wouldn’t be too much harder, but it’s
significantly jarring to human readers. (At least, to this one.)

On May 28, 9:59 am, Caleb C. [email protected] wrote:

If the end is misaligned, it will cause a warning on ruby 1.9.

Does it? That’s pretty cool.

I think
the language is already moving in the direction of encouraging you to
properly indent everything anyway.
We’ve had changes to ruby before which break old code before, and I
think it’s ok for this to be one too.

Well… keep in mind that this would break a lot of code. One of the
things I had my script do (the one which computed ~16% non-blank lines
were nothing but ‘end’) was try to figure out how many instances there
were of ‘end’ not being aligned with the block it began. The answer
was: relatively few, in the single-digit percentages, but that still
worked out to several hundred instances.

Always maintaining
backwards compatibility is one of the things that makes systems with a
long history kludgy, and I’m glad ruby doesn’t have that hangup.

I completely agree with you in principle, and I’m generally in favor
of breaking with the past when such a break is warranted. But
listening to the objections people have raised in this thread, for a
lot of people it boils down to “I don’t like it.” Well, there’s no
accounting for taste, and while I happen to feel that many of these
people would change their tune once they’ve had a little experience
with syntactic indentation, I think the best way to assuage their
doubts would be for any change to have absolutely no impact on
existing code, with a clear distinction between old-style syntax and
new-style indentation-aware syntax.

Besides, I kind of like the look of the colon as a block-start
delimiter. It emphasizes the separation between the loop or
conditional or method call and the block it controls.

Let’s look at an example:

if foo: #new-style indentation block
while bar #old-style ended block
baz
end

Maybe you have something else in mind, idunno. To me, this just looks
totally weird, and I wouldn’t expect it to work at all.

Yep, looks totally weird to me too. Okay, so: no end-delimited blocks
nested inside indentation-delimited blocks. Once you’re inside a scope
where the parser is paying attention to your indentation, it will
continue paying attention to your indendation until you leave that
scope.

On May 28, 2009, at 12:39 PM, Caleb C. wrote:

On 5/28/09, Eleanor McHugh [email protected] wrote:

The real answer to this issue is to have a pythonic indentation
pre-processor for those who find that of use, and to leave the
core language syntax alone.

I did, already.

I’m not convinced of the general utility of a pre-processor like
this. Obviously it can be used ‘privately’, but code that is
written with the expectation of being pre-processed is not
portable unless the wider Ruby community (or even other developers
in your organization) are prepared to run it through a preprocessor.

For that to work there has to be a ‘standard’ of some sort or
you’ll have to distribute the pre-processor along with your code.
And that argues for it to be part of the language and not a
pre-processor.

Yes, it is technically possible for this to ‘work’ but I just don’t
see the value in expending all that community energy to make it work.
I’d rather see better documentation, better implementations of the
existing syntax, gem compatibility with 1.9, frameworks that improve
my productivity, and so on.

Even if there was some way to prove that pythonic-indentation for
Ruby was ‘better’ than the current syntax rules any benefit has
to offset the costs of transition also (tools, developer workflow,
IDE re-writes, books, tutorials, bugs introduced in a mixed code
base, new idioms, etc.)

It just doesn’t seem worth it to me.

Gary W.

On 28 May 2009, at 17:39, Caleb C. wrote:

I feel that I did address this point adequately already, so it’s
annoying to see both you and Joshua B. bring it up again,
without even acknowledging that I had done so. Perhaps you don’t agree
with my points, but as you haven’t tried to rebut them, I think it
more likely that you did not read what I had to say at all.

Possibly not. This has been a long thread and I’m sure I’ve either
missed posts or misunderstood their intent.

exception like this is ok. Very occasionally, you have to type end in
endless ruby. Very occasionally, you have to type pass in python. BFD.

And as I’ve said elsewhere, how you handle things in a preprocessor is
entirely up to you. I’m not your target audience as I don’t like
pythonic indentation - it’s one of the reasons I didn’t take to the
language despite several attempts to get into it - but I’m sure those
who are won’t mind writing the occasional end statement.

I disagree that indentation sensitivity ‘considerably’ complicates the
lexer. endless.rb has about 180 lines of code, of which at least 25%
are concerned with the mechanics of pre-processing and wouldn’t be
necessary if it were fully integrated into the lexer. It was
moderately difficult, but not bad. Quite a few other ruby features
(here documents, utf-8 in identifiers, the ambiguity of some
operators) were what I would call complicated; by comparison
indentation was pretty easy.

Until I have time to study your code I’ll take your word for that.
Have you tried framing endless.rb as a patch for MRI’s lexer? and if
so how significant are the changes?

The real answer to this issue is to have a pythonic indentation
pre-processor for those who find that of use, and to leave the
core language syntax alone.

I did, already.

Then put out an [ANN] and if the community finds pythonic indentation
valuable, it’ll catch on.

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

On May 28, 2009, at 12:39 PM, Caleb C. wrote:

unreasonable to ask the user to type an extra (normally unnecessary)
end in those cases. 99% of ends can still be eliminated. If that
proportion were lower – 50% or 75% or even 90%, I would say that the
convenience of not having to type end is outweighed by the additional
burden of remembering that at least a tenth of the time you do anyway.
But when you get it down to the less than once a week level, a small
exception like this is ok. Very occasionally, you have to type end in
endless ruby. Very occasionally, you have to type pass in python. BFD.

Exactly, this is a bit of a straw man issue. Nobody is saying get
rid of all
ends. Use end where it makes sense, but if it’s just wasting space
there is
no need.

considerably more complicated lexer that might have other
implications.

I disagree that indentation sensitivity ‘considerably’ complicates the
lexer. endless.rb has about 180 lines of code, of which at least 25%
are concerned with the mechanics of pre-processing and wouldn’t be
necessary if it were fully integrated into the lexer. It was
moderately difficult, but not bad. Quite a few other ruby features
(here documents, utf-8 in identifiers, the ambiguity of some
operators) were what I would call complicated; by comparison
indentation was pretty easy.

That code was pretty standard looking. It had some functions in a
class. The
nesting wasn’t particularly deep. Are classes with functions now
considered
code smell? Also, if I recall correctly the Ruby core libraries are
about 16%
end. That ought to be, relatively speaking, pretty good Ruby. So
ironically, if Ruby added this capability it’s own code base would
shrink.

The real answer to this issue is to have a pythonic indentation
pre-processor for those who find that of use, and to leave the
core language syntax alone.

I did, already.

I think I’ll use it. That said, the case for giving core Ruby this
capability
seems to be getting stronger.

On 28 May 2009, at 16:05, Pascal J. Bourguignon wrote:

The real answer to this issue is to have a pythonic indentation pre-
processor for those who find that of use, and to leave the core
language syntax alone.

Exactly. The concrete syntax should be purely a question of
presentation and user interface, that should be dealt with by the
editor. Source files could be kept in s-exp form, like it was
intended originally (50 years ago).

I got pulled from a project a couple of years ago for saying the same
thing about DNS records. Some people really do lack a sense of humour :wink:

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

It’s interesting that Ruby doesn’t force you to use semicolons all
over the
place like C or Java. You only use them were they are actually
necessary
disambiguate something. Ruby also doesn’t force you to use
parentheses for
function definitions or function calls. Again, you only need them when
something is ambiguous. Ruby accomplishes these tricks by using
significant
whitespace. Optional ‘end’ seems very consistent with this philosophy.

On 5/28/09, J Haas [email protected] wrote:

I completely agree with you in principle, and I’m generally in favor
of breaking with the past when such a break is warranted. But
listening to the objections people have raised in this thread, for a
lot of people it boils down to “I don’t like it.” Well, there’s no
accounting for taste, and while I happen to feel that many of these
people would change their tune once they’ve had a little experience
with syntactic indentation, I think the best way to assuage their
doubts would be for any change to have absolutely no impact on
existing code, with a clear distinction between old-style syntax and
new-style indentation-aware syntax.

You’re making an argument based on other people’s opinions, for which
you have no evidence. I rather doubt that the naysayers on this thread
will be much convinced by such a fine point.

Besides, I kind of like the look of the colon as a block-start
delimiter. It emphasizes the separation between the loop or
conditional or method call and the block it controls.

Ah, now we get down to it. It’s really all just a matter of taste.

Would a semicolon make you just as happy? :wink: It looks almost the same
and can be used where you want a colon already:

while foo;
bar

Yep, looks totally weird to me too. Okay, so: no end-delimited blocks
nested inside indentation-delimited blocks. Once you’re inside a scope
where the parser is paying attention to your indentation, it will
continue paying attention to your indendation until you leave that
scope.

Can you give an example of when you would want indentation to be
ignored that’s not overly weird looking to you?