Possble bug in Ruby parser (Fixnum#times within "case" statement)

Hi, I cannot find an explanation for the following issue so I think it’s
a bug:


def kk

while case @state

when :init
  10.times do |index|
    puts "index = #{index}"
  end
  @state = :end
  true

 when :end
   puts "END"

 end

end

end

@state = :init
kk

Running the above code gives an error:

~# ruby1.9.3 kk.rb

kk.rb:6: syntax error, unexpected keyword_do_cond, expecting keyword_end
10.times do |index|
^
kk.rb:7: syntax error, unexpected tSTRING_BEG, expecting keyword_do or
‘{’ or ‘(’
puts “index = #{index}”
^
kk.rb:12: syntax error, unexpected keyword_when, expecting $end
when :end
^

Why??? If I replace:

  10.times do |index|
    puts "index = #{index}"
  end

with:

  10.times { |index|
    puts "index = #{index}"
  }

but of course using “do - end” is valid for Fixnum#times !!
What is happening?

This issue occurs in Ruby 1.8 and 1.9.

Any comment please?

On Aug 21, 2011, at 16:12 , Iaki Baz C. wrote:

Hi, I cannot find an explanation for the following issue so I think it’s a bug:

reformatted to be idiomatic and therefore readable:

    end

kk.rb:6: syntax error, unexpected keyword_do_cond, expecting keyword_end
10.times do |index|
^
Why???

Because it is nonsensical and confuses the parser. I’m not really sure
WHY you have a while in there at all but I can describe the components
involved:

while expr [do] body end

That ‘do’ is important here… You’ve started a while loop, started the
condition expression, left it hanging, and then hit the do in a place
that doesn’t make any sense to the condition expression… You have an
unterminated case/when. You can see this in all of its grubby detail by
running ‘ruby -ycw kk.rb’

Since I can’t make any sense of your code, I can only guess as to your
intent and offer suggestions:

loop do

… case, et al

end

or

def kk
while (case @state
when :init then
10.times do |index|
puts “index = #{index}”
end
@state = :end
true
when :end then
puts “END”
end) do
# do nothing?
end
end

or just get rid of your loop as it contributes nothing.

On Aug 21, 2011, at 7:12 PM, Iaki Baz C. wrote:

 end

 10.times do |index|

Why??? If I replace:

but of course using “do - end” is valid for Fixnum#times !!
What is happening?

“while” cannot take braces, only do-end. Your entire case expression
is the clause of the while loop, so the “do” is begin confused for the
“do” that optionally comes between the “while”'s condition and its body.
Since a brace never comes between a while condition and a while body,
there’s no ambiguity.

Ruby’s grammar is expressive, but when you do things like put many-line
case statements in a while condition, you’re going to get weird corner
cases in the grammar.

Michael E.
[email protected]
http://carboni.ca/

2011/8/22 Ryan D. [email protected]:

    end) do

do nothing?

end
end

That was exactly the point, thanks a lot. I understand it, and
honestly I don’t know how my code run before :slight_smile:

or just get rid of your loop as it contributes nothing.

The provided code was just a simplification of my real code. In my
real code I do need such a while to check the return value of the case
statement to continue looping or not.

Thanks a lot.

On Aug 21, 2011, at 16:56 , Iaki Baz C. wrote:

or just get rid of your loop as it contributes nothing.

The provided code was just a simplification of my real code. In my
real code I do need such a while to check the return value of the case
statement to continue looping or not.

I still say it contributes nothing and only serves to obfuscate your
code. You can always pull it out to something MUCH more readable:

kk until valid?

2011/8/22 Ryan D. [email protected]:

I still say it contributes nothing and only serves to obfuscate your code. You
can always pull it out to something MUCH more readable:

kk until valid?

Yes, “kk” method is useless, already said that in my previous example.
My real code looks more like the following:


def receive_data(data)
  @buffer << data

  while (case @state
    when :init
      @parser.reset
      @parser_nbytes = 0
      @state = :headers

    when :headers
      parse_headers

    when :body
      get_body

    when :finished
      process_request
      @state = :init
      # return true to continue processing possible TCP remaining 

data.
true

    when :ignore
      false
    end) do
  end  # while

end

2011/8/22 Michael E. [email protected]:

“while” cannot take braces, only do-end. Your entire case expression is the
clause of the while loop, so the “do” is begin confused for the “do” that
optionally comes between the “while”'s condition and its body. Since a brace never
comes between a while condition and a while body, there’s no ambiguity.

Ruby’s grammar is expressive, but when you do things like put many-line case
statements in a while condition, you’re going to get weird corner cases in the
grammar.

Thanks a lot. I understand it now. My code was just wrong from the
beginning (but worked in some specific case just by chance).

Thanks.

When I use case/when, I tend to use:

loop {

}

rather than while.

I found it is much more readable that way.

loop {

user_input = $stdin.gets.chomp

case user_input
when 'foo'
when 'bar'
when 'q','exit','quit'
  exit
end

}

Seems to read much better than:

while foo > bla

2011/8/22 Ryan D. [email protected]:

get_body
process_request
end

catch :done do
loop do
receive_data get_data
end
end

This is a very simplistic and urrealistic approach for a parsing on
top of TCP in which you can receive incomplete data in each TCP
package so must wait until the event reactor (EventMachine in this
case) calls receive_data(data) again.

For example, the method parse_headers() reads data from the buffer and
determines if it’s a complete protocol message (just headers, as in a
HTTP request) or not. If so, it returns and sets @state=:body, true so
the case statement gets true and the loop continues, but now it goes
to get_body() method. This method, similary, checks whether there are
Content-Length bytes in the buffer. If so, it has the complete body,
sets @state=:finished and returns true. If not, it returns false so
the while ends. Next time the TCP server receives pending data, the
reactor calls receive_data(data) again so the case enters again in
get_body() method and reads, probably, the entire buffer to complete
the body.

It’s much more complex than your code which cannot work in a real TCP
server in which nobody can expect to receive the entire protocol
message in a single TCP package.

On Aug 22, 2011, at 01:43 , Iaki Baz C. wrote:

     @parser.reset
     process_request
     @state = :init
     # return true to continue processing possible TCP remaining data.
     true

   when :ignore
     false
   end) do
 end  # while

end

Needless complexity is needless and will always bite you sooner or
later.

Compare:

def receive_data(data)
@buffer << data

@parser.reset
@parser_nbytes = 0
parse_headers
get_body
process_request
end

catch :done do
loop do
receive_data get_data
end
end

On Aug 22, 2011, at 12:55 , Iaki Baz C. wrote:

It’s much more complex than your code which cannot work in a real TCP
server in which nobody can expect to receive the entire protocol
message in a single TCP package.

There is nothing in my code example that implies that you receive
the entire payload. You’re inferring that on your own. The complexity
you introduce at the level you do it at is unnecessary.

2011/8/22 Ryan D. [email protected]:

It’s much more complex than your code which cannot work in a real TCP
server in which nobody can expect to receive the entire protocol
message in a single TCP package.

There is nothing in my code example that implies that you receive the entire
payload. You’re inferring that on your own. The complexity you introduce at the
level you do it at is unnecessary.

Your code calls get_body() after the function parse_headers(), in any
case:

def receive_data(data)
@buffer << data
@parser.reset
@parser_nbytes = 0
parse_headers
get_body
process_request
end

Why should you call get_body() when maybe parse_header() method has
not finished receiving all the headers?

2011/8/24 Ruby I. [email protected]:

Which suggests the parser is confused about which to evaluate first, the
while or the case block. But I don’t think it’s ambiguous, since the
case/when must be evaluated first, since the when only goes with the case.

In that code it’s clear that the “case” statement goes between ( ) so
its “end” is the one before “)”.

So I see:

a) while ( EXPRESSION ) ; end

or:

b) while ( EXPRESSION ) ; do CODE ; end

both are valid. But in case EXPRESSION contains some “end” keyword and
option a) is used, then ensure that EXPRESSION goes between ( ).

Yes, the parens let the parser know when the case statement ends.
If it sees the while statement, and then the case, I think when it hits
the “end” it applies it to the while ?, and there the compilation fails.
Oddly, it does work with the braces, a quirk I’ve personally experienced
also.

My preference would be that {…} === do … end
in every case. if one fails, the other fails, and vice versa.

In this case, because you have while case …
you would think it could handle the case statement first and then apply
it against the while. But it’s a lot to ask – other languages have an
enforced (…) after while, plus semicolons. Ruby doesn’t complain much,
but there are these edge cases where we see a little inconsistency. In
my experience, after a few months using the language pretty often, I’ve
stopped having trouble with these sorts of things for the most part.

another example, this works:

for y in (1…3) do
puts ‘y’
end

but this fails:

for y in (1…3) {
puts ‘y’
}

???

Dave

So this refactoring works:

def kk

while ( case @state
when :init
10.times do |index|
puts “index = #{index}”
end
@state = :end
true

  when :end
    puts "END"

  end )
end

end

@state = :init
kk

Which suggests the parser is confused about which to evaluate first, the
while or the case block. But I don’t think it’s ambiguous, since the
case/when must be evaluated first, since the when only goes with the
case.

I wonder if there’s a ‘case’ to be made for a fix here.

David