Forum: Ruby-core [ruby-trunk - Bug #7255][Open] Ripper inside class_eval ommits :string_content between new line and

Issue #7255 has been reported by brueckenschlaeger23 (Micha J).

----------------------------------------
Bug #7255: Ripper inside class_eval ommits :string_content between new 
line and :string_embexpr
https://bugs.ruby-lang.org/issues/7255

Author: brueckenschlaeger23 (Micha J)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:
ruby -v: 1.9.3p286


See the following Code parsed by Ripper using

    content = 'class_eval <<-RUBY, __FILE__, __LINE__+1

        def #{parent}contents(*args)
          ""
        end
    RUBY
    '

    Ripper::SexpBuilder.new(content).parse

The resulting sexp ommits truncates the first string_add to "        \n" 
instead of "        \n        def "

            - - :string_literal
              - - :string_add
                - - :string_add
                  - - :string_add
                    - - :string_content
                    - - :@tstring_content
                      - "        \n"
                      - - 2
                        - 0

If you remove the empty line, it works fine and the "     def " is part 
of the string. As well it works without any embed #{}.

Issue #7255 has been updated by brueckenschlaeger23 (Micha J).


I noticed, that this bug reaches a bit wider:

    'class_eval <<-RUBY, __FILE__, __LINE__+1

        def #{parent}_contents(*args)
          test = ""
          test ||= "#{parent}"
        end
    RUBY
    '

'      test ||= "' also is omitted by ripper. This is the string between 
the "\n" of [:@tstring_content, "_contents(*args)\n          test = 
\"\"\n", [3, 21]]] the next embed.
----------------------------------------
Bug #7255: Ripper inside class_eval ommits :string_content between new 
line and :string_embexpr
https://bugs.ruby-lang.org/issues/7255#change-32164

Author: brueckenschlaeger23 (Micha J)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:
ruby -v: 1.9.3p286


See the following Code parsed by Ripper using

    content = 'class_eval <<-RUBY, __FILE__, __LINE__+1

        def #{parent}contents(*args)
          ""
        end
    RUBY
    '

    Ripper::SexpBuilder.new(content).parse

The resulting sexp ommits truncates the first string_add to "        \n" 
instead of "        \n        def "

            - - :string_literal
              - - :string_add
                - - :string_add
                  - - :string_add
                    - - :string_content
                    - - :@tstring_content
                      - "        \n"
                      - - 2
                        - 0

If you remove the empty line, it works fine and the "     def " is part 
of the string. As well it works without any embed #{}.

Issue #7255 has been updated by brueckenschlaeger23 (Micha J).


Another update: It's not connected to class_eval but to the <<-RUBY and 
<<RUBY multiline String versions. This is my only use case for this 
format so I didn't notice.

----------------------------------------
Bug #7255: Ripper inside class_eval ommits :string_content between new 
line and :string_embexpr
https://bugs.ruby-lang.org/issues/7255#change-32165

Author: brueckenschlaeger23 (Micha J)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:
ruby -v: 1.9.3p286


See the following Code parsed by Ripper using

    content = 'class_eval <<-RUBY, __FILE__, __LINE__+1

        def #{parent}contents(*args)
          ""
        end
    RUBY
    '

    Ripper::SexpBuilder.new(content).parse

The resulting sexp ommits truncates the first string_add to "        \n" 
instead of "        \n        def "

            - - :string_literal
              - - :string_add
                - - :string_add
                  - - :string_add
                    - - :string_content
                    - - :@tstring_content
                      - "        \n"
                      - - 2
                        - 0

If you remove the empty line, it works fine and the "     def " is part 
of the string. As well it works without any embed #{}.

Issue #7255 has been updated by brueckenschlaeger23 (Micha J).


I did some further digging regarding the parser/scanner events.

Basically the cause for the error can be seen here (parsing the example 
from the description).

    [:@ident, "class_eval", [1, 0]]
    [:@sp, " ", [1, 10]]
    [:@heredoc_beg, "<<-RUBY", [1, 11]]
    [:string_content]
    [:@tstring_content, "        \n", [2, 0]]
    [:string_add, [:string_content], [:@tstring_content, "        \n", 
[2, 0]]]
    [:@embexpr_beg, "        def \#{", [3, 0]]   <------- this line
    ...

The scanner-event :@embexpr_beg contains the missing text which then 
will be discarded to create the string_embedexp from the sexp.
----------------------------------------
Bug #7255: Ripper inside class_eval ommits :string_content between new 
line and :string_embexpr
https://bugs.ruby-lang.org/issues/7255#change-32386

Author: brueckenschlaeger23 (Micha J)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:
ruby -v: 1.9.3p286


See the following Code parsed by Ripper using

    content = 'class_eval <<-RUBY, __FILE__, __LINE__+1

        def #{parent}contents(*args)
          ""
        end
    RUBY
    '

    Ripper::SexpBuilder.new(content).parse

The resulting sexp ommits truncates the first string_add to "        \n" 
instead of "        \n        def "

            - - :string_literal
              - - :string_add
                - - :string_add
                  - - :string_add
                    - - :string_content
                    - - :@tstring_content
                      - "        \n"
                      - - 2
                        - 0

If you remove the empty line, it works fine and the "     def " is part 
of the string. As well it works without any embed #{}.

Issue #7255 has been updated by zzak (Zachary Scott).

Category set to ext
Assignee set to nobu (Nobuyoshi Nakada)


----------------------------------------
Bug #7255: Ripper inside class_eval ommits :string_content between new 
line and :string_embexpr
https://bugs.ruby-lang.org/issues/7255#change-32440

Author: brueckenschlaeger23 (Micha J)
Status: Open
Priority: Normal
Assignee: nobu (Nobuyoshi Nakada)
Category: ext
Target version:
ruby -v: 1.9.3p286


See the following Code parsed by Ripper using

    content = 'class_eval <<-RUBY, __FILE__, __LINE__+1

        def #{parent}contents(*args)
          ""
        end
    RUBY
    '

    Ripper::SexpBuilder.new(content).parse

The resulting sexp ommits truncates the first string_add to "        \n" 
instead of "        \n        def "

            - - :string_literal
              - - :string_add
                - - :string_add
                  - - :string_add
                    - - :string_content
                    - - :@tstring_content
                      - "        \n"
                      - - 2
                        - 0

If you remove the empty line, it works fine and the "     def " is part 
of the string. As well it works without any embed #{}.