Ripper and Ruby 1.8

Hi all,

For the past two days I’ve been trying to get Ripper to compile with
Ruby 1.8
After searching and semi-translating some Japanese messages [1] with
Babelfish I managed to compile Ripper for 1.8 but when using it I get an
error-message: [Ripper - FATAL] : Unknown token 357 (or something like
that)

Does anyone here have experience with getting Ripper working under 1.8?
Does anyone know the specifics as to why Ripper states it needs 1.9?
Does it have to do with parse.y?

Any information on the subject is really really appreciated.

With kind regards,
Jonathan

[1] : http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-dev/26851

Hi,

At Sun, 20 Aug 2006 20:07:16 +0900,
Jonathan M. wrote in [ruby-talk:209477]:

Does anyone know the specifics as to why Ripper states it needs 1.9?
Does it have to do with parse.y?

Ripper needs bison, but 1.8 can be compiled with other yaccs.

[email protected] wrote:

Hi,

Ripper needs bison, but 1.8 can be compiled with other yaccs.

Thanks for the reply but I’m a bit at a loss here. I’m a total noob
regarding Ruby’s compilation process so I’m sorry if this is a stupid
question but if I have bison available why should I not be able to
compile Ripper against 1.8?

Thanks,
Jonathan

Hi,

On Sun, 20 Aug 2006 17:39:40 +0200, Jonathan M.
[email protected]
wrote:

Thanks for the reply but I’m a bit at a loss here. I’m a total noob
regarding Ruby’s compilation process so I’m sorry if this is a stupid
question but if I have bison available why should I not be able to
compile Ripper against 1.8?

I don’t know how to get Ripper working, but depending on what you want
to
do, there might be better solutions like RubyNode or ParseTree.

For example with RubyNode (http://rubynode.rubyforge.org/) you can do:

pp “3.times { puts ‘Ruby’ }”.parse_to_nodes.transform
[:iter,
{:var=>false,
:iter=>[:call, {:args=>false, :mid=>:times, :recv=>[:lit,
{:lit=>3}]}],
:body=>[:fcall, {:args=>[:array, [[:str, {:lit=>“Ruby”}]]],
:mid=>:puts}]}]
=> nil

The only thing that RubyNode or ParseTree won’t give you are the
comments
and whitespace in the code.

Dominik

Dominik B. wrote:

I don’t know how to get Ripper working, but depending on what you want
to do, there might be better solutions like RubyNode or ParseTree.

–snip–

It’s what we currently use in FreeRIDE and I kinda like the way Ripper
is used. Currently we only use it to aquire all methods/modules/classes.
I guess I could look into working with a different parser but I’d prefer
Ripper.

The only thing that RubyNode or ParseTree won’t give you are the
comments and whitespace in the code.

That’s a bit of a letdown, nothing too serious. Probably easy to work
around by wrapping the IO-source.
Do you have any idea as to how memory-intensive either are?

Anyhow, thanks for the suggestion. I’ll look into it if Ripper takes too
long to get running again.

Jonathan

On Sun, 20 Aug 2006 20:37:58 +0200, Jonathan M.
[email protected]
wrote:

I guess I could look into working with a different parser but I’d prefer
Ripper.

Ah okay, I thought you were trying to get Ripper running for the first
time, for a new project. If you already have code then it might be
easier
to get Ripper working.

Anyway, here is some basic code to get methods and classes with RubyNode
(it’s really basic and doesn’t work with nested classes, just to get you
started):

Lets say we have this file:

$ cat test.rb
class A
def a
1
end
def b
2
end
end

class B
def c
3
end
end

Then this code:

require “rubynode”
require “pp”

tree = IO.read(“test.rb”).parse_to_nodes.transform

def statements(block_or_single_node)
if block_or_single_node.first == :block
block_or_single_node.last
else
[block_or_single_node] # only one statement
end
end

statements(tree).each { |s|
if s.first == :class
p s.last[:cpath]
statements(s.last[:body].last[:next]).each { |d|
if d.first == :defn
p d.last[:mid]
pp d.last[:defn]
end
}
end
}

outputs:

[:colon2, {:mid=>:A, :head=>false}]
:a
[:scope,
{:rval=>false,
:tbl=>nil,
:next=>
[:block, [[:args, {:cnt=>0, :opt=>false, :rest=>-1}], [:lit,
{:lit=>1}]]]}]
:b
[:scope,
{:rval=>false,
:tbl=>nil,
:next=>
[:block, [[:args, {:cnt=>0, :opt=>false, :rest=>-1}], [:lit,
{:lit=>2}]]]}]
[:colon2, {:mid=>:B, :head=>false}]
:c
[:scope,
{:rval=>false,
:tbl=>nil,
:next=>
[:block, [[:args, {:cnt=>0, :opt=>false, :rest=>-1}], [:lit,
{:lit=>3}]]]}]

The only thing that RubyNode or ParseTree won’t give you are the
comments and whitespace in the code.

That’s a bit of a letdown, nothing too serious. Probably easy to work
around by wrapping the IO-source.
Do you have any idea as to how memory-intensive either are?

They both wrap Ruby’s internal NODEs. ParseTree converts them into
s-expressions (nested arrays), RubyNode’s transform converts them to
something similar (see above, a mix of arrays and hashes). So they use
as
much memory as their output needs.

If you really want to avoid the transformation to s-expressions then you
can use RubyNodes directly without calling transform (see
documentation),
but that usually shouldn’t be necessary.

And btw. if you want to get the nodes without evaling the code then you
currently have to use RubyNode, ParseTree doesn’t support that (yet).

Dominik

Hi,

At Mon, 21 Aug 2006 00:39:40 +0900,
Jonathan M. wrote in [ruby-talk:209505]:

Thanks for the reply but I’m a bit at a loss here. I’m a total noob
regarding Ruby’s compilation process so I’m sorry if this is a stupid
question but if I have bison available why should I not be able to
compile Ripper against 1.8?

Ripper needs a bison specific feature, but parse.y in 1.8 can’t
and doesn’t use it due to the compatibility.