Antlr3 1.7.2 ( improved ANTLR v3 language recognition for Ruby )

There have been a hand full of releases since I have posted an
announcement
here. Since then, I have implemented a simple ERB-based template output
mode,
in lieu of the StringTemplate-based template mode provided by ANTLR’s
Java and
Python targets. I have also fixed several bugs, made a number of code
tweaks,
and authored a significant amount of usage and API documentation. Scroll
on
down for a catalog of significant changes since version 1.3.

Gem installation: gem install antlr3
Homepage: http://antlr.ohboyohboyohboy.org
Rubygems.org: antlr3 | RubyGems.org | your community gem host
GitHub: GitHub - ohboyohboyohboy/antlr3: Fully-featured ruby parser generation using ANTLR version 3

== About

Fully-featured ANTLR 3 parser generation for Ruby.

ANTLR (ANother Tool for Language Recognition) is a tool that is used to
generate code for performing a variety of language recognition tasks:
lexing,
parsing, abstract syntax tree construction and manipulation, tree
structure
recognition, and input translation. The tool operates similarly to other
parser generators, taking in a grammar specification written in the
special
ANTLR metalanguage and producing source code that implements the
recognition
functionality.

While the tool itself is implemented in Java, it has an extensible
design that
allows for code generation in other programming languages. To implement
an
ANTLR language target, a developer may supply a set of templates written
in
the StringTemplate (http://www.stringtemplate.org) language.

ANTLR is currently distributed with a fairly limited Ruby target
implementation. While it does provide implementation for basic lexer and
parser classes, the target does not provide any implementation for
abstract
syntax tree construction, tree parser class generation, input
translation, or
a number of the other ANTLR features that give the program an edge over
traditional code generators.

This gem packages together a complete implementation of the majority of
features ANTLR provides for other language targets, such as Java and
Python.
It contains:

  • A customized version of the latest ANTLR program, bundling all
    necessary
    java code and templates for producing fully featured language
    recognition
    in ruby code

  • a ruby run-time library that collects classes used throughout the code
    that
    ANTLR generates

  • a wrapper script, `antlr4ruby’, which executes the ANTLR command line
    tool
    after ensuring the ruby-tailored ANTLR jar is in java’s class path

== Features

  • powerful LALR lexer, parser, and tree-parser class generation
  • rapid development with ANTLR features like LL(*) prediction,
    backtracking w/ memoization, and predicates
  • ANTLR run-time library code that aims to be more intuitive and
    stylized for
    the conventions ruby developers are accustomed to
  • integrated Abstract Syntax Tree (AST) construction, rewriting, and
    parsing
  • an ERB-based template output mode to simplify source translation and
    extraction
  • built-in test driver code to quickly try out grammar output
  • debug and profile modes to help develop and optimize grammars
  • extra utilities such as DOT-based graph generation and tree building
    wizards to help with the development process

== History

=== 1.7.2 / 05-16-10

  • Minor Enhancements

    • added new #hold convenience method to ANTLR3::CommonTokenStream
  • Bug Fixes

    • corrected a typo in ANTLR3::Error#FailedPredicate which caused a
      NoMethodError to raise instead of FailedPredicate when semantic
      predicates failed
  • Documentation

=== 1.7.0 / 04-20-10

  • Minor Enhancements

    • added new #each_on_channel and #walk convenience methods to
      ANTLR3::CommonTokenStream
  • Bug Fixes

    • discovered and fixed a rare but serious bug in the way the output
      code
      evaluates
      syntactic predicates
  • Documentation

=== 1.6.3 / 02-12-10

  • 4 Minor Enhancements
    • added --help and --version options to antlr4ruby
    • documented the antlr4ruby source and added license information
    • prettied up the source code in all of the library files a little
    • added a hoe-based rakefile with the gem distribution of the source
      code

=== 1.6.0 / 01-27-10

  • 4 Minor Enhancements to run-time library

    • CommonTokenStream and subclasses can now also accept another
      CommonTokenStream
      object, serving sort of as a copy constructor. This provides a
      reasonable
      way
      to cast a CommonTokenStream into a TokenRewriteStream and vice-versa
    • CommonTreeNodeStream can take an options hash at the end of any of
      its
      various
      signatures to provide values for attributes like token_stream
    • If a parser is created with a CharacterStream object (i.e.,
      StringStream),
      it
      will also try to cast it by checking for an associated lexer (like
      it
      currently
      does when created with String or IO objects).
    • Template-output mode parsers have a new class method
      “load_templates( path
      )”
      which will attempt to load the template group file at +path+ and, if
      successful,
      set it to the parser’s default template library. (See the CMinus
      example
      set for
      an example of how this can be used)
  • 2 bug fixes / 1 change to templates

    • backtracking was broken in situations where recognition failed when
      a semantic predicate was encountered as the template code did not
      throw BacktrackingFailed when it was in backtracking mode instead of
      PredicateFailed
    • rule memoization also had a major bug in that rules that were
      memoized
      after failure did not throw BacktrackingFailed when it was needed
    • I changed attribute scope' structures (not rule return scopes, but the groups of attributes you can assign to rules with the scope’
      keyword),
      from being implemented as named constants in the class body to class
      variables belonging to the recognizer. As class variables do not
      need
      to follow naming conventions, this provides more freedom in naming
      without
      all of the ugly name mangling. These are generally used privately by
      parsers,
      so it shouldn’t be too drastic of a change as to break end user’s
      code.

=== 1.5.0 / 01-25-10

  • 2 minor enhancements

    • Added BaseTree#walk, which iterates in a top-down fashion across all
      notes
      in a tree.
    • Added BaseTree#prune, which stops decent into the current node
      during
      BaseTree#walk.
  • 1 optimization

    • changed closure loops' ( uses of the *’ or +' modifiers ) from using Kernel#loop to while true … end’ style loops, which should be
      slightly
      faster
  • new test code

    • test/functional/template-output/template-output.rb serves as a basic
      functional test implementation for the new template output mode

=== 1.4.0 / 01-17-10

  • 1 major new feature

    • finished basic implementation of a template output mode.
    • lib/antlr3/template.rb defines a number of classes for
      ERB-based template output translation
    • templates/ST.stg defines the Ruby target templates to
      generate template output mode recognizers
  • new test code

    • added template unit tests at test/unit/test-template.rb
    • test/functional/template-output/template-output.rb serves as a basic
      functional test implementation for the new template output mode

Best Regards,
Kyle Yetter