There have been a hand full of releases since I have posted an
announcement
here. Since then, I have implemented a simple ERB-based template output
mode,
in lieu of the StringTemplate-based template mode provided by ANTLR’s
Java and
Python targets. I have also fixed several bugs, made a number of code
tweaks,
and authored a significant amount of usage and API documentation. Scroll
on
down for a catalog of significant changes since version 1.3.
Gem installation: gem install antlr3
Homepage: http://antlr.ohboyohboyohboy.org
Rubygems.org: antlr3 | RubyGems.org | your community gem host
GitHub: GitHub - ohboyohboyohboy/antlr3: Fully-featured ruby parser generation using ANTLR version 3
== About
Fully-featured ANTLR 3 parser generation for Ruby.
ANTLR (ANother Tool for Language Recognition) is a tool that is used to
generate code for performing a variety of language recognition tasks:
lexing,
parsing, abstract syntax tree construction and manipulation, tree
structure
recognition, and input translation. The tool operates similarly to other
parser generators, taking in a grammar specification written in the
special
ANTLR metalanguage and producing source code that implements the
recognition
functionality.
While the tool itself is implemented in Java, it has an extensible
design that
allows for code generation in other programming languages. To implement
an
ANTLR language target, a developer may supply a set of templates written
in
the StringTemplate (http://www.stringtemplate.org) language.
ANTLR is currently distributed with a fairly limited Ruby target
implementation. While it does provide implementation for basic lexer and
parser classes, the target does not provide any implementation for
abstract
syntax tree construction, tree parser class generation, input
translation, or
a number of the other ANTLR features that give the program an edge over
traditional code generators.
This gem packages together a complete implementation of the majority of
features ANTLR provides for other language targets, such as Java and
Python.
It contains:
-
A customized version of the latest ANTLR program, bundling all
necessary
java code and templates for producing fully featured language
recognition
in ruby code -
a ruby run-time library that collects classes used throughout the code
that
ANTLR generates -
a wrapper script, `antlr4ruby’, which executes the ANTLR command line
tool
after ensuring the ruby-tailored ANTLR jar is in java’s class path
== Features
- powerful LALR lexer, parser, and tree-parser class generation
- rapid development with ANTLR features like LL(*) prediction,
backtracking w/ memoization, and predicates - ANTLR run-time library code that aims to be more intuitive and
stylized for
the conventions ruby developers are accustomed to - integrated Abstract Syntax Tree (AST) construction, rewriting, and
parsing - an ERB-based template output mode to simplify source translation and
extraction - built-in test driver code to quickly try out grammar output
- debug and profile modes to help develop and optimize grammars
- extra utilities such as DOT-based graph generation and tree building
wizards to help with the development process
== History
=== 1.7.2 / 05-16-10
-
Minor Enhancements
- added new #hold convenience method to ANTLR3::CommonTokenStream
-
Bug Fixes
- corrected a typo in ANTLR3::Error#FailedPredicate which caused a
NoMethodError to raise instead of FailedPredicate when semantic
predicates failed
- corrected a typo in ANTLR3::Error#FailedPredicate which caused a
-
Documentation
- added more content to the ruby antlr3 guide
- set up an official project website at
http://antlr.ohboyohboyohboy.org
=== 1.7.0 / 04-20-10
-
Minor Enhancements
- added new #each_on_channel and #walk convenience methods to
ANTLR3::CommonTokenStream
- added new #each_on_channel and #walk convenience methods to
-
Bug Fixes
- discovered and fixed a rare but serious bug in the way the output
code
evaluates
syntactic predicates
- discovered and fixed a rare but serious bug in the way the output
-
Documentation
- began developing a more thorough usage guide for the package
- the guide is currently available on GitHub at
http://ohboyohboyohboy.github.com/antlr3
=== 1.6.3 / 02-12-10
- 4 Minor Enhancements
- added --help and --version options to antlr4ruby
- documented the antlr4ruby source and added license information
- prettied up the source code in all of the library files a little
- added a hoe-based rakefile with the gem distribution of the source
code
=== 1.6.0 / 01-27-10
-
4 Minor Enhancements to run-time library
- CommonTokenStream and subclasses can now also accept another
CommonTokenStream
object, serving sort of as a copy constructor. This provides a
reasonable
way
to cast a CommonTokenStream into a TokenRewriteStream and vice-versa - CommonTreeNodeStream can take an options hash at the end of any of
its
various
signatures to provide values for attributes like token_stream - If a parser is created with a CharacterStream object (i.e.,
StringStream),
it
will also try to cast it by checking for an associated lexer (like
it
currently
does when created with String or IO objects). - Template-output mode parsers have a new class method
“load_templates( path
)”
which will attempt to load the template group file at +path+ and, if
successful,
set it to the parser’s default template library. (See the CMinus
example
set for
an example of how this can be used)
- CommonTokenStream and subclasses can now also accept another
-
2 bug fixes / 1 change to templates
- backtracking was broken in situations where recognition failed when
a semantic predicate was encountered as the template code did not
throw BacktrackingFailed when it was in backtracking mode instead of
PredicateFailed - rule memoization also had a major bug in that rules that were
memoized
after failure did not throw BacktrackingFailed when it was needed - I changed
attribute scope' structures (not rule return scopes, but the groups of attributes you can assign to rules with the
scope’
keyword),
from being implemented as named constants in the class body to class
variables belonging to the recognizer. As class variables do not
need
to follow naming conventions, this provides more freedom in naming
without
all of the ugly name mangling. These are generally used privately by
parsers,
so it shouldn’t be too drastic of a change as to break end user’s
code.
- backtracking was broken in situations where recognition failed when
=== 1.5.0 / 01-25-10
-
2 minor enhancements
- Added BaseTree#walk, which iterates in a top-down fashion across all
notes
in a tree. - Added BaseTree#prune, which stops decent into the current node
during
BaseTree#walk.
- Added BaseTree#walk, which iterates in a top-down fashion across all
-
1 optimization
- changed
closure loops' ( uses of the
*’ or+' modifiers ) from using Kernel#loop to
while true … end’ style loops, which should be
slightly
faster
- changed
-
new test code
- test/functional/template-output/template-output.rb serves as a basic
functional test implementation for the new template output mode
- test/functional/template-output/template-output.rb serves as a basic
=== 1.4.0 / 01-17-10
-
1 major new feature
- finished basic implementation of a template output mode.
- lib/antlr3/template.rb defines a number of classes for
ERB-based template output translation - templates/ST.stg defines the Ruby target templates to
generate template output mode recognizers
-
new test code
- added template unit tests at test/unit/test-template.rb
- test/functional/template-output/template-output.rb serves as a basic
functional test implementation for the new template output mode
Best Regards,
Kyle Yetter