I bet you all have heard about ANTLR (ANother Tool for Language Recognition) and have some idea of what it does (hint: it’s a tool to generate parsers) but, if you’re anything like me, you never cared about learning more about ANTLR as you ruled it out as irrelevant from a software modeling perspective.

Yes, textual modeling is important but we already have tools like Xtext for that, aren’t we?  Well, yes and no. My good friend Federico Tomassetti  (we’ve had him here in this blog talking about creating DSLs on a budget with Kotlin , using ANTLR, no surprise here!) convinced me that I was too narrow-minded and keeping a very Eclipse-focused view of the problem.

There are several reasons you may prefer ANTLR over other more “modeling-oriented” solutions like Xtext to solve a language engineering problem:

• The ANTLR runtime is a small component, easy to integrate into online editors (anyone interested in building another web-based modeling editor?) or as the basis for building a full-blown compiler or model interpreter
• Xtext is based on Java, while you may need a parser that can be used in C#, Python or JavaScript. ANTLR has runtimes for all these languages.
• Xtext is based on OSGi. This has plenty of advantages in some cases but it also brings plenty of headaches and dependencies that make difficult to use Xtext parsers outside the Eclipse environment.
• And sometimes you just don’t want to run your parser within any kind of Eclipse-based environment. If so, it’s easier to directly use ANTLR.
• Or you may not even be interested in having any kind of IDE for your language, e.g. when transpiling your input language to another language. In those cases, all the powerful Xtext IDE features (like autocompletion) are irrelevant and therefore a simpler and lighter option as the one ANTLR provides is the way to go.

As usual, there is no one-size-fits-all tool but I’m now convinced that knowing ANTLR is a requirement for any language engineer, even if your goal is to develop domain-specific modeling languages. You never know when the flexibility of ANTLR will come in handy!.

ANTLR basics

Ok, I’m now convinced. Can you explain again what ANTLR is? Sure, ANTLR a powerful parser generator for reading, processing, executing, or translating structured text or binary files. Created by Terence Parr in 1989 and growing ever since. If you don’t believe me just take a look at the huge number of ANTLR grammars that have been contributed to this public ANTLR repository over the years.  There are a lot of languages (like C#, Java, Python, etc) for which we have a grammar ready to use.

ANTLR is more than a library or a way to describe grammars, ANTLR is a whole set of tools to help be productive: from creating to testing to debugging grammars to visualize the result of parsing. All this while trying to hide all complex parsing theory details.

Moreover, you can write just one grammar and let ANTLR generate parsers in many languages (Java, C#, Python, etc.). You use the same tool to define the grammar and then choose the appropriate runtime for your target environment.

Learning ANTLR

Nope. I’m not going to teach you ANTLR myself. My job was to explain to you why I think (after changing my mind) why taking a closer look at ANTLR was worth. I’ll now let Federico be your ANTLR teacher.

The free but time-consuming way to learn ANTLR

You can learn ANTLR on your own. The official ANTLR website and its reference book include plenty of useful information. But, without a doubt, the best way to learn ANTLR on your own is to head on this ANTLR mega tutorial where you’ll learn:

• the basics: what a parser is, what it can be used for
• see how to setup ANTLR to be used from Javascript, Python, Java and C#
• discuss how to test your parser
• present some advanced features

all with tons of examples

The most effective way to learn ANTLR

If you, like me, value your time, going over a lengthy tutorial is not an optimal way to learn ANTLR. To fix this, Strumenta (the company Federico co-founded together with his brother Gabriele)  has created the ANTLR video course “Using ANTLR like a professional”. 14 great video lessons (adding up to over 3h of videos!) revealing all secrets of ANTLR, also including a very didactic introduction to basic parsing concepts (very useful for me, last time I played with a lexer was during my CS degree and that was already the past century!).

The course has three different versions. The cheapest alternative, at only 77$ (just twice the cost of the official ANTLR reference book), is really a bargain. The intermediate one includes a 50+ pages manual with cheatsheets for ANTLR and 5 Example Grammars explained and commented ranging from data formats to programming languages. Buying the professional version of the ANTLR course gives you 2h of coaching with Federico and Gabriele.

In all versions, the lessons cover installing, creating, debugging and testing ANTLR parsers while also covering methodological aspects, e..g advising on how to best attack the challenge of writing a grammar (e.g. top-down vs bottom-up approaches) or how to integrate parsers in our programs (via listeners or visitors, depending on your need).

Lessons also cover more advanced content like semantic predicates for context-sensitive parsing or lexical modes (to handle multiple languages in the same grammar) and a variety of tips and tricks stemming from their large experience building ANTLR grammars so that you can, too, use “ANTLR like a professional” as they say and go from creating parsers to understanding full-blown compilers

I’ve been watching all lessons myself and I can attest you they are extremely didactic. The video shows the basic concepts in a slide while they talk over it to complement the key message

And of course, when necessary, they also show the actual code running (they recommend VS Code for that due to the high-quality ANTLR extensions available in the Visual Studio marketplace).

One way or the other, I hope you consider giving ANTLR a chance and adding it to your arsenal of tools to face new (modeling) language engineering challenges. And of course, if you do, we’ll be happy to get your opinion on ANTLR!

Jordi Cabot

FNR Pearl Chair. Head of the Software Engineering RDI Unit at LIST. Affiliate Professor at University of Luxembourg.  More about me.