This should be a good tutorial. I need to look into it:
http://www.ibm.com/developerworks/edu/os-dw-os-ecl-commplgin1.html
I know there is an excellent tutorial in Chinese. It demos how to craft a source code editor using Eclipse JFace Text Framework and a bit ANTLR. The link is here.
Here are some tutorial links for learning antlr v3:
If you are doing antlr v3 for Java, you can be very interested in this series of video tutorials. It demos how to develop with antlr3 IDE, and also the process of a small language. It credits to Marcel for the recommendation. Check it out at:
ANTLR 3.x Tutorials:
http://vimeo.com/groups/29150/videos
I started to experiment Domain-Specific Language (DSL) implementation two weeks ago. The tool I use is antlr v3. We know that Xtext, TCS, Boo are good DSL toolkit. The reason why we don’t use them for now, is because you may be restricted inside their available functions. For example, xtex is more a Domain-Specific Modeling tool than DSL tool, though they have overlaps. I will have to wait for future xtext versions as I see our requirements go beyonds its current features.
Right now have some experience that may be useful, which is from the coding debugging experience. The prototype I implemented is a dynamic script language. It has its own file structure, which defines input, process, and output options and rules.
The input part defines rules that will assign domain meaning to input data, based on evaluating conditions.
The process part supports arithmetical and string computing. It helps further yield, update data from input.
The output part supports dumping data into certain format like XML. Closure output is supported, since XML has such nature.
The following technologies are used in the prototype:
ANTLR works is a great GUI tool for grammar authoring and debugging. There are eclipse plug-ins for antlr, but I haven’t had time to test them. But more importantly, let’s talk about the grammars.
At antlr.org, we can download many sample grammars for popular languages. However, the parser grammar is only useful to say if an input is valid as the language. The main work is to define compiler/interpreter part in implementing a language. Generally there are two kinds of work in this part: (1). The supporting platform/framework to run the language; (2). The tree walker(s) to do staggered jobs, normally compiling or interpreting tasks. Or simply put, make the language runnable.
I defined 4 grammars, one for parser/lexer, and 3 for AST tree walking. When you DSL goes complicated, it helps to define separate groups functions in different grammar files. For example, we define an emitter class, which only deals with outputting XML results. You may say that there is performance problem if you walking through AST multiple times. I don’t have benchmark data so far and we will see. At least I can see that walking in-memory data structure should be quick and its time complexity is only increased linearly.
The expression calculation is in a separate grammar file, and the corresponding tree parser is a utility class. The statement executor is defined by another grammar file. By this way, it helps to refine the expression definition, and make it easy and quick to be reused in another project. In a production environment, it helps to define them in the same tree walker, but it helps to make it clear in current phase.
A beginner may encounter many issues going through the process. One possible issue is to generated runnable lexer/parser/walker. Since you embed action codes (java, c#, …) in grammar files, you may have to go back and forth multiple times to eliminate compilation errors. A good IDE (eclipse, visual studio) helps a lot in locating such issues.
Another issue is to understand how it works. I’d recommend: (1). debugging through parser/walker code to get famliar with it; (2). Read and run examples at antlr.org to explore wider problem space.
Wish the above stuff may help someone in some way. Feel free to comment if any question.
p.s. I found a bug with CommonTreeNodeStream#index() of antlr-v3.1.3. A bug report had been filed at antlr.org days ago. The bug will affect the following tasks:
I watched a good demo about M language from Microsoft. The link is listed below:
Modeling: Transformation and Constraints
http://www.langnetsymposium.com/2009/talks/37-PaulVickDavidLangworthy-M.html
As I watch the demo video, I start to realize that DSL and domain modeling may be two things, but with certern intersection. For example, if you really devise a computer language, it should have the abilities of executing programs. However, in the demo, oslo looks more like to describe your data or model, and lacks of executing support.
Another example is Xtext, which is a sub project of eclipse Textual Modeling Framework. There will release first version in June this year. I had a quick talk with some people of them. So far, to me, it looks more like to describe the strucutres of your data or domain model. In comparison, ANTLR v3 is supporting actions and templated transformations, AST and tree-based interpretion/transform. Therefore it can do more than just describe your stuff. You can define tree grammar to rewrite or translate your model to another model or even excutable stuff (bytecode, java code, c# code). Xtext people said that they plan to support actions in next version. Sounds like that they are going to be more like ANTLR v3. An interesting roadmap!
I am playing with DSLs. As a developer without too much language/compiler experience, it is not easy to get started. Fortunately, my friend Yu has a lot of experience, and Internet is another resort for resources. So, it becomes a bit easier for my evening learning.
Based on my recent reading, I would recommend the following resources for kick-off. If you are a DSL beginner who dones’t have plenty of experience (like me), it may be helpful:
1. There are two papers
a. On the Specification of Textual Syntaxes for Models
b. TCS: a DSL for the Specification of Textual Concrete Syntaxes in Model Engineering
The papers are written by committers of eclipse TCS project. They help to give you a feel and look in an academic perspective. I think it good to know about basic framework of methodology and a few terms in implementing a DSL.
2. ANTLR, get your hands dirty
Having knowledge with ANTLR will help your language definition journey. It is quite easy to master ANTLR with well-prepared documentations:
a. ANTLR Getting Start
b. Five minites introduction ANTLR v3
c. ANTLR v3 by Mark Volkmann
d. Some DSL posts in the Article section.
3. Some Links for Domain Specific Languages
This is a post I wrote earlier. It contains a few useful links:
http://www.frankdu.com/weblog/archives/21