Get dtg
Either install the DSLToolsGen dotnet tool and then run as dtg,
OR run it directly using dnx (dnx DSLToolsGen)
DSL Tools Generator
A tool that takes an ANTLR grammar of your language and generates a VSCode extension, and C# code for a language server and AST class hierarchy.
Overview
ANTLR grammar .g4
The provided ANTLR grammar can either be a combined grammar
(containing rules for both the lexer and the parser),
or be separated into a lexer grammar and a parser grammar that
references the lexer’s .tokens file via the tokenVocab grammar option.
The lexer rules are converted into a syntax highlighting grammar. The parser rules are used as the basis from which to infer the structure of the generated AST class hierarchy.
Configuration
The configuration file uses JSON (with Comments, sometimes abbreviated as jsonc).
Use dtg init to generate it (along with a dtg.schema.json which allows editors
such as VS Code to offer basic validation, hover, and completion support).
See Configuration docs.
dtg CLI
-
dtg initto create adtg.jsonconfig file -
dtg generateto run the configured generators (once)or
dtg watch, which automatically reruns the configured generators once their inputs change
ANTLR
DTG passes the grammar to ANTLR4 which generates the lexer and parser.
The command to run ANTLR (e.g. java -jar ./antlr/antlr-4.13.1-complete.jar)
is specified in the configuration file (Parser.AntlrCommand).
Lexer, Parser, Visitor, …
#TODO
Generated AST definitions
DTG generates a class hierarchy (in C#) for the abstract syntax tree.
It does this by analyzing the ANTLR grammar and creating a (record) class
for each parser rule. The generated classes are partial so that you can
add custom methods, properties, etc.
DTG also generates an AstBuilder class that constructs an AST from
an ANTLR parse tree.
var antlrParseTree = parser.xyzFile();var ast = new AstBuilder().VisitXyzFile(antlrParseTree);Custom language-specific analysis logic
After getting the AST of the document, you can perform any additional analysis on it, e.g. name binding (resolving references), type checking and other types of semantic validation, or even rendering a visual preview or alternative representation of the document to be displayed in a side view, etc.
The results can either be:
- pushed (“published”) to the client immediately (this is typical for diagnostics), or
- stored in the AST and later retrieved when responding to an LSP request made by the client (pull model)
- since the generated AST classes are
partial, you can add custom properties (and methods) to them
- since the generated AST classes are
Language server
DTG generates a lot of the boilerplate (in C#) needed to implement
an LSP server for the language.
For dealing with the protocol itself, the generated code depends on the
OmniSharp.Extensions.LanguageServer
library.
- live diagnostics (error checking, warnings with automatic code fix suggestions, etc.)
- lexical and syntax errors reported by ANTLR during parsing are automatically published as LSP diagnostics
- publish custom diagnostics (semantic errors, warnings, …) from inside your
AnalyzeDocumentoverride
- code completion
- use the generated
BasicCodeCompletionHandlerbase class
- use the generated
- semantic highlighting
- use the generated
BasicSemanticTokensHandlerbase class
- use the generated
- hover information
- use the generated
BasicHoverHandlerbase class
- use the generated
- document outline
- …
Don’t forget to register custom handlers using .WithHandler<T>() in RunAsync.
Syntax highlighting
DTG analyzes the provided ANTLR4 (lexer) grammar and generates a corresponding
TextMate grammar (.tmLanguage.json) for syntax highlighting in VS Code and other editors.
You can configure how to highlight a specific token,
which TextMate scopes to use, etc.
AST explorer
The generated VSCode extension can also (optionally) include a treeview for inspecting the AST of the currently opened document. Each AST node shows its children as well as other properties.
VS Code extension
DTG can generate the complete source code for a VSCode extension that uses the generated TextMate grammar for syntax highlighting and provides integration with the language server.
This (sub)generator is meant to be run only once, i.e. to quickly get an extension working with the DTG-generated language server code. After that, you can edit any part of the extension, e.g. to handle any non-standard/custom LSP requests and notifications.
Next steps
Read the docs
Learn more in the docs.