Generator
tip
For a quick how-to document on the MPS generator please check out the Generator Cookbook.
Generator is a part of language specification that defines the denotational semantics for the concepts in the language.
tip
Skip ahead to find out the details quickly:
MPS follows the model-to-model transformation approach. The MPS generator specifies translation of constructions encoded in the input language into constructions encoded in the output language. The process of model-to-model transformation may involve many intermediate models and ultimately results in sn output model, in which all constructions are in a language whose semantics are already defined elsewhere.
For instance, most concepts in baseLanguage (classes, methods etc) are "machine understandable", therefore baseLanguage is often used as the output language.
The target assets are created by applying model-to-text transformation, which must be supported by the output language. The language aspect to define model-to-text transformation is called TextGen and is available as a separate tab in concept's editor. MPS provides destructive update of generated assets only.
For instance, baseLanguage's TextGen aspect generates *.java files at the following location:
< generator output path >\< model name >\< ClassName >.java
where:
Generator output path - is specified in the module, which owns the input model.
Model name - is a path segment created by replacing '.' with the file separator in the input model's name.
Unlike any other language aspect, the generator aspect is not a single model. Generator specification can comprise many generator models as well as utility models. A Generator Model contains templates, mapping configurations and other constructions of the generator language.
A Generator Model is distinguished from a regular model by the model stereotype - 'generator' (shown after the model name as < name >@ generator).
The screenshot below shows the generator module of the smodel language as an example.
tip
You can research the smodel (and any other) language generator by yourself:
download MPS (here);
create new project (can be empty project);
use the Go To -> Go to Language command in the main menu to navigate to the smodel language (its full name is jetbrains.mps.lang.smodel)
A new generator is created by using the New -> Generator command in the language's popup menu.
It is possible to create more than one generator for a language. When a language has only a single generator, using that language implies using its one and only generator to generate code for the model. However, when using languages with multiple generators an extra step is needed by the user to select the generator to trigger. Currently this is done through generator plans.
When creating a new generator module, MPS will also create the generator model 'main@generator' containing an empty mapping configuration node.
As a module, generator can depend on other modules, have used languages and used devkits (refer to Module meta-information).
The generator properties dialog also has two additional properties:
depends on generators - specifies the dependencies on other generators; this allows the dependent generator to make references to templates defined in another generator;
mapping constraints - priority relationships between mapping rules can be specified. If such a relationship involves other generator rules, then declaring a dependency on that generator is also required.
MPS generator engine (or the Generator language runtime) uses mixed compilation/interpretation mode for transformation execution.
Templates are interpreted and filled at runtime, but all functions in rules, macros, and scripts must be pre-compiled.
To avoid any confusion, always follow this rule: after any changes made to the generator model, the model must be re-generated (Shift+F9). Even better is to use Ctrl+F9, which will re-generate all modified models in the generator module.
The transformation is described by means of templates. Templates are written using the output language and so can be edited with the same cell editor that would normally be used to write 'regular code' in that language. Therefore, without any additional effort the 'template editor' has the same level of tooling support right away - syntax/error highlighting, auto-completion, etc. The templates are then parametrized by referencing into the input model.
The applicability of individual templates is defined by #Generator Rules, which are grouped into #Mapping Configurations.
A Mapping Configuration is a minimal unit, which can form a single generation step. It contains #Generator Rules, defines mapping labels and may include pre- and post-processing scripts.
Applicability of each transformation is defined by generator rules.
There are six types of generator rules:
conditional root rule
root mapping rule
weaving rule
reduction rule
pattern rule
abandon root rule
drop attribute rule (new in 3.3)
Each generator rule consists of a premise and a consequence (except for the abandon root ruleand drop attribute rule, with predefined consequence that cannot be specified by the user).
All rules except for the conditional root rule contain a reference to the concept of the input node (or just input concept) in its premises. All rule premises also contain an optional condition function.
Rule consequence commonly contains a reference to an external template (that is a template declared as a root node in the same or different model) or so-called inline template (conditional root rule and root mapping rule can only have reference to an external template). There are also several other versions of consequences.
The following screenshot shows the contents of a generator model and a mapping configuration example.
The code in templates can be parameterized through macros. The generator language defines three kinds of macros:
property macro - computes a property value;
reference macro - computes the target (node) of a reference;
node macro - is used to control template filling at generation time. There are several versions of node macro - LOOP-macro is an example.
Macros implement a special kind of so-called annotation concept and can wrap property, reference or node cells (depending on the kind of macro) in the template code.
Code wrapping (that is the creation of a new macro) is done by pressing Ctrl+Shift+M or by applying the 'Create macro' intention.
The following screenshot shows an example of a property macro.
Macro functions and other parameterization options are edited in the inspector view. Property macro, for instance, requires specifying the value function, which will provide the value of the property at generation time. In the example above, output class node will get the same name that the input node has.
The node parameter in all functions of the generator language always represents the context node to which the transformation is currently being applied (the input node).
Some macros (such as LOOP and SWITCH-macro) can replace the input node with a new one, so that subsequent template code (that is code that is wrapped by those macros) will be applied to the new input node.
External templates are created as a root node in the generator model.
There are two kinds of external templates in MPS.
One of them is root template. Any root node created in generator model is treated as a root template unless this node is a part of the generator language (that is mapping configuration is not a root template). Root template is created as a normal root node (via Create Root Node menu in the model's popup).
The following screenshot shows an example of a root template.
This root template will transform input node (a Document) into a class (baseLanguage). The root template header is added automatically upon creation, but the concept of the input node is specified by the user.
It is a good practice to specify the input concept, because this allows MPS to perform static type checking in the code of the macro function.
A Root template (reference) can be used as a consequence in conditional root rules and root mapping rules. ( When used in a conditional root rule, the input node is not available).
The second kind of template is defined in the generator language and its concept name is 'TemplateDeclaration'. It is created via the 'template declaration' action in the Create Root Node menu.
The following screenshot shows an example of template declaration.
The actual template code is 'wrapped' in a template fragment. Any code outside template fragment is not used in transformation and serves as a context (for example you can have a Java class, but export only one of its method as a template).
Template declaration can have parameters, declared in the header. Parameters are accessible through the #generation context.
Template declaration is used in consequence of weaving, reduction and pattern rules. It is also used as an included template in INCLUDE-macro (only for templates without parameters) or as a callee in CALL-macro.
A template switch is used when two or more alternative transformations are possible in a certain place in template code. In that case, the template code that allows alternatives is wrapped in a SWITCH-macro, which has reference to a Template Switch. Template Switch is created as a root node in the generator model via the Create Root Node menu (this command can be seen in the 'menu' screenshot above).
The following screenshot shows an example of a template switch.
The Ant MPS generator task generate exposes properties configurable from the build script (parallel, threads, in-place, warnings). The build language uses the Ant Generate task under the hood to transform models during the build process. This task now exposes parameters familiar from the Generator settings page:
strict generation mode
parallel generation with configurable number of threads
option to enable in-place transformation
option to control generation warnings/errors.
These options are also exposed at build language with the BuildMps_GeneratorOptions concept, so that build scripts have more control over the process.
In addition to the MPS-own generate task and the regular Ant's javac task, MPS offers a new mps.make task that corresponds to the Make process known from the MPS IDE. It is responsible for complete transition of a model down to compiled code. The task combines both code generation and compilation. This saves time as MPS needs compiled classes anyway for its module classloading purposes as well as making both IDE and non-IDE command-line builds similar and more reliable.
If you're feeling like it's time for more practical experience, check out the generator demos.
The demos contain examples of usage of all the concepts discussed above.
Thanks for your feedback!