Tips for improving performance of your generator

Measure 1: Develop a clever runtime and generate code that leverages that runtime

Now, in retrospect, we followed an obvious pattern when initially designing the code generators. Basically, all the code was fed directly into the Generator Template, whether this was necessary or not. The following simple example can easily illustrate our former approach.

The Java class in the generator template of Figure 2 implements the IGenSelControlled interface directly. Boilerplate methods like saveAndValidate() or collectDelegateChanges() are directly in the template, although there is no variation in the behavior of the methods depending on the input model. The implementation of the methods does not vary in any way at all. In fact, the implementation is always the same. As one can also see in the same Figure 2, there are multiple methods of the same characteristic, not just one.

Figure 3 shows a refactored version of that generator template. The Java class has now extended another Table class. All that static code mentioned above was implemented in the extended class. Furthermore, behavior that is affected by the input model was implemented in the form of Boolean properties. This can be seen in the constructor declaration in Figure 3. Three different Boolean switches with property macros applied are available to configure the behavior of the Table class. Additionally, the former method selectionChanged() illustrates how a more complex generator fragment was refactored. While the static parts of the code moved to the runtime Table class, the dynamic parts moved into calcSelectionSummeryLineText(). The method comes with a parameter and a return value. The method itself is very compact. An expression calculating some domain logic is passed forward from the input model into the generated class. Now, that method is called by the Table class, resembling the same behavior from Figure 2.

Overall, similar refactorings in our own code generators reduced the number of lines generated from over one million down to 484,000 lines. By moving static code from the Generator Templates into a runtime, the amount of code to be generated and compiled was halved. This led to a generation and compile time improvement of approximately two minutes on our developer machines. A very significant improvement. Therefore, we strongly recommend building a powerful runtime environment in which to generate code.

Measure 2: Do not reference concepts by name

MPS has supported model generation performance reports for some time now. This report type gives generator designers performance insights on the various generation steps employed per model. The report is sorted in such a way that it lists the generator steps consuming the most time first. One can easily identify time spent on generator steps and the activities within a single step. It also shows the time spent on weaving rules, reduction rules, and delta changes. Of particular value to us was the time spent restoring references.

Restoring references is the process of looking up the actual target concept when working only with target names in reference macros. MPS reference macros are capable of returning target concepts (node<> or node-ptr<>) or simply names (strings). When working with plain names, MPS will take over the work of locating the referenced targets in the output model. However, this additional work might take some time. This is the same period of time that is reported as “restoring references”.

In the context of our MMS project, the time spent on restoring references was not significant at first glance. However, one or two seconds restoring references accumulates when you’re working with 140 models and multiple generation steps. Therefore, we tried to address referencing target concepts generally in another refactoring of our generators. There are multiple possible methods of doing this.

The primary measure around referencing by name is actively using generator mapping labels. As shown in Figure 4 below, the mapping label “DelegateFD” is attached to the loop macro, labeling each generated class field in the class. That labeling process creates a relationship between an input concept instance and a generated concept instance. Later, the generator designer can look up generated concepts by input concept via those labels. This is typically what reference macros are for. Based on a specific input concept instance, we want to reference a related generated target in the output model. In the example illustrated in Figure 4, the generator designer might look up one of those generated class fields.

Trying out mapping labels is a straightforward process. All you have to do is label some generated code and use those labels in reference macros – it’s as simple as that. As long as labeled generated code resides in the same output model as the code with reference macros applied, mapping labels do not add any additional burden. Things get a bit more complicated when referencing instances across models. By default, labels are not accessible across different input and output models. In the case of our generators, for example, such a situation quickly arises. In model A, a “service component” S is declared, whereby that service component S is also used and accessed from model B. The generated code in model B has to reference the generated code in model A. There are two solutions to this scenario.

Restoring references becomes obsolete when directly generating text instead of doing model-to-model transformations. This is what we have resorted to when referencing across models in our generators. In the superclass constructor call in Figure 4, a concept from the JetBrains BaseLanguageInternal language is used to determine the class instance of a Java class at runtime. The concept is different from the standard Java .class expression. It accepts a fully qualified name as text and a classifier to determine the preliminary type of the concept in the output model. The corresponding generator of that concept – a JetBrains TextGen – translates straight to Java text by appending a .class to the given name. As can be seen in Figure 4, a property macro is used instead of a reference macro. Thus, restoring references becomes obsolete when working with this BaseLanguageInternal concept.

There are more similar baseLanguageInternal concepts that translate directly to text with TextGen. We relied on some of those in our generators when cross-model references are needed. This saved us approximately another 45 seconds in the generation process for the entire MMS. While the BaseLanguageInternal stuff seems to be somewhat “internal”, we did not experience any significant changes to those concepts that would demand generator refactorings. This approach of generating text directly is a viable solution to the cross-model reference scenario in our situation. Be aware that no other transformations can be applied after text has been generated.

There is also a newer recommended approach to handle model cross-referencing situations: a generation plan in combination with checkpoints. Generation plans allow developers to specify the desired order of generation for their models explicitly and thus gain better control over the generation process. Generation plans list all the languages that should be included in the generation process, order them appropriately, and optionally specify checkpoints at which the generator shall preserve the current transient models. These models can then be used for automatic cross-model reference resolution further down the generation process. Although we specified “generators’ priorities” for our generators in the traditional approach, we additionally gave the generator plan a try. Setting up a generation plan is indeed quite easy. One can specify all the priorities by listing the generators from the top down. The complete plan for our language stack and its generators is shown in Figure 5.

As can also be seen, we did not specify any checkpoints in the plan since we rely on the BaseLanguageInternal approach when it comes to model cross-referencing. However, we created a simple language for demonstration purposes to resemble the situation where a “service component” S is declared in model A, whereby that service component S is also used and accessed from model B. Quite simplistic, the “service component” was transformed into a Java class with the generator. Thus, the generated code from model B has to reference that very class in the generated code of model A. With a checkpoint added in the generation plan (i.e., simply adding the “checkpoint” keyword after our demo language’s “apply generator” command, everything is already done. MPS automatically takes the checkpoint models into account when looking up generated concepts by label and input. The mechanism is exactly the same as with the intra-model reference resolution. As a generator designer, there are no additional things to consider. In fact, setting up a generation plan with checkpoints in our demo language turned out to be no hassle at all. Furthermore, cross-model referencing with labels worked out of the box.

Measure 3: Be careful with '.type' calculations in the generator

The jetbrains.mps.lang.typesystem language provides a .type operation to determine the type of an arbitrary node in the generator. This is extremely useful when handling expressions since various decisions are typically made on the basis of the expression type. For example, methods applicable to an expression depend on the type of that expression and not on whether the outermost concept of that expression is a variable reference or another method call.

It is not untypical to perform different transformations in a generator based on the type of a given node. However, type calculations with .type are rather CPU-intensive operations. We have taken some measurements on our developer machines. As a rough indicator, single .type calculations have taken 2–30 milliseconds. This does not appear to be particularly concerning at first, but when we consider that the whole MMS consists of 400,000 nodes, times for .type calculations in a generator might add up when not applied carefully.

The time spent on .type calculations in a generator is also reported in the aforementioned “model generation performance report” when we select “time spent on type calculations” in the settings. Unfortunately, in this case, the generator designer will be overwhelmed with performance information. As you can imagine, it’s quite a cumbersome process to check through all those performance hints.

One of our refactorings aimed specifically at minimizing the generation time by preventing type calculations for the very same expression multiple times. The case is as follows: One of our languages introduced arithmetic operations for non-primitive values (e.g., the Java BigDecimal class). The user of the language can simply use “+” instead of a method call to add(). The code generator will reformat the code slightly by inspecting the type of expressions in the code. If the type is of BigDecimal and “+” is applied, the expression has to be transformed.

Since we also allowed for comparison operations (e.g., == / < / <=, etc.) to be part of our language, they had to be translated to the calls to the compareTo() method. The same reformat logic had to be applied to all comparisons and all arithmetic operations in the language. In a naive approach, we created one different MPS generator reduction rule for each different method call to translate to (see Figure 6). Since each such reduction rule contained a condition with .type calculations, we ended up with multiple calculations on the same parts of expressions from the input model.

We introduced a step-by-step process with refactoring to minimize those calculations. When some form of operation with a left and a right part is applied to BigDecimal, we save the type calculated in a transient object in the generation context and reuse that information. That is to say, only one reduction rule with “inheritors” enabled for the general BinaryOperation is used to check the type for the left expression in its condition. If that type is of BigDecimal or Int, the type is stored, and the right expression is evaluated and stored. That information is passed forward to an MPS template switch, which transforms the operation accordingly. The template switch does not contain any .type calculations, nor do we have multiple calculations on the same expressions any longer.

.type calculations in generators are CPU-intensive operations, which can slow down generation significantly. In our case, we only experienced a small performance increase with this refactoring. However, the most significant change to our generators was that it minimized the total amount of code generated. Naturally, this, in turn, significantly reduced the number of inspected expressions during generation, which helped to alleviate the problem of repeated type calculations in the generation process.