HowTo -- Integrating into the MPS Make Framework
Like basically any build or make system, the MPS make executes a sequence of steps, or targets, to build an artifact. A global ordering of the necessary make steps is derived from relative priorities specified for each build targets (target A has to run before B, and B has to run before C, so the global order is A, B, C).
A complete build process may address several concerns, for example generating models into text, compiling these models, deploying them to the server, and generating .png files from graphviz source files. In MPS, such different build aspects are implemented with build facets. A facet is a collection of targets that address a common concern.

tip
Avoiding unnecessary file overwrites
The make process does not overwrite generated files that hold identical content to the one just generated. You can rely on the fact that only the modified files get updated on disk.
The targets within a facet can exchange configuration parameters. For example, a target that is declared to run early in the overall make process may collect configuration parameters and pass them to the second facet, which then uses the parameters. The mechanism to achieve this intra-facet parameter exchange is called properties. In addition, targets can use queries to obtain information from the user during the make process.

The overall make process is organized along the pipes and filters pattern. The targets act as filters, working on a stream of data being delivered to them. The data flowing among targets is called resources. There are different kinds of resources, all represented as different Java interfaces and tuples:

MResource contains MPS models created by users, those that are contained in the project's solutions and languages
GResource represents the results of the generation process, which includes the output models, that is the final state of the models after generation has completed. These are transient models, which may be inspected by using the Save Transient Models build option
TResource represents the result of text-gen
CResource represents a collection of Java classes
DResource represents a collection of delta changes to models (IDelta)
TextGenOutcomeResource represents the text files generated by textgen
warning
These resources interfaces have been deprecated:
IMResource contains MPS models created by users, those that are contained in the project's solutions and languages
IGResource represents the results of the generation process, which includes the output models, that is the final state of the models after generation has completed. These are transient models, which may be inspected by using the Save Transient Models build option
ITResource represents the text files generated by textgen towards the end of the make process
FResource
Build targets specify an interface. According to the pipes and filters pattern, the interface describes the kind of data that flows into and out of a make target. It is specified in terms of the resouce types mentioned above, as well as in terms of the kind of processing the target applies to these resources. The following four processing policies are defined:
transform is the default. This policy consumes instances of the input resource type and produces instances of the output resource type (e.g. it may consume
MResources
and produceTResources
.)consume consumes the declared input, but produces no output. * produce consumes nothing, but produces output
pass through does not access any resources, neither produce nor consume.
Note that the make process is more coarse grained than model generation. In other words, there is one facet that runs all the model generators. If one needs to "interject" additional targets into the MPS generation process (as opposed to doing something before or after model generation), this requires refactoring the generate
facets. This is beyond the scope of this discussion.
As part of the mbeddr.com project to build a C base language for MPS, the actual C compiler has to be integrated into the MPS build process. More specifically, programs written in the C base language contain a way to generate a Makefile
. This Makefile
has to be executed once it and all the corresponding .c and .h files have been generated, i.e. at the very end of the MPS make process.
To do this, we built a make facet with two targets. The first one inspects input models and collects the absolute paths of the directories that may contain a Makefile
after textgen. The second target then checks if there is actually a file called Makefile
in this directory and then runs make there. The two targets exchange the directories via properties, as discussed in the overview above.
tip
The sampleFacet sample project that comes bundled with MPS distributions provides a simple facet definition that you can take as a starting point for your adventure with make facets.
Facets live in the plugins
aspect of a language definition. Make sure you include the {{jetbrains.mps.make.facets} language into the plugins model, so you can create instances of FacetDeclaration
. A facet is executed as part of the make process of a model if that model uses
the language that declares the facet.
The facet is called runMake
. It depends on TextGen
and Generate
. The dependencies to those two facets has to be specified so we can then declare our targets' priorities relative to targets in those facets.
facet runMake extends <none>
Required: TextGen, Generate
The first target is called collectPaths
. It is specified as {{transform IMResource -> IMResource} in order to get in touch with the input models. The facet specifies, as priorities, after configure
and before generate
. The latter is obvious, since we want to get at the models before they are generated into text. The former priority essentially says that we want this target to run after the make process has been initialized (in other words: if you want to do something "at the beginning", use these two priorities.)
target collectPathes overrides <none> {
resources policy: transform IMResource -> IMResource
Dependencies:
after configure
before generate
We then declare a property pathes
which we use to store information about the modules that contain make files, and the paths to the directories in which the generated code will reside.
Properies:
list<[string, string]> pathes;
Let's now look at the implementation code of the target. Here is the basic structure. We first initialize the pathes
list. We then iterate of the input (which is a collection of resources) and do something with each input (explained below). We then use the output
statement to output the input data, in other words, we just pass through whatever came into out target. We use the success
statement to finish this target successfully (using success
at the end is optional, since this is the default). If something goes wrong, the failure
statement can be used to terminate the target unsuccessfully.
(input)->void {
pathes = new linkedlist<[string, string]>;
input.forEach({~inpt =>
(module, models) res = ((module, models)) inpt;
// do stuff. See below.
});
output input;
success;
}
The actual processing is straight forward Java programming against MPS data structures:
res.models.forEach({~model =>
string path = res.module.getFacet(GenerationTargetFacet.class).getOutputLocation() + "/" +
model.getLongName().replaceAll("\\.", "/");
string locationInfo = res.module.getModuleFqName() + "/" +
model.getLongName();
pathes.add([path, locationInfo]);
});
We use the getOutputLocation
method of the GenerationTargetFacet
facet of the module to get the path to which the particular module generates its code (this can be configured by the user in the model properties). We then get the model's dotted name and replace the dots to slashes, since this is where the generated files of a model in that module will end up (inspect any example MPS project to see this). We then store the module's name and the model's name, separated by a slash, as a way of improving the logging messages in our second target (via the variable locationInfo}). We add the two strings to the {{pathes
collection. This pathes
property is queried by the second target in the facet.
This one uses the pass through
policy since it does not have to deal with resources. All the input it needs it can get from the properties of the collectPaths
target discussed above. This second target runs after collectPaths}, {{after textGen
and before reconcile
. It is obvious that is has to run after collectPaths}, since it uses the property data populated by it. It has to run after {{textGen}, otherwise the make files aren't there yet. And it has to run before {{reconcile}, because basically everything has to run before {{reconcile
target callMake overrides <none>
resources policy: pass through
Dependencies:
after collectPathes
after textGen
before reconcile
let's now look at the implementation code. We start by grabbing all those entries from the collectPathes.pathes
property that actually contain a Makefile
. If none is found, we return with success
.
sequence<[string, string]> modelDirectoriesWithMakefile =
collectPathes.pathes.
where({~it => new File(it[0] + "/Makefile").exists();
});
if (modelDirectoriesWithMakefile.isEmpty) { success; }
We then use the progress indicator language to set up the progress bar with as many work units as we have directories with make files in them.
begin work "run make" covering ALL units of total work left,
expecting modelDirectoriesWithMakefile.size units;
We then iterate over all the entries in the {{modelDirectoriesWithMakefile} collection. In the loop we advance the progress indicator and then use Java standard APIs to run the make file.
foreach dirInfoTuple in modelDirectoriesWithMakefile {
try {
advance 1 units of "run make"
with comment "running make for " + dirInfoTuple[1];
Process process = Runtime.getRuntime().
exec("make", new string[0], new File(dirInfoTuple[0]));
if (process.waitFor() > 0) {
error "make failed with exit code " + process.exitValue() + " for " +
dirInfoTuple[1];
} else {
info "make finished successfully for " + dirInfoTuple[1];
}
} catch (Exception ex) {
error ex.getMessage(), ex;
}
}
To wrap up the target, we use the finish
statement to clean up the progress bar.
finish "run make";
Thanks for your feedback!