HowTo -- Integrating into the MPS Make Framework
Build Facets
Overview
Like basically any build or make system, the MPS make executes a sequence of steps, or targets, to build an artifact. A global ordering of the necessary make steps is derived from relative priorities specified for each build targets (target A has to run before B, and B has to run before C, so the global order is A, B, C).
A complete build process may address several concerns, for example generating models into text, compiling these models, deploying them to the server, and/or generating .png files from graphviz source files. In MPS, such different build aspects are implemented with build facets. A facet is a collection of targets that address a common concern.
The targets within a facet can exchange configuration parameters. For example, a target that is declared to run early in the overall make process may collect configuration parameters and pass them to the second facet, which then uses the parameters. The mechanism to achieve this intra-facet parameter exchange is called properties. In addition, targets can use queries to obtain information from the user during the make process.
The overall make process is organized along the pipes and filters pattern. The targets act as filters, working on a stream of data being delivered to them. The data flowing among targets is called resources. There are different kinds of resources, all represented as different Java interfaces and tuples:
MResource contains MPS models created by users, those that are contained in the project's solutions and languages
GResource represents the results of the generation process, which includes the output models, that is the final state of the models after generation has completed. These are transient models, which may be inspected by using the Save Transient Models build option
TResource represents the result of text-gen
CResource represents a collection of Java classes
DResource represents a collection of delta changes to models (IDelta)
TextGenOutcomeResource represents the text files generated by textgen
Build targets specify an interface. According to the pipes and filters pattern, the interface describes the kind of data that flows into and out of a make
target. It is specified in terms of the resouce types mentioned above, as well as in terms of the kind of processing the target applies to these resources. The following four processing policies are defined:
transform is the default. This policy consumes instances of the input resource type and produces instances of the output resource type (e.g. it may
consumeMResources
and produceTResources
.)consume consumes the declared input, but produces no output. * produce consumes nothing, but produces output
pass through does not access any resources, neither produce nor consume.
Note that the make process is more coarse grained than model generation. In other words, there is one facet that runs all the model generators. If one needs
to "interject" additional targets into the MPS generation process (as opposed to doing something before or after model generation), this requires refactoring
the generate
facets. This is beyond the scope of this discussion.
Building an Example Facet
As part of the mbeddr.com project to build a C base language for MPS, the actual C compiler has to be integrated into the MPS build process. More
specifically, programs written in the C base language contain a way to generate a Makefile
. This Makefile
has to be executed once it and all the
corresponding .c and .h files have been generated, i.e. at the very end of the MPS make process.
To do this, we built a make facet with two targets. The first one inspects input models and collects the absolute paths of the directories that may contain a
Makefile
after textgen. The second target then checks if there is actually a file called Makefile
in this directory and then runs make there. The two
targets exchange the directories via properties, as discussed in the overview above.
The first target: Collecting Directories
Facets live in the plugins
aspect of a language definition. Make sure you include the {{jetbrains.mps.make.facets} language into the plugins model,
so you can create instances of FacetDeclaration
. A facet is executed as part of the make process of a model if that model uses
the language that
declares the facet.
The facet is called runMake
. It depends on TextGen
and Generate
. The dependencies to those two facets has to be specified so we can then declare our targets' priorities relative to targets in those facets.
The first target is called collectPaths
. It is specified as {{transform IMResource -> IMResource} in order to get in touch with the input models. The
facet specifies, as priorities, after configure
and before generate
. The latter is obvious, since we want to get at the models before they are
generated into text. The former priority essentially says that we want this target to run after the make process has been initialized (in other words: if
you want to do something "at the beginning", use these two priorities.)
We then declare a property pathes
which we use to store information about the modules that contain make files, and the paths to the directories in which
the generated code will reside.
Let's now look at the implementation code of the target. Here is the basic structure. We first initialize the pathes
list. We then iterate of the
input (which is a collection of resources) and do something with each input (explained below). We then use the output
statement to output the input
data, i.e. we just pass through whatever came into out target. We use the success
statement to finish this target successfully (using success
at the end is optional, since this is the default). If something goes wrong, the failure
statement can be used to terminate the target unsuccessfully.
The actual processing is straight forward Java programming against MPS data structures:
We use the getGeneratorOutputPath
method to get the path to which the particular module generates its code (this can be configured by the user in the
model properties). We then get the model's dotted name and replace the dots to slashes, since this is where the generated files of a model in that module will
end up (inspect any example MPS project to see this). We then store the module's name and the model's name, separated by a slash, as a way of improving the
logging messages in our second target (via the variable locationInfo}). We add the two strings to the {{pathes
collection. This pathes
property
is queried by the second target in the facet.
The second Target: Running Make
This one uses the pass through
policy since it does not have to deal with resources. All the input it needs it can get from the properties of the collectPaths
target discussed above. This second target runs after collectPaths}, {{after textGen
and before reconcile
. It is obvious that is has to run after collectPaths}, since it uses the property data populated by it. It has to run after {{textGen}, otherwise the make files aren't there yet. And it has to run before {{reconcile}, because basically everything has to run before {{reconcile
Let us now look at the implementation code. We start by grabbing all those entries from the collectPathes.pathes
property that actually contain a
Makefile
. If none is found, we return with success
.
We then use the progress indicator language to set up the progress bar with as many work units as we have directories with make files in them.
We then iterate over all the entries in the {{modelDirectoriesWithMakefile} collection. In the loop we advance the progress indicator and then use Java
standard APIs to run the make file.
To wrap up the target, we use the finish
statement to clean up the progress bar.