Getting the dependencies right

Last modified: 11 February 2024

Motivation

Modules and models are typically interconnected by a network of dependencies of various types. Assuming you have understood the basic principles and categorisations of modules and models, as described at the MPS project structure page, we can now dive deeper as learn all the details.

Getting dependencies right in MPS is a frequent cause of frustration among inexperienced users as well as seasoned veterans. This page aims to solve the problem once and for all. You should be able to find all the relevant information categorised into sections by the various module and dependency types.

All programming languages have a notion of "imports". In Java you get packages and the "import" statement. In Ruby or Python you have "modules" and "require" or "import" statements. In MPS we provide a similar mechanism for packaging code and expressing dependencies in a way that works universally across languages - code is packages into models and these models can express mutual dependencies.

In addtion, since MPS is a multi-language development environment, models can specify the languages (aka syntaxes) enabled in them. This is different from when writing code in Java, Ruby or other languages, where the language to be used is given and fixed by conventions (e.g. the file extension).

Useful keyboard shortcuts

Whenever positioned on a model or a node in the left-hand-side Project Tool Window or when editing in the editor, you can invoke quick actions with the keyboard that will add dependencies or used languages into the current model as well as its containing solution.

- Add a used language
- Add a dependency
- Add a dependency that contains a root concept of a given name
- brings up a generic action-selection dialog, in which you can select the desired action applicable in the current context

Solution

Solutions represent programs written in one or more languages. They typically serve four purposes:

Regular solutions - these solutions hold end user code. The IDE typically generates the code on demand and compiles any generated Java code. The compiled code is not loaded into the IDE, but it can be run on demand in a separate Java process.
Sandbox solutions - these solutions are mostly used by language designers during the process of language design to play with the code in order to interactively evolve their languages. The IDE typically generates the code, but does not attempt to compile the generated code.
Runtime solutions - these solutions contain code that other modules (Solutions, Languages or Generators) depend on. The code can consist of MPS models as well as of Java classes, sources or jar files. The IDE will generate and compile the code and reload the classes whenever they get compiled or changed externally.
Plugin solutions - these solutions extend the IDE functionality in some way. They can contribute new menu entries, add side tool panel windows, define custom preference screens from the Project settings dialog, etc. Again, MPS will generate and compile the code and keep reloading the classes, whenever they change. Additionally, the IDE functionality will be updated accordingly.

We'll start with the properties valid for all solutions and then cover the specifics of sandbox, runtime and plugin solutions.

Common

Properties

Name - name of the solution
File path - path the module file
Generator output path - points to the folder, where generated sources should be placed
Module version - an internal version number of the module, should not be changed directly by a user
Left-side panel - contains model roots, each of which may hold one or more models.
Right-side panel - displays the directory structure under the model root currently selected in the left-side panel. Folders and jar files can be selected and marked/unmarked as being models of the current model root.

Model root types

Solutions contain model roots, which in turn contain models. Each model root typically points to a folder and the contained models lie in one or more sub-folders of that folder. Depending on the type of contained models, the model roots are of different kinds:

default - the standard MPS model root type holding MPS models
java_classes - a set of directories or jar files containing Java class files
javasource_stubs - a set of directories or jar files containing Java sources
tip
When included in the project as models, Java classes in directories or jar files will become first-class citizens of the MPS model pool and will become available for direct references from other models, which import these stub models. A second option to include classes and jars in MPS is to use the Java tab and define them as libraries. In that case the classes will be loaded, but not directly referenceble from MPS code. This is useful for libraries that are needed by the stub models.

Dependencies

The dependencies of a solutions are other solutions and languages, the models of which will be visible from within this solution.

The Export flag then specifies whether the dependency should be transitively added as a dependency to all modules that depend on the current solution. For example, of module A depends on B with export on and C depends on A, then C depends on B.

Used Languages

The languages as well as devkits that the solution's models may use are listed among used languages. Used languages are specified on the model level and the Used Languages tab on modules only shows a collection of used languages of its all models.

Java

This is where the different kinds of Solutions differ mostly.

The Java tab contains several options to customize the level of language features as well as the process of compilation and class-loading. Some of these options are only visible when the area with "more advanced" settings is expanded by clicking on the little arrow icon on the left side:

Not all combinations of these settings make sence so the UI will gray out the visual elements that are irrelevant with respect to the other selected options.

Language level - specifies the java language features that should be supported in the solution.
Compilation - specifies which compiler to use to compile the Java code generated from the module.
- with MPS - indicates, whether the generated artifacts should be compiled with the Java compiler directly in MPS as part of the generation process
- External compiler - use an external compiler. MPS doesn’t take any part in the compilation, it assumes an external compiler will take care of the compilation, but doesn’t mandate any compilation process to be performed. For example, for a java libraries containing compiled classes, there’s no compilation process, and the “external” option here means that MPS expects compilation to happen independently of MPS itself.
- Request external IDEA compilation - (for internal use by MPS developers) use IntelliJ IDEA for compilation. This option is only useful for developing MPS itself and requires an additional plugin to work. The -Dmps.internal=true vm option must be enabled for this option to show up.
- None - No compilation should be done. MPS does not expect this module to contain any classes and so will not include the module content in the classpath.
Compiler output path - specifies the folder in which to store the compiled code.
Class loader - specifies how the compiled classes should be loaded.
- MPS - It tells MPS to manage classpath of a module based on the information available in JavaModuleFacet (namely, the output classes location and the java libraries + the dependencies on other MPS modules). Modules coming from various contributors have the classpath of the contributor as a parent classloader and can access e.g. plugin libraries (in case of a plugin contributor). The difference from the “Module contributor” option below is that MPS creates a module classloader (that respects classes_gen + module dependencies + parent classpath) when classloading by MPS, while relies solely on contributor’s classpath in case of “Module Contributor” (i.e. no MPS dependencies or classes_gen).
- Module contributor - indicates that the code that provided the module is also responsible for providing a classloader for the module classes. Most common contributors are app/project libraries and plugins. E.g. a plugin contributor provides a classloader that adds the plugin classes (those in plugin-home/lib/*) in addition to the actual MPS module classes.
- N/A - do not load the classes.
Contributes extensions to MPS - indicates whether the compiled classes should be loaded into MPS as its extensions (e.g. actions, menus, etc.). It is mostly used for scenarios when the code needs to integrate with IDEA or another plugin, i.e. when some classes have to be visible to both IDEA and MPS as coming from the same classpath.
Source Paths - Java sources that should be made available to other Java code in the project, i.e. to get included into module compilation along with generated sources.
Libraries - Java classes and jars that are required to be included on classpath at compile-time as well as at run-time.

Facets

Custom Generation - allows attaching a generator plan to the module. More on explicit generator plans is to be found at the Generator plans documentation.
Java - checked, if the solution relies on Java on some way. Keep this checked in most cases.
Tests - checked, if the solution contains test models

Solution models

Solutions contain one or more models. Models can be mutually nested and form hierarchies, just like, for example, Java packages can. The properties dialog hides a few configuration options that can be tweaked:

Dependencies

Models from the current or imported modules can be listed here, so that their elements become accessible in code of this model.

Used languages

The languages used by this model must be listed here.

Advanced

A few extra options are listed on the Advanced tab:

Do not generate - exclude this model from code generation, perhaps because it cannot be meaningfully generated
File path - location of the model file
Languages engaged on generation - lists languages needed for proper generation of the model, if the languages are not directly or indirectly associated with any of the used languages and thus the generator fails finding these languages automatically

Virtual packages

Nodes in models can be logically organised into hierarchies of virtual packages. Use the Set Virtual Package option from the node's context pop-up menu and specify a name, possibly separating nested virtual folder names with the dot symbol.

Adding external Java classes and jars to a project - runtime solutions

Runtime solutions represent libraries of reusable code in MPS. They may contain models holding MPS code as well as stub models that refer to external Java sources, classes or jar files. To properly include external Java code in a project, you need to follow a few steps:

Create a new Solution in your project.
Copy the desired jar files, classes or Java sources into a directory under the solution's root folder, such as "/libs".
In the Solution's properties dialog () attach the Java code. This requires a few steps:
1. First, in the left-hand side panel of the Common tab, which is used to manage model roots (i.e. locations of models that belong to the module), delete the existing model root unless you want to preserve or create code that you write in MPS as part of your runtime library.
2. Second, also in the Common tab, click on Add Model Root and select the desired type of files:
  - Java Classes - for classes or jars
  - Java sources - for Java sources
  - Kotlin Common - for Kotlin/Common libraries
  - Kotlin JVM - for compiled Kotlin/JVM libraries
  Then navigate MPS to your lib folder (the folder holding the jar files or the folder holding the root of the package hierarchy of your Java classes or sources). This will add a new entry into the list of model roots. This entry will represent the "libs" folder.
3. Third, when the new model root is selected in the left-hand side panel, select the folder(s) or jar(s) listed in the right-side panel of the properties dialog and click the blue "Sources" button at the top of the right-hand side panel. The selected folders and jars will be shown with a blue icon next to them in the right-hand side panel.
4. Switch to the Dependencies tab and add the JDK solution as a dependency. Do not forget to turn on the Export flag. This will allow your jar files to refer to the JDK classes.
5. Then head to the Java tab to add all the jars or the classes root folders to the Libraries part of the window, otherwise the solution using the library classes would not be able to compile. When using java_sourcestubs, add the sources into the Source paths part of the Java tab window, instead. Also add any dependencies of the libraries that you have added to the module.
  tip
  To speed things up, you can actually reverse the order and first add the files on the Java tab, which will automatically (once you press the "Apply" button and possibly also close and reopen the dialog) create a corresponding model root entry on the Common tab.
  tip
  Although the paths shown in the dialog appear to be absolute, MPS stores paths relative to the module (provided they refer to locations under the project root directory) using the "${module}" macro or using a user-defined path variable, if available, so the paths adjust automatically to the MPS environment that opens your project.
6. Additionally, still on the Java> tab, expand the panel hiding behind the Regular MPS module without contributions of MPS extensions label and set Contributes extensions to MPS option to Yes to have MPS reload the classes properly.
A new folder named stubs should appear in your solution
.
Now after you import the solution into another module (solution, language, generator) the classes will become available in that module's models.
The languages that want to use the runtime solution as a runtime dependency of the code that gets generated by the language's generator will need to refer to it in the Runtime Solutions section of the Runtime tab of their module properties.
Include the runtime jars in the build models - If you use the MPS build scripts to build a plugin with your languages you should also package the runtime libraries in the plugin by copying them into the desired location within the target layout definition.
If you forget to do so, you'll be getting errors like "No such path in local layout" when rebuilding the build script's model.
If your runtime solution uses libraries that are located outside of the project folder, your build script should specify the path to the libraries with a folder macro.

Language

Languages represent a language definition and consist of several models, each of which represent a distinct aspect of the language. Languages may contain one or more Generator modules. The properties dialog for languages is in many ways similar to the one of Solutions. Below we will only mention the differences: