Custom Persistence Cookbook

Stub models

These are read-only models that typically represent library code. Suppose, for example, that we want to leverage MPS to describe one of the existing languages. BaseLanguage, for instance, is a good description of Java. In such a case, the libraries written for that original language should be accessible from within MPS. It is the stubs aspect of a language that helps to create stub models for such libraries. Stub models are then used as targets for references that need to reference the code outside of MPS.

Persistence for your language in your own custom format

Sometimes the default MPS persistence format for models is too heavyweight for your needs or you may want to use the persisted models in some other tools external to MPS, and so you want to customize the persistence format. Another useful example: if all you need from MPS is to merely edit files of your own DSL using the MPS editor, it is useful to store the model in text format so that it could be edited in any text editor.

Generic persistence for any MPS models in your own custom format

When you want to implement a generic persistence that is in some respect superior to the three formats provided by MPS out-of-the-box.

Each of these cases requires slightly different handling.

SModel and EditableSModel (org.jetbrains.mps.openapi.model)

Interfaces that can represent a model or an editable model inside MPS. Can be loaded from persistence storage. Editable models can be saved to a persistent storage.

Noteworthy implementations:

• SModelBase - a default SModel implementation

• EditableSModelBase - a default EditableSModel implementation

• RegularModelDescriptor - should be extended from for stub models

• EditableModelDescriptor - should be extended from for custom persistence

SModelData (jetbrains.mps.extapi.model)

An interface that represents the model contents in memory. It holds and can manipulate its roots and nodes.

Noteworthy implementations:

• jetbrains.mps.smodel.SModel - a default all purpose implementation

• jetbrains.mps.smodel.DefaultSModel - an implementation that allows read optimization through the use of “headers”.

DataSource (org.jetbrains.mps.openapi.persistence)

Represents a location of the persisted data that represent a model. Files, internet content or database content are examples of possible data sources. FileDataSource and StreamDataSource are the most frequently utilized implementations in custom persistence.

DataSourceType (org.jetbrains.mps.openapi.persistence.datasource)

Uniquely identifies a Data Source kind. FileExtensionDataSourceType implementation class is used to represent data sources attached to files.

ModelRoot (org.jetbrains.mps.openapi.persistence)

Represents a logically connected group of models that come from a related physical origin, such as a file or a directory. It manages these models - creates, loads and saves them.

Noteworthy implementations:

• ModelRootBase

• FileBasedModelRoot

Memento

A generic abstraction of a hierarchical configuration information storage that preserves the information between sessions. Mementos, in essence, resemble hierarchically organized hashmaps.

SourceRootKind

Model roots can be of several kinds - typically sources, tests and excluded. This interface is used to communicate the desired/supported kinds of roots. The SourceTootKinds enumeration contains some predefined kinds.

ModelFactory (org.jetbrains.mps.openapi.persistence)

Represents a data source loading/saving/upgrading strategy. Its load(), create() and save() methods work with instances of the SModel and the DataSource interfaces.

ModelRootFactory

A factory for model roots. It must be registered in the plugin.xml file in order to take effect.

ModelRootEntry

A UI element that interacts with the user when a new model root is being specified.

ModelRootEntryFactory

A factory for ModelRootEntries. It must be registered in the plugin.xml file in order to take effect.

ModelLoadResult (jetbrains.mps.smodel)

Represents the result of model loading. It holds the loaded ModelData and a state of the loaded model data (jetbrains.mps.smodel.loading.ModelLoadingState)

Model id

Uniquely identifies a model. Other models use the id to represent the referenced model.

There are two options:

• A model can derive its id from some reproducible fact, such as the location of its datasource.

• A model can have an arbitrary id and store it in its datasource, typically at the beginning of a file. The model must reconstruct its id each time it is loaded.

The PersistenceFacade.createModelId() can be used to create an id from a description string. The string must have a special format 'factoryDesignator:text' (for example 'path:a/b/c.xml'), where:

• factoryDesignator is an 'identification' of an id factory to use for constructing the id. These factories must have been registered through the PersistenceFacade.setModelIdFactory() method.

  There are some factories already available:

  • 'i' (IntegerSModelId) - creates an IntegerSModelId which gives models an integer value (parsed as hex from 'text'), models should persist the id themselves.

  • 'path' (RelativePathSModelId) - creates a RelativePathSModelId which identifies a model with its relative path as specified in 'text'. The id can be reconstructed from the model’s datasource path.

  • 'r' (RegularSModelId) - Uses a UUID parsed from 'text', models must take care of storing the ide.

  • 'f' (ForeignSModelId) - Uses the string stored in 'text' directly, models must take care of storing the ide.

• text - the text to pass to the factory as a seed information to construct the id.

SModelName (org.jetbrains.mps.openapi.model)

Represents a name of a model (for example jetbrains.mps.samples.xmlPersistence@generator), which normally consists of several parts:

• Namespace (jetbrains.mps.samples)

• Simple name (xmlPersistence)

• Stereotype (@generator)

Model references

Represent a model within its containing module. They are part of the internal API and should be eliminated from the public API eventually.

PersistenceFacade

A singleton class that represents a registry of models and model root factories. It also provides helper methods to transform String to/from model.module references and node ids.

ModelSaveException

Thrown when the model does not fit the persistence format.

ModelLoadException

Thrown when the persistence format does not contain all the necessary data to construct the desired model.

ModelReadException

Used to report errors when loading a model.

Problem (org.jetbrains.mps.openapi.model.SModel)

Represents a persistence problem. It is typically reported through an exception and MPS will visualize the problem to the user eventually. It keeps references to the location of the problem as well as the node.

Noteworthy implementation:

• jetbrains.mps.extapi.model.PersistenceProblem

Using the custom persistence plugin in MPS

After installing the generated plugin into MPS, when you are creating new models you are able to specify the XML file persistence provider for them:

Just as we specify in the XmlPersistenceModelDescriptor.importedLanguageIds() method, the new model will have the jetbrains.mps.core.xml language added as a used language and an empty Root Node will be added to it:

Whatever you type into the root node will be persisted immediately into a corresponding xml file. Changes to the underlying xml file will be reflected in the model upon opening in the editor.

Project structure

If you open the xmlPersistence sample project, you will see three solutions, each of which fulfills a separate role in the puzzle.

The xmlPersistence module defines the actual persistence logic, xmlPersistence.build contains a build script and xmlPersistence.ideaPlugin contains a customized plugin descriptor. Although the build script normally provides a reasonable plugin descriptor by itself, this time we need to customize the descriptor, thus we include it in the project explicitly.

The plugin descriptor provides the standard plugin information plus it registers our jetbrains.mps.persistence.XmlModelPersistence class as mps.ModelFactoryProvider. This is the additional bit that requires us to provide an explicit plugin descriptor.

XmlPersistenceModelDescriptor

This class extends EditableModelDescriptor and will represent models in MPS.

These are the noteworthy points:

• The importedLanguageIds() method ensures that instances of this model have the jetbrains.mps.core.xml language imported.

• The save() method assumes a single root in the model, which will be persisted into the model file. It also assumes StreamDataSource. It leverages a helper RegularTextUnit class that converts a model in the jetbrains.mps.core.xml language into xml text.

• The createModel() method checks whether the model file exists.

  • If not, an new empty in-memory model (SModel instance) is returned.

  • If the file exists, it is read, parsed and converted to an SModel instance with a single root XmlFile in it.

XmlModelPersistence

This class extends ModelFactory and handles the creation and persistence lifecycle of XmlPersistenceModelDescriptor instances.

supports()

The supports() method checks the type of the data source. We only accept FileSystemBasedDataSource and StreamDataSource at the same time. The method must be called from all persistence-related methods manually to prevent ClassCastExceptions from within these methods.

getPreferredDataSourceTypes()

This method returns a collection of DataSourceTypes that this ModelFactory handles. This implementation returns its own XML_TYPE, which is a type based on a file extension.

getType()

This method returns an instance of ModelFactoryType that describes the model factory. There are three built-in types in MPS, all defined in the PreinstalledModelFactoryTypes enum.

Our model factory defines its own type:

The getFormatTitle() method is worth a special mention here, since the string it returns will be used to represent the storage format to the future users.

create()

This method will create a new instance of XmlPersistenceModelDescriptor and initialize it with path-based id. Then it adds a new root node with the name identical to the name of the model to represent an empty model.

load()

This method will create a new instance of XmlPersistenceModelDescriptor and initialize it with path-based id. The model itself is not loaded at this point.

save()

This method delegates to the save() method of the model.

Using the custom stub persistence

Once the plugin is deployed to MPS, users will be able to set a new type of model root in their solutions - Property files.

If this model root is selected, all folders marked by the user as 'Sources' (in the right-hand side panel) that contain property files (files with the .properties extension) will become models.

These models (named 'client' and 'server' in the image above) are stub models for the property files. They are read-only and other models in the project can reference them. The language used to describe property files in MPS, called jetbrains.mps.samples.PropertyDefinition, is also part of the propertyPersistence plugin.

The plugin project structure

The project contains the jetbrains.mps.samples.PropertyDefinition language and a sandbox solution to test the language on simple property definitions.

The jetbrains.mps.samples.propertyPersistence.build solution contains a build script that packages three of the modules into an MPS plugin.

Custom plugin.xml definition is used.

It registers our custom factories as mps.modelRootFactory and mps.modelRootEntry as extensions to MPS so that they can take effect. Additionally, the location of the packaged language is specified with mps.languageLibrary.

Stub Models implementation

The actual implementation of all the necessary classes is in the jetbrains.mps.samples.propertyPersistenceDef solution.

Each of the classes has a well-defined purpose:

• PropertyFilesDataSource - a FolderDataSource extension that only specifies that files with the .properties extension from the current folder should be included in the data source.

• PropertyFilesStubModelRootEntryFactory - a factory that creates instances of PropertyFilesStubModelRootEntry when asked by MPS.

• PropertyFilesStubModelRootEntry - ensures that the user can specify and configure this kind of model root in the UI of a module. Since our model root uses files and folders, this class only delegates to FileBasedModelRootEntry.

• PropertyFilesStubModelDescriptor - the model implementation that extends RegularModelDescriptor. It parses property files, creates root nodes and populates freshly created models with them. Similarly to the xmlPersistence the model descriptor class also ensures that a proper language is imported (jetbrains.mps.samples.PropertyDefinition) into the models when created. Since stub models are read-only, there is no need to save modes.

• PropertyFilesStubModelRootFactory - a factory that creates instances of PropertyFilesStubModelRoots when asked by MPS.

• PropertyFilesStubModelRoot - the actual implementation of a model root. It extends FileBasedModelRoot

PropertyFilesStubModelRoot

Some more details about this class:

• loadModels() - this method gets all the directories marked by the user as SOURCE and for each directory recursively creates PropertyFilesStubModelDescriptor instance.

• getType() - this method returns a string identifier of the root type. The identifier is used in the plugin.xml to identify associated factories.

• getSupportedFileKinds1() - returns all supported file kinds (SOURCES, TESTS, EXCLUDED, etc.) which the user can choose from to mark files and directories with when setting up the model root in the module settings dialog.

• load(Memento) - reads the path of the model root from the memento. It has been saved by the parent FileBasedModelRoot class previously so we rely on it being present in the memento. Using the path it then sets the contentDirectory and source root properties. Note: This method serves as illustration of working with mementos. In fact it is not needed in this example as the parent class already reads the memento and sets the contentDirectory as well as source root.