MPS 2022.3 Help

Migration Guide

When opening a project in a newer version of MPS, all the automated migration scripts that come with that version of MPS will be executed to bring the project up to date with the languages and APIs. Sometimes additional manual steps might be needed or some special care must be taken by the user. This page lists such instructions for each past version of MPS.

Before you update your MPS version, it is advised to run the Pre-Update Check against your project . The Pre-Update Check action can be started from the Main Menu | Migration | Run Pre-Update Check menu item. You run it in the old MPS instance to verify that there are no unmigrated left-overs in the project that the new version of MPS may have issues with. Namely, it invokes all check() methods of all migrations scripts of the languages used in the project. The action will attempt to fix all probable problems of these cases:

  • New nodes written using old language versions were merged into a migrated branch from an unmigrated branch

  • Some nodes weren't fully migrated, typically because the migration gave up migrating them for being too complex and instead recommended a manual migration for these nodes, which then never happened.

Migration to MPS 2022.3

Java Module Facet settings

A dedicated project migration saves Java Module Facet settings in the module descriptors as explicit values rather than unrelated values scattered around the descriptor (e.g. compileInMps and pluginKind were attributes of a module, and external classloading was managed by an additional module facet).

Extract gnu.trove and org.jdom libraries

A project migration extracts gnu.trove and org.jdom libraries out of MPS.Core. There are 2 new stub modules for these libraries. The migration redirects references to their stub classes that used to go through MPS.Core right to the new modules. These modules can be used independently of MPS.Core, if necessary.

Make facets, TextGen and Generate moved

Make facets, TextGen and Generate, used to reside in a plugin aspect of the j.m.lang.core language and now got moved out into a dedicated module, j.m.make.facetsi>. You may notice updated Make Target names in your generated sources if you reference these facets. Although unlikely, you may need to adjust model imports in the model with your custom Make Facet in case you reference TextGen/Generate as a dependency.

JNA library update

The generic distribution of MPS now contains two versions of the native jna library, each held in a separate subfolder of the /lib/jna/ folder:

  • amd64 - for the Intel-based Mac computers

  • aarch64 - for the Apple hardware Mac computers

The build script wizard that creates build scripts for standalone MPS IDEs, already provides an additional MPS-macos-aarch64.zip layout that utilizes the aarch64 variant of the jna library, while the other three, previously existing, layouts have been updated to correctly include their corresponding jna libraries from the new location.
Existing build scripts shall be updated manually to accommodate for the change in location of the jna library as well as to adjust the set of binary files according to the new platform.

Migration to MPS 2022.2

Stop supporting the old format for enumeration values

Although the structure language's enumerations and their persistence format had changed many releases ago, the old format was still supported for compatibility reasons. This supports ended with MPS 2022.2, which only supports the new format. This may cause problems reading models in MPS 2022.2, if the models have not been migrated properly in the previouos versions of MPS. To fix this, re-run the migration "Update enumeration properties" that has been introduced intoo the structure language in MPS 2019.1. You have to run the migration in an older MPS before opening the project in MPS 2022.2.

MPS Kotlin language updates

Automatic migrations have been provided to reflect the recent improvements in the MPS Kotlin language:

  • Statement support has been made generic in the IStatementHolder interface; other concepts using statements on their own have been migrated to use this concept.

  • String literal and multiline literal have been merged into a single string literal concept.

  • Inheritance modifier (final, abstract, etc.) on function declarations is now mandatory because of the usage of the Inheritable interface; declarations will be migrated accordingly.

  • Support for deconstructing variables has been grouped under the IDeconstructingDeclarations interface; for statement and lambda multi-parameter are migrated to use it.

  • Concepts with receiver types no longer use the ReceiverType concept (which is an expression referring to a type) and hold the type directly.

  • Unsigned, Long and Integer literal concepts have been grouped under the IntegerLiteral concept (which uses boolean flags). Moreover, negative number literals are no longer supported and must be created using the unary minus concept.

Using JDK 17

MPS 2022.2 has been developed and tested with JDK 17, so it is recommended to use JDK 17 for running MPS as well as for running standalone IDEs built with MPS. There may potentially be some problems with the visibility of different classes that your code may be accessing using Java reflection API. You can solve the issues in the same way we solved it for the JDK classes when running MPS. Use the --add-opens JVM argument to add the respective packages:

--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.desktop/java.awt=ALL-UNNAMED --add-opens=java.desktop/java.awt.dnd.peer=ALL-UNNAMED --add-opens=java.desktop/java.awt.event=ALL-UNNAMED --add-opens=java.desktop/java.awt.image=ALL-UNNAMED --add-opens=java.desktop/java.awt.peer=ALL-UNNAMED --add-opens=java.desktop/javax.swing=ALL-UNNAMED --add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED --add-opens=java.desktop/sun.awt.datatransfer=ALL-UNNAMED --add-opens=java.desktop/sun.awt.image=ALL-UNNAMED --add-opens=java.desktop/sun.awt.windows=ALL-UNNAMED --add-opens=java.desktop/sun.awt.X11=ALL-UNNAMED --add-opens=java.desktop/sun.awt=ALL-UNNAMED --add-opens=java.desktop/sun.font=ALL-UNNAMED --add-opens=java.desktop/sun.java2d=ALL-UNNAMED --add-opens=java.desktop/sun.swing=ALL-UNNAMED --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-opens=jdk.jdi/com.sun.tools.jdi=ALL-UNNAMED --add-opens=java.desktop/com.apple.laf=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt.event=ALL-UNNAMED

Changes in logging configuration

Due to abandoned log4j, bin/log.xml configuration is no longer in use. There’s no mechanism to migrate your settings from log.xml, you need to re-create them in log.properies or to use IDE actions to configure logging. MPS no longer relies on configuration files for default log configuration (done in code, instead), and you need to turn on the use of configuration file explicitly through command-line switches.
For a hand-written logging code, it’s recommended to replace log4j code with jetbrains.mps.logging.Logger.getLogger(Class<>) to minimize the changes. Users of jetbrains.mps.baseLanguage.logging language don’t need to bother, just regenerate the code with the new templates to get updated.

MPS.Core no longer exports unrelated stubs

MPS.Core no longer exports stubs available from MPS.Generator and MPS.Textgen. MPS.Generator and MPS.TextGen were introduced a year ago in 21.2, to keep dependencies focuse and to prevent MPS.Core growing into a monster hog with a lot of unrelated functionality. For a few releases, MPS.Core exported java stubs for the Generator and TextGen subsystems, along with designated MPS.Generator and MPS.TextGen stub solutions. In this release, we finalize the split and MPS.Core no longer exports stubs for these subsystems. There’s re-runnable migration to re-route references to MPS.Generator and MPS.TextGen stubs you may need to run it again in case your models still reference Generator/TextGen stubs through MPS.Core solution.

New api in lang.plugin to register shortcuts

You may notice changes in the generated code if you declare custom shortcut sets or override the existing shortcuts for your actions. This streamlined and simplifies the generated code.

False dependencies need to be corrected

The fact ‘deps.cp’ keeps a snapshot of actual module compile dependencies at transformation time may ruin hacks/workarounds when an unused dependency was added to a module/model to satisfy compilation-time (Java classpath) dependency. Instead, this dependency has to be introduced in a proper way (in majority of cases, it’s just a misconfigured runtime of an engaged language).

Changes to pathToFile in the Make facet

In the Make facet, the pathToFile(String,IFile) function parameter gone. It served few very specific cases and was unlikely you ever needed it. If you do, however, there has been a replacement, pathToFile(IFile, IFile), available for a few releases already.

Deputy concept in aspect declaration

EXPERIMENTAL: ‘Deputy’ concept in aspect declaration. You may notice these if you declare your own aspects. This is a work towards meaningful @descriptor models. MPS employs this for the ‘structure’ aspect now. If you feel this is the right move (or a wrong one), please share your opinion.

mpsCore import in build scripts (MPS-34554)

MPS used to re-export mpsCore project and the respective plug-in from mpsStandalone project. If the build project for your RCP used to bundle/import mpsCore, you need to add a direct project dependency to mpsCore for import to get resolved again.

An additional dependency needed for the build scripts

The lib/util_rt.jar file has to be added in the boot classpath of the MpsStartupScript roots of the existing projects that use the build language to create standalone IDEs. Otherwise, exception like this one may occur when you attempt to start your standalone IDE:

Exception in thread "main" java.lang.NoClassDefFoundError: com/intellij/openapi/util/SystemInfoRt at com.intellij.openapi.application.PathManager.getHomePath(PathManager.java:101) at com.intellij.openapi.application.PathManager.getHomePath(PathManager.java:68) at com.intellij.openapi.application.PathManager.getBinPath(PathManager.java:186) at jetbrains.mps.Launcher.main(Launcher.java:28)

Build script migration

VM options changed

The VM options used to start MPS IDEs have changed. The existing build scripts for standalone IDEs have to be updated manually to provide all the necessary VM options. An intention is in the works that will add all the missing depedencies. Until it is in place, the recommended way is to generate a new build script and copy the just created MpsStartupScript root node into your build script, or just use the list of options below:

# Common IntelliJ Platform options: -Xms256m -Xmx2048m -XX:ReservedCodeCacheSize=512m -XX:+UseG1GC -XX:SoftRefLRUPolicyMSPerMB=50 -XX:CICompilerCount=2 -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -XX:+IgnoreUnrecognizedVMOptions -ea -Dsun.io.useCanonCaches=false -Dsun.java2d.metal=true -Djdk.http.auth.tunneling.disabledSchemes="" -Djdk.attach.allowAttachSelf=true -Djdk.module.illegalAccess.silent=true -Dkotlinx.coroutines.debug=off -Dsun.tools.attach.tmp.only=true # Additional MPS options: -client -Dfile.encoding=UTF-8 -Dapple.awt.graphics.UseQuartz=true -Dide.mac.message.dialogs.as.sheets=false -Didea.invalidate.caches.invalidates.vfs=true -Didea.trust.disabled=false -Dfreeze.reporter.enabled=false -Didea.indices.psi.dependent.default=false # -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5071

Re-create run configurations

Existing run configurations will not work because the VM options required to start Java processes in MPS have changed. You can either create new run configurations, which will inherit updated VM options automatically. Or you can update the value in the run configuration's Virtual Machine Parameters field to

--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt.event=ALL-UNNAMED --add-opens=java.desktop/com.apple.laf=ALL-UNNAMED --add-opens=java.desktop/java.awt=ALL-UNNAMED --add-opens=java.desktop/java.awt.dnd.peer=ALL-UNNAMED --add-opens=java.desktop/java.awt.event=ALL-UNNAMED --add-opens=java.desktop/java.awt.image=ALL-UNNAMED --add-opens=java.desktop/java.awt.peer=ALL-UNNAMED --add-opens=java.desktop/javax.swing=ALL-UNNAMED --add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED --add-opens=java.desktop/sun.awt.datatransfer=ALL-UNNAMED --add-opens=java.desktop/sun.awt.image=ALL-UNNAMED --add-opens=java.desktop/sun.awt.windows=ALL-UNNAMED --add-opens=java.desktop/sun.awt=ALL-UNNAMED --add-opens=java.desktop/sun.font=ALL-UNNAMED --add-opens=java.desktop/sun.java2d=ALL-UNNAMED --add-opens=java.desktop/sun.swing=ALL-UNNAMED --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-opens=jdk.jdi/com.sun.tools.jdi=ALL-UNNAMED --add-opens=java.desktop/sun.awt.X11=ALL-UNNAMED
.

Editor hints allowed only from extended languages

In the previous versions, when defining editors and assigning editor hints to them, MPS tolerated editor hints from other language modules available through mere 'default' dependency. Now MPS only allows editors to use editor hints from languages explicitly "extended" by the language holding the editors.

Obsolete custom VM options need to be removed

If you installed MPS 2021.2 EAP 2 from the toolbox you may have ended up with three custom VM options set:

-Dsplash=true -DactionSystem.update.actions.async=false -DactionSystem.update.actions.async.ui=false
These options are now obsolete and you are highly recommended to remove them. Just choose "Edit Custom VM Options" from the "Help" menu and remove the lines manually.

Replace the deprecated needNoWriteAction property with accessMode

The node test cases got a new accessMode property. Previously, the node tests were executed wrapped in either runWithinCommand or runWithinRead methods of BaseTestBody. Now the tests are executed under runWithinCommand, runWithinRead or without any wrapping depending on the value of the accessMode property.
The ExecutionModeNodesTestCase migration removes the needsNoWriteAction deprecated property and sets the accessMode property to either command or nonedepending on the previous value of needsNoWriteAction.

Custom extensions of the test language and/or the node test cases that provide their own test methods should pay special attention in their generators to proper wrapping of the code of the custom test methods with locking. The runtime class BaseTestBody now provides two additional methods runWithinCommand and runWithinRead that should be used just like they are used in the switch_Test2Method template switch.

Migration to MPS 2021.3

Add log4j to the build script

Build scripts created by the Build Script Wizzard in earlier versions of MPS may have problems finding the log4j jar file in 2021.3.

Migration build1
To fix this problem the lib/3rd-party-rt.jar must be added into the boot classpath section of the MpsStartupScript node of the build model.
Migration build2

Re-route references to util.jar in MPS.IDEA to MPS.Boot

On our way towards replacing huge single-point-of-entry stub modules (like MPS.Core) with slim modules that target narrow usage scenarios, we extracted the MPS.Boot solutiion. It holds classes related to the startup of MPS as well as a few IntelliJ stubs related to the IDE startup logic, which used to be in MPS.IDEA. The migration re-targets the references pointing to the corresponding stub models of the solution MPS.IDEA to the new stub solution.

Replace ToRemove and ScheduledForRemoval with Deprecated

The ToRemove and ScheduledForRemoval annotation both mark deprecated code. The standard Deprecated annotation will be used insed of those. The version parameter of the two annotations is migrated into the "since" parameted of Deprecated. The migration only works for instances of IBLDeprecatable.

Adopt asynchronous update mechanism for UI Actions

There’s a major change in IDEA Platform to update UI actions in an asynchronous manner. Data necessary to update action state is collected and consumed asynchronously and independently. There’s a promise of more responsive UI and general performance improvements. MPS started to adopt the change, and unless you have some hand-written action code or custom UI components that serve as DataProviders, you should not notice anything. Actions generated from j.m.lang.plugin are not affected, if you stick to MPS actions/UI abstractions, your code doesn’t need additional attention. However, if you deal explicitly with IDEA’s DataProvider/DataContext APIs (e.g. provide custom UI view implementing DataProvider), you might need to apply some extra migration efforts.

MPS 2021.3 still comes with asynchronous update turned off by default (EDITORIAL: check MPS-33702 to make sure this holds true), mostly as a precaution, however, one can try asynchronous update mode by editing respective settings in the Registry.

Delayed model loading

There's experimental functionality to load models only when requested. MPS used to load all module’s models the moment the module becomes known (aka ‘attached’) to a repository. Now, publishing a module doesn’t force loading of its models, only when there’s a need to access models, MPS makes an attempt to load them. Generally, this happens quite soon after a module is published, the change is just a first step to make the model's lifecycle more independent of its module.

If you experience any issues with delayed loading of models, you can turn legacy behavior on with a ‘mps.models.force=true’ system property.

Notable API changes

  • UI: The NewSolutionDialog has been replaced with a generic NewModuleDialog

  • SModel: [openapi] SNode.setReference(SReferenceLink, SReference) has been deprecated, with few alternatives available.

  • SModel: [impl] SNode.toString and node.presentation operation

  • Make: changes in the runtime modules for few j.m.make.* languages. You might notice them if you write custom make facets.

  • BaseLanguage: Support for the "[package]Classifier" reference notation has been removed

Migration to MPS 2021.2

NodeTestCases need regeneration

When upgrading to 2021.2.6 all NodeTestCases have to be regenerated for them to run successfully. Otherwise you will be getting exceptions like

java.lang.reflect.InvocationTargetException at jdk.internal.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.intellij.execution.process.ProcessHandler$2.invoke(ProcessHandler.java:247) at com.sun.proxy.$Proxy32.onTextAvailable(Unknown Source) at com.intellij.execution.process.ProcessHandler.notifyTextAvailable(ProcessHandler.java:221) at jetbrains.mps.execution.configurations.implementation.plugin.plugin.FakeProcessHandler$BlockingReader.onTextAvailable(FakeProcessHandler.java:155) at com.intellij.util.io.BaseOutputReader.sendText(BaseOutputReader.java:205)

Replace implicit node.toString() with explicit smodel operation

A new node.presentation operation and a migration have been introduced to jetbrains.mps.lang.smodel. The migration replaces the usages of a node in string concatenation ("string" + node and node + "string") with the explicit node.presentation operation. At the moment, this operation delegates to the getPresentation() behavior method, which is what node.toString() used to do. Since MPS ceased to use the getPresentation() behavior method in the toString() method implementation, it is advised to use explicit node operations (like node.presentation, node.getPresentation() or any other tailored behavior) when you intend to show the node presentation to the end-user.

Split MPS.Core stub models into distinct modules

The stub models exposed through the MPS.Core module (lib/mps-core.jar, lib/mps-generator.jar and lib/mps-textgen.jar) have been separated into MPS.Generator and MPS.Textgen. This mostly impacts users who use the generator or the textgen APIs directly.

An automated migration merely updates references to use affected stubs to be used from the new modules. If it fails, the MPS.Generator and MPS.TextGen can be imported manually and then the classes from the proper modules must be selected in code using the code completion dialog.

For compatibility, MPS.Core still exposes the jars, but it will cease to expose them in 2021.3.

Plugin descriptor placement

The IntelliJ platform now loads plugins only if their plugin descriptor (the plugin.xml file) is located in a specific place in the plugin layout: plugin_folder/lib/one_of_the_plugins.jar!/META-INF/plugin.xml Previously, the META-INF/plugin.xml was allowed to be placed anywhere in the plugin folder.

Unless the plugin descriptor is mentioned explicitly in the plugin layout at the right location, an automatic transaction will generate it when the build script is generated. It will add the required jar file to the plugin layout and will make it hold hold the plugin descriptor. The code intended for adding during generation is displayed in a read-only section of the plugin layout:

Plugin descriptor1
An intention called "Override Plugin Descriptor Layout" is available that allows the build script author to override the default behavior with her own - the intention generates the required part of the layout and enables it for manual editing:
Plugin descriptor2
An inspection will report a warning whenever the plugin descriptor cannot be found in the layout definition.
Plugin descriptor3

Migration to MPS 2020.1

Improved core.text language

The jetbrains.mps.core.text language has been improved and is now being used for comments in BaseLanguage. The CommentPart and TextCommentPart concepts have been deprecated. If your language utilizes these deprecated concepts, consider using the concepts like Text, Line and Word from the jetbrains.mps.core.text language instead. You'll then need to provide migrations for the users of your language that will migrate the text held in TextCommentParts to be transformed to instances of Words and Lines. The easiest way to implement such migrations is to:

  • instantiate a Line

  • instantiate a Word

  • add the word to the line using Line.addTextElement()

  • set the Word.value property to the text of a TextCommentLine

  • call the Word.normalize() method to have the text of the word split properly into words, all belonging to the same line

The new migrations

MPS 2020.1 comes with four migrations that will update your projects to the latest versions of jetbrains.mps.baseLanguage, jetbrains.mps.lang.quotation and jetbrains.mps.lang.editor:

  • UpdateSingleLineCommentToUseLinePerComment will update instances of SingleLineComment to use a different child to hold the textual description of the comment. The comment can now only hold a single line of text, so the migration has to split single line comments that spread across several lines.

  • MigrateTryStatement will update instances of TryCatchStatement and TryFinallyStatement to TryUniversalStatement, which allows for JDK 8 features, such as multiple exceptions in a single catch clause and with-resource capabilities.

  • LightQuotation_InitProperyExpression wraps property initialisers of light quotations into instances of NodeBuilderPropertyExpression.

  • Migrate_MergeNamedAndDefaultMenus will upside definitions of transformation and substitute menus to the new concepts. Transformation menus (TM) and Substitute menus (SM) were refactored so that Named and Default concept variants were merged into one concept each: TransformationMenu and SubstitutionMenu, respectively. The distinction between default and named variants is still there, but is now handled by the type property of each concept instead of having a separate root concept for them.

All four migrations are re-runnable and so can be run repetitively if a merge brings non-migrated code into your branch.

Update the VCS settings

The MPS merge driver has been improved. Make sure your merge driver as well as the VCS settings are up-to-date before merging conflicting branches. The VCS subsystem of MPS needs Git to be configured in a certain way in order to function properly. An outdated VCS configuration of a project is indicated to the user upon opening the project.

The user can trigger the "Update" action from the automatically displayed popup or from the VCS -> Install MPS VCS Add-ons menu.

Vcs addons1
Vcs addons install

When the installation completes, you can rely on the VCS system to be ready.

Recommended migration strategy

Merging two branches that both have been migrated to MPS 2020.1 separately is likely to give you headaches. Avoid it! Otherwise this will happen to you and your loved ones:

  • All single line comments that spread across several lines will be split into new instances of single line comments, which will be different nodes in each branch and will be in conflict.

  • New instances of TryUniversalStatement in both branches will be in conflict if there are additional changes inside this statement.

  • Light quotations that the migration enhances with the NodeBuilderPropertyExpression wrappers, will be in conflict.

  • All Transformation menus and Substitute menus will be in conflict and you will need to accept one or the other (root node's concept has changed and there are new child nodes in the menus). This is because the root nodes of Transform/Substitute menus changed their concepts and are in fact different instances. You can merge them one by one and in case the menu was not changed in either branch OR if the menu changed in just one branch you can use the Accept Yours or Accept Theirs strategy as appropriate.

We strongly recommend you to migrate all your branches to MPS 2020.1 in a single step.

This document overrides the information given in the Migration with branching documentation page and should be followed when migrating to MPS 2020.1.

The prefered scenario is:

  • Merge all the branches to master.

  • Migrate the master using the migration assistant.

  • Run the Migrate try statement enhancement script.

  • Start new branches off the migrated master.

The "Run the Migrate try statement enhancement script" step needs a bit of explanation. The normal migration of try statements works in a defensive way and avoids migrating in situations that can potentially lead to broken code. These are typically try statements in generator template fragments, macros, commented out nodes or inside quotations. To avoid the need to manually update the code that the migration skips (the not-migrated nodes are reported by the migration, so you will see a list of them) we provide an enhancement script called Migrate try statement to attempt to migrate those complicated cases. It may, however, lead to broken code, so manual verification of the result is needed.
Mig23

Alternative migration strategy

If you can't help it and have to merge a not-migrated branch into a migrated master, you should follow the scenario described in this section:

  • Resolve merge conflicts

  • Execute re-runable migrations

  • Manually check the diff

  • Run the "Migrate try statement" enhancement script

  • Manually check the diff

  • Commit

We will discuss these topics in more detail now.

Resolve merge conflicts

Conflicts in editor.mps file (jetbrains.mps.lang.editor)

This may happen if you try to merge a non-migrated branch to a migrated branch and your non-migrated branch contains some changes to the same menu that you have migrated in the other branch. In this case it's recommended to Accept Yours (keep the migrated menu) and apply the modifications done in the other branch manually.

Conflicts in SingleLineComment (jetbrains.mps.baseLanguage)

A merge conflict will be reported for all single-line comments that have been modified in the branch.
Blc3
The single-line comments that have not been changed in the branch will be taken from the master and so will have been migrated already. These can be auto-resolved using the appropriate tool button.
Blc4
The remaining conflicting changes have to be migrated manually, perhaps by accepting the changes from the branch (accept theirs). The not migrated instances of SingleLineComment will thus be re-inserted into the code.
Blc5
Special care must be taken for single-line comments spreading across multiple lines, as is in the last comment in the illustrating code above. Since the migration separated the additional lines into standalone instances of SingleLineComment, which are hanging below the original SingleLineComment instance, and since merging the changes from the branch will only affect the original (top) instance of SingleLineComment, the lines below will be duplicating the text of the comment.
Blc6
The duplicating trailing single-line comments can be deleted. Some of the lines, however, may hold changes to the text of the comment coming from the master branch. These must be manually copied into the single-line comment and only then the trailing comment can be deleted.
Blc7
Blc8
Blc9
This way you resolve all the conflicts and close the dialog. The code now, however, contains not-migrated instances of SingleLineComment coming from the branch.
Blc10

Conflicts between not-migrated TryCatchStatement/TryFinallyStatement and already migrated TryUniversalStatement (jetbrains.mps.baseLanguage)

A merge conflict will be reported if there were changes inside the non-migrated or the migrated try statement. There are 3 possible cases:
  • changes were only inside the migrated try statement - This is the simplest case and there should be no conflict. Just accept changes if they were not accepted automatically.

  • changes were only inside not-migrated try statement - Here you should accept the changes from the not-migrated branch and ignore those from the migrated one (actually there are no meaningful changes to loose, just the migration that you'll repeat in the next step, anyway).

  • changes were inside the try statements in both branches - This is the most complicated case. Choose one of the branches (not-migrated or migrated) from which you'll accept the changes. We suggest to use the migrated branch but sometimes most of the changes are in not-migrated, in which case you should perhaps choose that one. Then accept the changes in try statements from the chosen branch and ignore the changes in try statements from the other branch. Finally manually copy/paste the changes from the other branch. You can conveniently do it directly in the 3-way merge dialog: copy the statements from the right or the left editor and then paste into the center one.

Conflicts between not-migrated and migrated light quotations

Depending on the changes made to either of the branches or both of them, handle the situation in a way similar to the previous case.

Execute re-runable migrations

Since the master after merge may contain code that has not been migrated, the migrations have to be re-run to upgrade the code to MPS 2020.1. Run the Execute Re-Runnable Migrations action from the menu to get that done easily.

Mig111

Manually check the diff

Verify manually that the changes above did not introduce any errors into your code.

Run the 'Migrate try statement' enhancement script

Just like in the normal migration scenario, run the Migrate try statement enhancement script, to attempt to migrate automatically the cases that the migration skipped to avoid breaking code.

Mig23

Manually check the diff again

Since the previous step could have broken some code, you should inspect the changes manually and correct them eventually.

Commit

You are done. The code can now be committed.

Migration to MPS 2019.2

Migration of structure's enumeration declarations

MPS 2019.2 features a new way of declaring structure enumerations. Therefore the existing enumeration declarations become deprecated by this feature, MPS provides a set of migration scripts to replace them as well as their usages with the new constructions. This article precisely describes how these migrations are processed.

Upgrading enumeration declaration

Firstly, the migration replaces the old enumeration declarations with the featured declarations. For each deprecated enumeration in a migrated language the migration creates a new enumeration using the featured concept. Also, the migration places the old enumeration declaration in a migration's particular attribute marking that this declaration is no longer operating and stays there only for documenting and as migration's auxiliary.

Since the new declaration holds different kinds of properties (means in general sense, not a concept's property), mapping old declaration's properties to the new ones might be non-trivial. Here we describe how the migration handles each property of the old enumeration declaration:

  • name of the old enumeration are trivially copied to the name to the new declaration;

  • "has default" and "default member" became just "default member";

  • "empty text", "member datatype" and "member identifier policy" are no longer supported;

  • For each old enumeration's member, the migration creates a member in the new declaration. Member's "external value" became "presentation" (that is the optional text line in 2nd column of new enumeration declaration). Member's "internal value" and "identifier" no longer are supported but they are used to choose a suitable name for new enumeration member.

Upgrading enumeration properties

For each migrated enumeration declaration, the migration replaces property declarations of the old enumeration type with a new property declaration of the new enumeration type. The migration places the old property declaration in a migration's attribute to handle the migration of its usages properly.

Migration of smodel and aspect languages' code

MPS 2019.2 provides concise experience working with the new structure's enumerations. Therefore the SModel language presents new enumeration's property values with "enummember<_>" type in smodel queries instead of raw values. The raw values were essential for old enumeration members but had lack of design-time support in smodel queries and therefore were dropped with the new enumerations and replaced with "enummember<_>" types. A new approach guides the language designer towards writing correct code with more restrictive types for property access operations and reduced boilerplate via providing elegant switch operation over enum members. However, this approach is not compatible wild the smodel code behaviour for old enumerations where the language designer was forced to use "raw value" of an enumeration member. Therefore, additional migration is required to update smodel queries to work with the new enumerations.

Migrations of smodel and aspects code are triggered when the code refers to enumeration properties or enumeration declarations that were replaced with the new alternatives by the structure aspect migrations. Basically, for each such reference, one of these migrations updates it so that it refers to the new counterpart. The migration also adjusts the code nearby to preserve code's semantics. There are several cases where such procedure might change code significantly, so we describe them in more detail below.

Node's property access operations for enumeration properties represents one such situation where the migration changes the code a lot, since the typing rules of these expressions are different between the old and the new enumerations. There are general rules which migration follows when handling smodel code:

  • If the enumeration declaration had a "boolean" member datatype, then smodel code semantics is restored by use of "is" checks on enumeration members.

  • If the enumeration declaration had "integer" member datatype and its members form ordered ordinal number sequence starting from 0 or 1, then the migration generates "indexOf" operations upon the enumeration's member list.

  • If the enumeration declaration had "string" member datatype, the migration could generate "name" or "presentation" operation on enumeration member, if the given names/presentations for new member declarations match their past internal values. Usually, this is the case unless the internal values do not satisfy naming constraints and don’t match member presentations.

  • Otherwise, the migration produces calls to special methods that emulate the semantics of an initial smodel code. The migration script creates such methods beforehand in a distinct model called "enumMigration" inside the enumeration-owning language.

The migration script applies similar transformations in all places where type mismatch might occur due to upgrades to some enumeration property. Here is a list of such places:

  • Node's property read and write accesses

  • Generator's property macro queries

  • Property constraint's queries

  • Editor's queries for transactional property cells

  • Property initializers for node builders (a.k.a. light quotations)

  • Property antiquotations in node quotations

  • Usages of property pattern variables in node patterns

Another place where migration has to transform code significantly to keep original semantics is old "name" and "value" operations on enumeration member. The migration handles such places in a way similar to described above.

As there are potentially lots of places where a transformations might be used, the migration script also detects patterns in the code where such transformation are not required or can be simplified in order to produce fewer changes to the user codebase.

Manual migrations

Although MPS offers automatic migration for new enumeration, a user might end up doing manual migration of his/her code. For example, this may happen during resolving merge conflicts. Here we describe common scenarios of a manual migration.

If you encounter non-migrated enumeration or non-migrated enum property in a language on which MPS has already executed migrations consider to rerun migrations scripts via `Migration -> Migrations -> Execute Re-Runnable Migrations`.

Non-migrated smodel code can be handled with this action also. However, you also can update smodel code manually by replacing references on properties, enumerations or its members in smodel code from old declarations to new ones. After doing that, type errors might occur, so you have to resolve them manually as well.

Sometimes smodel code might depend on property not by directly referencing it but rather indirectly via property attribute. To migrate property attributes manually, you have to invoke "Migrate by hand" intention on the attribute and fix probable type errors afterwards.

Code clean-up

Enumeration migration produces lots of additional attributes during the migration process. The migration framework requires such attributes for processing correctly further migrations on depending modules and projects. Although, language designer might not expect to have such depending modules outside his/her project and hence would like to remove migration attributes. This chapter describes how to do it and in which situations it is a safe change.

There are three kinds of artefacts that the migration process produces: migration attributes on new enumerations, migration attributes on enumeration properties and auxiliary methods in "enumMigration" model.

Attributes on the migrated enumerations provide data for correct migration of smodel code that refers to particular enumeration and of property instances in models written in the owning language. Generally, it means that if the language that owns such enumeration is published externally, migration attribute should be kept to perform the migration correctly. However, it's not the case for sandbox/test languages or languages in internal projects. For these scenarios, a language designer might legitimately want to drop enumeration's migration attributes by invoking "Drop Enumeration Migration's Attribute" intention on a migrated enumeration.

Attributes on migrated enumeration properties are required only for handling dependent smodel code. That means that if a language designer doesn't expect anyone to extend his/her language or maintain smodel code that operates with the enumeration property, then such attribute can be dropped safely. To make this, consider invoking "Drop Enum Property Migration's Attribute".

Likewise, auxiliary methods in "enumMigration" model are only needed for migrating smodel code. If the designer doesn't expect any smodel code operating with the migrated enumeration's members outside his/her project and has no occurrences inside the project, he/she might remove such methods manually.

Note that removing attributes is up to your choice. The MPS team, for example, schedules to perform automatic code clean up after a few releases.

Implementing automatic migrations for meta-languages

This chapter is mainly for MPS extensions' writers who implement either a new aspects or extend existing aspects (e.g. editor). Here we describe how they should update their extensions to tailor changes in MPS meta-languages.

This chapter describes the MPS metalanguages API for migrating enumeration data types and therefore contains lots of link to MPS source code that refer to either handy utility methods or examples for migrations. Open MPS 2019.2 application to be able to navigate by these links.

Generally speaking, if some MPS extension provides a concept that refers to a property declarations or a datatype declarations or depends on them by other means, the designer of this concept has to provide a migration to update the concept's instances. Otherwise, after applying the MPS enumeration migrations, nodes of this concept become to depend on outdated code, so a user becomes forced to fix them manually. The migration might differ between different concept so MPS can't provide a general solution for all possible extensions. Instead, MPS provides a set of utility functions that help extension's author to implement migration efficiently. Below we describe a few possible common scenarios.

If a concept has a reference link on a property declaration:

For such a concept, a language designer has to write a migration script that updates it on a new declaration if this property is of enumeration type that has been migrated. To do so, the extension's author might use the auxiliary method `EnumUsagesMigration#migratePropertyReference`. It accepts a node that has a reference for migrating and a reference link that targets PropertyDeclaration instances. The method returns a new target node of a given reference if it has been updated, or null otherwise. To use this auxiliary, the extension designer needs to import `jetbrains.mps.lang.structure.migration` model into the extension's migration aspect.

A canonical example here is a migration for editor cell properties.

If a concept has a reference link to a datatype declaration:

Similarly to the scenario above, a language designer has to implement a migration script to update such concept's nodes. He/she might use `EnumUsagesMigration#migrateEnumReference` to achieve this. The method accepts a node that has a reference for migrating and a reference link that targets to EnumerationDataTypeDeclaration_Old or its superconcepts. It returns a new target node for the given reference if it has been updated, or null otherwise.

Migration for “enummember<_>” instances might be an example of “migrateEnumReference” usage.

If a concept has a reference link to an enumeration's member declaration:

To migrate such concept, the language designer might use `EnumerationMemberDeclaration_Old#findReplacement()` behavior method to get an EnumerationMemberDeclaration instance that replaces old declaration. As `EnumerationMemberDeclaration_Old` and `EnumerationMemberDeclaration` concepts have no essential common superconcept, there is no option to reuse one reference link to refer to nodes of both concepts. Therefore the language designer has to consider introducing a new link in the concept and migrate from the old reference link to the new one. If a concept has the meaning of a smart reference, as an alternative, the language designer can introduce a new concept with the new link and migrate old concept's instances to new concept's instances.

Migration for deprecated “is” operation over enumeration properties can serve as an example.

If the typing rules of BL expressions or queries depend on a datatype:

Some extensions might cooperate with baseLanguage and smodel languages by introducing new expressions or queries. For such concepts, a language designer writes typing rules which might consider a baseLanguage's type of a referenced property or enumeration. MPS 2019.2 now provides a utility to evaluate a baseLanguage type for given structure's datatype - `jetbrains.mps.lang.smodel.typesystem.RulesUtil#datatypeBLType(node<DataTypeDeclaration>)`. This utility should be used in the typesystem aspect's rules to acquire proper property value's BL type by a given datatype.

If some concept is being migrated from the old enumerations to the new ones as in the first two described scenarios, and it has typing rules that manipulate with property value's BL type, then a node of this concept might end up having type errors even if this node was correct before the migration.

Lets return back to an example of migration of smodel code for the “WeekDays” enumeration described in “Migration of smodel and aspect languages' code” section. When the smodel migration updates a property reference in the property access operation, the BL type of such operation suddenly changes from `string` to `enummember<WeekDays>` and hence the smodel migration additionally wraps this property read with a call to the `name` operation that extracts the raw value from the enumeration member so that the overall expression behaves the same way as it did before the migration was applied. The smodel migration script does not actually implement the creation of such a call but delegates to an auxiliary class `jetbrains.mps.lang.smodel.migration.EnumExpressionsMigration` instead. This class provides a set of methods that the language designer can use to pass through similar problems in his/her extensions.

Particularly, it provides several methods:

  • `downgradeExpressionType(node<EnumerationDeclaration> enumeration, node<Expression> expression)` - transforms a given $expression$ of `enummember<$enumeration$>` type to evaluates it to a raw value of the originally evaluated enumeration member.

  • `upgradeExpressionType(node<EnumerationDeclaration> enumeration, node<Expression> expression)` - transforms a given $expression$ that evaluates to a raw value by means of the old declaration of $enumeration$ to evaluate it to an actual enumeration member that represents such a raw value.

  • `upgradeQueryReturnExpressions(node<EnumerationDeclaration> enumeration, node<IMethodLike> query)` - transforms a given $query$ body that evaluates to a raw value by means of the old declaration of $enumeration$ to evaluate it to an actual enumeration member that represents such araw value. Technically it collects all return clauses of the query and transforms them with `upgradeExpressionType` functionality.

  • `optimize()` - attempts to optimize the code around an expression that has been transformed with this the `EnumExpressionsMigration` instance. A general workflow with `EnumExpressionsMigration` follows these steps: it is instantiated at the beginning of a migration script, it uses the 3 methods described above to correct the user models and then the `optimize()` method is invoked to clean up the transformed code.

Example: Migration script for property constraints highlights the usages of EnumExpressionsMigration utility.

If a concept extends PropertyAttribute:

Even if the `PropertyAttribute` concept has no direct reference to `PropertyDeclaration`, the language designer might discover such instances through a deployed `SProperty` instance via `PropertyAttribute#getProperty()`. Since 2019.2, the `PropertyAttribute` concept provides a `#getPropertyDeclaration()` method that is aware of the enumeration migration, and for enumeration properties it provides the old property declaration if the attribute hasn't been migrated yet and the new property declaration if the migration has already been applied.

If a concept extends `PropertyAttribute` then there should be a migration that updates attributes of such a concept. The migration script has to invoke `EnumUsagesMigration#migrateEnumPropertyAttribute(node<PropertyAttribute>)` to update it. The method returns a new enumeration property declaration if the attribute is for the property of an enum type and the attribute hasn't been migrated yet. Otherwise, the method returns null.

The generator’s property macros is a good example of ‘PropertyAttribute#getPropertyDeclaration’ usage and property attribute migration.

Last modified: 06 September 2023