Migrate Space Automation to TeamCity
To successfully transfer your CI/CD configuration from Space Automation to TeamCity, it's essential that you have a solid grasp of the CI/CD concept and in-depth knowledge of both Space Automation and TeamCity. If you are new to TeamCity, please study its documentation first.
Prerequisites
TeamCity Cloud or a TeamCity Server version 2024.03.1 and newer.
To complete the migration, you should have:
System administrator rights in TeamCity.
Project administrator rights in Space projects which automation configuration will be migrated to TeamCity.
Transferring your CI/CD configuration from Space Automation to TeamCity is done in a series of steps, on both Space and TeamCity sides.
Connecting a Space project to TeamCity
We begin by creating a connection with you Space instance and setting up a target project in TeamCity. For optimal organization, consider creating separate TeamCity subprojects for each Space repository containing .space.kts
files. This guide will walk you through creating TeamCity builds with Kotlin-based configuration, but note that TeamCity also offers the option to configure builds entirely through the user interface.
Follow the instruction to configure a dedicated connection to your Space instance.
In the upper right corner, click Administration and choose Create Project.
The Create Project page will be displayed.
On the "Create Project" interface, select "From JetBrains Space". Choose the Space project and repository containing
.space.kts
. Once you select the Space project, you'll be prompted to create a new Space Application and authorize it within the source Space project. This application allows TeamCity to interact with repositories, report build statuses back to Space, post messages in MRs, and so on.In the New Project dialog, assign job's name you want to migrate first to the Build Configuration name - this will automatically create an empty build configuration for your job. By default, TeamCity build gets triggered by Git pushes to the default branch. You can insert additional
branchFilters
> from your job'sgitPush
definition into the Branch specification field.Once your TeamCity project is set up, move on to initializing and committing your TeamCity configuration as code setup. Open the Build Configuration Settings and navigate to the Versioned Settings tab. Now, toggle the Synchronization enabled option, select the source repository and set Kotlin as your Settings format. Then click Apply and wait until configuration files are committed into the repository.
Migrating shellScript block
The Space Automation shellScript
action has a direct equivalent in TeamCity: the build step script
.
Please note that TeamCity provides specialized build steps (or Build Runners) for various build or test tools. Using these dedicated build steps unlocks advanced functionality such as integrated test results support. So it's recommended to use dedicated build steps rather than invoking build tools within a generic script step.
If you have a complicated Space Automation shellScript
, you can translate it into several TeamCity build steps. Besides, you can easily mix simple script
steps with specific build runners like maven
or gradle
.
Migrating jobs with multiple steps
Migrating from Space Automation to TeamCity may require some refactoring. A thorough comprehension of the core concepts in both systems is crucial for a seamless transition.
Space Automation. In Space Automation, a Job is the basic building block of CI/CD processes, representing a single entity triggered for a specific git repository revision. A Job comprises multiple steps, each executing independently on a separate virtual machine. The step sequence determines the execution order, unless steps are located within a parallel block. A host step can encompass multiple actions executed sequentially on the same VM, while a container step always contains a single action.
TeamCity
The basic CI/CD component in TeamCity is a Build. Like Steps in Space Automation, each Build runs on its own VM. The difference between Space Automation Steps and TeamCity Builds is that in Space Automation a Step can't be run independently, it should be always defined in a context of a Job, that encapsulates triggers and establishes VCS snapshots. In TeamCity, each Build is independent and can run on its own.
A counterpart to a Job in TeamCity is a Build Chain. TeamCity Build Chain connects a sequence of Builds, linked via the Snapshot Dependency. A trigger is usually defined on the last Build in a chain. All the depended Builds will be triggered automatically and the Snapshot Dependency mechanism will guarantee that all Builds are triggered on the same VCS snapshot.
A TeamCity Build consists of a series of Build Steps. Build Steps line up directly with the host actions in Space Automation. The container step in Space Automation directly equates to a TeamCity Build with a single Build Step.
An important thing to consider during the migration is that you might be able to consolidate multiple Space Automation Steps into a single TeamCity Build. This consolidation is beneficial as it helps reduce the overhead associated with setting up new VMs for each additional Automation Step or TeamCity Build.
There are some characteristics of Space Automation, which might have encouraged you to create multiple steps, may not apply directly to TeamCity due to its different structure:
In Space Automation, each
container
step could only perform a single action. In TeamCity, you can merge these steps into a single Build, since TeamCity Build supports multiple Build Steps running within the same container on the same VM.In Space Automation, there is no out-of-the-box way to run multiple actions wrapped in different containers in a single host step. In TeamCity, each Build Step can be encapsulated in a different container.
In Space Automation, passing parameters between actions isn't possible and it led to extraction of parameter-calculation actions to a separate steps. TeamCity doesn't have this limitation. All parameters generated in TeamCity are immediately available to subsequent Build Steps.
However, if you previously ran steps concurrently in Space Automation, that should remain unchanged in TeamCity by translating those as separate Builds.
Example of migrating a multi-step job
The following demonstrates the migration of a simple multistep job from Space Automation to TeamCity. Note how the last 2 steps run in parallel. The challenge in TeamCity migration lies in the absence of a singular final step that depends on previous steps and will manage trigger definitions. So it might be unclear how to connect all steps into a TeamCity Build Chain.
To resolve this, TeamCity offers a special Build Type known as the Composite Build. This type doesn't perform any tasks on its own but instead, provides place to configure triggers and track other Builds.
The resulting Build Chain in TeamCity will look like this:
Publishing artifacts and dockers images
Space Automation integrates seamlessly with Space Packages. Each job execution provisioned automatically with Automation Service credentials, which by default have the authority to push to any package repository associated with the project that the job is running within. And Docker login occurs before the start of each job automatically, so in any Docker, push or pull works without any additional setup.
When migrating to TeamCity, it becomes necessary to manually establish credentials to support pushing artifacts or container images to Space Packages. In Space, you need to set up a Space Application, on which behalf artifacts or containers images will be pushed. To set it up, follow these steps:
Create new application. Navigate to the Extensions menu and select Applications Installed. Here, create a new application.
Authorize packages access. Switch to the Authorization tab and assign the permissions
Read package repositories
andWrite package repositories
for your desired project.Generate client secret. Go to the Authentication tab to generate a Client Secret. Additionally, you'll need a Client ID which acts as the client's username in most tools.
Store client secret in TeamCity. In the Project Settings, select Generate token for a secure value in the Actions drop-down menu. This approach allows referencing stored secrets in a Kotlin configuration-as-code. See more details on managing tokens in TeamCity.
Publishing Maven artifacts
Maven publish integration in Space Automation might look like the example described here. The crucial part is the following:
Note how the default Automation Service JB_SPACE_CLIENT_ID
and JB_SPACE_CLIENT_SECRET
are used to deploy a package. This build will be converted to the TeamCity format in the following way:
Note that in this example, we use the basic script
step for publishing Maven artifacts to show the most generic principle of how publication secret should be provided to the TeamCity build. In real-life cases it's better to use dedicated TeamCity Step Runners, such as Maven Build Runner.
Publishing Docker images
TeamCity has a special feature that automatically logins to a specified Docker Registry on build start, thus the approach in a TeamCity configuration is very similar to the approach in Space Automation, the only difference is that in TeamCity you have to manually setup credentials for the Docker integration feature.
The following is the Space Automation example of a simple project where the first action builds an application, and the second one builds and pushes the docker image from a Dockerfile located in a repository root.
This example translates to the TeamCity configuration as follows:
Migrating Parameters and Environment Variables
Handling parameters is very similar in Space Automation and TeamCity, though there are some caveats. Let's take a look at the main scenarios.
Passing Parameters into DSL and Environment Variables
Space Automation allows referencing parameters in most string definitions, but environment variables must be explicitly declared. TeamCity follows the same approach, where parameters prefixed with env.
are automatically passed as environment variables. For instance:
Passing Parameters between Steps
In Space Automation, you can create parameters on the fly and pass them between steps. Here's an example where we calculate the build version in the first step and use it in the second one:
There are two ways to translate this job to TeamCity. You can either merge the Space Automation steps into a single TeamCity build, or you can translate each step into a separate build in TeamCity. The parameters mechanism varies slightly between these approaches.
Passing Parameters between TeamCity Build Steps
When passing parameters between TeamCity Build Steps, you only need to reference the newly set parameter in the next steps. Be aware that you need to declare the parameter before starting the build, otherwise the build won't start due to a missing dependency error. The previous example can be translated into single TeamCity build as follows:
Passing Parameters between TeamCity Builds
In Space Automation, all parameters created in previous steps are automatically available in subsequent steps. In TeamCity, parameters need to be explicitly referenced from previous builds using the Dependency Parameters mechanism. The initial example can be translated as follows into multiple TeamCity builds:
Using secrets
Secret management is very similar in Space Automation and TeamCity. In Space Automation, you can add project secrets and reference them in your DSL. In TeamCity, you can also store secret values for a project and reference them in a DSL by a specifically generated token. To store secret in TeamCity, go to Project Settings page and select Generate token for a secure value in the Actions drop-down menu. Consider following this migration example:
Predefined Parameters Mapping
Both Space Automation and TeamCity have a list of predefined parameters that are automatically available in your builds. The most important parameters are mapped below:
Space Automation | TeamCity |
---|---|
{{ run:id }} or JB_SPACE_EXECUTION_ID ENV var |
|
|
|
{{ run:number }} or JB_SPACE_EXECUTION_NUMBER ENV var |
|
{{ run:url }} or JB_SPACE_EXECUTION_URL` ENV var |
|
{{ run:git-checkout.ref }} or JB_SPACE_GIT_BRANCH ENV var |
|
{{ run:git-checkout.commit }} or JB_SPACE_GIT_REVISION ENV var |
|
Migrating kotlinScript
Space Automation provides a unique kotlinScript
feature, that allows seamlessly write parts of a build using Kotlin without any additional setup. While TeamCity also supports native Kotlin script runner, it doesn't provide all the utility and helper wrappers out of the box. This results in Kotlin parts of a build being less convenient to use when working with TeamCity and, in some cases, it would be more advantageous to entirely rewrite them as shell scripts during migration.
There are two ways to define Kotlin Script runner, either by embedding script content directly using kotlinScript
build step, or by referencing <file>.main.kts
file using kotlinFile build step:
Working with Parameters in kotlinScript
Reading and setting parameters in Space Automation is done via the api.parameters helper
. In TeamCity, you can use environment variables to pass parameters and service messages to write parameters. Here's an example:
Using Space SDK
In Space Automation's kotlinScript
, you can use api.space()
to interact with Space. In TeamCity, the same functionality can be accessed by using the Space SDK library.
To work with the Space SDK, you are required to:
Use recent Kotlin version. The Space SDK may require a more recent version of Kotlin than that of the default TeamCity Cloud build agents. See more details in the Customizing Kotlin Version section.
Provide Space Credentials. You need to provide credentials for a Space client to access Space server.
Provide required dependencies. You need to include the following dependency definitions in your Kotlin script:
@file:Repository( "https://maven.pkg.jetbrains.space/public/p/space/maven", // this is where the Space SDK is "https://repo.maven.apache.org/maven2/", // Maven Central, for other dependencies ) @file:DependsOn( // You can find current Space SDK version version by going "Extensions" -> "API Playground". Select "Kotlin SDK" instead of "cURL" and click "Set up dependency...". "org.jetbrains:space-sdk-jvm:2023.3-172916", "io.ktor:ktor-client-java-jvm:2.3.3", "org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3", )A complete example:
/* TeamCity settings.kts */ object KotlinScriptSpaceSDK : BuildType({ name = "Kotlin Script Space SDK" vcs { root(DslContext.settingsRoot) } params { password("env.JB_SPACE_CLIENT_ID", "credentialsJSON:3b51efdf-0d3c-4b12-8db9-e5da56711108") password("env.JB_SPACE_CLIENT_SECRET", "credentialsJSON:9d2a713d-5a75-4900-b392-981c5ebc6afc") } steps { kotlinFile { path = "./use-space-sdk.main.kts" } } }) /* use-space-sdk.main.kts */ @file:Repository( "https://maven.pkg.jetbrains.space/public/p/space/maven", "https://repo.maven.apache.org/maven2/", ) @file:DependsOn( "org.jetbrains:space-sdk-jvm:2023.3-172916", "io.ktor:ktor-client-java-jvm:2.3.3", "org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3", ) import space.jetbrains.api.runtime.* import space.jetbrains.api.runtime.resources.* import space.jetbrains.api.runtime.types.* import kotlinx.coroutines.* val space = SpaceClient( appInstance = SpaceAppInstance( clientId = System.getenv("JB_SPACE_CLIENT_ID") ?: error("JB_SPACE_CLIENT_ID missing"), clientSecret = System.getenv("JB_SPACE_CLIENT_SECRET") ?: error("JB_SPACE_CLIENT_SECRET missing"), spaceServerUrl = "https://myorganizationjetbrains.space", ), auth = SpaceAuth.ClientCredentials(), ) // The SDK uses mostly suspend functions, so we wrap everything in this runBlocking runBlocking { val projectIdentifier = ProjectIdentifier.Key("MY-APPLICATION") val p = space.projects.getProject(projectIdentifier) { name() repos { name() } } println("Project name: ${p.name}") println("Project repositories: ${p.repos.map { it.name }}") }
Running External Processes
In Space Automation, utility methods like api.gradle()
are provided to run external processes. In Space Automation's ScriptApi
, some convenience methods to run other processes like Gradle were provided. TeamCity doesn't provide any special library, so only vanilla Kotlin Scripting API is present and you need to invoke basic JVM APIs to run processes, like ProcessBuilder
. Here's an example of how to define convenient functions and run gradlew process from Kotlin script:
ScripApi Migration Reference
Space Automation | TeamCity |
---|---|
|
|
|
|
|
|
|
|
|
|
| Space-specific. If necessary, pass this as project parameter or hardcode. |
| Space-specific. If necessary, pass this as project parameter or hardcode. |
| Space-specific. If necessary, pass this as project parameter or hardcode. |
| Space-specific. If necessary, pass this as project parameter or hardcode. |
| Space-specific. If necessary, pass this as project parameter or hardcode. |
| See "Using Space SDK" section. |
| See "Using Space SDK" section. |
| See "Using Space SDK" section. |
| See "Using Space SDK" section. |
| See "Running External Processes" section. |
| See "Running External Processes" section. |
| See "Running External Processes" section. |
| Use regular TeamCity artifacts. |
| See "Working with Parameters in |
| See "Working with Parameters in |
Customizing Kotlin Version
By default, TeamCity Cloud agents use older Kotlin version (1.7.10 at the time of writing). You can change the version by installing a different version on the agent and specifying its location in kotlinScript
/ kotlinFile
.