Create Pipeline

This tutorial assumes that you have already started your TeamCity instance. We also suggest that you learn how to run a simple build.

A build chain or a pipeline is a sequence of consecutively run build configurations. In TeamCity, these configurations can belong to different projects, and pass files (artifacts) from one build to another.

This tutorial walks you through setting up the following project:

The pipeline above cycles through the following stages:

1 — Building the sample Spring Boot application 2 — Creating a Docker image for this app 3 and 4 — Running two sets of parallel tests

Import Sample Project

This tutorial uses a sample project with five separate build configurations which we are about to connect. To follow the tutorial, you can use the sample repository and repeat the steps below on your TeamCity server.

To import the sample project:

Go to Administration | Projects and click Create project.
In the Repository URL field, enter the sample repo URL and click Proceed.
TeamCity will detect the .teamcity/settings.kts file, which corresponds to a TeamCity project's settings saved in Kotlin format. Leave the default settings and proceed.
Choose import without synchronization when prompted. Otherwise, your project can become non-editable via TeamCity UI.
TeamCity will import the sample project's settings and redirect you to its General Settings page. Here, you can scroll a bit and see the TodoBackend subproject. Click it to view all the created build configurations.

Now we can start chaining them!

Configure Snapshot Dependency

The TodoApp build configuration compiles a .jar application and publishes it to the build/libs/ directory. The TodoImage configuration has to build a Docker image out of this .jar.

To create a synchronized pipeline, or chain, you need to connect these builds with a snapshot dependency. We use the word snapshot to describe a specific state of the project's sources, or basically a specific commit. If you connect multiple builds with snapshot dependencies, they are guaranteed to process the same sources.

A dependency determines how one build depends on another, and thus is created in the settings of the dependent build. In our case, it's TodoImage. Let's go to its settings and add a snapshot dependency:

Open the Dependencies settings tab (you might need to click Show more to display this item) and click Add new snapshot dependency.
Select TodoApp as a build config to depend on.
Leave the default settings and save the dependency.

Configure Artifact Dependency

To pass the .jar from one configuration to another, we need to create an artifact dependency between them. This way, when each new TodoApp build finishes and produces an artifact, TeamCity will use this artifact in the following TodoImage build.

To add an artifact dependency in TodoImage:

Open the Dependencies settings tab and click Add new artifact dependency.
Select TodoApp as a build configuration to depend on.
Choose to get artifacts from the build from the same chain.
In Artifacts rules, specify that we want to import the specific artifact as todo.jar — enter todo.jar => build/libs/todo.jar.

You can read about patterns of artifact rules and other details related to artifact dependencies here.

It is important to remember that a build chain is a sequence of builds connected with snapshot dependencies. Some of the builds may also be connected with artifact dependencies, but this is not a mandatory condition.

Run Simple Chain

At this point, the first two builds are already chained together, and you can run your first chain. Note that to compose a Docker image, a TeamCity agent needs to have Docker installed and running on its machine, so make sure to install it in advance.

When you run any build from a chain, whether it's the last one or medium one, TeamCity gathers all the other chained builds into a sequence, according to their dependencies. As you saw on our sample chain's scheme, TodoImage always runs after TodoApp; Test1 and Test2 start only after TodoImage finishes and run in parallel to each other.

Let's run the TodoImage build with the Run button. Go to Project Home to see how TeamCity automatically runs a new TodoApp build first and, after its finish, launches the following TodoImage build. As a result of this chain, TeamCity will produce a Docker image according to the Dockerfile.

To view the statuses of all chained builds, go to Build Configuration Home of any chained build and open the Dependencies tab of Build Results:

The Sakura UI offers three modes of representing dependencies: timeline, list, and chain. When you create more advanced chains, try monitoring them with each of these modes and choose the most convenient one for your tasks.

Configure Trigger and Checkout Rules

You already know the basics of creating chains, or pipelines, in TeamCity. However, to become effective and production-ready, a chain needs to be automated further.

Add VCS Trigger

As we explained in the first build guide, TeamCity offers a variety of build triggers. Triggers run builds automatically if certain conditions are satisfied. The most popular type of trigger is a VCS trigger, and that's the trigger we will use in this tutorial.

A VCS trigger starts a new build whenever it detects changes in the project's sources. You can define what repository and even the exact files it will monitor. Let's add a trigger in the TodoImage settings:

Open the Triggers page and click Add new trigger.
Open advanced options and then enable the option to trigger a build on changes in snapshot dependencies. This way, this trigger will also react to the changes relevant to the TodoApp config.

It's often convenient to add a single trigger at the end of the chain and enable this option to consider the previous builds. Whenever you want to change the triggering settings, you will be able to do this in one place.
Leave other settings default and save the trigger.

Now, if you change the sample project's code, TeamCity will detect it and run the chain.

Restrict Checkout Scope

Every chain stage is responsible for its own task. And in some cases, different build configurations need to monitor different parts of the source project. You can configure custom checkout rules for a configuration, and its builds will only be triggered by changes that satisfy these rules.

For example, let's exclude Dockerfile from the checkout scope of TodoApp. This way, when you change the Docker settings, only TodoImage will be triggered. Without such restrictions, TeamCity would start both of these builds per any change in the source repo, which will waste resources and could cause a mess.

You can define the scope of monitored sources in a build configuration's Version Control Settings:

Opposite our only VCS root, click Edit checkout rules.
Enter the -:docker rule to exclude the docker directory from the checkout scope. In the future, use this syntax to specify these rules.
Save the rules.

Complete Chain with Tests

Builds in a chain can run in parallel. Let's explore this on the example of tests.

The project's General Settings list three other build configurations: Test1, Test2, and TestReport. According to our target scheme, Test1 and Test2 should depend on TodoImage, which means you need to create a snapshot dependency on it in both of these builds. If there are at least two suitable build agents on your server, TeamCity will be able to run these builds in parallel to each other; otherwise, it will start one after another.

As you might remember, our VCS trigger in TodoImage considers only preceding builds (that is TodoApp) and won't be able to launch tests. We can add triggers in both test builds, but TeamCity provides a more straightforward option — creating an extra composite build, that is TestReport. A composite build can run without an agent and accumulate results of the preceding builds in a chain. Moreover, it will aggregate and report the results of Test1 and Test2 in one place. Just what we need.

So, to complete this tutorial:

Add snapshot dependencies from Test1 and Test2 on TodoImage and from TestReport on Test1 and Test2.
Add a VCS trigger in TestReport, similarly to how we did it for TodoImage. After that, you can safely remove the trigger from TodoImage, as the new one will trigger the whole chain.

As Test2 contains a failing test, you will see that TestReport will fail as well. Expand any test or build problem to quickly preview the related part of the build log.

The build chain mechanism in TeamCity is very flexible and designed to satisfy the needs of every project. You will also notice that build chains are much easier to monitor than scattered builds. Detailed statuses of all chained builds are displayed in the Dependencies tab of Build Results.

Proceed with our getting started tutorials to learn about the other type of build configuration — deployment.

Takeaway

A build chain is a sequence of builds connected with snapshot dependencies. A snapshot corresponds to a certain commit in the source code.
Builds in a chain can pass artifacts to each other if you configure artifact dependencies between them.
Builds in a chain can run sequentially or in parallel. You can create chains with dozens of builds, and only the number of available build agents limits how many of them can run simultaneously.
When any chained build is triggered, TeamCity composes and runs the whole chain from start to finish. As triggers can only consider preceding builds, it is convenient to add one VCS trigger in the very last build of a chain.
You can limit what scopes of the source projects are relevant to each build configuration. This prevents excessive build runs.
You can create a logical composite configuration to gather the results of multiple dependency builds. Such a configuration doesn't require a build agent and only serves as an aggregator.

Last modified: 01 October 2024