Storing Build Artifacts in Amazon S3

TeamCity comes bundled with the Amazon S3 Artifact Storage plugin which allows storing build artifacts in an Amazon S3 bucket.

It is possible to replace the TeamCity built-in artifacts' storage with Amazon S3 at a project level. When an S3 artifacts storage is configured, it:

allows uploading to, downloading, and removing artifacts from S3;
handles resolution of artifact dependencies as well as clean-up of artifacts;
displays artifacts located externally in the TeamCity UI.

Migrating Artifacts To a Different Storage

TeamCity provides a command-line tool dedicated to automatic migration of build artifacts from one storage to another. To get the tool, go to Project Settings | Artifacts Storage and use the Download artifacts migration tool link in the top right corner of the page. Currently, the tool supports migration from the built-in storage to Amazon S3. We're working on supporting other cloud storage options as well.

Configuring Amazon S3 Artifacts Storage

To enable an external artifact storage in an AWS S3 bucket:

Open Project Settings | Artifacts Storage. The built-in TeamCity artifacts storage is displayed by default and marked as active.
Click Add new storage. Select S3 Storage as the storage type.
Storage ID is filled out automatically. You can modify it with your own unique ID.
Enter an optional name for your storage.
Select the AWS environment and provide the required settings.
Provide your AWS security credentials.
Specify an existing S3 bucket to store artifacts.
Save your settings.

The configured S3 storage will appear on the Artifacts Storage page. To enable it in this project, change its state to Active.

Now, new artifacts produced by builds of this project and its subprojects will be stored in the specified AWS S3 bucket.

Path Prefix

You can set an S3 path prefix. This allows using the same S3 bucket for all TeamCity projects and configure prefix-based permissions.

Virtual Host Addressing

You can enable the virtual host addressing for S3 buckets. Currently, both hosted-style and path-style requests are supported by TeamCity. Note that Amazon stopped supporting path-style access for new buckets since September 2020.

Permissions

When the "Use Pre-Signed URLs for upload" option is enabled, the provided AWS credentials or IAM role on the TeamCity server should have permissions: DeleteObject, ListAllMyBuckets, GetBucketLocation, GetObject, ListBucket, PutObject.

When the "Use Pre-Signed URLs for upload" option is disabled:

the provided AWS credentials or IAM role on the TeamCity server should have permissions: DeleteObject, ListAllMyBuckets, GetBucketLocation, GetObject
either AWS credentials should be specified and have ListBucket, PutObject permissions, or IAM role on all the TeamCity agents should have permissions: ListBucket, PutObject

Multipart Upload

To optimize the upload of large files to Amazon S3, you can initiate multipart upload instead of regular upload. To do this, set the multipart upload threshold in the Connection Settings block. The minimum allowed value is 5MB. Supported suffixes: KB, MB, GB, TB. If you leave this field empty, multipart upload will be initiated automatically for all files larger than 8 MB (8MB is the default value).

Additionally, you can configure the maximum allowed size of each uploaded file part. The minimum value is 5MB. If left empty, TeamCity will use 8MB as the default value.

Transferring Artifacts via CloudFront

Amazon CloudFront is a content delivery network that offers low latency and high transfer speeds. Enabling its support for an S3 storage will allow TeamCity to transfer artifacts through the closest CloudFront server. If your S3 bucket is located in a different region than your TeamCity infrastructure, this could significantly speed up the artifacts' upload/download and reduce expenses.

Prerequisites

TeamCity can set up CloudFront integration for you, or you can set up all the settings manually.

The CloudFront integration requires configuring:

an OAI user
two CloudFront distributions, one for uploading and downloading artifacts.
a trusted key group
an SSH-2 RSA key pair (public key + private key) in PEM format

CloudFront Settings

To enable the CloudFront support for the current S3 bucket, activate the Use CloudFront to transport artifacts option in the storage settings.

In each of the dropdowns, Distribution for uploads and Distribution for downloads, choose an available distribution if it was manually created in your Amazon profile, or click to configure settings automatically.

For Cloudfront settings to work properly TeamCity needs new permissions:

cloudfront:ListDistributions
cloudfront:ListKeyGroups
cloudfront:ListPublicKeys

Automatic CloudFront Setup

TeamCity can configure the settings automatically. This involves:

Generating a key pair and uploading a public key to CloudFront.
Creating a new key group in CloudFront.
Creating two new distributions with:
- the Use all edge locations price class.
- a new Origin Access Identity that can access the current bucket.
- the default behaviour defining
  - allowed HTTP methods for uploading artifacts: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE, for downloading: GET, HEAD, OPTIONS;
  - a new key group with the viewer access;
  - a custom cache policy that allows passing all query strings.
Adding a new policy to an S3 bucket to allow the new distributions access to it. See the policy example.

Automatic setup requires giving Teamcity additional permissions:

cloudfront:CreateDistribution
cloudfront:CreateKeyGroup
cloudfront:CreatePublicKey
cloudfront:CreateOriginRequestPolicy
cloudfront:CreateCloudFrontOriginAccessIdentity
cloudfront:CreateCachePolicy
cloudfront:DeleteKeyGroup
cloudfront:DeletePublicKey
cloudfront:ListCloudFrontOriginAccessIdentities
cloudfront:ListCachePolicies
cloudfront:ListOriginRequestPolicies
cloudfront:GetDistribution
cloudfront:GetPublicKey
s3:GetBucketPolicy
s3:PutBucketPolicy

Example policy providing all necessary permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "cloudfront:CreatePublicKey",
                "cloudfront:CreateOriginRequestPolicy",
                "cloudfront:ListCloudFrontOriginAccessIdentities",
                "cloudfront:DeleteKeyGroup",
                "cloudfront:GetPublicKey",
                "cloudfront:ListCachePolicies",
                "cloudfront:CreateDistribution",
                "cloudfront:ListOriginRequestPolicies",
                "cloudfront:DeletePublicKey",
                "cloudfront:CreateCloudFrontOriginAccessIdentity",
                "cloudfront:CreateKeyGroup",
                "cloudfront:CreateCachePolicy",
                "cloudfront:GetDistribution",
                "cloudfront:ListPublicKeys",
                "s3:ListAllMyBuckets",
                "cloudfront:ListKeyGroups",
                "cloudfront:ListDistributions"
            ],
            "Resource": "*"
        },
        {
            "Sid": "2",
            "Effect": "Allow",
            "Action": [
                "s3:PutBucketPolicy",
                "s3:GetBucketPolicy"
            ],
            "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>"
        }
    ]
}

Manual CloudFront Setup

For security reasons, we recommend configuring two separate distributions for uploading and downloading artifacts. For each distribution:

Generate a key pair in SSH-2 RSA key format.
Upload the public key from the pair to CloudFront.
Add a new key group in CloudFront and add the created public key to this group.
Create a new cache policy with Cache key settings | Query strings set to All.
If you use a private bucket, create a new OAI user.
Create a distribution and attach your key group to it:
- Make sure to choose the same S3 bucket as specified in TeamCity.
- Allowed HTTP methods for uploading artifacts GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE; for downloading GET, HEAD, OPTIONS
- Restrict viewer access: yes
- Trusted authorization type: trusted key groups
- Cache key and origin requests: Cache policy and origin request policy
- For private buckets, enable the use OAI option and configure OAI with the following setting:
  - Bucket policy: No, I will update the bucket policy
- For public buckets, disable the Block public access option.
Add a new policy to your S3 bucket. See the policy example.

When configured, the distributions should automatically appear in the Distribution for uploads and Distribution for downloads drop-down menus.

Select the target CloudFront distribution.
In CloudFront public key, select the public key associated with this distribution.
In Private SSH key, upload the private key from the pair.
Save the storage settings.

S3 Policy Example

For accessing a private bucket with OAI:

{
"Sid": "1",
"Effect": "Allow",
"Principal": {
      "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <OAI ID"
  },
  "Action": [
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject"
  ],
  "Resource": "arn:aws:s3:::<S3 bucket name/*"
}

For accessing a public bucket:

{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
  "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion"
    ],
  "Resource": "arn:aws:s3::<BUCKET_NAME>/*"
}

Last modified: 16 March 2023