Git

Native Git for VCS-related operations on the server

TeamCity can now use native Git as the default option for Git operations on the server. Switching to native Git improves the performance of the checking for changes operations on the server in comparison with the previously used JGit implementation. It also fixes a number of issues related to large Git repositories.

Before switching, make sure a native Git client version 2.29 or later is installed on your server machine, and the path to its executable is specified in the PATH environment variable. Alternatively, you can set the full path to the executable via the teamcity.server.git.executable.path internal property (no server restart is required). On Windows, remember to use double backslashes in the path.

To switch your TeamCity server to native Git, go to Administration | Diagnostics and open the Git tab. Here you can test the connection via native Git in any VCS root on your server. If you choose to test all VCS roots, TeamCity will check whether they successfully connect via JGit and then test their connection via native Git. This measure helps ensure that none of your pipelines will break after switching to native Git. If the connection test is successful, you can enable the native Git support on your server(s).

General Settings

• Fetch URL — the URL of the remote Git repository used for fetching data from the repository.

  You can override the fetch URL for individual agents to allow them to use a closer proxy instead of the original VCS hosting. To do so, open a required agent's conf/buildAgent.properties file and add the redirection rule as follows: teamcity.git.fetchUrlMapping.<name> = <source URL> => <target URL>. For example:

  teamcity.git.fetchUrlMapping.firstrule = https://example.com/org/test.git => http://proxy.com/test.git

  You can use partial addresses and the asterisk (*) wildcard to set up proxies for all fetch URLs that match the pattern. For example, the following rule allows an agent to use the http://proxy.com/test/test.git URL instead of the original https://example.com/org/test/test.git:

  teamcity.git.fetchUrlMapping.secondrule = https://example.com/org/* => http://proxy.com/

  Note that along with a fetch URL a VCS root also stores authentication settings required to access a repository. As such, replacing fetch URL is in effect only for VCS roots that use Anonymous or Private key authentication modes. Other modes do not guarantee an agent will be able to access a repository (for example, a refreshable token issued for a source VCS will not be accepted by a proxy hosting).

• Push URL — the URL of the target remote Git repository used for pushing annotated tags created via VCS labeling build feature to the remote repository. If blank, the fetch URL is used.

• Default branch - the default branch. Parameter references are supported here. Default value is refs/heads/master.

• Branch specification — lists the patterns for branch names, required for feature branches support. The matched branches are monitored for changes in addition to the default branch. The syntax is similar to checkout rules: +|-:branch_name, where branch_name is specific to the VCS, i.e. refs/heads/ in Git (with the optional * placeholder).

• Use tags as branches — allows monitoring / checking out git tags as branches making branch specification match tag names as well as branches (for example, +|-:refs/tags/<tag_name>). By default, tags are ignored.

• Username style — defines a way TeamCity reports username for a VCS change. Changing the username style will affect only newly collected changes. Old changes will continue to be stored with the style that was active at the time of collecting changes.

• Submodules — specifies whether submodule repositories should be checked out. For multi-level submodule setup, you can choose between the "Checkout" (recursively fetches the entire repository tree) and "Non-recursive checkout" (fetches only those submodules that are directly referenced by the main repository) modes. In the diagram below, submodules A and B are available in both modes, whereas submodules C and D require the recursive "Checkout" mode.

  Main_Repo | |_________ Tier 1 Submodule A | | | |_________ Tier 2 Submodule C | |_________ Tier 1 Submodule B | |_________ Tier 2 Submodule D

  Submodule repositories should either not require authentication or use the same protocol and accept the same authentication as configured in the VCS root. Otherwise, you need to configure additional credentials to access these repositories as described in the LFS and Submodules Support section.

• Username for tags/merge — a custom username used for labeling.

Authentication Settings

• Anonymous — select this option to clone a repository with anonymous read access.

• Password / personal access token — specify a valid username (if there is no username in the clone URL, the username specified in this field overrides the username from the URL) and a password to be used to clone the repository.
  For the agent-side checkout, it is supported only if Git 1.7.3\+ client is installed on the agent. See TW-18711.
  For Git hosted from Team Foundation Server 2013, specify NTLM credentials here.

  You can use a personal access token instead of a password to authenticate in GitHub, Azure DevOps Services, GitLab, JetBrains Space, and Bitbucket. When connecting to Azure DevOps, remember to set the Code access scope to Code (read) / Code (read and write) for versioned settings in the repositories you are about to access from TeamCity.

  When using an existing Bitbucket Cloud, Bitbucket Server, GitLab or Azure DevOps Services connection to create a VCS Root, TeamCity will use a refreshable token instead of the password.

• Refreshable token — if a VCS root that fetches data from a GitHub, GitHub App, Bitbucket Server, Bitbucket Cloud, Azure DevOps, GitLab, or JetBrains Space was configured using a TeamCity connection, refreshable tokens are enabled by default.

  Refreshable access tokens are short-lived tokens acquired by TeamCity from a required VCS provider via existing OAuth connections (as opposed to static PAT tokens issued manually by users on a VCS hosting side). See the following article for more information on generating and using refreshable tokens: Manage Refreshable Access Tokens.

  You can specify a username if the clone (fetch) URL does not have it. The username specified here overrides the username from the URL.

• Private Key — valid only for SSH protocol. A private key must be in the OpenSSH format.

  Select one of the options from the Private Key list and specify a valid username (if there is no username in the clone URL; the username specified here overrides the username from the URL).
  Available Private Key options:

  • Uploaded Key — select this option to utilize the key(s) uploaded to the project.

  • Default Private Key — select this option to utilize the keys available on the file system in the default locations used by common ssh tools: the mapping specified in <USER_HOME>/.ssh/config if the file exists or the private key file <USER_HOME>/.ssh/id_rsa (the files are required to be present on the server and also on the agent if the agent-side checkout is used).

  • Custom Private Key — supported only for server-side checkout. Fill the Private Key Path field with an absolute path to the private key file on the server machine. If the key is encrypted, specify the passphrase in the corresponding field.

For all available options to connect to GitHub, see the comment.

Server Settings

These are the settings used in case of the server-side checkout.

Agent Settings

These are the settings used in case of the agent-side checkout.
Note that the agent-side checkout has limited support for SSH. The only supported authentication methods are "Default Private Key" and "Uploaded Private Key".
If you plan to use the agent-side checkout, you need to have Git 1.6.4+ installed on the agents.

Configuring Git Garbage Collection on Server

TeamCity server maintains a local clone for every Git repository used in the VCS roots configured on the server. Since the server performs fetch in those clones many times a day, the clone needs regular optimization to maintain predictable performance. If the Git garbage collection for the clone was not run for a long time, the process of collecting changes may slow down or start to report memory-related errors.
TeamCity can automatically run git gc periodically when the native Git client can be found on the server. Inability to run Git GC results in a related health report.

To fix the warning / meet automatic git gc requirements, perform the following:

1. Install a native Git client manually on the TeamCity server.

2. Specify the path to the Git executable:

   • Add the directory with the executable to the PATH environment variable and restart the server, or

   • Set the full path to the executable in the teamcity.server.git.executable.path internal property without the server restart. On Windows, remember to use double backslashes in the path.

When TeamCity runs Git garbage collection, the details are logged into the teamcity-cleanup.log. If git garbage collection fails, a corresponding warning is displayed.

TeamCity executes Git garbage collection until the total time doesn't exceed 5 hours quota; the quota can be changed using the teamcity.server.git.gc.quota.minutes internal property.
Git garbage collection is executed every night at 2 AM. This can be changed by specifying the internal property with a cron expression like this: teamcity.git.cleanupCron=0 0 2 * * ?. If the git gc process works slowly and cannot be completed in the allotted time, check the git-repack configuration in the default Git configuration files (for example, you can increase --window-memory to improve the git gc performance).

If the local Git clones need some kind of manual maintenance, you can find them under the <TeamCity Data Directory>/system/caches/git directory. The map file in the directory contains mapping between the repository URL and the subdirectory storing the bare clone of the repository.

LFS and Submodules Support

Internal Properties

For Git VCS, it is possible to configure the following internal properties:

Build parameters for Git:

Agent-side checkout rules limitations

The Git plugin uses git sparse-checkout to check out Git files on an agent. The plugin is able to perform only simple file mapping operations which limits the set of supported VCS checkout rules for Git.

The following rules are supported:

Note that rules must not remap files. That is, the following rule is not supported: +:dirA/dirA1 => dirA/dirA2.

If you specify multiple checkout rules for one root, make sure their checkout directories (the right part of the rule) have a common parent directory ([prefix/]). Only the rules +:dirA => [prefix/]dirA are supported for agent-side checkout, and the [prefix/] must be the same for all rules.

For example:

Note that the following rule is not supported: +:dirA=>[prefix/]dirA/postfix. If you append [/postfix] to the checkout directory path and the configuration's VCS checkout mode is set to "Always checkout files on agent", new builds will be unable to start.

Known Issues

• java.lang.OutOfMemoryError while fetching from a repository in case teamcity.git.fetch.process.max.memory property is specified. Since TeamCity 2019.2, the recommended approach is to disable this property thus delegating the automatic memory management to TeamCity.

• TeamCity running as a Windows service cannot access a network mapped drives, so you cannot work with git repositories located on such drives. To make this work, run TeamCity using teamcity-server.bat.

• Inflation using streams in JGit prevents OutOfMemoryError, but might get time-consuming (see the TW-14947 issue related to the problem). If you notice a similar behavior, try to increase teamcity.git.stream.file.threshold.mb. Additionally, it is recommended to increase the overall amount of memory dedicated for TeamCity to prevent OutOfMemoryError.