Server Health

The Server Health report contains results of the server inspection for any configuration issues which impact or could potentially impact the performance. Such issues, the so called server health items, are collectively reported by TeamCity on the Server Health page in the Administration area.

The Project Administrator permissions at least are required to see the report.

Scope and Severity

The report enables you to define the analysis scope: you can select to analyze the global configuration or report on project-related items. The scope available to you depends on the level of the View Project permission granted to you. Note that the report is not available for archived projects.

The Server Health analysis also employs the severity rating, depending on the issue impact on the configuration of the system on the whole or an individual project.

Visibility Options

By default, the warning and error level results pertaining to the global configuration will be displayed on the report page as well as at the top of each page in TeamCity.

Besides those, some items will be displayed in-place: depending on the object causing the issue, the server health item will be reported on the Build Configuration, Template or VCS root settings page.

Only active items are displayed on the TeamCity pages. To remove an item from display, use the hide option next to an item on the report page. For global items, this option is available in every server health message.

Hidden items will be removed from the TeamCity pages, and will be displayed on the Server Health page below the active items. Their description will be greyed out.

To return an item to display, use the Show option.

The visibility changes will be listed on the Audit page in the Administration area.

Issue Categories

Health items cover a wide range of server functionality to allow administrators to easily monitor the TeamCity overall status.

The main configuration issue categories are listed below:

Global Configuration Items

TeamCity displays a notification on the availability of the new TeamCity version and a prompt to upgrade. A warning is displayed if any of the licenses are incompatible with this new version. The notification is visible to system administrators only and they can use the link in the "Some Licenses are incompatible" message to quickly navigate to the Licenses page, where all incompatible licenses will have a warning icon.

Agent Configuration

Since TeamCity 10.0, TeamCity displays a notification if agents are not running the recommended Java 8: this report shows all of the agents running under Java earlier than version 1.8.

WebSocket connection issues

The WebSocket protocol is used to get web UI updated for events, running builds updates and statistics counters.

In case of any problems preventing WebSocket connection from working, a warning will be displayed. TeamCity will automatically switch to the legacy update mode (usual HTTP request polling used by TeamCity before version 9.0) and you will be able to continue using TeamCity in this mode.

However, it is recommended to make the following adjustments to benefit from faster web UI updates as well as reduced unnecessary network traffic and latency:

Proxy Server Configuration

If a reverse proxy is used in front of the TeamCity server, it needs to be configured to support the WebSocket protocol.

All URLs used by browsers that do not support the WebSocket connection are listed in the corresponding health report.

BIO Connector Adjustment

If Tomcat is configured to use the BIO connector, the WebSocket protocol is automatically disabled. It is recommended to change the Tomcat Connector settings to use the NIO connector.

Critical Errors

This category shows the following errors:

• errors in project configuration files - occur if the server detects some inconsistency or a broken configuration while it loads configuration files from the disk

• errors raised by the disk space watcher

• warnings from the TeamCity Server Memory Monitor

Database Related Problems

TeamCity will warn you if the server currently uses the internal database. A standalone database is recommended as the storage engine.

As TeamCity does not support Sybase as an external database, a warning message will be displayed if you are using Sybase.

Build Configuration Items

Dependency problems

Since version 10.0.3, TeamCity detects build configurations dependent on a missing build configuration (e.g. on a build configuration deleted after the dependency was configured).

Configurations with Large Build Logs

Large Build Logs (more than 200 MB by default) can reduce the server performance as they could be too heavy to parse and are hard to view in the browser.

The build script can be tuned to print less output if this inspection fails frequently for some Build Configuration.

Inefficient Artifacts Publishing

Since TeamCity 10.0, TeamCity detects a build publishing many small artifact files and suggests publishing them as a single .zip archive.

VCS Root Related Problems

Unused VCS Roots

TeamCity will show you the VCS roots defined in a project and will offer to delete the unused roots.

Similar VCS roots

TeamCity qualifies VCS roots as identical when their major settings (e.g. URLs, branch settings) are the same even if some of their settings (e.g. user name, password) are different.

The report will show you identical roots and will leave it up to you whether to merge them or not.

Similar VCS Root Usages (AKA instances)

You can define values for VCS root settings or use parameter references to various parameters defined at different levels.

When the referenced VCS roots parameters are resolved to the same values as the values defined, such cases will be reported as identical VCS root usages.

The general recommendation is to use parameter references for root settings, thus optimizing the amount of VCS roots.

Trigger Rules for Unattached VCS roots

TeamCity displays a warning if a rule of a VCS Trigger or Schedule Trigger references a VCS root which is not attached to any build configuration.

Redundant Trigger

The report will show cases when a build trigger is redundant, for example:

• There are two build configurations A and B

• A snapshot depends on B

• Both have VCS triggers, A with the Trigger on changes in snapshot dependencies option enabled.

In this case, the VCS trigger in B is redundant and causes builds of A to be put into the queue several times.

Multiple identical build triggers

The warning is displayed if there are two or more enabled triggers of the same type with identical sets of parameter values. Disabled triggers are not taken into account.

Effective Quiet Period is Bigger Than Specified

When a VCS trigger for a build configuration has a quiet period, TeamCity will wait the specified time after the last detected change before triggering the build. During this time, all VCS Roots which affect this build configuration are checked for changes. If other VCS Roots have checking for changes interval bigger than the quiet period, the effective quiet period will be equal to the maximum checking for changes interval of the involved VCS Roots (it could be a VCS Roots from the dependencies).

Possible fix could be one of the following:

• Use commit hooks to trigger checking for changes operations

• Increase quiet period in the VCS trigger so it is larger than checking for changes interval of the related VCS Roots

• Reduce checking for changes interval for the problematic VCS Roots

VCS checkout 

Possible Frequent Clean Checkout 

This section of the report will show possible frequent clean checkout, which may be caused by the following two reasons:

Custom Checkout Directory 

Build configurations having different VCS settings but the same custom checkout directory may lead to frequent clean checkouts, slowing down the performance and hindering the consistency of incremental sources updates.

Build Files Cleaner (Swabra) Settings 

Enabling the Build files cleaner (Swabra) build feature in several build configurations may cause extra clean checkouts. This may happen if builds from these configurations alternately run on the same agents and have identical Version Control Settings or the same custom checkout directory specified.

Possible frequent clean checkout (Swabra case) server health report shows such incorrectly set up configurations grouped by Swabra settings.

Optimal recommended checkout

Since TeamCity 10.0, a report is shown for large server-side patches with a recommendation to switch to the agent-side checkout.

Default auto-checkout on agent 

Since TeamCity 10.0, if the default agent-side checkout is not possible, TeamCity will display a corresponding health report item and will use the server-side checkout.

Integrations-related Items

• Since TeamCity 10.0, if a project or build configuration has secured parameters and is configured to build GitHub pull requests, this report will raise the warning, because malicious code submitted via the pull request can obtain these secured parameters

• Since TeamCity 10.0, if a VCS root points to GitHub or Bitbucket, a suggestion to configure a corresponding issue tracker is displayed.

Agents Health

agentUpgradeFailedSome agents cannot upgrade

The report helps to find agents which failed to upgrade. Build agent logs should help identify the cause of the issue.

Cloud Agents

If a user removes an image from a profile, a warning is displayed that the instances already started by TeamCity will not be automatically stopped.

Unused Build Agents

The report is displayed for the agents not used for 3 days and more, if

• you have more than 3 agents in your environment

• your agents have been registered for more than 3 days

• if the builds were run on the server during these 3 days

Incorrect proxying server configuration

The report displays detected misconfiguration of the proxy server that is used to access the TeamCity web interface.

See our recommendations how to set up a proxy server with TeamCity.

Suggested Settings

TeamCity analyzes the current settings of a build configuration and suggests additional options, e.g. adding a VCS trigger, a build step, etc. Besides the server health report, the suggestions for a specific build configuration are shown right on the configuration settings pages.

Extensibility

The default Server Health report provided by TeamCity might cover either too many, or not all the items required by you. Depending on your infrastructure, configuration, performance aspects, etc. that you need to analyze, a custom Server Health report can be needed. TeamCity enables you to write a plugin which will report on specific items.