TeamCity Cloud
 
2022.12
Get TeamCity Cloud
You are viewing the documentation for an earlier version of TeamCity Cloud.
  1. Managing Build Configurations
  2. Creating and Editing Build Configurations
  3. Configuring Build Steps
  4. .NET
  5. Deprecated .NET Runners
  6. Visual Studio Tests

Visual Studio Tests

Edit page Last modified: 17 January 2022

note

Since TeamCity 2019.2.3, we have stopped providing active support for the Visual Studio Tests runner. Please use the .NET runner with the vstest command instead. It offers the same scope of functionality and adds new features such as Docker support.

For more details on migration, refer to the .NET runner description.

The Visual Studio Tests build runner integrates functionality of the MSTest framework and VSTest console runner. Support for both frameworks enables TeamCity to execute tests and automatically import their test results.

The Visual Studio Test runner requires Visual Studio Test Agent or Microsoft Visual Studio installed on the build agent.

Visual Studio Tests runner settings

Option

Description

Test engine type

Select the tool used to run tests: VSTest or MSTest.

Test engine version

Select the version of the tool from the drop-down menu. By default, the available VSTest and MSTest installations are autodetected by TeamCity:

  • MSTest versions 2005-2017 are supported

  • VSTest versions 2012-2017 are supported

You can specify a custom path to the test runner here as well. TeamCity parameters are supported.

tip

You can specify the agent property teamcity.dotnet.vstest.VS_VERSION.install.dir pointing to a VSTest distribution path and use it here, where VS_VERSION can be 11.0, 12.0, 14.0, 15.0, or 16.0. The property will have priority over registry values.

Test file names

This field is mandatory for VSTest and optional for MSTest.
Specify the new-line separated list of paths to assemblies to run tests on in the included assemblies list. Exclude assemblies from the test run by specifying paths to them in the corresponding field. Wildcards are supported.
Paths to the assemblies must be relative to the build checkout directory.

Run configuration file

(Optional) Specify the typical .runsettings file for VSTest or .testsettings file for MSTtest. You can use the file browser VCS-browserIcon.png.

Additional command line parameters

Enter additional command line parameters for the selected test engine.
Note that tests run with MSTest are not reported on-the-fly.
The Microsoft Developer Network lists the available options for VSTest and MSTest.

The rest of settings will vary depending on the engine to run tests with:

VSTest Settings

Option

Description

Target platform

Select the platform bitness. Note that specifying x64 target platform will force the vstest.console process to be run in the isolated mode

Framework

If the default is specified, vstest.console will select the target framework automatically. You can also choose the .NET platform manually using the drop-down.

Test names

(optional) Of all tests discovered in the included assemblies, only the tests with the names matching the provided values will be run. For multiple values, use new line.

If the field is empty, all tests will be run. See details in the Microsoft documentation. Cannot be used with the option below.

Test case filter

Run tests that match the given expression. See details in the Microsoft documentation. Cannot be used with the option above.

Run in isolation

Run the tests in an isolated process

Use real-time test reporting

  • If this checkbox is selected, a custom TeamCity test logger will be used for real-time reporting. See the next section for details.

  • If not selected, VSTest will report tests to TeamCity after the run (the discovery of custom logger will not be attempted).

note

When using this option, it is recommended to check the number of tests in the produced trx file against the build log. In case of incosistencies, switch to the custom logger.

Custom test logger

VSTest.Console supports custom loggers, that is libraries that can handle events that occur when tests are being executed.
TeamCity has a custom logger that provides real-time test reporting. The logger must be installed manually on the agent machine, as it requires dlls to be copied to the Extensions directory of the VSTest.Console. No agent restart is needed when the custom logger is installed.

To install the custom logger:

1. Download the custom logger.

2. Extract the contents of the downloaded archive on the agent machine to the following path:

  • for VisualStudio 2019 update:

PROGRAM_FILES(x86)\Microsoft Visual Studio\2019<Edition>\Common7\IDE\Extensions\TestPlatform\Extensions

  • for VisualStudio 2017 update 5 onwards:

PROGRAM_FILES(x86)\Microsoft Visual Studio\2017<Edition>\Common7\IDE\Extensions\TestPlatform\Extensions

  • for VisualStudio 2017 up to update 4:

PROGRAM_FILES(x86)\Microsoft Visual Studio\2017<Edition>\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions

  • for VisualStudio 2015:

PROGRAM_FILES\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions

  • for VisualStudio 2013:

PROGRAM_FILES\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions

  • for VisualStudio 2012:

PROGRAM_FILES\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions

3. Check that the custom logger was installed correctly by executing vstest.console.exe /ListLoggers in the console on the agent machine. If the logger was installed correctly, you will see the logger with FriendlyName TeamCity listed:

VSTest.TeamCityLogger.TeamCityLogger
Uri: logger://TeamCityLogger
FriendlyName: TeamCity{info}

MSTest settings

Option

Description

MSTest metadata

Enter a value for /testmetadata:file argument. See details in the Microsoft documentation.

Testlist from metadata to run

Edit the Testlist. Every line will be translated into /testlist:line argument. See details in the Microsoft documentation.

Test

Names of individual tests to run. This option will be translated into the series of /test: arguments. See details in the Microsoft documentation.

Unique

Run the test only if one unique match is found for any specified test in test section.

Results file

Enter a value for the /resultsfile:file_name command-line argument. Parameter references are supported. It is recommended to leave the field blank to generate a temporary \*.trx file in the build temporary directory. Note that the directory may be cleaned between the builds.

To save the test run results to a named non-default file, enter a value for the /resultsfile:file_name command-line argument.

  • If the path specified is relative, it will be relative to the build checkout directory. If the specified file already exists in the checkout directory, the build agent will attempt to delete the file. If the agent fails to delete it, the build will fail.
    To create a *.trx report in the build temporary directory, use %system.teamcity.build.tempDir%.

note

If you need to generate your results file in the checkout directory, consider adding the Swabra build feature to your configuration.

  • If the path specified is absolute, it will be used "as is".