Visual Studio Tests
The Visual Studio Tests runner, available since TeamCity 9.1, integrates MSTest runner and VSTest console runner formerly provided as an external plugin. The Visual Studio Tests runner provides support for both frameworks enabling TeamCity to execute tests and automatically import their test results.
Prior to TeamCity 9.1.5, the runner requires the appropriate Microsoft Visual Studio edition installed on the build agent. Since TeamCity 9.1.5, the Visual Studio Test Runner supports Visual Studio Test Agent installed on the agent machine (Microsoft Visual Studio is no longer required).
Visual Studio Tests runner settings
Option | Description |
---|---|
Test engine type | Select the tool used to run tests: VSTest or MSTest. |
Test engine version | Select the version of the tool from the drop-down. By default, the available VSTest and MSTest installations are autodetected by TeamCity. MSTest versions 2005 through 2015 are supported, VSTest 2012, 2013 and 2015 are supported. You can specify a custom path to the test runner here as well. TeamCity parameters are supported. |
Test file names | This field is mandatory for VSTest and optional for MSTest. Specify the new-line separated list of paths to assemblies to run tests on in the included assemblies list. Exclude assemblies from the test run by specifying paths to them in the corresponding field. Wildcards are supported. Paths to the assemblies must be relative to the build checkout directory. |
Run configuration file | (Optional) Specify the t ypical |
Additional command line parameters | Enter additional command line parameters for the selected test engine. Note that tests run with MSTest are not reported on-the-fly. The Microsoft Developer Network lists the available options for VSTest and MSTest. |
The rest of settings will vary depending on the engine to run tests with:
VSTest Settings
Option | Description |
---|---|
Target platform | Select the platform bitness. Note that specifying x64 target platform will force the |
Framework | If the default is specified, vstest.console will select the target framework automatically. You can also choose the .Net platform manually using the drop-down. |
Test names | (optional) O f all tests discovered in the included assemblies, only the tests with the names matching the provided values will be run. For multiple values, use new line. If the field is empty, all tests will be run. See details in the Microsoft documentation. Cannot be used with the option below. |
Test case filter | Run tests that match the given expression. See details in the Microsoft documentation. Cannot be used with the option above. |
Run in isolation | Run the tests in an isolated process |
Use real-time test reporting | If this checkbox is selected, a custom TeamCity test logger will be used for real-time reporting. See the next section for details. If not selected, the t ests will be reported after the run ( the discovery of custom logger will not be attempted). |
Custom test logger
VSTest.Console
supports custom loggers, i.e. libraries that can handle events that occur when tests are being executed. TeamCity 9.0+ has a custom logger that provides real-time test reporting. The logger must be installed manually on the agent machine, as it requires dlls to be copied to the Extensions
folder of the VSTest.Console. No agent restart is needed when the custom logger is installed.
To install the custom logger:
Download the custom logger
Extract the contents of the downloaded archive on the agent machine:
for VisualStudio 2015 - to
PROGRAM_FILES\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions
for VisualStudio 2013 - to
PROGRAM_FILES\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions
for VisualStudio 2012 - to
PROGRAM_FILES\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions
Check that the custom logger was installed correctly by executing
vstest.console.exe /ListLoggers
in the console on the agent machine. If the logger was installed correctly, you will see the logger with FriendlyNameTeamCity
listed:VSTest.TeamCityLogger.TeamCityLogger Uri: logger://TeamCityLogger FriendlyName: TeamCity{info}
MSTest settings
Option | Description |
---|---|
MSTest metadata | Enter a value for |
Testlist from metadata to run | Edit the Testlist. Every line will be translated into |
Test | Names of individual tests to run. This option will be translated into the series of |
Unique | Run the test only if one unique match is found for any specified test in test section |
Results file | Enter a value for the To save the test run results to a named non-default file, enter a value for the
|