Risk Tests Reordering in Custom Test Runner

In TeamCity, you can instruct the system to Running Risk Group Tests First.

To implement risk group tests reordering feature for your own custom test runner, TeamCity provides following special properties:

• teamcity.tests.runRiskGroupTestsFirst — this property value contains groups of tests to run before others. Accordingly there are two groups: recentlyFailed and newAndModified. If more than one group specified they are separated with comma. This property is provided only if corresponding settings are selected on build runner page.

• teamcity.tests.recentlyFailedTests.file — this property value contains full path to a file with recently failed tests. The property is provided only if recentlyFailed group is selected. The file contains tests separated by new line character. For Java like tests full class names are stored in the file (without test method name). In other cases full name of the test will be stored in the file as it was reported by the tests runner.

• teamcity.build.changedFiles.file — this property is useful if you want to support running of new and modified tests in your tests runner. This property contains full path to a file with information about changed files included in the build. You can use this file to determine whether any tests were modified and run them before others. The file contains lines separated by new line. Each line corresponds to one file and has the following format: <relative file path>:<change type>:<revision> where:

  • <relative file path> is a path to a file relative to the current checkout directory.

  • <change type> is a type of modification and can have the following values: CHANGED, ADDED, REMOVED, NOT_CHANGED, DIRECTORY_CHANGED, DIRECTORY_ADDED, DIRECTORY_REMOVED

  • <revision> is a file revision in repository. If file is a part of change list started via remote run then string <personal> will be written instead of file revision.

• teamcity.build.checkoutDir — this property contains path to a build checkout directory. It is useful if you need to convert relative paths to modified files to absolute ones.

note

TeamCity will pass teamcity.tests.runRiskGroupTestsFirst, teamcity.tests.recentlyFailedTests.file and teamcity.build.changedFiles.file properties to the build process, but if process starts additional JVM or other processes, these properties won't be passed to them automatically.

For example, if you are using Ant runner, you will have access to these properties from the Ant build.xml. But if your build.xml starts new JVM (or <junit/> task with fork="yes" attribute), and you want to access these properties from this JVM, you'll have to modify your build script and pass them explicitly.