NUnit Support in TeamCity

TeamCity provides its own NUnit tests launcher that can be used from the command line. The tests are run according to the passed parameters and, if the process is run inside the TeamCity build agent environment, the results are reported to the TeamCity agent.

tip

Tips:

• If you need to access the path to the TeamCity NUnit launcher from some process, you can add the %system.teamcity.dotnet.nunitlauncher% environment variable.

• Values surrounded with % in custom scripts in the Command Line runner are treated as TeamCity references.

You can pass the following command-line options to the TeamCity NUnit Test Launcher:

${teamcity.dotnet.nunitlauncher} <.NET Framework> <platform> <NUnit vers.> [/category-include:<list>] [/category-exclude:<list>] [/addin:<list>] <assemblies to test>

%teamcity.dotnet.nunitlauncher% v2.0 x64 NUnit-2.2.10 Assembly.dll

%teamcity.dotnet.nunitlauncher% v2.0 x64 NUnit-2.2.10 /category-include:C1 /category-exclude:C2 Assembly.dll

%teamcity.dotnet.nunitlauncher% v2.0 x64 NUnit-2.5.0 /addin:Addin1.dll;Addin2.dll Assembly.dll Assebly2.dll

This section assumes, that you already have a NAnt build script with the configured nunit2 task in it, and want TeamCity to track test reports without making any changes to the existing build script. Otherwise, consider adding the NUnit build runner as one of the steps for your build configuration.

To track tests defined in a NAnt build via the standard nunt2 task, TeamCity provides a custom task implementation and automatically replaces the original <nunit2> task with its own task. When the build is triggered, TeamCity starts TeamCity NUnit Test Launcher using own implementation of <nunit2>. This allows you to leave your the build script without changes and receive on-the-fly test reports in TeamCity.

If you don't want TeamCity to replace the original nunit2 task, consider the following options:

• Use the NUnit console with TeamCity Add-in for NUnit.

• Import XML test results via the XML Test Report plugin.

• Use the command-line NUnit Test Launcher.

• Configure reporting tests manually via service messages.

• To disable nunit2 task replacement, set the teamcity.dotnet.nant.replaceTasks system property to false.

The nunt2 task implementation in TeamCity supports additional options that can be specified either as NAnt <property> tasks in the build script, or as system properties under Build Configuration | Build Parameters.

The following options are supported for the TeamCity <nunit2> task implementation:

TeamCity NUnit test launcher will run tests in the .NET Framework, which is specified by the NAnt target framework: that is, on .NET Framework 1.1, 2.0 or 4.0 runtime. TeamCity also supports test categories for the <nunit2> task.

Adding the listed properties to the NAnt build script makes it TeamCity-dependent. To avoid this, specify properties as system properties under Build Configuration, or consider adding the <if> task.

tip

If you need a TeamCity test runner to support third-party NUnit add-ins, refer to the NUnit Add-ins Support section for the details.

Examples

Start tests form a single assembly files under x64 mode on .NET 2.0.

<property name="teamcity.dotnet.nant.nunit2.platform" value="x64" />
<nunit2>
    <formatter type="Plain" />
    <test assemblyname="MyProject.Tests.dll" />
</nunit2>

Run all tests from category C1, but not C2.

<nunit2 verbose="true" haltonfailure="false" failonerror="true">
         <formatter type="Plain" />
          <test>
            <assemblies>
              <include name="dll.dll" />
            </assemblies>
            <categories>
               <include name="C1" />
               <exclude name="C2"/>
            </categories>
          </test>
        </nunit2>

Explicitly specify the version on NUnit to run tests with.
Note that in this case, the following property should be added before the nunit2 task call.

<property name="teamcity.dotnet.nant.nunit2.version" value="NUnit-2.4.10" />
<nunit2> <!--....--> </nunit2>

This section describes how to use NUnit from MSBuild.

• For NUnit prior to 3.0

• For NUnit 3.0 and above

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="NUnitTeamCity" AssemblyFile="$(teamcity_dotnet_nunitlauncher_msbuild_task)"/>

  <Target Name="SayHello">
     <NUnitTeamCity Assemblies="!!!*put here item group of assemblies to run tests on*!!!"/>
  </Target>
</Project>

<Target Name="build_01">
     <!-- start tests for NUnit-2.2.10 -->
     <NUnitTeamCity Assemblies="@(TestAssembly)" NUnitVersion="NUnit-2.2.10"/>

     <!-- start tests for NUnit-2.4.6 -->
     <NUnitTeamCity Assemblies="@(TestAssembly)" NUnitVersion="NUnit-2.4.8"/>
  </Target>

<Target Name="build">
     <NUnitTeamCity Assemblies="@(TestAssembly)" Addins="NUnitExtension.RowTest.AddIn.dll" NUnitVersion="NUnit-2.4.6"/>
  </Target>

<Target Name="build">
     <NUnitTeamCity Assemblies="@(TestAssembly)" Addins="NUnitExtension.RowTest.AddIn.dll" NUnitVersion="NUnit-2.4.6" RunProcessPerAssembly="True"/>
</Target>

<NUnitTeamCity ... Condition=" '$(TEAMCITY_VERSION)' != '' "/>

For MSBuild, TeamCity provides the following service tasks that implement the same options as the Build Script Interaction:

TeamCitySetBuildNumber allows changing the build number:

<TeamCitySetBuildNumber BuildNumber="1.3_{build.number}" />

It is possible to use {build.number} as a placeholder for older build number.

TeamCityProgressMessage allows writing a progress message:

<TeamCityProgressMessage Text="Progress message text" />

TeamCityPublishArtifacts allows publishing all artifacts from the MSBuild item group:

<ItemGroup>
    <Files Include="*.dll" />
  </ItemGroup>
  <TeamCityPublishArtifacts SourceFiles="@(Files-> '%(FullPath)' )" Condition=" '$(TEAMCITY_VERSION)' != '' "/>

TeamCityReportStatsValue is a handy task for publishing statistic values:

<TeamCityReportStatsValue Key="StatsValueType" Value="42" />

TeamCityBuildProblem reports a build problem which actually fails the build. Build problems appear on the Build Results page and also affect build status text:

<TeamCityBuildProblem description="description" identity="identity"/>

• The mandatory description attribute is a human-readable text describing the build problem. By default, description appears in the build status text.

• identity is an optional attribute and characterizes a particular build problem instance. It shouldn't change throughout builds if the same problem occurs: for example, the same compilation error. It should be a valid Java ID up to 60 characters long. By default, identity is calculated based on description.

TeamCitySetStatus is a task to change current build status text.

<TeamCitySetStatus Status="<status value>" Text="{build.status.text} and some aftertext" />

{build.status.text is substituted with an older status text. The status can have the SUCCESS value.

NUnit Add-in is an extension that plugs in to the NUnit core and changes the way it operates. Refer to the NUnit add-ins page for more information. This section covers description of the NUnit add-ins support for NAnt, MSBuild, and NUnit Console Launcher.

<property name="teamcity.dotnet.nant.nunit2.addins" value="<list of paths>" />

<property name="teamcity.dotnet.nant.nunit2.addins" value="../tools/addins/MyFirst.AddIn.dll;MySecond.AddIn.dll" />

${teamcity.dotnet.nunitlauncher} /addin:../tools/addins/MyFirst.AddIn.dll;nunit-addins/MySecond.AddIn.dll

Addins="<list>"

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="build">
  <ItemGroup>
    <TestAssembly Include="$(MSBuildProjectDirectory)/MyTests.dll" />
  </ItemGroup>
  <Target Name="build">
    <NUnitTeamCity Assemblies="@(TestAssembly)" Addins="../tools/addins/MyFirst.AddIn.dll;nunit-addins/MySecond.AddIn.dll" />
  </Target>
</Project>

TeamCity NUnit Add-in supports NUnit prior to version 3.0. For later versions, refer to the this section.

If you run NUnit tests via the NUnit console and want TeamCity to track the test results without having to launch the TeamCity test runner, the best solution is to use TeamCity Add-in for NUnit. You can plug this add-in into NUnit, and the tests will be automatically reported to the TeamCity server.

Alternatively, you can opt to use the XML Report Processing build feature, or manually configure test reporting by means of service messages.

To be able to review test results in TeamCity:

1. In your build, set the path to the TeamCity Add-in to the system property teamcity.dotnet.nunitaddin (for MSBuild, it would be teamcity_dotnet_nunitaddin), and add the version of NUnit at the end of this path. For example:

   • For NUnit 2.4.X, use ${teamcity.dotnet.nunitaddin}-2.4.X.dll (for MSBuild: $(teamcity_dotnet_nunitaddin)-2.4.X.dll).
     Example for NUnit 2.4.7: NAnt: ${teamcity.dotnet.nunitaddin}-2.4.7.dll, MSBuild: $(teamcity_dotnet_nunitaddin)-2.4.7.dll.

   • For NUnit 2.5.0 alpha 4, use ${teamcity.dotnet.nunitaddin}-2.5.0.dll (for MSBuild: $(teamcity_dotnet_nunitaddin)-2.5.0.dll).

2. Copy the .dll and .pdb TeamCity add-in files to the NUnit add-in directory.

Although you can copy these files once, it is highly recommended configuring your builds so that the TeamCity add-in files are copied to the NUnit add-in directory for each build, because these files could be updated by TeamCity.

The following example shows how to use the NUnit console runner with the TeamCity Add-in for NUnit 2.4.7 (on MSBuild):

<ItemGroup>
  <NUnitAddinFiles Include="$(teamcity_dotnet_nunitaddin)-2.4.7.*" />
</ItemGroup>

<Target Name="RunTests">
  <MakeDir Directories="$(NUnitHome)/bin/addins" />
  <Copy SourceFiles="@(NUnitAddinFiles)" DestinationFolder="$(NUnitHome)/bin/addins" />
  <Exec Command="$(NUnitHome)/bin/NUnit-Console.exe $(NUnitFileName)" />
</Target>

If you need to configure environment variables for NUnit explicitly, specify an environment variable with the value reference of %system.teamcity.dotnet.nunitaddin%. See this article for details.

NUnit 2.4.8 Issue
NUnit 2.4.8 has the following known issue: NUnit 2.4.8 runner tries to load an assembly according to the created AssemblyName object. However, the addins folder of NUnit 2.4.8 is not included in application probe paths. As a result, NUnit 2.4.8 fails to load any add-in in the console mode.
To solve the problem, we suggest that you use any of the following workarounds:

• Copy the TeamCity add-in assembly both to the NUnit bin and bin/addins directories.

• Patch NUnit-Console.exe.config to include the add-ins to application probe paths. Add the following code into the config/runtime element:

  <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <probing privatePath="addins"/>
  </assemblyBinding>