CTest

Last modified: 11 February 2024

CTest is a test automation tool distributed as part of CMake.

note

The minimal supported version of CTest is 3.14.

CLion uses the ctest executable located in the same folder as the toolchain's cmake executable. Selecting an arbitrary ctest executable is not supported.

CTest run/debug configuration

When you open a project that uses CTest, CLion detects the tests and automatically creates a run/debug configuration for them. This configuration is called All Tests and is based on the CTest Application template.

You can edit the automatically created configuration and add more configurations of the same type in the Edit Configurations dialog.

The automatically created configuration for CTests

Create CTest configurations

  1. In the main menu, go to Run | Edit Configurations, click and select CTest Application from the list of templates:

    Adding a CTest Application configuration

  2. Provide the configuration name and specify the settings.

    • In the Test list to run field, select the tests to be included. You can use smart search within the List of Available Tests for easier navigation:

      List of the available tests in CTest Application

      CLion will notify you if the version of ctest you are using is less than 3.14, or if there were no tests detected in the project.

      Test detection fails warning

      In this case, check the version of CMake in Settings | Build, Execution, Deployment | Toolchain and make sure CTest tests are enabled in your CMakeLists.txt files.

      note

      To filter the list of tests using CTest options with regular expressions, use the CTest arguments field.

    • In the CTest arguments field, you can provide additional flags. For example, specify --parallel to run the tests in parallel if your tests support that.

    Other fields of the configuration settings are described on the reference page.

Launch CTest tests

To run or debug tests, use one of the following options:

  • Select the configuration in the switcher and click one of the toolbar icons, or use the corresponding shortcuts.

    Configuration switcher and toolbar

  • Open the CMakeLists.txt file and find the add_test command that corresponds to the desired test. Click the left-hand gutter icon next to it and select from the list of options:

    Gutter menu for tests

    When you run a test using gutter icons, CLion creates a temporary CTest Application configuration, which is greyed out in the list of configurations. To save it, select it in the Edit Configurations dialog and press :

    Saving a temporary configuration

Debug CTest tests

  • If you start a debug session for a CTest test, CLion will run the test itself under debug instead of launching the ctest child process.

    In case of a multi-test configuration, you will be prompted to select a single test from the list:

    Lists of tests for debugging

Dynamic analysis

  • You can use the tools of dynamic analysis for your tests to eliminate issues at runtime. CLion integrates with Valgrind Memcheck, Google Sanitizers, CPU Profiler, and Code Coverage, providing them with visualized output and handy features to help you work with the results. For more information, refer to the corresponding articles.

    Profiling in the gutter menu for tests

    note

    Unlike the case of debugging, dynamic analysis is applied to execution under ctest.

Explore the results

  • When you run tests, CLion shows the results and the process in the built-in test runner window. This window includes:

    • progress bar with the percentage of tests executed so far,

    • tree view of all the running tests with their status and duration,

    • console output stream,

    • toolbar with the options to rerun failed tests, export or open previous results saved automatically , sort the tests alphabetically to easily find a particular test, or sort them by duration to understand which test ran longer than others,

    • context menu with actions like Jump to Source, which navigates you to the CMakeLists.txt file where the test is declared.

    Test runner window

Skip tests

  • Test failure can be ignored by CTest in particular cases. To configure this, set the SKIP_RETURN_CODE or SKIP_REGULAR_EXPRESSION property. CLion will detect these properties and mark the tests as Skipped in the output tree:

    Skipping tests

See also

Procedures

Reference

Getting Started