IntelliJ IDEA 2024.3 Help

Run/Debug Configuration: TestNG

The TestNG run/debug configuration is used to launch the tests that comply with the TestNG framework. The dialog consists of several tabs.

Common options

Name

Specify a name for the run configuration to quickly identify it among others when editing or running.

Allow multiple instances

Allow running multiple instances of this run configuration in parallel.

By default, it is disabled, and when you start this configuration while another instance is still running, IntelliJ IDEA suggests stopping the running instance and starting another one. This is helpful when a run configuration consumes a lot of resources and there is no good reason to run multiple instances.

Store as project file

Save the file with the run configuration settings to share it with other team members. The default location is .idea/runConfigurations. However, if you do not want to share the .idea directory, you can save the configuration to any other directory within the project.

By default, it is disabled, and IntelliJ IDEA stores run configuration settings in .idea/workspace.xml.

Configuration tab

The composition of this tab depends on the selected testing scope: package, project, and so on.

Item

Description

Test kind

  • All in package – run all tests in a package. In the Package field, specify the fully qualified name of the package.

  • Pattern: run a set of test classes or specific methods within test classes. This set may include classes located in the same or different directories, packages, or modules. Declarations must be separated with ||.

    Each class in this field must be represented by its fully qualified name:

    • Include the package name and class name

    • Use , as a separator for methods

    • Use $ as a separator for inner classes

    You can type class names manually, or click the Add button on the right (or press Shift+Enter) and search for classes you want to add in the dialog that opens.

    Example: packageName.ClassName$InnerClassName,methodName

    You may also specify the required classes using regular expressions.

    For example, if you want to exclude all integration tests that have IT in their names, type ^(?!.*IT.*).*$.

    You can create a suite test, that is, a bundle of several test classes that will be run together. To create a suite test class, click the Expand button on the right and type the test classes you want to be run as a suite. As a result, a new class will be created with the @Suite annotation.

  • Group – run a test group. In the Group field, specify the group you want to run.

  • Suite – run a test suite. In the Suite field, enter the corresponding testng.xml file.

    • In whole project – the IDE will look for the tests in all the modules

    • In single module – the IDE will look for the tests only in the module that is selected in the Use classpath of module field.

    • Across module dependencies – the same as the previous option plus the modules that depend on that module.

  • Class – run a test class. In the Class field, enter the fully qualified name of the test class you want to run.

  • Method – run a test method.

Output directory

The directory in which test reports will be generated.

JDK Settings

Item

Description

VM options

Specify the options to be passed to the Java virtual machine when launching the application, for example, -mx, -verbose, and so on.

When specifying JVM options, follow these rules:

  • Use spaces to separate individual options.

  • If the value of an option includes spaces, enclose either the value or the actual spaces with double quotes.

  • If an option includes double quotes as part of the value, escape the double quotes using backslashes.

  • You can pass environment variable values to custom Java properties.

-Xmx1024m -Dspaces="some arg" -Dmy.prop=\"quoted_value\" -Dfoo=${MY_ENV_VAR}

Use code completion in this field: start typing the name of a flag, and the IDE suggests a list of available command line options. This works for -XX: and -X options and some standard options that are not configured by IntelliJ IDEA automatically, like -ea, but not for -cp or –release.

The -classpath option specified in this field overrides the classpath of the module.

Test runner params

Arguments to be passed to the test runner.

Use the same rules as for specifying the VM options.

Working directory

Specify the working directory to be used for running the application. This directory is the starting point for all relative input and output paths. By default, the working directory is the project root.

Environment variables

Click the Browse button to open the Environment Variables dialog where you can create variables and specify their values.

Use module path

This option is available for projects with JPMS modules (Java 9 and later). It allows you to select how to run your tests: on a class or on a module path.

JRE

The JRE to be used.

Shorten command line

Select a method that will be used to shorten the command line if the classpath gets too long, or you have many VM arguments that exceed your OS command line length limitation. The choice of option depends on the class loader implementation. Note that some frameworks do not support JAR manifest, while other frameworks with custom class loaders will not work well with the classpath.file option.

  • none: IntelliJ IDEA will not shorten a long classpath. If the command line exceeds the OS limitation, IntelliJ IDEA will be unable to run your application and will display a message suggesting that you specify the shortening method.

  • JAR manifest: IntelliJ IDEA will pass a long classpath via a temporary classpath.jar. The original classpath is defined in the manifest file as a class-path attribute in classpath.jar. Note that you will be able to preview the full command line if it was shortened using this method, not just the classpath of the temporary classpath.jar.

  • classpath.file: IntelliJ IDEA will write a long classpath into a text file.

  • @argFiles (Java 9+): the command-line length limitations will be overcome using the @argument files option.

Parameters

Item

Description

Parameters

Properties file

Specify the .properties file to be passed to TestNG.

Name - Value

Additional parameters as key - value pairs.

Listeners

 

Use these icons to make up a list of listeners.

Code Coverage tab

Use this tab to configure code coverage monitoring options.

Item

Description

Packages and classes to include in coverage data

Click the Add button and select the Add Class button Add Class or the Add Package button Add Package to specify classes and packages to be measured. You can also remove classes and packages from the list by selecting them in the list and clicking the remove the package button.

Packages and classes to exclude from coverage data

Click the Add button and select the Add Class button Add Class or the Add Package button Add Package to specify classes and packages that should not be measured. You can also remove classes and packages from the list by selecting them in the list and clicking the remove the package button.

Logs

The following options are related to logging the execution of this configuration. For more information, refer to Logs.

Item

Description

Log files to be shown in console

Specify which log files to display while running the application.

Click the Add button to add a new log. In the Edit Log Files Aliases dialog, configure the following:

  • Alias: The name of the tab where the log will be displayed.

  • Log File Location: Specify the path to the log file or an Ant pattern. If several files of a rolling log match the pattern, IntelliJ IDEA will display the most recent one.

  • Show all files coverable by pattern: Show all logs that match the pattern.

For logs in the table, you can configure the following options:

  • Is Active: Display the specified log file.

  • Skip Content: Do not display old log messages from previous runs.

Save console output to file

Save the console output to the specified location. Type the path manually or click the browse button and point to the desired location in the dialog that opens.

Show console when a message is printed to standard output stream

Activate the console when the application writes to the standard output stream.

Show console when a message is printed to standard error stream

Activate the console when the application writes to the standard error stream.

Before Launch

In this area, you can specify tasks to be performed before starting the selected run/debug configuration. The tasks are performed in the order they appear in the list.

Item

Description

Add before launch task

Enable this option to add one of the following available tasks:

  • Launch Web Browser: select this option to have a browser started. In the dialog that opens, select the type of the browser and provide the start URL. Also, specify if you want the browser be launched with JavaScript debugger.

  • Run External tool: run an external application. In the dialog that opens, select one or multiple applications you want to run. If it is not defined in IntelliJ IDEA yet, add its definition. For more information, refer to External tools.

  • Run Another Configuration: select to execute another run/debug configuration and wait until it finishes before starting the current configuration. If you want to run several configurations in parallel, use a compound run/debug configuration.

  • Run Remote External Tool: add a remote SSH external tool.

  • Run Gradle task: run a Gradle task. In the dialog that opens, specify the task and provide additional configuration if necessary.

  • Build: select to compile the specified module. The Build Module action will be executed.

    If an error occurs during compilation, IntelliJ IDEA won't attempt to start the run/debug configuration.

  • Build Project: select to compile the entire project. The Build Project action will be executed.

    If an error occurs during compilation, IntelliJ IDEA won't attempt to start the run/debug configuration.

  • Build, no error check: the same as the Build option, but IntelliJ IDEA will try to start the run/debug configuration irrespective of the compilation results.

  • Build Artifacts: select this option to build an artifact or artifacts. In the dialog that opens, select the artifact or artifacts that should be built.

  • Run Maven Goal: select this option to run a Maven goal. In the dialog that opens, select the goal to be run.

  • Run Grunt task: select this option to run a Grunt task.

    In the Grunt task dialog that opens, specify the Gruntfile.js where the required task is defined, select the task to execute, and specify the arguments to pass to the Grunt tool.

    Specify the location of the Node.js interpreter, the parameters to pass to it, and the path to the grunt-cli package.

  • Run gulp task: select this option to run a Gulp task.

    In the Gulp task dialog that opens, specify the Gulpfile.js where the required task is defined, select the task to execute, and specify the arguments to pass to the Gulp tool.

    Specify the location of the Node.js interpreter, the parameters to pass to it, and the path to the gulp package.

  • Run npm script: select this option to execute an npm script.

    In the NPM Script dialog that opens, specify the npm run/debug configuration settings.

  • Compile TypeScript: select to run the built-in TypeScript compiler and thus make sure that all the changes you made to your TypeScript code are reflected in the generated JavaScript files. In the TypeScript Compile Settings dialog that opens, select or clear the Check errors checkbox to configure the behaviour of the compiler in case any errors are detected:

    • If the Check errors checkbox is selected, the compiler will show all the errors and the run configuration will not start.

    • If the Check errors checkbox is cleared, the compiler will show all the detected errors but the run configuration still will be launched.

  • Disconnect Data Source: select this option if you want to disrupt the connection to a data source before the run/debug configuration is run.

Activate tool window

Depending on the type of configuration, open the Run, Debug, or Services tool window when you start this run configuration. If this option is disabled, you can open the tool window manually:

  • View | Tool Windows | Run or Alt+4

  • View | Tool Windows | Debug or Alt+5

  • View | Tool Windows | Services or Alt+8

Focus tool window

Focus on the run configuration tool window when the tests are running.

Show this page

Show the run configuration settings before actually starting it.

Last modified: 23 July 2024
TestNGGetting started with Spock