Run Python tests
Generally, Aqua runs and debugs Python tests in the same way as other Python applications, by running the run/debug configurations you have created. When doing so, it passes the specified test classes or methods to the test runner.
If you run a test for which there is no permanent run/debug configuration, a temporary configuration is created. You can then save such a configuration using the Run/debug configuration dialog if you want to reuse it later.
The tests run in the background, so you can execute several tests at the same time.
Each running configuration gets its own tab in the Run tool window (the Test Results tab). One tab can aggregate several tests.
Run all Python tests in a directory
In the Project tool window, select the directory that contains tests to be executed.
From the context menu, select the corresponding run command.
If the directory contains tests that belong to the different testing frameworks, select the configuration to be used.
For example, select .
Explore results in the test runner.
By default, all tests are executed one by one. You can change this behavior for the pytest testing framework and execute your tests in parallel.
Run Python tests in parallel
Enable test multiprocessing to optimize execution of your pytest tests.
To explicitly specify the number of CPUs for test execution:
Install the pytest-xdist.
Specify pytest as the project testing framework. See Choose your testing framework for more details.
Select Edit configurations... from the list of the run/debug configurations on the main toolbar. In the Run/Debug Configurations dialog, expand the Pytest tests group, and select pytest in <directory name>.
Aqua creates this configuration when you run all tests in a directory for the very first time. If you haven't executed the tests yet, click the
icon and specify the run/debug Configuration parameters.In the Run/Debug Configurations dialog, in the Additional Arguments field specify the number of the CPUs to run the tests:
-n <number of CPUs>and save the changes.
Now run all the tests in the directory again and inspect the output in the Run tool window. In the shown example, The total execution time is 12s 79ms as compared to 30s 13ms when running the same tests consequentially. The test report provides information about the CPUs used to run the tests and execution time.
Alternatively, you can specify the number of CPUs to run your tests in the pytest.ini file. For example,
If you can stop a running tests, all running tests stop immediately. Icons of tests in the Run tool window reflect status of the test (passed, failed, aborted).