Code coverage
Code coverage uses generated reports to calculate the overall code coverage inside a method, class, and file. It also reports on the issues associated with the missing coverage in these entities.
This feature is available under the Ultimate and Ultimate Plus licenses in the following linters:
Linter | Code coverage tool | Supported report file extensions |
---|---|---|
IntelliJ IDEA Code Coverage Agent is the recommended tool |
| |
| ||
| ||
| ||
| ||
| ||
|
|
note
Code coverage for files is available only for Qodana for JS, Qodana for PHP, and Qodana for Python linters.
For the missing code coverage issues, the predefined threshold in Qodana is currently set to 50%.
Code coverage uses several inspections that are already included in the qodana.recommended
and qodana.starter
default inspection profiles, so you do not need to enable them:
Once analysis is complete, reports are available in Qodana reports and JetBrains IDEs.
Qodana calculates code coverage based on the number of code lines containing logic with function, method, and class statements being ignored. Here is the snippet containing comments on how it works:
function divide(a, b) { // Not analyzed by the code coverage
return a / b; // Analyzed by the code coverage
}
module.exports = divide; // Analyzed by the code coverage
tip
You can find configuration examples on GitHub.
Configure your code coverage tool. Jest code coverage reports should contain paths relative to a project root. For example, if your codebase files are contained in the
<project-root>/src/
directory, then reports should containsrc/<file-name>
file paths.Use your code coverage tool to generate and save code coverage reports to the
<project-root>/.qodana/code-coverage
directory. To learn how to override this directory, see the recommendation from the Run code coverage chapter. For a monorepo project containing multiple repositories, this directory should be created in each repository.Prepare your project. If you have a monorepo project, save Qodana configuration for each repository in a separate
qodana.yaml
file. You can put these files in repository directories, or give them custom names and save them in the root directory of a project.For the Qodana for .NET linter, add the
coverlet.msbuild
andcoverlet.collector
packages to the test project.
note
You can run Qodana over a single test coverage report generated by a single code coverage tool at a time.
note
The Qodana for Go linter requires that your project contains no
.idea
directory.
To learn about running code coverage using the Qodana for .NET linter, skip to the Qodana for .NET section of this page.
Map the directory containing code coverage reports to the /data/coverage
directory and a project token using the QODANA_TOKEN
variable. Here are the Docker and Qodana CLI command samples:
$docker run \ -v $(pwd):/data/project/ \ -v .qodana/code-coverage/:/data/coverage \ -e QODANA_TOKEN="<qodana-cloud-token>" \ jetbrains/qodana-<linter>
$qodana scan \ -v .qodana/code-coverage/:/data/coverage \ -e QODANA_TOKEN="<qodana-cloud-token>"
If you have a monorepo project, use the -i <path-relative-to-project-root>
option to point a repository directory. If you saved Qodana configuration files under custom names, use the --config <path-relative-to-project-root>
option. To override the default code coverage report directory, use the --coverage-dir <path-relative-to-project-root>
option.
Create the pipeline that will store all code coverage output files in the <project-root-dir>/.qodana/code-coverage
directory. You can find various examples of the GitHub Actions configurations on the GitHub website.
Below is the pipeline configuration example for the Qodana for JS linter running in the JS/jest
directory of a repository:
name: JavaScript - Jest Test
on:
workflow_dispatch:
pull_request:
push:
branches:
- main
- 'releases/*'
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Use Node.js 18.x
uses: actions/setup-node@v2
with:
node-version: 18.x
- name: Install dependencies
run: npm ci
working-directory: JS/jest
- name: Run tests
run: npm test
working-directory: JS/jest
- name: Archive coverage data # Archive data for use by Qodana
uses: actions/upload-artifact@v2
with:
name: jest-coverage-data
path: JS/jest/.qodana/code-coverage
- name: Qodana Scan # Run Qodana
uses: JetBrains/qodana-action@main
env:
QODANA_TOKEN: ${{ secrets.QODANA_TOKEN_JS }}
with:
args: "-i,JS/jest,--linter,jetbrains/qodana-js:2024.3"
pr-mode: false
If you have a monorepo project and saved Qodana configuration files under custom names, then in the args
block use the --config,<path-relative-to-project-root>
option. To override the default code coverage report directory, use the --coverage-dir,<path-relative-to-project-root>
option.
Create a GitLab CI/CD pipeline that will store all code coverage output files in the <project-root-dir>/coverage
directory:
qodana:
image:
name: jetbrains/qodana-<linter>
entrypoint: [""]
cache:
- key: qodana-2024.3-$CI_DEFAULT_BRANCH-$CI_COMMIT_REF_SLUG
fallback_keys:
- qodana-2024.3-$CI_DEFAULT_BRANCH-
- qodana-2024.3-
paths:
- .qodana/cache
variables:
QODANA_TOKEN: $qodana_token
script:
- qodana --save-report
--results-dir=$CI_PROJECT_DIR/.qodana/results
--coverage-dir=$CI_PROJECT_DIR/.qodana/code-coverage
artifacts:
paths:
- .qodana/results/report
expose_as: 'Qodana report'
The variables
block uses the QODANA_TOKEN
variable to contain a project token. The --coverage-dir=$CI_PROJECT_DIR/.qodana/code-coverage
in the script
block enables Qodana to read code coverage reports from a specific directory. If you have a monorepo project and saved Qodana configuration files under custom names, then in the script
block use the --config <path-relative-to-project-root>
option.
tip
Configuration examples for code coverage are available on the GitHub website.
Here is an example of the qodana.yaml
file configuration for the Qodana for .NET linter:
dotnet:
solution: <your-solution-file>
bootstrap: dotnet build; cd <path-to-dir-with-test-project-file> && \\
dotnet add package coverlet.msbuild && dotnet add package coverlet.collector && \\
((dotnet test /p:CollectCoverage=true /p:CoverletOutput=<your-project-folder>/.qodana/code-coverage /p:CoverletOutputFormat=lcov))
Here, the dotnet
option configures the solution file.
The bootstrap
key performs several steps before running Qodana:
Command step | Description |
---|---|
| Build a project or a solution |
| Navigate to the directory containing the project test file |
| Add the |
| Execute tests in the project, and: |
| Enable code coverage |
| Collect code coverage results to a specific directory |
| Specify the code coverage output format |
Code coverage inspection results for the Qodana for .NET linter are available in Qodana Cloud.
Fresh code is the code contained in a GitHub pull request. Qodana can calculate fresh code coverage and display the results.
note
While working with fresh code, Qodana cannot analyze coverage issues for missing coverage in methods, classes, and files.
To enable the fresh code feature, configure the PR-mode in your GitHub workflow.
Here is the sample for inspecting the JavaScript fresh code:
name: Code coverage fresh code
on:
workflow_dispatch:
pull_request:
push:
branches:
- main
- 'releases/*'
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Use Node.js 18.x
uses: actions/setup-node@v2
with:
node-version: 18.x
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test
- name: Archive coverage data
uses: actions/upload-artifact@v2
with:
name: jest-coverage-data
path: .qodana/code-coverage
- name: Qodana Scan
uses: JetBrains/qodana-action@main
env:
QODANA_TOKEN: ${{ secrets.QODANA_TOKEN_JS }}
with:
pr-mode: true # Enable the pull-request mode
After you have prepared the project and ran the code coverage, you can view code coverage reports in Qodana Cloud or using your IDE.
You can find code coverage statistics in the upper-right corner of the Qodana report UI. It also lists the inspections used by the feature.
You can view code coverage reports using IntelliJ IDEA, WebStorm, PhpStorm, PyCharm, and GoLand IDEs starting from version 2023.2. This feature is available for reports retrieved from Qodana Cloud after linking, or reports from local storage.
note
Currently, code coverage overview is not available for XML-formatted reports generated by IntelliJ IDEA Code Coverage Agent and JaCoCo, and LCOV-formatted reports generated by coverlet.msbuild.
In your IDE, navigate to Tools | Qodana | Log in to Qodana.
On the Settings dialog, click Log in.
This will redirect you to the authentication page.
In the Settings dialog, search for the project you would like to link with.
You can view code coverage reports based locally using JetBrains IDEs.
In your IDE, navigate to Run | Show coverage data and open the file containing a code coverage report.
In the Coverage tool window, you can view the test coverage report. This report shows the percentage of the code that has been executed or covered by tests.
The IDE highlights the codebase test coverage using color marking. By default, the green color means that a particular line was covered, and the red color means the uncovered line of code.
note
If you see that code coverage results look incomplete, you probably need to reconfigure your code coverage tool and generate a new code coverage report.
The report shows coverage for the lines that implement the logic of a method, function, or a class, but not for the function, method, or class declaration. The image below shows that code coverage is not applicable to line 7, while line 8 is not covered.
Thanks for your feedback!