Testing

Share:

These questions were only shown to the respondents who said they work as a Tester / QA Engineer or are involved in testing as part of their jobs.

79%

of all respondents say testing plays an integral role in their development, which is four percentage points more than last year.

63%

of surveyed respondents say they are personally involved in test automation.

What types of tests do you have in your projects?

67%

Unit

45%

Integration

33%

End-to-End

31%

Performance

2%

Other

15%

None

85%

of those who have unit tests in their projects write them on their own.

How do you perform testing for your product?

96%

In-house testing

10%

Outsource testing

7%

Assessors

1%

Other

46%

of respondents use test case design as part of their testing process.

Which test design techniques do you use?

61%

Use case testing

47%

User story testing

30%

Boundary value analysis

23%

Condition testing

21%

Ad hoc testing

20%

Data-flow testing

20%

I’m not sure

59%

of respondents report that – in their companies – the same people take part in both test design and test execution.

The usage of performance, regression, and stress testing rose compared to last year, while functional and usability testing was used less often despite still being in the top-5 testing types.

Which types of testing are used in your project?

72%

77%

Functional testing

48%

43%

Performance testing

48%

44%

Regression testing

42%

45%

Acceptance testing

32%

39%

Usability testing

63%

of those who use unit tests also use test coverage metrics.

Which test coverage metrics do you use for unit testing?

44%

LoC / Methods / Classes / Packages

27%

Branches

17%

Paths

15%

Predicates of Conditions

10%

Data Flow metrics

1%

Other

37%

None

Test / QA Engineers have become more in-demand and highly skilled. The share of projects where there is more than 1 QA per 10 developers increased by four percentage points compared to last year. The same growth is seen among projects where 76-100% of QAs have programming skills. Respectively, the share of those who do only manual testing has declined.

What is the ratio of the number of testers / QA Engineers to software developers in your project?

40%

44%

Less than 1 QA per 10 developers

31%

29%

1–3 QAs per 10 developers

7%

5%

4–6 QAs per 10 developers

1%

1%

7–8 QAs per 10 developers

2%

3%

9–10 QAs per 10 developers

1%

1%

More QAs than developers

18%

18%

I'm not sure

How many testers / QA Engineers have programming skills in your project?

33%

35%

0–25%

12%

12%

26–50%

7%

7%

51–75%

24%

20%

76–100%

23%

26%

I don't know

How many testers / QA Engineers do only manual testing in your project?

38%

34%

0–25%

12%

10%

26–50%

10%

9%

51–75%

15%

20%

76–100%

24%

27%

I don't know

What tools do you use to store test cases?

42%

Office documents

32%

I don’t use any specific tools

22%

Special test case management tools

11%

Other

The share of those who use specific tools for test case storage increased by nine percentage points compared to last year. Test automation coverage did not change much compared to last year. The share of projects with at least half of the tests automated is still slightly higher than 50%.

Please provide an estimate of how many of your project tests are automated?

9%

All

26%

Most

13%

Half

31%

Only some

12%

None

10%

I'm not sure

Which test frameworks / tools / technologies do you use?

34%

29%

Postman

31%

34%

JUnit

25%

23%

Jest

20%

15%

Pytest

19%

20%

Selenium WebDriver

Which programming languages do you use for test automation in your project?

36%

JavaScript/Typescript

31%

Python

29%

Java

13%

C#

13%

PHP

12%

Bash

The top five languages used for test automation didn’t change from last year and still are JavaScript/TypeScript, Python, Java, C#, and PHP.

Which BDD (Behavior Driven Development) technologies do you use?

83%

I don’t use BDD

10%

Cucumber

6%

Gherkin

2%

JBehave

1%

Behat software tools

1%

Lettuce

1%

Gauge

2%

Other

14%

of respondents say they use AI/ML in testing their projects, which is six percentage points more than last year.

Behavior Driven Development is becoming more and more widespread, adding five percentage points since last year. Now, 17% of respondents say they use some BDD technologies.

JetBrains AquaNew

Looking for a powerful tool for test automation? Get JetBrains Aqua and boost your productivity by developing more reliable automated tests which require less maintenance.

Based on your valuable feedback, Aqua supports the most popular programming languages and automation testing frameworks. Subscribe to our blog to receive news about the latest improvements, and be sure to follow us on Twitter!

Testing:

2022

Thank you for your time!

We hope you found our report useful. Share this report with your friends and colleagues.

If you have any questions or suggestions, please contact us at surveys@jetbrains.com.