Risk-based testing: prioritising test effort with a risk matrix

What risk-based testing is

ISTQB describes risk as a combination of two factors. Likelihood says how likely it is that a module, function or change contains a defect. Impact says what happens if this defect occurs in production: financial loss, affected customers, consequences with the supervisor, disruption of operations.

This is about product risks, the risks in the software itself. Project risks such as staff shortages or late deliveries belong in project planning, not in test prioritisation.

Risk-based testing is not a shortcut. It makes visible the decision every project makes anyway: where to look closely and where not.

Five steps to a risk matrix

1. Break the system into test items that can be tested separately: modules, business transactions or the changes in a release. 10 to 40 items is a good range.
2. Rate the likelihood from 1 to 5. Indicators are defects in earlier releases, the size and type of changes, complexity, the number of interfaces, new technology and the team’s experience with the module.
3. Have the business rate the impact from 1 to 5. How much money is at stake, how many customers notice, does the defect have to be reported to the supervisor, do operations stop?
4. Calculate the risk as likelihood × impact. This gives values from 1 to 25.
5. Assign each risk class a test depth and a place in the order.

Example

An insurer ships a release of its customer portal. Testing and the business rate five test items; the matrix above shows the result.

The calculation for the premium calculation: it was rebuilt for the new tariff and had two defects in the last release, so likelihood 4. A calculation error hits every new customer and is relevant to the supervisor, so impact 5. 4 × 5 = 20 gives the top priority. The change of address, by contrast, was barely touched, and a defect would be annoying but quickly fixed: 2 × 2 = 4, low.

The classes in the example are 15–25 high, 8–14 medium and 1–7 low. Each project sets its own thresholds and records them in the test strategy.

You can build the matrix for your own modules with the risk matrix calculator. It runs only in your browser and stores nothing.

From class to test depth

Class	Order	Test depth
high	first, already in early test levels	systematic test techniques such as boundary value analysis and decision tables, review of requirements, tests on several test levels, automated regression, acceptance by the business
medium	next	standard techniques, positive and the most important negative cases, regression for the core flows
low	last, cut first under time pressure	sample, smoke test or exploratory test

One rule protects against a wrong conclusion. The product treats 5 × 1 and 1 × 5 the same, but a rare defect with devastating effect is different from a frequent one with little effect. Many projects therefore decide that impact 5 is always at least “medium”, however low the likelihood.

Re-rate before every release

The matrix is a snapshot. New changes shift the likelihood, new products shift the impact. Re-rate before every release and record who rated what and when. That way the business, internal audit and supervisors can trace every priority. For banks, this documentation supports the testing programme that DORA requires.

Limits

Ratings are judgements, and two people arrive at different values. Rate as a team and with clear criteria for each level.

Low risk does not mean “no test”, but less depth, decided on purpose. And the method does not replace mandatory tests: what law, supervisors or contracts require is tested regardless of its value in the matrix.

Gut-feeling likelihoods are the weakest part. Ratings become more robust with data from defect history and changes. How AI can help and where it reaches its limits is covered in Estimating defect likelihood with AI.