Which data can be used
| Data source | Example | What it says about defects |
|---|---|---|
| Defect history | Tickets from Jira, ALM or Xray, mapped to modules | Where defects occurred before, they often occur again |
| Code changes | Commits, changed lines and files per release from Git | A lot of change in a short time raises the risk |
| Complexity | Size, nesting, dependencies | Complex code is more error-prone |
| Interfaces | Number of connected systems, changed contracts | Transitions between systems are typical sources of defects |
| Configuration | Changed parameters, tariffs, rules | Defects often arise in data, not in code |
| Test results | Unstable or frequently failing tests | Signs of turbulent areas |
| Usage data | Calls per function in production | Important for weighting, more about impact than likelihood |
The data must be mappable to a module. In practice, this is often the largest effort.
How a model turns this into a likelihood
Classic machine learning methods are often sufficient, for example logistic regression or decision trees. They learn from past releases which characteristics preceded defects.
Language models help where text is involved. They map defect descriptions to modules, summarise changes or recognise similar earlier defects.
The result is a value per module that is mapped to levels 1 to 5 of the risk matrix. How the matrix is built is explained in Risk-based testing: prioritising test effort with a risk matrix.
How to check whether the estimate is any good
The first check is a look back: would the model have ranked the modules with actual defects at the top in past releases? This gives a hit rate, the share of defects found later that sat in the modules of the top levels.
The comparison with the team’s estimate is also revealing. Where model and team are far apart, it is worth asking why. After each release, estimate and outcome are compared and the model is adjusted.
Limits
Without history there is no estimate. New modules or systems have no past; there, the experts’ judgement remains. Bad data also leads to bad estimates, for example wrongly assigned tickets or missing commits.
The model only sees what is measured. Business changes, new regulations or a new team do not appear in any log. And impact remains a human judgement: what a defect means for customers, money and the supervisor is decided by the business.
Internal audit and supervisors must be able to trace which data led to which rating and who approved it. Models whose results nobody can explain are unsuitable for this. Defect tickets and code can also contain confidential information and do not belong in public AI services.
How Cubio works
We start with what your project already has: defect tickets, commits and test results from recent releases. First we check whether this data can be mapped to modules and whether it is enough for an estimate. If it is not, the team’s rating remains, supported by clear criteria for each level.
If the data is sufficient, we start with simple, explainable methods and check them against past releases. Only when the estimate holds up there do we use it for the next release. Your business rates the impact, and test management and the business approve the rating together.
The AI follows the same guard rails as in AI in testing: no customer data in public models, operation in the EU or on your servers, every rating documented and verifiable. The risk matrix calculator shows what the matrix for your modules could look like.
