It seems to be that risk based testing is an approach that everyone knows they should be doing – but when asked to justify that on the spot, it can be hard to come up with concise answers.
In my experience, there are lots of myths and misconceptions in the testing industry around how risk based testing should be implemented, so I’m going to cover why and how to take a risk based approach.
What I’ll cover about the risk based testing strategy:
- Why your business needs risk based testing
- What is risk based testing?
- What are the benefits of risk based testing?
- Where do you start when implementing a risk based testing approach?
- When should you implement risk based testing?
- Analysing the success of your risk based testing
Why your business needs risk based testing
As testers, all of our work revolves around risk. Software testing aims to alleviate product risk (which you can read more about here), but how do you know which tests are most important?
Often testers will start with the standard tests that they’re most familiar with or are easiest to facilitate. You may think you can guess a project’s biggest risks from your previous experience, but in actuality, every product has a different set of faults and failures that even the most experienced tester might not predict. By using risk based testing, you’ll zero in on what that specific software’s most likely and most impactful risks are (not necessarily always the same thing) and prioritise those tests.
What is risk based testing?
By spending the extra time at the start of the project to create concise and comprehensive risk information early on, potential issues with the product can be predicted and prevented.
What are the benefits of risk based testing?
If you’re not taking this approach, you might be wasting your time. The key benefits of risk based testing are:
- Prevent bugs before they happen. By focusing on likely risks early you can predict and prevent problems down the line.
- Demonstrate your value as a tester. If your work is preventing issues in the development of the software and saving time and money at the start of the project, you’ll have more sway later on if you need to push back a go live date to run more tests.
- Be more likely to be finished by the go live date. By focusing on the most risky areas first and maybe even descoping unnecessary tests, you’ll be more likely to have completed all the necessary tests before your deadline – so even if you have more testing that could be done, it will be on non-critical areas.
- Be more confident in your testing. Knowing that you’re going to cover all of the greatest risks means you can have greater confidence that your testing is comprehensive enough.
- Defend your decisions more easily. By creating a detailed risk profile, you’ll be able to justify your testing decisions quickly and easily if they’re challenged.
Risk based testing makes sense for any project. In my opinion it is simple: if you’re not assessing and understanding risk before you’re testing, then you’re not testing right.
Where do you start when implementing a risk based testing approach?
You want to be starting your risk based testing approach as early as the planning stage. Ideally put in a process to identify possible risks in conjunction with business users and business analysts. To create a developed risk based testing strategy, it’s vital to have buy-in from interested stakeholders and get their involvement at this stage. If you don’t, you’re more likely to run into problems and push-back later down the line.
When categorising risk, it makes the most sense to examine what’s most important to the end user. This way you’ll be able to assess the priority of each risk based on factors that are truly relevant to your project, instead of your own personal ideas on what is or isn’t necessary.
An easy way to prioritise risk: impact and likelihood
The easiest and quickest way I’ve found to prioritise your product risks is to create a risk analysis chart by identifying and multiplying the impact and likelihood of each risk.
The impact of the risk
Firstly, you need to look at the impact of that risk: if it happened, how badly would it impact the business on a scale of 1-5? If it means you can no longer sell anything through your website, that’s a 5. If it’s a possibility that you might have a spelling mistake on page 3 of your website, it’s a 1.
This is where your stakeholder involvement comes in. Different stakeholders will have different opinions on what constitutes a 1 or a 5. Combining their inputs will ensure everyone feels heard and should mean that you have to spend less time justifying your testing choices.
The likelihood of the risk
The second factor to consider is likelihood: how likely is that the risk will manifest? While it’s tempting to work this out by relying on your knowledge of past projects, make sure you are evaluating the context of the risk and the product involved. This will give you the most accurate assessment.
Consider things like:
- Is that feature integral to the software?
- Is it just a small update or a new and untried technology?
Thinking through questions like this and evaluating the software’s history should give you a decent prediction of the likelihood of each risk.
Then same as above, rate this from 1-5 where 5 is highly likely and 1 is very unlikely.
The final product: an overall risk factor
Multiplying these two numbers (impact and likelihood) will give you a clear overall risk factor. This is your new priority list. Instead of going through the motions of your standard testing process, prioritise the highest risk factor tests first.
Implementing a risk based testing approach in agile
The risk based testing approach suits the agile way of working because you front-load the most important work first. If done correctly, the Agile way of working is inherently risk-based anyway, so it lends itself to this approach.
In Agile projects, quality risk analysis should happen in two places:
- Release planning where business representatives who know the features in the release provide a high-level overview of the risks and the whole team, including the testers, can help to identify and assess the risks.
- Iteration planning where the whole team identifies and assesses the product (quality) risks to be covered in the iteration.
In theory, both high and low risks are planned for in terms of the number of story points (time and effort) they will involve. Iteration planning needs to cover all the risks as it normally determines what the go-live date is. If you develop and test the high risk items last, you’re likely to get less time to put the wrong things right.
When should you implement risk based testing?
It’s best to implement your risk based testing strategy as early as resources allow within the product life cycle.
Retroactively fitting risk based testing to a project past the planning stage is better than nothing, but it’s often harder to secure stakeholder buy-in and you may lose some of the benefits of preventing bugs before they happen.
Risk based testing is not effective as a firefighting tool, though I do see testers regularly try to use it as one. The earlier you implement a risk based testing approach, the better.
Analysing the success of your risk based testing
As testers we’re always assessing the success of a product, but it’s also vital to check in on your own testing process and analyse that too. When you’re trying to evaluate how your risk based testing strategy is going, consider these questions:
- Are you the only person assessing the risk?
From my experience, if the testers are the only people assessing the risks based on their knowledge, unexpected issues usually crop up. Don’t be afraid to get multiple stakeholder input on how they rank the risks you’ve identified.
- Are you providing a nuanced picture of your progress?
Don’t forget to make the most of your prioritising when creating your reports. Does your reporting make use of the relative priorities of tests to give a more qualitative view of progress, rather than just a quantitative one?
- Are you analysing your output?
It should be standard practice to report on the output of your testing efforts, especially in regards to prioritisation. Obviously give this to stakeholders, but make sure you’re also using it to iteratively improve your process too.
- Are you communicating regularly with the rest of the team?
The most important part of our job is communicating problems. If you have outstanding tests that are high priority, you can’t go live without completing those tests. The earlier your team knows, the better.
- Have you tested all your high priority risks?
Even if you’re using an agile approach, you probably have a set date for go live. By checking back on your prioritisation document, you can know all of the most important tests are covered off before you get to that deadline.
Want more advice on how to implement a successful risk based testing process?
Experimentus have been using the risk based testing approach for over a decade, so we’re happy to field any more questions you’ve got on how best to implement it for your current project.
Our tried and tested Experimentus Risk Based Testing method can help you get the most out of your testing efforts. We can work with you to tailor this approach to your specific needs, and then teach and train your team on how to implement a risk based testing strategy correctly.
We’ll even be on hand throughout the implementation process to help and support you to ensure your success. If that sounds like something that would improve your project, contact us using the form below for a no obligation discussion of your needs.
Leave a Reply