In this third blog, we look in more detail about how return on investment (ROI) for a test process improvement initiative can be calculated.

There are numerous ways of working out the potential, and actual, financial benefits of implementing test process improvements. Here we show one way, based on the method Experimentus uses and for a TMMi driven initiative.

Below is an example of a spreadsheet used to help calculate and document the costs and benefits for a project improvement plan (PIP) for a real client:

• For each area of improvement, we estimate how many person-days we believe it will take to make the improvement.
• We then agree with the client what the estimated average cost of a person day should be
  • This will include both the client’s resource costs and Experimentus’ resource costs – dependent on our level of involvement.
• These are then used to calculate the estimated cost of implementation. Here we have estimated £1,000 per day.
• We then calculate the anticipated benefit, and, by subtracting the cost, we can estimate the net potential benefit for each of the activities.

In the table above, for PIP ID 5, you can see that we have an improvement with a potential benefit of zero against it. Often we find that there are some improvements for which there are no financial benefit. The benefit they do provide is that they are foundation stones for the other changes. If these are not implemented, the benefits of the other changes would not be delivered.

These items still need to be in the costings, and as long as the overall calculation still demonstrates a sufficient benefit, there is not an issue. The advantage is that by including these zero-benefit items with the positive-amount benefits, the overall cost of the implementation is fully understood.

The above is an example of a potential benefits summary where there is positive value-add to the business.

We have included a confidence factor (in other words, how confident we are that the benefits will be realised) which can be adjusted to each situation. The implementation factor (how much could implementation costs vary from the amount originally estimated) can be positive or negative.

For this example, from experience with this particular client, we have estimated an extra 50% could be needed. The final net potential benefit is calculated by subtracting the adjusted cost from the adjusted benefit.

We hope you enjoyed this series of blogs on the need for and benefits of test process improvement. It can be difficult to justify a test process improvement initiative, but it will provide a lot of benefit, both direct and indirect. For more information on how we can help you, please take a look at our test process improvement page and feel free to contact us.

Typical benefits of process improvement

In this second blog on test process improvement, we look at the benefits that you can expect to realise as you implement new and improved ways of working.

Test process improvement benefits can be broken down into direct and indirect types. Sometimes the indirect benefits are larger, or more important, than the direct ones, so they should not be ignored. It is well worth taking into account the indirect benefits when the overall value of an improvement program is being calculated.

Because long-term investments can be allocated to improvement programs, it is also critical that the return on investment (ROI) of an improvement program is calculated accurately and consistently.

Direct benefits of process improvement

Direct benefits include:

• A rise in productivity
• Early defect detection resulting in a reduced need for live fixing

Indirect benefits of process improvement

Indirect benefits include:

• Improved staff experiences leading to increased morale and motivation and as a result, better quality work
• Increased customer loyalty
• Allows greater movement of employees within projects
• Improved working environments
• Recognition of testing as a profession
• Integration of testing into the development process, irrespective of the methodology being followed (for example following a traditional model like v- model or where agile is used).
• TMMi will help guide the introduction and use of a structured and controlled set of test processes.

Return on investment

Return on investment (ROI) is calculated by direct costs and benefits, since it is easier to quantify direct costs than indirect ones.

Our next blog will give more detail on how ROI can be calculated for a test process improvement initiative.

General benefits

While there are specific benefits, there are also general ones to consider that can be just as important and valuable, so they should not be ignored. General benefits include:

• Organisations are enabled to be more effective and efficient
• Product quality can be improved
• Increased testing productivity – potentially leading to shorter delivery time frames
• TMMi suits any development life cycle model
• There is a shift-left paradigm, moving from defect detection to defect prevention

This is not a full list, but just to set a scene for the kinds of improvements that may be expected.

As yet, there is little published information about costs and benefits from assessments. In fact, most organisations will want to keep that information confidential, particularly those that are not performing well.

Take a look at our test process improvement page for more detail on how we can help you realise these benefits.