Time for improvement: 10 questions CIOs must ask their test manager

Software testing is the backbone of your software development process. Without a good testing process, you put the quality, reliability and – ultimately – the reputation of both your software and your organisation at risk.

That’s why, as CIO, you need to maintain communication with your test manager: to make sure you keep your testing process working and evolving.

Ask the following questions, and you’ll know all you need to know about the testing process, as well as how you can help to improve it.

10 questions CIOs must ask their test manager

1. What are the testing team doing to improve IT quality?

You want to know:

• How testing is helping to improve IT quality at each stage of the software development process.

2. How much does testing cost me?

You want to know:

• How testing cost is measured.
• If cost is increasing, decreasing or remaining stable.
• Where the ‘hot spots’ of testing cost are.
• What the forecasts and projections are for the next project.

3. What are we doing to train, develop and certify the test team?

You want to know:

• Whether all testing staff are certified by the International Software Testing Qualifications Board (ISQTB).
• What ongoing training is in place to make sure staff maintain this certification.

4. How do you measure testing performance?

You want to know:

• How individual and team performance is being tracked.
• Whether test analysis is being conducted and how it affects the design of future tests.
• If defect leakage is being tracked and noted.

5. How can we reduce the cost of testing?

You want to know:

• The plan to prevent further defect leakage.
• If there is a plan in place to ‘shift left and compress’.
• How the methodology for each testing project is being formed and whether costs are being taken into account.
• How testing is being automated and whether the tools used are reducing costs.
• Whether the balance of permanent to contract-based testers needs addressing.
• If risk-based testing is being implemented and whether it is effectively managing costs.

6. Do we have an up-to-date, stable, consistent testing process?

You want to know:

• If the test team is using a maturity model like TMMi to monitor the effectiveness of software testing.
• If the team do have a maturity model in place, how the test manager plans to progress the testing process on that maturity model.

7. How is the testing process being improved?

You want to know:

• Improvements which are underway.
• Improvements which have been planned.
• Improvements which are in consideration.

8. What is the departmental testing strategy?

You want to know:

• The departmental strategy for the next year.
• The departmental strategy for the next three years.

9. What can the software development team do to help testing efficiencies?

You want to know:

• How the software development team is working alongside the testing team.
• Whether the testing team require more support from the software development team.

10. What can senior IT and business management do to help testing efficiencies?

You want to know:

• How you and members of senior management in contact with the testing team can help improve testing.
  (Encourage your test manager to be honest with you. Senior management may not directly intervene in the testing process, but management can help or hinder the testing team depending on how they work together.)

Help your test manager to help you

If you maintain good communication with your test manager, you can ensure that your testing process won’t just function, it’ll get better.

It’s possible to have cheaper testing and better software quality, and a good CIO knows that supporting the test manager is the key to making that happen.

(Hat tip to Tsahi Levent-Levi for the image)