Experimentus Blue Banner

Smarter Testing Solutions

I was asked very recently by a colleague from Korea whether I would be interested in helping his staff attain their highest level of recognition as a software tester, which was to have worked as a software tester in the UK. Yes you read this right; the Koreans put a lot of value in working in the UK!

As the conversation continued we started to discuss the skill set that I needed, at this point I was asked what the common entry level for a System tester in the UK was, I sarcastically said “Oh if they can say TEST, then they will get a job”, we laughed and he explained how, in Korea, a Software Tester would on average need to hold a Software Engineering degree to get a job.

We agreed how we might help each other, but the conversation started me thinking seriously about some of the issues that face the Software testing industry in the UK today. I began questioning:

1. How correct was I regarding anyone who can say “Test” being skilled enough for the UK System Test market? It was a joke but actually it resonated with me, and

2. If that is the case why do people employ such unqualified testers?

I also recently attended the ANZTB conference in Australia and remember one particular presentation by Vipul Kocher from India, on the Future of Testing. He gave an excellent view of what might become of our industry and some of the issues we all face today and will face tomorrow. Then he went quiet and as a finale asked the audience “But what is the biggest issue that will stop the test industry progressing?” Many answers flowed from the audience – developers, requirements, senior management and the like. Vipul allowed this to carry on for a while before looking out at the audience and saying “None of these, it’s you the testers; you are the biggest risk to testing today.” An interesting observation I thought.

So I started pulling these two very different situations together and what follows in this whitepaper is a very personal view of where we are today and why I now believe we are going to fail as an industry if things don’t change.

Maybe I should start with a warning here; what I have written is a broad brush view, so in no way reflects what I feel about every tester and every buyer of test resource.

I began by looking at why we had some resources working within testing that were perhaps less than qualified. The rot has set in for three reasons:

1. Year 2000, when testers of any sort were in short supply, so anyone with an interest could become a testing contractor.

2. The fact that there were (before the recession) better opportunities in the software testing contract market than any other IT contract market.

3. “Stupid buyers” who don’t understand what testing is about and just employ to effectively satisfy their conscious that they think they have testing covered.

Now, we cannot blame the test contractor for trying to earn their daily bread, nor the test resource suppliers, but I do have a real issue with what I have termed “stupid buyers” who are effectively fuelling this lowering of the entry gate for testing skills and knowledge.

A couple of anecdotes that perhaps illustrate my point:

During the Year 2000 I employed a temp whose job it was to collate and store all test artefacts for my test team. She was very efficient and when her time came to leave I sent her on her way with a glowing recommendation for her administration skills. I was however very surprised, when 2 years later, I was asked to attend an ISEB Foundation training course to review it and found this person in the audience for the course. I asked her what she was doing and she told me she was the lead tester on a major bank system upgrade for one of the big five (now three I guess) banks. She had gone there straight from working for me, as a test contractor. I asked her what skills she demonstrated to get her the job, and here is the trick, she replied:

“I learnt some of the buzz words whilst working within your team, so talked my way in because the Manager interviewing me knew nothing about testing”

Whose fault was that?

On another occasion I was asked to review why, after testing had completed on a project for a potential client, they had had serious issues in production. Having reviewed the tests the issue was very clear, the organisation had asked for test coverage to be at least 90% and they received confirmation of the fact that 95% coverage had been achieved.

So you may ask why there was an issue. The issue was in the understanding of coverage. The organisation didn’t understand it at all, they just had heard that it’s a good measurement, and the tester’s view of coverage was the percentage of tests they had run that had passed.

When quizzed on what tests have been run and what coverage of the requirements they had written tests for, he said he didn’t understand what we were talking about, he had written 100 tests and 95 had passed. “What tests?” we asked “The ones I thought we needed to run based on my knowledge of the system” he answered.

“So did you read the requirements?” we asked... “No” was the answer.

Some would say the organisation got what they deserved. What they had done is employ someone who was not qualified to do the testing and trusted him to make the decisions for them!

A significant amount of test resource today has been brought kicking and screaming out of the business itself, so they knew the systems they were testing, a lot more than they knew how to test. So they managed to get the job done but whether they did it efficiently and effectively is not very clear.

My final observation is that there is a general perception that being a tester is easy and for some reason companies feel that although they employ administration staff and expect them to understand the business very quickly, they don't think testers are that clever. As it seems that understanding of the business (I had a job spec the other day with the only must have requirement being experience in Cheque Clearing – testing knowledge was in the nice to have list!) are bought in preference to testing skills.

Now there are a few notable exceptions, but in general it would take about 2 weeks to train someone up to understand the business, but it would, in my estimation, take at least two to five years of real hard earned experience and training to become a good, proficient and efficient test resource.

It is fair to suggest that anyone can test, we all do it every day, but testing software is a specialist skill which ensures that code is tested efficiently and effectively.

So there’s the issue; we have “stupid buyers” buying untrained and unskilled resources from resource providers who sell anything to anyone, so how would I suggest things get resolved?

1. The first step is easy – go to the root cause of the issue and educate the buyers! Oh such a small sentence but such a complex target. Before you can educate them we need to make them realise they need to learn. So to point 2...

2. Resource providers need to stop bowing to the buyers’ less than efficient demands and actually try to add more value by working with the clients to understand what is required and how efficiently they could resource it. They need to understand where certain skills are needed. Unfortunately there is big money in selling to the ignorant and I have seen good resource companies who have tried to influence a buyer being gazumped by another ruthless resource provider who will do what the client wants and charge less to win the business, even if it doesn’t help the client just for the few pounds they will earn!

However, there are suppliers in the System Integrator market who are moving into the test arena and refreshingly are trying to provide the best possible solution for the client. There will always be a place for bodies to do testing, so resource providers out - there I am not saying you shouldn’t be doing business; just that you should do the right business.

3. The other most important area is that as an industry we understand the different roles that do exist within testing. Some of these roles need to be undertaken by skilled resources others could be delivered using less skilled resources. They are, for example:

a) Test Architect – defines how testing will be undertaken during a project, works with the other Architects and designers on the project to ensure prevention of defects is considered along with detection where appropriate. This is a highly skilled role, requiring not only good testing and project management qualifications but years of real experience. The use of the right resource for this role will determine the success or failure of the testing activity.

b) Test Director/Programme Manager – ensures that testing is well planned across a series of projects (in line with the approach defined by the Test Architect or Company test approach/framework or strategy). A highly skilled role requiring a degree of test knowledge and experience but realistically needing significant skills in the Project Management. A good Programme Manager who is willing to learn enough about testing could do this, but a good tester with a little Project Management experience would struggle.

c) Test Manager/Lead –Manages one test project or many small unrelated projects

d) Test Analyst – These resources need to be expert analysts and have an in-depth knowledge of testing. Their role is to define the test conditions, test cases and test scripts

e)Test executers – dependent on the detail included in the test scripts these resources could be unskilled in testing. They are simply the admin resources who execute the tests in line with the instructions they have and record the child results

Most organisations believe testing is manned by Test executers, who just push buttons and want to see tests being run from day one. How naïve; and all of those who believe this obviously don’t really consider quality as an important software attribute! Look at Microsoft - they have learnt - I understand not so long ago they reviewed their test resources and now employ only those with the right skills.

If some believe testing is all about pressing buttons as soon as possible, I would ask the question “what other significant activity in any walk of life would anyone undertake without understanding the approach and plan?/span>” Even when you simply go shopping, in your head you plan the time you will leave and what will happen when you get home. Why would anyone consider it right that a test executer with no real knowledge of testing, or maybe only holding the ISTQB Foundation certificate (meaning they understand the terms but it is no measure of their capability to test), should be expected to run all of the tests needed to validate and verify the software.

If the testing profession stands any chance of having a successful future, one of the major contributors will be the need to be looking at Smarter Testing Solutions.

Geoff Thompson, Delivery Director