As a tester, some of the most irritating questions I have heard is, “How many test cases have you written? Is the testplan execution complete? Only this many test cases in your testplan?” and so on. I have never understood the mindset of the people who go by the number of the testcases executed for a feature to be declared tested. How can a finite number of tests be performed on a system to be qualified as bug free? A question always pops up in my mind – “Is the test case required?”
All testers will have their opinion, but this question has the ability to start a war between the various stakeholders. In my opinion the answer to the above question is both “Yes” and “No”, not necessarily in that order. And I have come to the conclusion after working nearly 4 years as a tester. Lets see how I came to this conclusion.
I was never trained to be a tester. One fine day, I was asked to join a testing team and work as a tester. I did not know much about testing apart from few terms like unit testing and integration testing. I had a few sessions of training on the application and was handed over a testplan. I was supposed to read the test cases and execute them manually marking them PASS/FAIL. I was relieved. I thought I got an easy work. Gradually, over a period of few weeks, I was able to test the whole application without looking at the testplan. It had became a habit. And with it the work became boring. I had to repeat the same steps every time.
Then one day I decided not to follow the testplan. Till that day I was ignoring all other things in the application which were not part of the testplan. That day I found some bugs. This was my first step against following the testplan. Now I did not follow the testplan more often. And the outcome seemed to be good. I realised that a tester need not confine himself to any set criteria or limit. And I also started forming an opinion that writing the testplan is a waste of one’s time. I stopped writing the testplans. This worked for quite a good time, until one day a bug was found in Production. My manager came up to me and asked if that particular testcase was tested? I told him “Yes”. He then asked me to send the testplan as the client wanted to review it. Shock. I somehow wrote a few test cases and sent him.
Two days later, another shock. I was asked to explain why all scenarios were not covered in testing? I tried to explain that all are covered. But no one agreed as they were not there on the test plan. The client was now convinced that the testing was not done properly. As they were readying for a patch release to fix the bug, I was asked to again do the testing for the whole application again, just because my test plan did not have complete test cases. And this time they wanted to see the testplan as well. I had to slog, but did it. This experience made me realise who are the people who actually want the testplan.
As a tester , we may not want to follow the testplan when we do the testing. We might want to announce ourselves as ‘Exploratory Testers’, but there are certain stakeholders who look at the testplan in a different way. For the client, it is the proof that this many testcases have been executed and what is the result. Based on that he may take the decision of releasing or not releasing the product. He may want the QA team to do another round of testing. So, I again started writing the testplans. But now this had a catch too. If any bug was discovered in Production, again we were blamed that the testplan/testing was not proper. So something had to be done.
We decided to get the tesplan reviewed by client, before the testing starts. Now, we write the testplan based on the requirement specs that we get. After a few rounds of clarification on the specs and updating the testplan, it is sent for review. The review process is sometimes lengthy as the testplan keeps on changing hands quite often as the review comments are implemented and the review done again. Before the actual testing starts the test plan is signed off and finalised. The first round of testing is usually done by following the testplan. But once it has been done, for further rounds we usually do not follow it. As the testing cycle progresses, sometimes more tests are done than present in the testplan. We include those too in the testplan. When the testing is done, the tesplan is one of the deliverable from the QA’s end.
So, I might not be using the testplan for testing, but keeping an updated one helps me in taking a stock of the features for which testing is covered. It also helps me to keep a track of the automation. If any new team member joins, I provide him the same to refer for his understanding. Also, now no one can blame me as the testplan was reviewed and signed off. 🙂