In response to my What I Love About Kanban As A Tester #1 post, Anonymous stated:
“The whole purpose of documenting test cases…[is]…to be able to run [them] by testers who don’t have required knowledge of the functionality.”
Yeah, that’s what most of my prior test managers told me, too…
“if a new tester has to take over your testing responsibilities, they’ll need test cases”
I wouldn’t be surprised if a secret QA manager handbook went out to all QA managers, stating the above as the paramount purpose of test cases. It was only recently that I came to understand how wrong all those managers were.
Before I go on, let me clarify what I mean by “test cases”. When I say “test cases”, I’m talking about something with steps, like this:
- Drag ItemA from the catalog screen to the new order screen.
- Change the item quantity to “3” on the new order screen.
- Click the “Submit Order” button.
Here’s where I go on:
- When test cases sit around, they get stale. Everything changes…except your test cases. Giving these to n00bs is likely to result in false fails (and maybe even rejected bug reports).
- When test cases are blindly followed, we miss the house burning down right next to the house that just passed our inspection.
- When test cases are followed, we are only doing confirmatory testing. Even negative (AKA “unhappy”) paths are confirmatory testing. If that’s all we can do, we are one step closer to shutting down our careers as testers.
- Testing is waaaay more than following steps. To channel Bolton, a test is something that goes on in your brain. Testing is more than answering the question, “pass or fail?”. Testing is sometimes answering the question, “Is there a problem here?”.
- If our project mandates that testers follow test cases, for Pete’s sake, let the n00b’s write their own test cases. It may force them to learn the domain.
- Along with test cases comes administrative work. Perhaps time is better spent testing.
- If the goal is valuable testing from the n00b, wouldn’t that best be achieved by the lead tester coaching the n00b? And if that lead tester didn’t have to write test cases for a hypothetical n00b, wouldn’t that lead tester have more time to coach the hypothetical n00b, should she appear. Here’s a secret: she never will appear. You will have a stack of test cases that nobody cares about; not even your manager.
In my next post I’ll tell you when test cases might be a good idea.
RSS
What I do is have my senior QA folks create the test scenario's and review them with business and technology. Once they approve them I will turn them to newer resources to script out the steps as a chance to learn since they might be involved with the execution window. Since identifying what needs to be tested is the critical piece I only have senior people do that.
Not me!
Eh, I somewhat disagree with this post. You're right, we shouldn't be teaching noobs that, oh, if you just test everything on this list, and if everything passes, you're golden! It is better to have a senior tester mentor the noobs than just give them a test case. But, I think that test cases are valuable for noobs. Here are a few reasons why.
We don't have documentation for our software for the noobs to look at because we don't have a technical document writer, and nobody in the company has time to do it. At least with the test cases, even though they might be out of date, there is some accurate information in there. If I just walked someone through that part of the program, they might forget some of the nitty gritty details of how the software works because it was not written down, or maybe I have even forgotten them when teaching.
Another reason is, the test case serves as a guide. Maybe if you didn't have the test case, the noob wouldn't have thought to test an obscure feature because you didn't know about it.
Finally, as a more senior tester, I still use test cases, and find them valuable as jumping off points. I'll start running the test case and realize, hey, I totally forgot about that feature... Let's see if it works if I do some oddball thing. I think that noobs should be taught to use them in the same way, that test cases are not the be all end all, but to use them to inspire more testing, and in some cases remind you about functionality of the program that you have forgotten.
Everyone on the team helps keep the test cases up to date. We have found that the best time to update them is, you guessed it, right when you are running them!
In any case, the noobs should definitely be doing their testing in conjunction with a mentor.... Just handing them test cases and saying, go learn, is asking for trouble.
Thanks for the good, thought provoking post!
Interesting thoughts...
Well it's a hard fight - starting with myself - in this way that I'm raised with TMap Next on one hand, but never really saw it work on the other hand - and with the world - who tends to like being embedded security.
The problem I still think to find in your view, Anonymous, is the time to update all those test cases. With this time you
1. Bore the tester to death - which is not an effective state to find bugs.
2. Take away precious time to find bugs.
I understand your problem very well on the other hand. But I think there are better solutions for those.
Something I did is I kept some kind of a user manual up to date (that I didn't take responsibility for) and work with very simple test cases that you list up as you execute them? This is something you can keep up to date as well, if you need re-usable test cases.
A mind map is sometimes a great tool to do this.
I still think following test scripts is keeping you too much 'on track' and brings you into the ' my ass is covered mode', even though I appreciate your view on seeing test cases as a starting point. I, without realizing it, as a TMapper, tended to do the same. Yet I feel like it's wrong... or maybe it is a way to get people across the bridge that the fake security test case preparation provides...