It’s a cliché, I know. But it really gave me pause when I heard Jeff “Cheezy” Morgan say it during his excellent STAReast track session, “Android Mobile Testing: Right Before Your Eyes”. He said something like, “instead of looking for bugs, why not focus on preventing them?”.
Cheezy demonstrated Acceptance Test Driven Development (ATDD) by giving a live demo, writing Ruby tests via Cucumber, for product code that didn’t exist. The tests failed until David Shah, Cheezy’s programmer, wrote the product code to make them pass.
(Actually, the tests never passed, which they later blamed on incompatible Ruby versions…ouch. But I’ll give these two guys the benefit of the doubt. )
Now back to my blog post title. I find this mindshift appealing for several reasons, some of which Cheezy pointed out and some of which he did not:
- Per Cheezy’s rough estimate 8/10 bugs involve the UI. There is tremendous benefit to the programmer knowing about these UI bugs while the programmer is writing the UI initially. Thus, why not have our testers begin performing exploratory testing before the Story is code complete?
- Programmers are often incentivized to get something code completed so the testers can have it (and so the programmers can work on the next thing). What if we could convince programmers it’s not code complete until it’s tested?
- Maybe the best time to review a Story is when the team is actually about to start working on it; not at the beginning of a Sprint. And what do we mean when we say the team is actually about to start working on it?
- First we (Tester, Programmer, Business Analyst) write a bunch of acceptance tests.
- Then, we start writing code as we start executing those tests.
- Yes, this is ATDD, but I don’t think automation is as important as the consultants say. More on that in a future post.
- Logging bugs is soooooo time consuming and can lead to dysfunction. The bug reports have to be managed and routed appropriately. People can’t help but count them and use them as measurements for something…success or failure. If we are doing bug prevention, we never need to create bug reports.
Okay, I’m starting to bore myself, so I’ll stop. Next time I want to explore Manual ATDD.
RSS
As wonderful as your observation sounds, in the current software environment (especially with highly diverse and globally located teams) bugs are almost a certainty. And in my experience, most UI bugs are (for lack of a better term) superficial (labels, placement, colors, graphics, etc.) and not functional (the user cannot select the yes radio button, the address validation is failing, etc). And where there are "functional" bugs, they are more the result of requirement change/creep than poor development.
Anonymous, good point. It sounds like you are saying, no matter how early you test, additional changes can not be prevented. I agree. People don't know what they want until they start using it, right?
However, change/creep is not what I am talking about. I would argue, change/creep bugs are not the kind of bugs testers attempt to find. Change/creep bugs, by definition, normally can not be found until the product is released.
In this post, I'm suggesting (and I think Cheezy is suggesting) bugs that would otherwise be coded then hunted for, could possibly be prevented from existing in even the first build of the product.
See the difference?
Nice post, Eric. That's exactly what other industries have done for quite some time.
I think that is one way to go to be able to catch apparent stuff before it gets into the development.
Another way is to "help" the users to not being able to work wrongly. It's no magic, but its a small step towards a more smart and efficient way of working instead of looking for errors after the work has been done.
"There is tremendous benefit to the programmer knowing about these UI bugs while the programmer is writing the UI initially. Thus, why not have our testers begin performing exploratory testing before the Story is code complete?"
Why aren't programmers doing this?
In my world I see programmers making the same mistakes over and over again. I've brought these issues to the attention of the programmer and management and yet nothing ever changes. I know this is the reality for a lot of testers.
How do testers train programmers and management to wake up and stop this?
I'm not trying to knock a good post - I liked your blog. But I really think you are just pointing out the obvious. Having testers create bugs based on scenarios for code not yet created will definitely help the developer because he/she can make sure that their code accounts for those scenarios. That makes perfect sense. And that is really why there is a need for thorough requirements before a developer starts to code.
In today's environment where documented requirements seem to be falling by the wayside, developers more and more are relying on the testers to guide them. To me, that is what the point of your post is although it's not really said that way. I would correct that problem by not putting the burden on the tester to create scenarios for the developer but would instead make sure that the developer had thorough requirements.