If you find an escape (i.e., a bug for something marked “Done”), you may want to develop an automated check for it. In a meeting today, there was a discussion about when the automated check needed to be developed? Someone asked, “Should we put a task on the product backlog?”. IMO:
The automated check should be developed when the bug fix is developed. It should be part of the “Done” criteria for the bug.
Apply the above heuristically. If your bug gets deferred to a future Sprint, deffer the automated check to that future Sprint. If your bug gets fixed in the current Sprint, develop your automated check in the current Sprint.
3 comments:
Subscribe to:
Post Comments (Atom)
RSS
Good idea, but how do you decide whether it's worthwhile to have an automated check for the bug? Consider the opportunity cost - could there be a test automation task that's likely to find a more severe bug, or a bug that's more likely to occur than a bug we just fixed?
Good point Danny. This is definitely something I struggle with, and worthy of a post or larger discussion. It seems to me, those automated checks for the more important problems should already be written at this stage (i.e., when the Story is "Done"). Therefore, the remaining work is to add the check for said escape. Make sense?
My experience has been that there are practically always gaps in the test coverage, especially in para-functional areas like reliability. So it's easy to find a testing task with higher ROI than automating bug regression checks.
You can get a good return on the investment in regression testing if your development process allows for an unusual number of regressions. But you can get a better results by addressing the root cause of the regressions instead. Even shifting focus from system testing to unit testing is often a better use of resources.