I love all the insightful responses to my To Bug Or Not To Bug post. Contrary to the voting results, the comments indicated most testers would log the bug.
“When in doubt, log it”
“Always error on the side of logging it”
“Log everything”
“The tester needs to log the bug so you have a record of the issue at the very least”
These comments almost convinced me to change my own opinion. If I were the tester, I would not have logged said bug. I have a model in my head of the type of user that uses my AUTs. My model user knows the difference between a double-click and a triple-click. And if they get it wrong, they are humbled enough not to blame the software.
But the specifics on this bug are not my main thought here.
Within the last 6 months, I’ve started to disagree with some of the above comments; comments I used to make, myself. As testers, it’s not up to us to decide which bugs to fix. I agree. But since we have the power to log any bug we choose, we need to make sure we don't abuse this power.
- Bugs create overhead. They have to be logged properly with repro steps, read and understood in triage meetings, tracked, and assigned destinations. Bugs linger in developer bug queues, sometimes with negative connotations. All these things nickel and dime the team’s available work time.
- Your reputation as a tester is partly determined by the kinds of bugs you log.
RSS
Being one of those who said "log the defect", I see your point of view as well. So I guess it comes down to more of a CYA activity.
I've been on the end of the argument where the testers knew about the problem, but didn't log the issue for the same reasons you propose, only to have it come back later and haunt us as a production test escape. At the very least if the problem is reported up front, everyone has an opportunity to reject the defect based on being informed. Then if it's found to be a more urgent issue in production, we have a starting point for repro steps, etc. and can reopen the defect to address.
Sorry, Joe! Whilst walking my dogs in the rain, operating my iphone through a ziplock bag, I accidentally pressed comment "Reject" instead "Publish". Apparently Blogger does not give you a second chance. Here is your comment.
"I’m giving this tester the benefit of the doubt. She has a very good track record of predicting user behavior. Her actual decision was to log the bug and offer to reject it if users don’t encounter the issue during UAT."
What does "reject it" mean here? If we mean "we can decide not to fix it later", then I agree.
"Another approach would have been to not log the bug unless the users notice it. Which is less work?"
Is "less work" the right question here?
In most shops, waiting until users notice it would be a "bad thing". Otherwise, I suppose you could skip the testing altogether (that's likely to be "less work", at least for the tester), and just wait for users to complain.
Perhaps in this shop, the dynamic is different?
Here's a framework for how I think about the report-the-bug-or-not question, in a different context:
http://strazzere.blogspot.com/2010/04/non-reproducible-bugs.html
I apply a similar reasoning here - report the bug!
Joe,
Yes, that's what I (and the tester) meant by rejecting the bug.
IMO, "less work" is the right question, if the outcome is the same. I didn't explain it well. In this scenario the bug got logged but the team agreed to only fix it if the users catch it. Users don't benefit by a logged bug unless it's fixed, right? So if you are not going to fix a bug until a user finds it, you gain nothing by logging it.
I like your framework but the alleged bug in my scenario has clear repro steps. We just couldn't agree if it was a bug or not.
"Users don't benefit by a logged bug unless it's fixed, right?"
Actually, there's often lots of value.
- My bug reports indicate how to reproduce the bug. Users/customers may report superficial symptoms but not know how to reproduce it
- Bug reports may have been investigated before being deferred. The results of the investigation are usually captured in the bug report.
- Knowing when the bug first appeared (in which build, before the initial release or after subsequent patches, etc) is often an important debugging step
Any or all of these can lead to more rapid fixes.
The only value to not reporting a known bug seems to come from
- avoiding the work involved with physically writing the bug report
- losing the plausible deniability ("No, we didn't know that..")
Basically, I don't want my testers pre-filtering bugs based on what they suspect is not important, or they suspect won't get fixed. They may get lucky, and guess right. But they may guess wrong. We can avoid the problem by simply reporting the bug.
Perhaps someone at Google decided not to bother to report the "lack of a second chance" bug?
I think its the tester's responsibility not only to log bugs (just to cover his act) but also to work with the developers to get it fixed. Otherwise its not of much use.
It's funny how the developer tries to justify that this is a "bad habit" of the user!
It the solution the developer is proposing is too convoluted, there probably is another way to solve this issue.
If the tester thinks that this issue will lead to a bad opinion about the software quality, it sure is a bug but has to be pushed by the tester to get fixed. Mostly developers do not consider cosmetic bugs as serious enough to be resolved but a simple typo can significantly affect a user's opinion about the software.
Surya Dodd
Coroware