Invigorated by the comments in my last post, I’ll revisit the topic.
I don’t think we can increase our tester reputations by sticking to the credo:
“Raise every bug, no matter how trivial”
Notice, I’m using the language “raise” instead of “log”. This is an effort to include teams that have matured to the point of replacing bug reports with conversations. I used the term “share” in my previous post but I like “raise” better. I think Michael Bolton uses it.
Here are a couple problems with said credo:
- Identifying bugs is so complex that one cannot commit to raising them all. As we test, there are countless evaluations our brains are making; “That screen seems slow today, that control might be better a hair to the right, why isn’t there a flag in the DB to persist that data?”. We are constantly making decisions of which observations are worth spending time on. The counter argument to my previous post seems to be, just raise everything and let the stakeholders decide. I argue, everything is too much. Instead, the more experience and skill a tester gains, the better she will know what to raise. And yes, she should be raising a lot, documenting bugs/issues as quickly as she can. I still think, with skill, she can skip the trivial ones.
- Raising trivial bugs hurts your reputation as a tester. I facilitate bug triage meetings with product owners. Trivial bugs are often mocked before being rejected": “Ha! Does this need to be fixed because it’s bugging the tester or the user? Reject it! Why would anyone log that?”. Important bugs have the opposite reaction. Sorry. That’s the way it is.
- Time is finite. If I’m testing something where bugs are rare, I’ll be more inclined to raise trivial bugs. If I’m testing something where bugs are common, I’ll be more inclined to spend my time on (what I think) are the most important bugs.
It’s not the tester’s job to decide what is important. Yes, in general I agree. But I’m not dogmatic about this. Maybe if I share some examples of trivial bugs (IMO), it will help:
- Your product has an administrative screen that only can be used by a handful of tech support people. They use it once a year. As a tester, you notice the admin screen does not scroll with your scroll wheel. Instead, one must use the scroll bar. Trivial bug.
- Your product includes a screen with two radio buttons. You notice that if you toggle between the radio buttons 10 times and then try to close the screen less than a second later, a system error gets logged behind the scenes. Trivial bug.
- Your product includes 100 different reports users can generate. These have been in production for 5 years without user complaints. You notice some of these reports include a horizontal line above the footer while others do not. Trivial bug.
- The stakeholders have given your development team 1 million dollars to build a new module. They have expressed their expectations that all energy be spent on the new module and they do not want you working on any bugs in the legacy module unless they report the bug themselves and specifically request its fix. You find a bug in the legacy module and can’t help but raise it…
You laugh, but the drive to raise bugs is stronger than you may think. I would like to think there is more to our jobs than “Raise every bug, no matter how trivial”.
RSS
Maybe we should remember what a bugs is. Personally I like to use Jame Bach's definition “A bug is something that bugs somebody who matters.”
By using that definition, if it's a trivial bug, it's something that doesn't matter to somebody that matters. So it's ok, for me, to not report it.
We work with others, we became to understand them. So, when we decide not to report a trivial bug, it's because we can presume there's nothing that really matter to anyone.
In fact, each time a reported bug is rejected/cancelled, we usually adapt our testing to it. The same when a bug is mark as important. It reorients our testing efforts where it provides value the most. We will "ignore" some bugs simply by not testing for them.
Francis, you have really got my head spinning now. Because, I also prefer that definition, "A bug is something that bugs somebody who matters."
If we use that definition then the trivial um..."mistakes" we are discussing, aren't "bugs", right? If it doesn't bug someone who matters, it's not a bug.
You have just invalidated both my posts! Perhaps the Context-Driven-Test leaders have already worked this out. My blog post could have been titled, "Coding Mistakes That Don't Bug People Who Matter Are Not Bugs, So Don't Raise Them" (or a more concise version of that).
I also realize, the trivial mistakes I have been thinking of are mostly coding mistakes. These have often been classified as "bugs" under a more traditional definition of "bug". The problem is, most testers still think of these as "bugs" even if they don't bug someone who matters.
Sigh...
I think my points are still relevant for most testers. The lack of a common accepted tester language makes the discussion awkward.
The only "trivial" bug that makes sense not to log (based on your and/or M.B.'s opinion) is :"The stakeholders have given your development team 1 million dollars to build a new module. They have expressed their expectations that all energy be spent on the new module and they do not want you working on any bugs in the legacy module unless they report the bug themselves and specifically request its fix. You find a bug in the legacy module and can’t help but raise it…"
I like how someone else put it. QA has to CYA. A conversation about a trivial bug will be quickly forgotten. At the end of the day (or when the release gets to Prod), someone may point the finger at QA for "missing" a "trival bug"...again, especially when the end-user is annoyed by it or reports it. Any bugs found in Prod is a reflection on QA and the quality of the produce...however "trival."
JJW,
Have you considered that by NOT raising trivial problems, the quality of your product may increase?
That is certainly my opinion and that is why I am proposing this change in tester behavior. Less time spent on trivial problems = more time spent on non-trivial problems.
Again, if a user reports a problem you failed to raise, you screwed up. I am not suggesting to suppress problems that might disrupt users.
I think part of what makes a great tester is being able to draw your own line for what to report and what to not report. Since I'm not familiar with your field, a couple of your examples bother me - maybe you'd like to elaborate on them.
"Your product includes a screen with two radio buttons. You notice that if you toggle between the radio buttons 10 times and then try to close the screen less than a second later, a system error gets logged behind the scenes. Trivial bug."
All I can think here is "Are you sure?" What does the error say? Is it possible there are other situations where we may encounter this error. This one seems too full of unknown unknowns to immediately reject as trivial.
"The stakeholders have given your development team 1 million dollars to build a new module. They have expressed their expectations that all energy be spent on the new module and they do not want you working on any bugs in the legacy module unless they report the bug themselves and specifically request its fix. You find a bug in the legacy module and can’t help but raise it…"
How bad is the bug? Is it something that could result in a liability? What if you think it could impact the new module eventually but can't prove it right now? I'm comfortable enough with my work and my cross-team and -company relationships that I might tell them they're wrong. What I won't do is call a bug trivial just because someone else told me to ignore that area.
Judgement calls are a crucial part of our work and, as testers, we need to be aware of that and aware of curating our reputations. I have found that if I do log a bug which is probably trivial, but indicate that it's probably trivial, it doesn't hurt my reputation at all, doesn't hurt the quality of my product, and I also sleep well at night.
I think your statement about worrying about trivial bugs may actual hinder that overall quality of the product. Its important to remember that not only our time as testers are limited but the developer coding the project is as well. Do I want them wasting time on trivial bugs? NO!
Whenever I am working on a project that I know will go through UAT, I figure if there are small issues that are not functional related I'll let them decide if its worth reporting. If it bothers the user then OK, lets fix it. Otherwise let it go. If there is no UAT then I bring them up towards the end of testing but I make sure the developers know they are low priority.
Thanks, Patrick. Yes, you get it! This is also a conversation about time wasted. Some testers don't get the trade-off, spending time on trivial bugs brings us closer to missing the important ones.
Severity of a defect is in the eye of the beholder.
I tested a web application once and noticed that the name of my company was not spelled correctly. And on another page, the name of the product had the same issue. So I see these as minor and did not report them til the end of the project when I was not finding critical defects. The product owner agreed they were minor and sent the product to release. During the release checks, marketing reviewed the open defects and rejected the entire project back to engineering. They said that branding is the most important part of the product, which included getting the names spelled correctly. And so the names were updated and everything needed a full regression to verify nothing broke. IE: the project shipped late. Lesson learned and so I set up automation to check for all the common spelling issues at the start of a project for the company name and the product names. An interesting note, the automation checked the raw HTML and so it found the same issue in comment blocks a lot more often than in visible text.
Also, I have found that reporting small issues is good even if they are not resolved. On the next project, the same developers will then have those trivial issues in their mind and maybe not make those mistakes to begin with. This way all the products improve in the long term.