Well…yes. I would.
The most prolific bug finder on my team is struggling with this question. The less the team decides to fix her bugs, the less interested she grows in reporting them. Can you relate?
There is little satisfaction reporting bugs that nobody wants to hear about or fix. In fact, it can be quite frustrating. Nevertheless, when our stakeholders choose not to fix certain classes of bugs, they are sending us a message about what is important to them right now. And as my friend and mentor, Michael Bolton likes to say:
If they decide not to fix my bug, it means one of two things:
- Either I’m not explaining the bug well enough for them to understand its impact,
- or it’s not important enough for them to fix.
So as long as you’re practicing good bug advocacy, it must be the second bullet above. And IMO, the customer is always right.
Nevertheless, we are testers. It is our job to report bugs despite adversity. If we report 10 for every 1 that gets fixed, so be it. We should not take this personally. However, we may want to:
- Adjust our testing as we learn more about what our stakeholders really care about.
- Determine a non-traditional method of informing our team/stakeholders our bugs.
- Individual bug reports are expense because they slowly suck everyone’s time as they flow through or sit in the bug repository. We wouldn’t want to knowingly start filling our bug report repository with bugs that won’t be fixed.
- One approach would be a verbal debrief with the team/stakeholders after testing sessions. Your testing notes should have enough information to explain the bugs.
- Another approach could be a “supper bug report”; one bug report that lists several bugs. Any deemed important can get fixed or spun off into separate bug reports if you like.
RSS
I agree that they should continue reporting them. While the issues might not be considered important by the customer today, one never knows what the future holds.
For example, if I file a bug based on accessibility, maybe the customer is less concerned about that right now, but that doesn't mean they won't care about it later. If they decide that's a market differentiation that could give them an advantage, having that older bug report could be valuable.
I agree that they should continue reporting them. While the issues might not be considered important by the customer today, one never knows what the future holds.
For example, if I file a bug based on accessibility, maybe the customer is less concerned about that right now, but that doesn't mean they won't care about it later. If they decide that's a market differentiation that could give them an advantage, having that older bug report could be valuable.
Are they being rejected outright, or placed in a lower priority queue?
Good rapport with programmers has helped lower "rejected" numbers on my projects in recent years to almost zero.
Of course, it depends on the attitude of management. Some places I've worked just let the low priority defects pile up. In that situation it's better to open them up and let them fester, and not worry about it.
In my experience, if I found a bug during test and it was deemed not important enough to fix, then the same issue was found in production - the first question was always "Why didn't we find it in test?"
So in this case it's CYA. If I hadn't reported it, even though I found it, it wouldn't matter.
If I reported it and it was never found in prod, it still wouldn't matter.
Anonymous, the answer to your question is: Both. In some cases placed as low priority with no intention to fix in the foreseeable future. In other cases, told not to bother logging it because we need time to market more than quality.
We run into this issue all the time. It's discouraging to enter issues you know we don't have the bandwidth to fix at the moment but hope that we will in the future. Also, we want to have the issue logged so that in the future if the issue is brought up you can cya and say you reported it already.
Hi Eric,
Now this is a tricky question that you have answered in your article. From my personal experience, I would recommend that each bug that a tester founds should be reported. When I've started working as a software tester, I used to submit all the issues that I could found. After a period due to the fact that the devs wouldn't fix all my issues, I've started to not report all the bugs that I was finding or I used to report them verbally to the project supervisor. This was a bad idea because some of the issues were reported by the customer and looked bad at my monthly evaluation.
At first I was very disappointed that all the bugs that were submitted by me weren't fixed by the developers, and I've felt that I wasn't contributing to the development of the application, but I've found out that those were added to the development queue for future fixes.
So I can say that it's better to submit all the bugs that you've found and give your best, than let some bugs reach to the customer and affect the trust of your team mates in your work.
Also submitting all the bugs that you find is a plus when working as a freelance tester because most of the freelancing companies pay for bug reported, and more bugs means more income to the tester.
Deciding to report a bug (and to report it well) is typically within our control.
Deciding to fix it (or not) is typically not within our control.
I choose to worry about things I can control, and not so much on things I cannot control.
That said, they might not be fixing some bugs now but might very well choose to fix them later (perhaps when time allows, perhaps when the bugs are more immediately impactful, perhaps when a new Product Manager arrives, etc. That's a business decision.
In my opinion, knowing that a bug exists, and not bothering to report it is virtually always a bad thing. While not exactly the same thing, this might help: http://www.allthingsquality.com/2010/04/non-reproducible-bugs.html
Thanks for your thoughts, Eric. That's a question I've pondered a lot recently, though I settled on my answer years ago. If you have to ask this question (that is - should I bother to report this bug), then it may be that you're asking the wrong question.
When I was a development manager, I did everything I could to encourage reporting bugs as well as other observations and suggestions about the software. If there's a storm brewing, closing the curtains is not going to stop it, at least in most cases.
If a team member sends me a 20 page weekly status report, that's a problem for me. It is wasteful for me to spend my time distilling a poorly written report. But too much feedback is a worthy challenge for any software organization to contend with. There are options for easing this burden without gagging your most important stakeholders.
Development leadership should help contributors understand that reporting a perceived bug doesn't guarantee immediate attention, or any attention at all for that matter, but that it is appreciated and expected. So for me the real question becomes:
If these people don't want to hear from me, is it time to find a new vendor/employer/group/role?
It would be a failure to do our jobs as testers if we didn't report every bug we found. It's not our call to decide which bugs are not worth fixing, and by not reporting one, we are doing just that.
In addition to reporting bugs in hopes they will be fixed, we also report them so they are recorded somewhere. These records can be shared in release notes, but also protect our testing teams in the event a user or client comes back to is, very unhappy that a bug was not fixed.
By reporting everything, we provide several services to ourselves:
* We show our worth to our company.
* When a user is upset about a known bug, the heat falls on the people who chose not to fix it, rather than our own teams (for either failing to find a bug, or for deciding it was not worthy of being reported), and the problem can be addressed in the appropriate spot.
* This knowledge may cause our development teams to think about how to prevent these types of bugs in future releases.
Reporting a lot of bugs even if they are not being fixed can aid in identifying the real problem.
Bug reports are often symptoms of a deeper problem, be it in the way the team works, a team member in trouble or a troublesome third party module in the solution, for example.
Analysing the data, the bug collection, could / should help revealing the real issue.
So keep reporting them bugs but don't forget to also use them, analyse the data!
We see that too.. often related to content/data. This starts wars.
That is what both side should consider
For testers:
* A bug is a bug is a bug. When you see it, report it. Let your test or project manager know that you’ll report these kind of issues even if they are potentially content-related. Only this way, you can be sure not to miss anything important.
* Draw conclusions from previously reported issues and comments.
* Try to get a feeling for your project and make sure not to address problems you're actually causing yourself.
For project and test managers:
* Don’t dismiss these issues as minor troubles you don’t want to bother with. They can easily turn into big ones at any stage of your project. Before go-live, you want everything to be fixed, not only functional problems.
* Problems that appear to be data-related at first sight might in fact be functional. Handling them within the regular test process facilitates their correct fixing.
* Don’t hesitate to tell your testers if they are causing the problem themselves. Granted, this might be an uncomfortable situation at first, but it will definitely trigger an Aha! effect everyone in the team can benefit from.
* Give content-related issues a low severity. This way, you can focus on troubles that are more pressing and, at the same time, ensure that nothing gets lost and QA will still retest everything.
* Inform your test team about the system’s architecture and how content and date influence its behavior.
* Also let your test team know when certain data aren’t available yet and when you don’t want the resulting problems reported. Testing is way easier when having such information in advance.
We create Parent tasks or a category for these.
So bugs which might or might not get fixed are assigned to this pool of "nice to have fixed" and sometimes they are even moved to patches or small releases if there is a enough dev time.
If it is never intended to be fixed, there is still a catagory for them so future testers coming across them in ad hoc testing will not log a duplicate.