We all have different versions of our product on different environments, right? For example: If Iteration 10 is in Production, Iteration 11 is in one or more QA environments. When bugs exist in both Iterations, we have BIMIs (Bugs-In-Multiple-Iterations).
One of my project teams just found a gap in our process that resulted in a BIMI hitching a ride all the way to production. That means our users found a bug, we fixed it, and then our users found the same bug four weeks later (and then we fixed it again!). Our process for handling bugs had always been to log one bug report per bug. Here is the problem.
- Let’s say we have a production bug (in Iteration 10).
- Said bug gets a bug report logged.
- Our bug gets fixed and tested in our post-production environment (i.e., test environment with same bits as production).
- Finally, the fix deploys to production and all is well, right? The bug report status changes to “Closed”.
- Now we can get back to testing Iteration 11.
What did we forget?
…well it’s probably not clear from my narrative but our process gap is that the above bug fix code never got deployed to Iteration 11 and the testers didn’t test for it because “Closed” bugs are already fixed in prod, and thus, off the testers’ radar.
If our product was feasible to automate, we could have added a new test to our automation suite to cover this. But in our mostly manual process, we have to remember to test for BIMIs. The fact is, the same bug fix can be Verified in one iteration and Failed in another. The bug can take a different path in each environment or, like in my case, fail to deploy to the expected environment at all.
This iteration we are experimenting with a solution. For BIMIs, we are making a separate copy of the bug report and calling it a “clone”. This may fly in the face of leaner documentation teams but we think it’s a good idea based on our history.
What’s your solution for making sure bug fixes make it into the next build?
RSS
We have had similar situations where bugs were present in different projects which use partly shared code. In some cases the bug was fixed for product X but not for product Y.
Our solution to this was to simply create copies of the bug report for the several projects which makes sure that they are solved 'for good'.
This might create some extra work in some cases but it's worth it if it keeps bugs out of production
Hi,
Surely add this test to your automation tests is the best thing to do, but there's also another good solution:
if you use git, or any other DVCS, it's usually very easy to open a branch and to merge. Therefore, our iteration 10, will get a branch called Iteration10, and the next branch is on master (or iteration11).
After fixing the bug(s) on Interation10, you just need to go the current branch (master or interation11), and do merge from iteration10.
This will make sure all your fixes from Iteration10 will be on your current branch. If you have any conflicts, it will show you them.
if you have let's say 4 different iterations out there, then you have to make the merge 3 times. From the first to the second, 2nd -> 3rd, 3rd -> 4th. It works like charm, and this is one of the real power of git over svn, or any other non Distributed version control
Making copies for all versions where the bug should be fixed is probably the easiest to implement. Those can then be left open until you know when next release for each version is done.
Even more I would like to have a system where there would be only one bug report but it would be possible mark which versions are affected, in which versions the bug has been fixed and how (can vary if program has evolved in between), and how it has been tested in each.
I don't know if such systems are available already or if it would have to be implemented first.
@Yaniv: Your suggestion is a good approach when you are working with small products that don't change much between iterations. Unfortunately this is not always the case, when Iteration 11 starts to look quite different from Iteration 10 (or you have to merge a bug from It. 1 to It. 11), then your auto-merging starts to fail, and then it can be complicated to actually understand what was fixed (In Iteration 10), thus also hard to see if the scenario is still applicable to It. 11 or how it should be fixed in It. 11, because you ususally are missing the reference to the original bug report.
For large products with long histories and a large bug history I would suggest having a process in place to handle these cases. It might be duplicating the bug for each applicable iteration, or simply creating tasks that the bug fix needs to be merged - but in general, a solid process is key in making sure fixes are merged correctly!
Yaniv,
It looks like you're talking about version control for the code. I'm talking about something different; a process for Testers to make sure the programmers used the software you're talking about properly.