What is the relationship between these two objects?
How about these two?
This, I’m afraid, is how testers (myself included) often see software modules…like black boxes. Their relationships are hidden from us. We know the programmer just changed something related to the seeds inside the orange, so we ask ourselves, “How could changing the seeds inside the orange affect the toaster?”. Hmmmm. “Well, it sure seems like it couldn’t”. Then, after a deployment, we’re shocked to discover the toaster starts burning all the toast.
Why didn’t the programmer warn us? Well, just because the programmer understands the innards of the orange, doesn’t mean they understand the innards of the toaster. In fact, based on my experiences, if there is enough work to do on the orange, the orange programmer will happily take it over learning to code for the toaster.
So here we are, left with nothing better to do than regression test the toaster, the jointer, and the flash light, every time the orange changes. No wonder we spend so much time regression testing.
In conclusion, maybe the more we learn about the architecture behind our software system, the fewer regression tests we’ll need to execute. Instead, we’ll better focus our testing and make the invisible relationships visible for the rest of the team…before development even begins.
RSS
Eric,
Awesome analogies!
My experience has shown that the better the tester, the more that these hidden connections are understood, even if the developers should know they exist, but don't.
This has been the reason I constantly defend the need for testers on projects and not to rely solely on dev testing. Developers build things, testers test - and the foci of each do not necessarily intersect.
This is a nice metaphor you made, but it's quite exaggerated.
Usually, if two pieces of software have dependencies, developers and testers that work on either one of them know, or should know about them and testing can be done on the area that reacts to the modified section of the other piece of software.
Also, if, say, the toaster depends on the orange and the orange depends on the flashlight, if the flashlight has been modified, and the orange has been tested and has absolutely no functionality changes, there shouldn't be any need to test the toaster, since the orange and the toaster have been tested before, and the changes done to the flashlight have not affected the orange in any way.
P.S.: It's always nice to read your blog posts. Keep them coming :).
I totally agree, but you should still have automated safe guards in place to make sure client side breakages don't fly under the radar.
Lots of people are using services like phantom.js or selenium to help them out these days, but http://html5testing.co/ is the best and cheapest way to keep things in check.
Hi Mark,
I've had different experiences. It may be that my products are more complex than yours. Developers and testers may be aware of say 9 dependencies, but there is often a 10th that surprises us.
Nice post.
I totally agree with the point that testers need to know a good amount of internals in order to frame better tests. That is also one of the reason, why they are in the team right from the beginning. I mean even while designing the architecture, their questions and suggestions are vital. They can ask vital questions at those early stages itself. But not all projects can have the luxury, that might be one of the reasons for lack of knowledge or lack of exposure of the internals.
Regards
Rajaraman R
Programmer's Motivation
Hi Eric,
Sometimes the regression testing is a nightmare for testers. The worst situation is when a developer makes a small modification to a module and we are forced to test the entire application because that module affects key features.
If the application developed is not a huge one, and the regression testing can be performed by 1-2 testers, I would suggest that this type of testing should be performed by rotation. Why? I've faced situations when testers missed bugs in regression testing because they've lost interest and hated doing the same test cases over and over just like robots.
Also I think that each developer that works on a module can create a good documentation for testers, so this way the testers can figure it out if there are any relations between the modules.My opinion is that we have good documentation and information we can reduce the number of hours performing regression tests.
If the development schedule is very permissive and there are no pressures from the customer, a good thing would be that regression testing can be scheduled to be performed at certain dates and not each time a developer makes an important implementation.
Another solution would be to implement automation testing in order to speed up the process.
A big question is: when should we do regression testing? Do we choose the perfect time to do regression testing? We don't know the relations behind modules, so doing regression testing each time a module has modifications doesn't seems to be quite time efficient.