During a recent exchange about the value of automated checks, someone rhetorically asked:

“Is automation about finding lots of bugs or triggering investigation?”

Well…the later, right?

  • When an automated check passes consistently for months then suddenly fails, it’s an indication the system-under-test (SUT) probably unexpectedly changed.  Investigate!  The SUT change may not be directly related to the check but who cares, you can still pat the check on the back and say, “thank you automated check, for warning me about the SUT change”.
  • When you design/code an automated check, you are learning how to interact with your SUT and investigating it.  If there are bugs uncovered during the automated check design/coding, you report them now and assume the automated checks should happily PASS for the rest of their existence.
  • If someone is organized enough to tell you the SUT is about to change, you should test the change and assess the impact on your automated checks and make necessary updates.  Doing so requires investigating said SUT changes.

In conclusion, one can argue, even the lamest of automated checks can still provide value.  Then again, one can argue most anything.

1 comments:

  1. Kevin said...

    This is a good point. I worked in a group that thought automation should replace a tester and if it didnt find defects then it was a failure. But automation is an extension of the tester. It points to where the tester should focus to find the most defects. This makes more efficient use of the tester's time. And allows a tester to give the quickest feedback to developers.

    If a set of automated test cases always pass. Good, they have successfully steered the tester away from looking at that component of the system. Even if the failures point to changes in the SUT, it helps to keep the tester informed of the state of that system.



Copyright 2006| Blogger Templates by GeckoandFly modified and converted to Blogger Beta by Blogcrowds.
No part of the content or the blog may be reproduced without prior written permission.