As with manual tests, for each automated test we write, we must make a decision about where the test begins/ends and how much stuff it attempts to verify.

My previous post uses an example test of logging into an app then editing an account. My perfect manual test would stick with the main test goal and only verify the account edit. But if I were to automate this, I would verify both the log-in page and account edit. In fact, I would repeat the log-in page verification even if it occurred in every test. I do this for two reasons.

1. Automated verification is more efficient than manual verification.
2. Automated tests force extra steps that can’t determine bug workarounds.

In most cases, automated verification is more efficient than manual verification. Once the verification rules are programmed into the automated test, one no longer has to mentally determine whether said verification passes or fails. Sure, one can write just as many rules into a manual test, but it still takes a significant amount of time to visually check the UI. Worse yet, it takes a great deal of administrative work to record the results of manual verifications. So much time that I often get lazy and assume I will remember what I observed.

With manual tests, we can think on the fly and use our AUT knowledge to get the correct precondition state for each test. However, automated tests do not think on the fly. Thus, we have to ensure each automated test begins and ends in some known state (e.g., AUT is closed). This forces our automated tests to have a great deal more steps than our manual tests. An upstream bug may not have much impact on a test if a human finds a workaround. However, that same upstream bug will easily break an automated test if the test author did not plan for it. Thus, multiple verifications per test can help us debug our automated tests and easily spot upstream bugs (in both our AUT and our automated test library).