I chose to attend Hans Buwalda’s full day tutorial, “Delivering Test Automation Success Through People, Methods and Tools” at STPCon. Hans credits himself as the inventor of “Action Based Testing”, which is similar to what the industry calls “Keyword-Driven Testing”. He also says (with a straight face) he has automated 95% of the tests for every project he has worked on. Ah yes, you say, but this 95% goal is silly because if all your tests suck in the first place, your automated tests will suck even worse. Read on…
Hans spent most of his time discussing test design rather than how to actually “do” the automation. After my experiences with test automation, I completely agree with Hans. The test design is the hardest part, and there is something about automation that magnifies poor test design.
Manual testing allows for dynamic test design, automated testing does not. A manual tester can read an AUT’s modified validation message and determine if it will make sense to a user. An automated test cannot.
Per Hans, when thinking about automated test design, the first two questions should be:
1.) What am I checking?
2.) How will I check it?
These seemingly simple questions are often more difficult than automating the test itself. And that may be why these questions are often neglected. It is more fun to automate for the sake of automation than for the sake of making valuable tests.
To eliminate this problem, Hans counters that Test Automation Engineers should never see a single test case. And Test Designers should never see a single piece of automation code. Hmmm…I’m still not sure how I feel about this. Mostly, because I want to do both!
In the next post, I’ll share some of Mr. Buwalda’s test design ideas that apply to manual and automated testing.
Hans spent most of his time discussing test design rather than how to actually “do” the automation. After my experiences with test automation, I completely agree with Hans. The test design is the hardest part, and there is something about automation that magnifies poor test design.
Manual testing allows for dynamic test design, automated testing does not. A manual tester can read an AUT’s modified validation message and determine if it will make sense to a user. An automated test cannot.
Per Hans, when thinking about automated test design, the first two questions should be:
1.) What am I checking?
2.) How will I check it?
These seemingly simple questions are often more difficult than automating the test itself. And that may be why these questions are often neglected. It is more fun to automate for the sake of automation than for the sake of making valuable tests.
To eliminate this problem, Hans counters that Test Automation Engineers should never see a single test case. And Test Designers should never see a single piece of automation code. Hmmm…I’m still not sure how I feel about this. Mostly, because I want to do both!
In the next post, I’ll share some of Mr. Buwalda’s test design ideas that apply to manual and automated testing.
3 comments:
Subscribe to:
Post Comments (Atom)
RSS
What were his arguments that automation developers shouldn't see test designs and testers shouldn't see automation code?
Hm... It's very interesting to see Mr. Buwalda’s test design ideas.
Waiting for your further articles.
ido,
Hans believes if the automation engineer designs the tests, he/she may alter the integrity of the test in order to bypass a challenging automation problem. It also improves speed to separate responsibilities because the engineer and desiger need not worry about each other's goals.