Dear Test Automators,
The next time you discuss automation results, please consider qualifying the context of the word “bug”.
If automation fails, it means one of two things:
- There is a bug in the product-under-test.
- There is a bug in the automation.
The former is waaaaaay more important than the latter. Maybe not to you, but certainly for your audience.
Instead of saying,
“This automated check failed”,
consider saying,
“This automated check failed because of a bug in the product-under-test”.
Instead of saying,
“I’m working on a bug”,
consider saying,
“I’m working on a bug in the automation”.
Your world is arguably more complex than that of testers who don’t use automation. You must test twice as many programs (the automation and the product-under-test). Please consider being precise when you communicate.
5 comments:
Subscribe to:
Post Comments (Atom)
RSS
Hi Eric,,,
Instead of saying,
“I’m working on a bug”,
consider saying,
“I’m working on a bug in the automation”.
this has to be said by the people who are automated testers only ..?
Nice! Good points.
I don't believe I've seen the phrase "Bug in the test", but I like it.
Kotla,
Yes, I think so. People who are not writing test automation should not attempt to fix bugs in test automation code.
That being said, a manual tester should sometimes use a similar phrase, "I am working on a bug in the test"...just like Joe's re-phrasing in his comment.
Tests performed by humans often have "bugs" in them too, right? Although, we typically hear other phrases like, "ooops, user problem" or "I didn't undertand the spec" or "sorry, I had the wrong expected results". It seems to me, those are like bugs in a test.
I hear what you're saying with "You must test twice as many programs. . ." but I see the automation and the target application being two reciprocal parts of a greater whole.
To elaborate, the two must work together which is why I don't think you need a "test for your tests". The test for your tests is that application they are testing.
Also, I would just say that "I'm diagnosing an issue" instead of saying that I'm working on a bug in the product or automation script.
Pedantic yes, but my two bits. Great blog btw.
Matt, I couldn't follow the first portion of your comment. Can you say it a different way?
As far as the second portion of your comment, calling it an "issue" instead of a "bug" is fine. The point I am trying to make is that your statement would be more helpful if you said, "I'm diagnosing an issue with the automation" or "I'm diagnosing an issue with the product-under-test".
It seems to me, the team might take a completely different action depending on where the issue is. "I'm diagnosing an issue" is almost not worth saying. Is it an issue with your email? Is it an issue with your car?
Okay, I'm getting silly. But think about this: Teams often go through rituals like daily stand-ups because they heard it is what you're supposed to do. Members may give reports that nobody actually cares about.