What if the test documentation is written per the detail level in this post…
Document enough detail such that you can explain the testing you did, in-person, up to a year later.
…and the tester who wrote it left the company?
Problem #1:
How are new testers going to use the old test documentation for new testing? They aren’t, and that’s a good thing.
Add test detail as late as possible because things change. If you agree with that heuristic, you can probably see how a tester leaving the company would not cause problems for new testers.
- If test documentation was written some time ago, and our system-under-test (SUT) changed, that documentation might be wrong.
- Let’s suppose things didn’t change. In that case, it doesn’t matter if the tester left because we don’t have to test anything.
- How about the SUT didn’t change but an interfacing system did. In this case, we may feel comfortable using the old test documentation (in a regression test capacity). In other words, let’s talk about when the write-tests-as-late-as-possible heuristic is wrong and the test documentation author left the company. If you agree that a test is something that happens in one’s brain and not a document, wouldn’t we be better off asking our testers to learn the SUT instead of copying someone else’s detailed steps? Documentation, at this level of detail, might be a helpful training aid, but it will not allow an unskilled tester to turn off their brain. Get it?
How are people going to understand old test documentation if the tester who left, must be available to interpret? They won’t understand it in full. They will probably understand parts of it. An organization with high tester turnover and lots of audit-type inquiries into past testing, may need more than this level of detail.
But consider this: Planning all activities around the risk that an employee might leave, is expensive. A major trade-off of writing detailed test documentation is lower quality.
5 comments:
Subscribe to:
Post Comments (Atom)
RSS
"Write tests as late as possible because things change."
I've always objected to this. Nobody says to programmers "write code as late as possible because things change."
I've always had to struggle to get my QA teams involved in early requirements review / design because I was told that things might change too much. But it was perfectly fine for the programmers to have to make these same changes.
Anonymous, in your case, the early requirements review might be "as late as possible". This might mean writing vague tests early but leaving the details out until as late as possible...we don't know if it will be a dropdown or free-form text field but we can write a vague test or "test idea", as some like to call it.
"Write tests as late as possible because things change."
You will end up with no TCs and QA cycle starting. There is a need to get QA involved in the beginning and them starting writing TCs asap. They can start with black box style and then move to white box one when reqs are full and available.
Otherwise, you will have QA sitting there waiting for reqs and company is paying for dead time.
I hear this issue often, "If test documentation was written some time ago, and our system-under-test (SUT) changed, that documentation might be wrong."
My response is usually something along the lines of, "If the SUT is changing, then so should the documentation..." If we refactor code, change underlying logic or functionality, etc., then tests need to be made to ensure that the change is what was expected. The test would change from the original test, so the test case would change... this produces updated documentation.
If the documentation is wrong, then it isn't being updated as it should, and if it's important to the org, then it's the test managers responsibility to make sure that test cases are updated. Especially in high churn departments.
Anonymous, consider standing behind your ideas with conviction, rather than hiding behind "Anonymous". There is nothing wrong with your ideas, nothing to be afraid of. Why not build a reputation?
Per the comments so far, I see that I have not explained my position clearly. I just updated the post to say: add *details* to tests as late as possible (in bold).
I agree that testers should usually do up-front test planning, which may include writing "test ideas", tests without detail.
Does that help?
Per my experiences, when the SUT changes, supporting documentation (e.g., specs, test cases) does not always change with it. People are lazy and said docs are secondary to the SUT. I have a feeling that is why Agile developers like to say the best documentation is the source code.
But even for a team that is diligent enough to keep support docs in synch with SUT changes, adding details to tests as late as possible, would certainly be less work...IMO.