In my experiences, 95% of the test cases we write are read and executed only by ourselves. If we generally target ourselves as the audience, we should strive…
…to write the least possible to remember the test.
I like the term “test case fragment” for this. I heard it in my Rapid Software Testing class. On the 5% chance someone asks us about a particular test, we should be able to confidently translate our chicken scratches into a detailed test. That’s my target.
If we agree with the above, couldn’t we improve our efficiency even more by coming up with some type of test case short hand?
For example:
- Instead of “Expected Results”, I write… “E:”.
- Instead of writing statements like “the user who cancelled their order, blocks the user who logged in first”, I prefer to assign variables, “UserA blocks UserB”.
- Instead of multiple steps, I prefer one step test cases. To get there I make some shortcuts.
- Rather than specifying how to get to the start condition (e.g., find existing data vs. create data), I prefer the flexibility of word “with” as in “With a pending order”. How the order becomes pending is not important for this test.
- To quickly remember the spirit of the test, I prefer state-action-expected as in “With a pending order, delete the ordered product. E: user message indicates product no longer exists.”
- Deliberately vague is good enough. When I don’t have enough information to plug in state-action-expected, I capture the vague notion of the test, as in “Attempt to corrupt an order.”. In that case I drop the “E:” because it is understood, right? Expected Result: corruption handled gracefully.
It may be a stretch to call these examples “shorthand”, but I think you get the idea.
What test case short hand do you use?
5 comments:
Subscribe to:
Post Comments (Atom)
RSS
You make some excellent points. Using our own shorthand, provided it is not so esoteric as to remove all meaning, can speed up the whole test process.
Long-hand test cases still have their place, particularly with training junior testers. It helps them to think about what they are doing, and helps you to understand what they were thinking!
My biggest assumption nowadays is that another tester (instead of a non-tester), with reasonable knowledge of the product, may run my tests. Therefore, I can assume a level of knowledge.
I think I will start looking into streamlining my tests a bit more with common phrases being made into short-hand.
I call them "test ideas".
I write one sentence that captures the essence of what should be tested.
Your examples would be "Verify appropriate user message when deleting a pending product order" and "investigate handling when corrupting order (network disruption, database corruption etc.)"
The vagueness leaves freedom, and makes the test less fragile to changes.
I try to write in plain English, because if you write these condense and understandable, you can get 95% of them reviewed!
In this way you also get a lot of test ideas to use as inspiration for other products or areas, especially generic test ideas like "make sure performance is reasonable", "think about usability while performing the tests" etc.
Stuff like this is done by many testers in the world, but it is rarely spoken about.
(shameless plug: http://www.thetesteye.com/papers/redgren_moreandbettertestideas.pdf)
I like these ideas. Thanx:-)
Is this compliant with the Agile Metodology? I find that is a little bit troublesome, in case you go to another project, get fired or whatever reason puts you out of the map in that particular project. If someone who has no idea of that shortcuts has to run that test (without you to ask how), that person will be in trouble...
I tend to use a lot of tables and spreadsheets when I write tests. It doesn't work for all types of testing, but it frequently works for the types I'm doing.
In using tables, I create column headers and then just fill in everything.
For example:
USER|TYPE|INPUT|ACTION|ASSERT|RESULT|COMMENTS
User A|new|X|cancel on confirmation|form cleared|P|[blank]
-|-|-|-|record not created|P|[blank]
-|-|-|-|notification "User has Canceled"|F|no notification message (include link to bug report)
User B|existing|change lname to: $3§♦♣K╢╖■|submit|confirmation page|lname updated; name saved as was input|P|[blank]
-|-|delete fname text|submit|error message: "Please enter a First Name"|P*|error message font color is hard to read (include link to bug report)
In this way, I can group together tests that have the same general structure. I can create more tabs for tests with other structures, and I can group tests on the same tab into chunks by using a border between a set of rows.
For example:
Row 0: 1 cell merged across, with text for section title
Row 1
Row 2
Row 3
_______
Row 4: 1 cell merged across, with text for section title
Row 5
Row 6
If I want to be able to hand it off to someone else, I can make a key of some sort:
1. Log in as USER
2. Pull up a TYPE record
3. Use INPUT in the form
4. Perform ACTION
5. Assert on ASSERTIONs
** "X" means "do not care"
** "N/A" means "not applicable"
** "-" means "no step necessary"
** "P*" means the exact test case passed but another issue was found in the process
I also like using spreadsheets because I can use formulas and macros. If I get data in a spreadsheet from a BA, I can design a test layout and then build a macro to populate it from the BA's sheet.
I can use formulas to copy data from one section of my sheet to another or perform minor logic or calculations.
I almost always make some sort of a summary sheet on the very front of my workbook that sums up all of the passes and fails (and whatever other types of results I'm using). Then I can just copy and paste that summary into reports I'm sending off to the team.