Is there a name for this? If not, I’m going to call it a “fire drill test”.
- A fire drill test would typically not be automated because it will probably only be used once.
- A fire drill test informs product design so it may be worth executing early.
- A fire drill test might be a good test candidate to delegated to a project team programmer.
Fire drill test examples:
- Our product ingests files from an ftp site daily. What if the files are not available for three days? Can our product catch up gracefully?
- Our product outputs a file to a shared directory. What if someone removes write permission to the shared directory for our product?
- Our product uses a nightly job to process data? If the nightly job fails due to off-hour server maintenance, how will we know? How will we recover?
- Our product displays data from an external web service. What happens if the web service is down?
Too often, us testers have so much functional testing to do, we overlook the non-functional testing or save it for the end. If we give these non-functional tests a catchy name like “Fire Drill Test”, maybe it will help us remember them during test brainstorming.
6 comments:
Subscribe to:
Post Comments (Atom)
RSS
Yep, I know this as testing on the 'Robustness atribute' of your list of non-functional testing. And Maintainability. But Fire-drill test is more appealing of course.
Anyway, mostly forgotten, no written requirements etc. This Post is a reminder to set up a few tests on this on each project as applicable. You can almost call this Test-driven requirements ;-)
Yep, I know this as testing on the 'Robustness atribute' of your list of non-functional testing. And Maintainability. But Fire-drill test is more appealing of course.
Anyway, mostly forgotten, no written requirements etc. This Post is a reminder to set up a few tests on this on each project as applicable. You can almost call this Test-driven requirements ;-)
Check out the Chaos Monkey concept Netflix came up with. http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html
the term "fire drill test" is frequently used when a company does a session of all-hands testing: every employee, no matter the department, uses the product. This provides a simple but poor way to load test a site, but more importantly it gets people manually interacting with the site.
best,
--derek
Derek, interesting, I don't think the name "fire drill test" is a good descriptor for what you describe. Sounds to me like an all-hands test session to provide load is an attempt to simulate normal usage on the product-under-test.
Yehuda, thanks for the link. Sounds totally cool, like a fire drill test for sure! I plan to research it a bit and see if it's applicable to any of my projects.