Last week we had an awesome Tester Lightning Talk session here at my company. Topics included:
- Mind Maps
- Cross-Browser Test Emulation
- How to Bribe Your Developers
- Performance Testing Defined
- Managing Multiple Agile Projects
- Integration Testing Sans Personal Pronouns
- Turning VSTS Test Results Files Into Test Reports
- Getting Back to Work After Leave
- Black Swans And Why Testers Should Care
The “Performance Testing Defined” talk inspired me to put my own twist on it and blog. Here goes…
The terms in the above graphic are often misused and interchanged. I will paraphrase from my lightning talk notes:
Baseline Testing – Less users than we expect in prod. This is like when manual testers perform a user scenario and use a stopwatch to time it. It could also be an automated load test where we are using less than the expected number of users to generate load.Load Testing – # of users we expect in prod. Real-world scenario. Realistic.
Stress Testing – More users than we expect in prod. Obscene amount of users. Used to determine the breaking point. After said test, the tester will be able to say “With more than 2000 users, the system starts to drag. With 5000 users, the system crashes.”
Stability Testing – Run the test continuously over a period of time (e.g., 24 hours, 1 week) to see if something happens. For example, you may find a memory leak.
Spike Testing – Think TicketMaster. What happens to your system when it suddenly jumps from 100 simultaneous users to 5000 simultaneous users for a short period of time?
There. Now you can talk like a performance tester and help your team discuss their needs.
As far as building these tests, at the most basic level, you really only need one check (AKA automated test). Said check should simulate something user-like, if possible. In the non-web-based world (which I live in) this check may be one or more service calls. In the non-web-based world, you probably do not want to use an automated check at the UI level; you would need an army of clients to load test. After all, your UI will only have a load of 1 user, right? What you’re concerned with is how the servers handle the load. So your check need only be concerned with the performance before the payload gets handed back to the client.
The check is probably the most challenging part of Performance testing. Once you have your check, the economies of scale begin. You can use that same check as the guts for most of your performance testing. The main variables in each are user load and duration.
Warning: I’m certainly an amateur when it comes to performance testing. Please chime in with your corrections and suggestions.
RSS
Thanks for a great straight forward summary! As someone with very little experience in performance testing I enjoyed it a lot.
Do you plan to write something similar for any of the other lightning talks?
/Erik
Great idea with the lightning talks! With regard to a couple of the terms described in your post, a baseline could also be a reference of how your current system is performing in production in order to effectively measure the impact of a change or addition to that system. Load testing would then be additional load on top of your baseline.
Eric, you are simply superb with your content. Yet another interesting note on Performance Testing. How about taking a step forward and write about performance tunning. I will look forward to your next post. Thanks for sharing this post.
Do you manage Performance Testers too? As a manager of Performance Testing, I'll tell you that there is a whole lot more to performance testing than people realize. Even more if you want it done right. I was very interested in some of the topics you mentioned and hoped you would go a little more into what you learned from them in this post.
Your information about performance testing is really interesting. Also I want to know the latest new techniques which are implemented in performance testing. can you update it in your website?
Software testing training in Chennai