For most of us, testing for “coolness” is not at the top of our quality list.  Our users don’t have to buy what we test.  Instead, they get forced to use it by their employer.  Nevertheless, coolness can’t hurt. 

As far as testing for it…good luck.  It does not appear to be as straightforward as some may think.

I attended a mini-UX Conference earlier this week and saw Karen Holtzblatt, CEO and founder of InContext, speak.  Her keynote was the highlight of the conference for me, mostly because she was fun to watch.  She described the findings of 90 interviews and 2000 survey results, where her company asked people to show them “cool” things and explain why they considered them cool.

Her conclusion was that software aesthetics are way less important than the following four aspects:

1. Accomplishments – When using your software, people need to feel a sense of accomplishment without disrupting the momentum of their lives.  They need to feel like they are getting something done that was otherwise difficult.  They need to do this without giving up any part of their life.  Example: Can they accomplish something while waiting in line?
2. Connection – When using your software, they should be motivated to connect with people they actually care about (e.g., not Facebook friends).  These connections should be enriched in some manner.  Example: Were they able to share it with Mom?  Did they talk about it over Thanksgiving dinner?
3. Identity - When using your software, they should feel like they’re not alone.  They should be asking themselves, “Who am I?”, “Do I fit in with these other people?”.  They should be able to share their identity with joy.
4. Sensation – When using your software, they should experience a core sensory pleasure.  Examples: Can they interact with it in a fresh way via some new interface?  Can they see or hear something delightful?

Here are a few other notes I took:

• Modern users have no tolerance for anything but the most amazing experience.
• The app should help them get from thought to action, nothing in between.
• Users expect software to gather all the data they need and think for them.

I guess maybe I’ll think twice the next time I feel like saying, “just publish the user procedures, they’ll get it eventually”.

Last week we had an awesome Tester Lightning Talk session here at my company.  Topics included:

• Mind Maps
• Cross-Browser Test Emulation
• How to Bribe Your Developers
• Performance Testing Defined
• Managing Multiple Agile Projects
• Integration Testing Sans Personal Pronouns
• Turning VSTS Test Results Files Into Test Reports
• Getting Back to Work After Leave
• Black Swans And Why Testers Should Care

The “Performance Testing Defined” talk inspired me to put my own twist on it and blog.  Here goes…

 

 

The terms in the above graphic are often misused and interchanged.  I will paraphrase from my lightning talk notes:

Baseline Testing – Less users than we expect in prod.  This is like when manual testers perform a user scenario and use a stopwatch to time it.  It could also be an automated load test where we are using less than the expected number of users to generate load.
Load Testing – # of users we expect in prod.  Real-world scenario.  Realistic.
Stress Testing – More users than we expect in prod. Obscene amount of users.  Used to determine the breaking point.  After said test, the tester will be able to say “With more than 2000 users, the system starts to drag.  With 5000 users, the system crashes.”
Stability Testing – Run the test continuously over a period of time (e.g., 24 hours, 1 week) to see if something happens.  For example, you may find a memory leak.
Spike Testing – Think TicketMaster.  What happens to your system when it suddenly jumps from 100 simultaneous users to 5000 simultaneous users for a short period of time?

There.  Now you can talk like a performance tester and help your team discuss their needs. 

As far as building these tests, at the most basic level, you really only need one check (AKA automated test).  Said check should simulate something user-like, if possible.  In the non-web-based world (which I live in) this check may be one or more service calls.  In the non-web-based world, you probably do not want to use an automated check at the UI level; you would need an army of clients to load test.  After all, your UI will only have a load of 1 user, right?  What you’re concerned with is how the servers handle the load.  So your check need only be concerned with the performance before the payload gets handed back to the client.

The check is probably the most challenging part of Performance testing.  Once you have your check, the economies of scale begin.  You can use that same check as the guts for most of your performance testing.  The main variables in each are user load and duration.

Warning: I’m certainly an amateur when it comes to performance testing.  Please chime in with your corrections and suggestions.