Last week, Alex Kell (Atlanta-based tester and my former boss) gave a fun talk at Software Testing Club Atlanta, ”The Oracle is Fallible: Recognizing, Understanding, and Evaluating the Assumptions that Testers Make”.

Here are the highlights from my notes:

• After showing John William Waterhouse’s famous 1884 painting, Consulting the Oracle (above), of 7 priestesses listening to an 8th priestess (playing the Oracle) interpret from the gods, Alex asked:
• “Assumptions, are they bad or good?”
• We make them because we’re lazy.
• Sometimes we know we’re making an assumption, sometimes we don’t know.
• After some discussion and examples of assumptions we realized we are all constantly making assumptions every waking moment and decided assumptions can be good or bad.
• Bad assumptions (or forgetting the Oracle is fallible):
• “The spec is correct.” – Be careful.  Remember Ron Jeffries “Three Cs”:
• The Spec (AKA “Card”) is a reminder to have a conversation about something.
• The Conversation is a discussion of the details that result in test confirmations.
• The Confirmation is acceptance criteria that can be turned into acceptance tests.
• “They know what they’re doing.” – What if everybody on the team is thinking this…group think?
• “I know what I’m doing.”
• “The software is working because we haven't seen a (red) failed test.”  (Dennis Stevens says “Every project starts red, and it stays red, until it is green.”)
• “The model is reality.” – A model is an abstraction.  All decisions based on models are based on assumptions.  You should never be surprised when a model does not turn out to reflect reality.  Author, Nassim Nicholas Taleb, coined the word “platonicity” to describe the human tendency to find patterns in everything.
  
  Alex gave a nearly literal example of where people fall victim to this bad assumption: He told of people on Craig’s List (or similar) paying money for things like actual cars one can drive, only to discover they had just purchased a scaled down model of a car.
• Good assumptions (I loved this and thought it was pretty bold to declare some assumptions being good for testers):
• “The estimates are accurate”. – Take what you did last time.  Use the estimate until it is no longer helpful.
• “The web service will honor its contract”. If testers didn’t make this assumption, might they be wasting time testing the wrong thing? 
• There were more good assumptions but I have a gap in my notes.  Maybe Alex will leave a comment with those I missed.
• Alex talked about J.B. Rainsberger’s “Integrated Tests Are a Scam” – In other words, if we don’t make some assumptions, we would have to code tests for the rest of our lives to make a dent in our coverage.
• Suggestions to deal with assumptions:
• Be explicit about your assumptions.
• Use truth tables for complex scenarios (Alex shared one he used for his own testing).
• System thinking – Testers should be able to explain the whole system.  This cuts down on bad assumptions.