GR Testers
11 March 2013
Defect Types: What are they, How and When to Test for them?
Bug Taxonomies..
Taxonomy: the study of the general principles of scientific classification (Websters)
Phil's opening gambit -
Kaner's Testing Computer Software - 400 types of defects
eCommerce - book w/ 60 types of defects
Binder's approach -
knowing in advance (or believing) the nature of the bugs one might find
Bach's comment - If there's a bug in the system, how am I going to find it?
* Memory leaks ? How do you find them?
* Usability ?
* Race Conditions - Deadlocks, on and on and on
Phil is passing around Copeland's book on "Practitioners Guide to Software Test Design"
Greg - Sometimes experience with past work can guide us in where to test and how.
Pete - Past experience is our ideal place to begin - Aaron chiming in as well, and Greg makes the point that sometimes /sometimes/ the mere communication of the idea or learning what is needed is a challenge in itself
learn how your souces communicate!
Also need to remember that this is the starting point, not an exclusive list.
Usability sometimes get short-shrft -
some shops don't care about it - devs say "its just a usability thing" (variation on WAD?)
others go the opposite - UI gets all the attention - COOL! - but the underlying framework is rubbish
** it helps if people have a clue how the app works in the wild. Sometimes, the installation/Customer Support folks have a better clue than we (tech/design/development)
"What we have here is a failure to communicate."
Customers using the s/w "wrong"
"communicating" with reps - not actual users - and people "communicating" may not really get the main purpose
What about the motivations of the people devolping/designing it - What is the reward model?
===
Why do we care about taxonomies?
-- they show gaps in our knowledge --
-- ideas / applications - metrics (measurement) considerations
==
So, what makes a defect a defect?
-- vulnerabilities - unexpected behavior
-- Legal constraints - Kaner's view is that 'defect" implies a compulsion that they be corrected?
Aaron's defintions
-- Bugs ... unexpected behavior
-- Defects -- more subjective
How do we use these classifications?
-- Phil - it opens your mind with "Oh Wow! I'd never have tested that..."
-- Wade - understanding trends allows you to develop your own taxonomy (eg., dev x has a problem with code checked in after 3:00 on a Fridayt) to guide your future testing
-- The question around this becomes understanding how these overlap
===
if you find a bug by one method of testing - how do you know which "taxonomy" a bug fits into? Bugs can exist in multiple... luck
Dice Game - an excellent example of evaluating patterns