GR Testers 

11 March 2013

Defect Types: What are they, How and When to Test for them?

Bug Taxonomies..

Taxonomy: the study of the general principles of scientific classification (Websters) 

    Phil's opening gambit - 
      Kaner's Testing Computer Software - 400 types of defects
      eCommerce - book w/ 60 types of defects
      Binder's approach -
          knowing in advance (or believing) the nature of the bugs one might find
          Bach's comment - If there's a bug in the system, how am I going to find it?
              * Memory leaks ?  How do you find them?
              * Usability ?  
              *  Race Conditions - Deadlocks, on and on and on
    Phil is passing around Copeland's book on "Practitioners Guide to Software Test Design"
    
    Wade citing http://quality-assurance-software-testing.blogspot.com/2005/08/defect-taxonomies.html on defining defects & defect sources.  (We're not sure if we as a group agree, but this is an interesting point for discussion.)
    
    Greg - Sometimes experience with past work can guide us in where to test and how.  
    
    Pete - Past experience is our ideal place to begin - Aaron chiming in as well, and Greg makes the point that sometimes /sometimes/ the mere communication of the idea or learning what is needed is a challenge in itself
    
    learn how your souces communicate!
     
Also need to remember that  this is the starting point, not an exclusive list.

Usability sometimes get short-shrft - 
  some shops don't care about it - devs say "its just a usability thing" (variation on WAD?)
  others go the opposite - UI gets all the attention - COOL! - but the underlying framework is rubbish
  
** it helps if people have a clue how the app works in the wild.  Sometimes, the installation/Customer Support folks have a better clue than we (tech/design/development) 

"What we have here is a failure to communicate."

Customers using the s/w "wrong"
"communicating" with reps - not actual users - and people "communicating" may not really get the main purpose

What about the motivations of the people devolping/designing it - What is the reward model?

===

Why do we care about taxonomies?

-- they show gaps in our knowledge --
-- ideas / applications - metrics (measurement) considerations

==

So, what makes a defect a defect?

-- vulnerabilities - unexpected behavior 

-- Legal constraints - Kaner's view is that 'defect" implies a compulsion that they be corrected?

Aaron's defintions 
-- Bugs ... unexpected behavior 
-- Defects -- more subjective

How do we use these classifications?
--  Phil - it opens your mind with "Oh Wow!  I'd never have tested that..."
-- Wade - understanding trends allows you to develop your own taxonomy (eg., dev x has a problem with code checked in after 3:00 on a Fridayt) to guide your future testing
-- The question around this becomes understanding how these overlap

===

if you find a bug by one method of testing - how do you know which "taxonomy" a bug fits into?  Bugs can exist in multiple... luck

Dice Game - an excellent example of evaluating patterns