I recall a presentation delivered by Dr Mark Pedersen, a colleague of mine at K.J. Ross & Associates titled "Selling Testing to the Business". In this presentation there was a slide which presented the findings of a study done by HP in the early 90's (Grady, 1994). I have included it below...
I've translated and calculated some more metric's into the table below. I like to look at things from a few angles.
Test Type | Defects Per Hours | 1 Defect every ... Hours | ... Defects detected in an 8 hours day |
Day to Day | 0.21 | 4.76 | 1.68 |
Black box | 0.28 | 3.57 | 2.24 |
Glass Box | 0.32 | 3.13 | 2.56 |
Inspection | 1.05 | 0.95 | 8.4 |
For the purpose of this post, I am focused on the 'Day to Day' vs the 'Black Box' metrics. Based on the above numbers it's reasonable to infer that by using Black box testing technic's (and all other things being equal) we will find slightly more defects than if we just followed the business processes.
My hypothesis as to why so many IT people don't respect testing as a profession, and were the old saying 'Anyone can do testing' comes from, is based on the observations and experiences were folks from all manner of backgrounds and vocations have been involved in testing and found defects just by using the system in the manner to which it was intended... And here in-lies the challenge to the current and next generation of testers to advance the profession by improving our effectiveness - or maybe it's a case of simply communicating the improvements we've already made????