I recall a presentation delivered by Dr Mark Pedersen, a colleague of mine at K.J. Ross & Associates titled "Selling Testing to the Business". In this presentation there was a slide which presented the findings of a study done by HP in the early 90's (Grady, 1994). I have included it below...
I've translated and calculated some more metric's into the table below. I like to look at things from a few angles.
| Test Type | Defects Per Hours | 1 Defect every ... Hours | ... Defects detected in an 8 hours day |
| Day to Day | 0.21 | 4.76 | 1.68 |
| Black box | 0.28 | 3.57 | 2.24 |
| Glass Box | 0.32 | 3.13 | 2.56 |
| Inspection | 1.05 | 0.95 | 8.4 |
For the purpose of this post, I am focused on the 'Day to Day' vs the 'Black Box' metrics. Based on the above numbers it's reasonable to infer that by using Black box testing technic's (and all other things being equal) we will find slightly more defects than if we just followed the business processes.
My hypothesis as to why so many IT people don't respect testing as a profession, and were the old saying 'Anyone can do testing' comes from, is based on the observations and experiences were folks from all manner of backgrounds and vocations have been involved in testing and found defects just by using the system in the manner to which it was intended... And here in-lies the challenge to the current and next generation of testers to advance the profession by improving our effectiveness - or maybe it's a case of simply communicating the improvements we've already made????
Hi Andrew
ReplyDeleteI'm surprised. Although I've had to do it in the past, I've not experienced the need to "sell testing benefit" in recent years and mused recently that perhaps we were over the worst of it.
I recall reading the following article by James Bach a few year back in my formative years and re-visit it from time to time, when I need to remember the unique potential value that we can provide. I'm sure the short copy/paste that I've included will get you clicking thru to read the rest.
- - - - - - - - - - - - - - - -
http://www.satisfice.com/articles/explaining.shtml
- - - - - - - - - - - - - - - -
Don't you love it when a programmer or manager makes ignorant statements about testing? You don't? Well, I do. At least they're talking. My experience is that most of my non-tester colleagues, no matter how smart and talented they are in their own work, are pretty mixed-up about my line of work. But if they don't say anything, there's not a lot I can do about it. So, in a way, I feel better when I hear one of them say something like "You play with each feature and see if it works, right? What's the big deal?"
Because if they talk, maybe they'll listen. If they listen, maybe I can offer them a more useful view of testing.
Maybe you think your co-workers should already understand how testing works. I feel your pain. Now, get over it.
- - - - - - - - - - - - - - - - - - - - - - - -
(definitely worth a read...)
_______________ ___ ___ ___ __ __ __ _ _
Cheers, Tim
RBD Testing
Hey, I agree it's not the testing I've had to sell - it's pretty well understood that it's required. Its the need for Professional Testers instead of another IT person stepping into the role. I remember the 'Lawn Mower' analogy which James A Whittaker presented. He said, everyone can mow a lawn but those who aren't serious about it might skip the edging. James said testing is the same!
ReplyDeleteI should say that some of the best testers I have worked with have a varied background, particularly those with a musical bent :-)