Are we testing the right things?

Recently on a plane back from Adelaide I read an article in the QANTAS in-flight magazine the Australian Way (as you do!) the article was written through the eyes of a traveller who was visiting somewhere in either Canada or North America. The bits of the article I found most thought provoking was the quotes of the tour guide who said something like 'focus on what you can see, stop and admire all that is around you; don't lament the that you didn't see a bear, be thankful you have seen over 100 different types of plants and animals.' The guide went on to say 'all too often tourists visit these parts and rush from place to place to see this and that, taking only photos of things, not creating a memory by experiencing being there'. This made perfect sense too me the phase "Stop and smell the roses" came to mind. By chance tonight I saw the tagline of another article Look Beyond the Lens on the Sydney Morning Herald's website and after only reading the tagline I pondered.... As testers, in testing do we sometimes focus too much on proving that a requirements have been implemented or that defects have been found as a result of our efforts - to prove we were there? Do we focus on finding the bear in the woods because it's big, scary and easy to see or not see, all the while missing the wondrous smaller wildlife, flowers and trees that are necessary parts of the ecosystem for the bear to survive or equally perish?

My initial thoughts were along the lines of do we focus too much on finding bugs and satisfying requirements (taking pictures - see it is there, I saw it) only to forsake to the things that our users actually need from the system to do there job or improve their efficiencies? And if this is the case, then how do we change our plans to allow us to stop for a while and just "be there"...  Somehow I think the need to take pictures and wiz off to the next location is a result of time constrains and a desire to see as much as possible in a given period of time. Equally in testing, time or lack of time is a huge factor. But it is time that maybe the ultimate measure for a picture only lasts for as long as the media is around and often fades over time (remember when they used to print pictures!) while a memory can live forever being pasted from one generation to another! It's time to choose which we value more, pictures or memories...  
  

Thoughts provoked by the Information Age (ACS magazine)

There were several articles that tweaked my interested and awoken the sleeping grey matter from its weekend slumber! The first was one titled "Investing in careers with SFIA" which describes how Westpac has incorporated the Skills Framework for the Information Age (SFIA) as part of managing the 'whole employment life-cycle'. SFIA has been at the front of my mind recently as it's featured in many discussions I've had with clients and service providers alike. Initially I was drawn into reading this article because of the relevance to my current activities-discussions which centre around the implementation and benefits of the framework but two-thrids of the way through I found the paragraph circled in the picture below. My first thought was of all the skills in SFIA (there are 89 odd skill areas) the Westpac executive choose to highlight testing, not Development, Not BA's and not Project Managers but Testing. The article didn't explain why testing and not the others, but what I liked was that his person understands that testing is a Professional Skill, with a career path just like any other the skills in the Framework. At the recent Test Managers Forum there was lots of discussion surrounding professionalising testing and I am now seeing evidence we are heading in the right direction! My thought is that the next step will be to have the difference streams of testing (Performance, Automation, Functional) recognised as skills in there own right - more too follow on that one.

The article can be found on page 38-39 of the Information Age.

The next article that captured my attention was  'ICT's biggest money wasters' (pg 46-49). Unfortunately in the software testing game we all too often see evidence of the number 1 cited cause of waste: "Dusty Software Licenses". For the most part I've seen this occur in the niche areas of software testing Automation and Performance Testing and they are usually accompanied by/or caused in part by number 1's closest friend, number 6: "ICT projects gone wild". 

All to often I have seen organisations spend Thousands of dollars on tools to make the testing process more efficient and end up delivering far less than was originally specified. There are many reasons for this result, the least of which is that within IT we often attempt upgrades and process improvement activities internally and we don't apply the same level of rigour and governance that we do as when we are delivering project for an external stakeholders. We often put the tool subject matter expert or super user in charge of the project, on top of there other duties and expert that it'll just happen. The lesson here is that all internal projects (upgrades, automation and performance testing) should be run as projects with milestones, reporting, have established baselines and be constantly measured, with individuals held to account for the time & $$ invested. 

Teleworking is a subject that is close to my heart, as I Telework for one of my jobs :o) It works really well for me (and my employer I hope!). I enjoy the flexibility to do what needs to be done in and around the other things in my life (Family & my other job). The part of the article that made me smile, and then ponder was the part were the author said the 'non-productive time' of the Teleworker can be put towards doing the household chores like cleaning, doing the dishes or preparing dinner - the tasks that usually get done during personal time. This made me go hmmmmm..... Initially I was like, well, that's not cool and as an employer and supervisor I'm not sure I agree. But on reflection, and when considering the alternative proposed in the article (surfing the net or chatting) I'm now thinking that if that is the case, well it'll save us some $$ on bandwidth and download fees and potentially stop the disruption to the other employees and causing them to be unproductive so maybe on balance it's not such a outlandish statement. Though, Boss, if your reading this, this is my only 'unproductive time' ;-p     

Defects are requirements?

It was a throw away line over a beer "Defects should be written as requirements..." The context of the conversion was talking about delivering a testing training course and the topic of defects.

At the time I had "Ding" moment where I thought its such a simple concept. All testers understand that defects are important, but as possibly the only tangible outcome of testing we should give them more focus.

When I think of the conversations about testing I've had of late at go, no go meetings, almost all have glossed over the test results and focused on the outstanding defects. If I calculated the amount time I spent translating and interpreting what the functional impacts were, the associated effects on users and risk to data - well it would be days. I think that if I approached the documentation of defects with much of the discipline I'd expect to find in the documentation of requirements then I'd be that much better off.

Speaking of requirements, there is a lesson we (as testers) should learn and its one we try to teach on a daily basis. The quality of requirements has a major impact on the quality of the system built and the testing conducted. Well, our defects are mini specifications on how particular system function should work and its content is of poor quality, then the probability that the fix implemented will be sub optimal is increased.

Lastly, reason for making an effort to document defects thoroughly, is that one day down the track you might have to retest or reproduce the defect. If you've only entered scant on details in your hast to raise the defect, then it makes your job that much harder! I know I've had many instances where I've read a defect report that I'd raised and had to scratch my head to fill in the gaps I left.

It's all about testing!

As I look back on the year that has been and ponder the year ahead I have a smug little smile on my face, and not just from the Christmas cheer that I've over dosed on!

The project that I am currently working on is like any other, multiple systems, tight deadlines, thorw in the mad rush to get all the loose ends tied up before the Christmas break and its pretty much situation normal.

So for several months now I've been telling anyone that will listen, that "the sooner they realise this project is all about testing and that my needs are the most important the better off they will be!" Self centred I know, however I see a large part of my role is keeping testing at the front of every ones minds. Some days I feel like a little kid saying "don't forget about me" but if that's what it takes to ensure that testing is not left out of the information/communication loop then I'm ok with that.

Well in the lead up to Christmas I received a couple of most welcome, and unexpected Christmas presents partly due to my nagging. Both the project manager, and the software engineering manager commented separately to me "This project is all about testing" and without too much of a "I told you so" look on my face I replied with "indeed it is". It's amazing how the acknowledgement of the importance of testing to the projects success lifted the spirits in the run to up Christmas break.

I've been able to establish several reasons for the realisation of the importance of testing to the project. Firstly, the sign off of several key testing areas (performance & SOE) are absolutely critical and not negotiable if we want to release our software into the production network. Secondly, acceptance of the software by the stakeholders is largley based on the results of the final acceptance testing; and sign off of the Acceptance Test Report is therefore a key milestone. Finally, because I said so, and will continue too remind them.