Great food, great company and thought provoking discussions what more could you ask for?This was my first KJRA Summer School and for me it brought back many memories of my formative years attending a boarding school in western NSW. The location of summer school was the 'The Women's Collage' at the university of Queensland, which is steeped in history. The halls of residence where we stayed opened in the late 1950's and reminded me of the accommodation I lived in whilst completing year 12. At the time, the year 12 accommodation was the pinnacle, single rooms with a balcony. Oh I waited patiently "doing my time" for 5 years to ascend into "Hindmarsh". The memories of hot summers night with only a fan for cooling, sleeping without any covering flooded on back. However the most confronting thing about attending summer school at a women's collage was that in the bathrooms there was full length mirrors, and round mirrors over the sinks which didn't just reflect, but magnified ones face so that all the blemished became blindingly obvious! Back in the day, the bathrooms where I went to school had only a single mirror over one of the sinks for those of us that needed to shave! So being confronted with ones unshaven, just woken up reflection was a memory I'd rather not have gained!
I do love the boarding house/summer school/residential school setup where the focus is on community and communal activities. At school, as at summer school you never ate alone there was always someone too share a conversation at meal time. Talking about the events of the day, sports or conquering great plans for the nights activities! and that's one of the other great things group activities that inspire laughter and cooperation. Back in the day we played pool in the common room, or went 'out bush' into the forest in our younger years building cubby houses for all manner of adventure, and during summer swimming in the mighty Murrumbidgee river was a great way to pass the time and keep cool. At summer school some of the old favourites were played the egg and spoon race, sack races and the three legged race began the evening. Then we fast forwarded to what seems to be the in thing at the moment with Guitar Hero - World tour. After several rounds, each increasing in difficulty Todd was crowned the guitar hero hero! That was Thursday night, Friday night was 80's night and it was fabulous to see some of the attendees dress up! Oh the fashion crimes that occurred in the 80's :-)
I finished summer school on Saturday, checking out of my room which incidentally bore a plarke on the door frame "The John Graham Miller room, endowed by Mrs Rina Miller" which I pondered for some time. It wasn't the only room to be named after a bloke oh if only those walls could talk!!!
I sat in on the Test Project Management course for the morning which was the source of much inspiration on the trip home! On Friday whilst delivering the Agile Testing Course I quoted a Prince2 trainer (somewhat strange I know given some peoples views on Prince2 and Agile) that I'd heard say "The Project Management should be like a modern day pilot (with all due respect) in that he/she should plan the flight, get the plane ready, take off and then switch on the auto pilot - monitoring the progress at key points making corrections as necessary. Then plan the landing, land and disembark the passengers". In the context of Prince2 it made a lot of sense and on the flight back from Brisbane I juxtaposed some of my experiences as a test manager to that of a pilot, so come fly with me.
Thinking of myself as a pilot is an interesting thought and gives a much better visual than when I tried to explain the role of testers (many years ago) to a group of developers as 'life guards' to which they replied "in speedo's? LOL". But anyway, my comparison starts with the pilot receiving instructions of the flight that he/she is to take charge of today. I imagine there are several key details that the instruction contains: the aircraft, destination, departure time, flight crew, number of passengers and expected weather conditions on route. In my experience as a test manager I've received similar briefs about the project I'm going to be managing; intended release date, commencement date, testing resources allocated, scope of testing and warnings about the political landscape!
After receiving the flight instructions even the most experienced of pilots will research the route, weather and all manner of other details (assuming the flight assignment details are allocated several days in advance of the flight) steadily building the plan of the flight. On the day of the flight, the flight plan will be finalised with details of the flight path, check points, , altitude, number of passengers, expected take off weight, boarding gate allocation, expected boarding time, flight deck and cabin crew assignments.
In a test management context, this period equates to the period after you've received the initial project brief through until your troops begin arriving on the ground (covering the development and documentation of a master test plan). You will have discovered all manner of details about the project, including who the key stakeholders are, key stage gates (i.e. change control windows), allocated manual, automated and performance testing resources, environment availability, development and requirements status. All of these (and many more) elements are all considered in the development of a test plan, and each influences how the plan is implemented.
Now with the plan matured it's time to implement. In the case of the pilot, a pre-flight briefing might be the vehicle to bring all the aircraft crew up to speed. Then it's off to the boarding gate to commence the pre-flight checks. The process of getting the plane off the ground involves numerous activities, in my comparison I've focused on the following; the boarding of the passengers (Test Cases), load the catering (Test Data), Storing the luggage below and most importantly load enough fuel (defects) to complete the flight! To me, the process and activities surrounding taking flight are somewhat similar pre-requisites for commencing testing. Ideally at the commencement of a testing program the parameters would be as well defined! Can you image knowing at the commencement of testing outer limits of scope and duration? With a plane its absolutely defined, as a plane can only fit X passengers, has a maximum take off weight and maximum distance it can travel, therefore limiting overall capacity.
The nature of testing often prevents such well defined boundaries, but that's not to say we shouldn't have a detailed understanding of the testing we are about to undertake. In my comparison I've compared the passengers to test cases, and the reason being is the success of a flight can be measured in terms of did it arrive on time which is a quantitative measure, and perhaps more importantly were the passengers satisfied with the quality of the flight in terms of service, comfort and timeliness. Test cases also have quantitative measures such as 'were they all run' which by its itself doesn't provide enough depth to make a definitive statement about success. One needs to look deeper into the results, how many passed, how many failed and what was the magnitude of the failure? More on that later.
Once all the passengers, luggage and catering is loaded its time to depart! Even as a test manager I still get excited when a new build arrives in the test environment, its like the anticipation of taking off once you've boarded a flight you know you ever so close to taking off and you just want to be in the sky! Once clearance to taxi is granted we push back from the gate and taxi to the runway. At this point in the testing process I like to have the test strategy signed off, test cases reviewed and accepted - essentially all my 'Entry Criteria' met so testing can commence. The take off, by far one of the most exciting or scary parts of the trip, as the plane accelerates down the runaway and the G-force pushes you back in your chair and then the point where the plane lifts off the ground - I'll never get tried of that feeling! Often there is a little bit turbulence or bumps as the plane assents into the sky, in the same way as the first delivery of the 'build' often experiences some difficulty being deployed into the test environment. And once your in the sky and the seat belt sign goes off you know you're on the way... This is the equivalent of getting the green light to start testing and so it begins!
Now we're in the sky climbing to the cruising altitude and constantly monitoring all manner of things - weather, wind, cabin pressure and fuel consumption. Fuel consumption I found has some interesting similarities to defects - what tha I here you say. But think about it this way, as the requirements and code are developed defects creep in and steadily build up, then once testing commences (at any level) these defects are discovered and removed. Compare this to the fuel for a plane. The fuel required to make the flight is loaded pre-take off, then as the flight progresses the amount of fuel consumed increases to the point where by the time the plane lands the fuel load is minimal. This is exactly the trend that we want to observe with our testing, that at the beginning of the testing cycle the defect detection rate is high and then as testing progresses the rate decreases, corresponding the number of residual defects decreasing. After all defects are the fuel of testing! While there are defects there will be testing :-) it's not a perfect model, but I like it.
Whilst in the air the passengers consume the catering and the plane steadily makes its way to the destination. The pilot (or co-pilot) monitors the progress along the planned flight path making adjustments as required and all if fine. Then out of no where, that clear trouble maker of the skies strikes, turbulence and has the capacity to effect all on board. The pilot has several options depending on the severity encountered, turn on the seat belt sign, adjust the altitude and/or the flight path (as happened on my flight). In terms of testing, turbulence could be equated to a change in scope mid way through testing, or test environment issues or requirements or code instability. Any of these (and many more) issues often result in the test manager having to make adjustments to the plan! and its just a natural part of the process.
Newtons Law states "What goes up, must come down" and as the destination draws closer its' time to prepare the cabin for landing. All passengers need to return to there seats and place luggage appropriately. Throughout the flight the pilot is in constant communication with Air Traffic control, reporting location and status, and as the pilot prepares to descend, permission to land needs to be sort. The air traffic controller can be compared to either the stakeholders or the project manager whom need to be kept update to constantly about the status of testing -also making decisions in regards to ending testing. In terms of testing this is the point in the cycle where the test manager starts to make sure all is in order, all test cases have been attempted, the defect status is published and understood, requirements coverage is assessed and the test report is in draft. As with the take off there are many factors that influence the 'smoothness' of the landing and the pilot must continually monitor and adjust whilst approaching the runway. In this period even minor issues can have disastrous effects, but more often that now the plane touches down with little or no noticeable issues to the passengers. Once safely on the ground its time to taxi to the terminal and disembark the passengers.
Disembarking the passengers is a key 'milestone' of the flight and its at this point where the quality of the flight can begin to be assessed. Flights like testing cycles have both qualitative and quantitative measures of success. quantitative measures like all statistics are open for interpretation and qualitative measures are subjective so its important to gather both for consideration of 'success'. Its also important to have a definition of what 'success' is at the commencement of the project. Many would argue that simply reached your destination is only part of a satisfactory outcome!
With the plane on the ground, passengers and their luggage disembarked the pilot must complete some paperwork and then its off onto the next assignment. As the test manager wrapping up the testing cycle, the test report is usually the way to close out the testing for this project...
