In my prior blog I explored the role of a software developer as being an architect, artist and craftsman. This blog will explore testing as it relates to a developer, ultimately examining the question: During testing, are developers – as Steve McConnell quoted – SOBs?
For the purposes of this discussion, I will focus on two categories of testing:
· Unit testing performed by a developer
· Formal, independent testing performed by Quality Assurance.
Unit Testing
Unit testing is something that a developer does before he or she hands the code off to Quality Assurance. Unit tests focus on testing very individual pieces of code – functions or methods – and are designed to ensure that the code is functioning as the developer intended.
These days automated unit testing – and integrating this testing into the build process – is becoming more and more important. Doing so ensures the continued, reliable operation of the software while work progresses over an extended period of time.
Good unit tests also aid as the code is re-factored (re-worked) over time. By having these tests as a measurement, code can be re-factored with confidence that it will not break existing applications, provided the unit tests pass before and after the re-factoring, naturally!
QA Testing
Ultimately, software must be validated from a business perspective by a set of tests that are geared towards the anticipated real-world use of the software. At AMS Services in Portland, Maine, these tests are driven from the user interface, mimicking the exact interaction that a user would take to drive the software.
As we progress towards a Web Service world, our tests are also taking the testing of Web Services themselves into account, minus any specific, defined user interface. This is because the Web Services are the “contract” by which a multitude of programs will gain access to business processing. These programs will either by our own or custom applications developed to leverage business processing that we at AMS Services provide.
Regardless of the particulars, any formal testing and sign-off of software should be performed by someone other than the developer who wrote the code. Why?
Why Developers Shouldn’t Test Their Own Code
Let’s say you were the developer in question. As the developer of a particular piece of software, you would possess intimate knowledge of the design and inner workings of the code, and as a result you would have a strong mental model in your head about how the software should flow. Your mindset is thus predisposed to operating the software in a particular way.
When other people use that software, however, they will likely interact with it in ways that you didn’t expect or anticipate! Those in the Quality Assurance area understand this and develop test cases with this in mind. Good QA engineers will mimic real-world scenarios by entering “strange” data (to determine if you validate data inputs to prevent “bad” data from being captured and/or otherwise handle the situation gracefully), or they will click on something out of sequence from what you expected.
As you can see, QA’s efforts are geared towards not only testing the core capability of the software, but all of the fringe areas where you as a developer might not have spent as much time as you could have or should have. While it takes additional time and effort, the engagement of good Quality Assurance personnel yields higher-quality software.
Are Developers Difficult?
As many a QA Engineer can attest, developers can arguably be thought of as testy during testing. I have seen developers who have rightly earned a reputation as very quarrelsome when their software is being tested, to the point where they vehemently argue their software is correct – despite clear and overwhelming evidence that it is not.
Even the most collaborative of developers can have their off days. Consider the following scenario:
You’ve spent the past five months as a developer on “Big Project X.” You’ve spent long days, working into the evenings and weekends to meet the project’s deadline. You’ve done all the right things, you’ve collaborated on design, and you’ve spent countless hours in front of your computer getting the program just right. A code review yields some minor issues that really should be incorporated into the code. You’re tired, but you see the wisdom of your peers’ suggestions, so you make the changes. You conduct one final series on unit tests before the code is promoted to be built for formal Quality Assurance testing.
Finally, all that is left is some testing to confirm that everything is good and you’re done! You’re confident in your design, confident in your knowledge and skills as a developer, confident in your unit testing, confident in your peers and that the code review will have caught any remaining issues. You’re looking forward to catching up on some sleep, maybe getting back to taking some evening strolls…
And within one or two days you’re besieged with high-priorities bugs with your code! It’s been raised to management that the latest build was “unstable and unsatisfactory,” and indirectly, the fingers point your way. How could this happen? Wouldn’t you feel just a tad mean and ornery? Believe me, as a developer at what is perceived to be the end of the cycle – often times with significant product delivery pressure being applied – this situation is frustrating and exasperating. (This is equally so for the QA folks – who usually end up being short-changed on their planned time for assuring product quality.)
Reality and Approaches
I wish that this scene was fictional. In reality, this is played out (many times only in degrees) all too often. It is a very real danger in the waterfall model of development, where every step in the process flows from one to another, with each preceding step considered final and complete before it is handed to the next step in the process.
What can go wrong? I’ve seen cases where there has been an issue of not being specific enough with the requirements, leaving things open to interpretation. Because Development and Quality Assurance keeps “optimized” by working (isolated) in their own domains, each group implements what they thought met the requirements. The problem is that differing views on what meets the requirements are possible.
It is possible to mitigate these problems in way one or another, even in a waterfall model. If you have a great business analyst, you can prevent the problem completely with well-defined, clearly articulated requirements. This same, great analyst also serves as the glue between Development and QA – preventing problems with interpretation from creeping into the equation. Other techniques, such as the use of “paper prototypes” (UI mockups on paper) engage people in a meaningful dialog during the design phase, helping to catch issues before development commences.
Other approaches, such as Agile development, seek to build software incrementally. Agile targets the delivery of working, testable software in short increments called “iterations.” These iterations are typically two-three weeks in length, and provide mutual benefits of being open and adaptable to change while stressing team collaboration – meaning that no one is left to interpret requirements in a vacuum.
Developers can be difficult, but this can be avoided…
In summation, the literal nature of computers and demanding projects with tight timeframes can turn the best developers into SOBs at times. Personally, I find the notion of short iterations and collaboration championed by Agile development a great way to solve this problem – unless you happen to have been born ornery…