Name
DEV-07: Get independent testers for functional sign-off.
Synopsis
Individual developers should and must be responsible for defining and executing unit tests on the programs they write (see [DEV-06: Set up and use formal unit testing procedures.]). Developers should not, on the other hand, be responsible for overall functional testing of their applications. There are several reasons for this:
We don’t own the requirements. We don’t decide when and if the system works properly. Our users or customers have this responsibility. They need to be intimately connected with, and drive, the functional tests.
Whenever we test our code, we follow the “pathways to success” without ever knowing it. In other words, the mindset we had when we wrote the code is the same mindset we have when testing the code. Other people, other eyes, need to run the software in complete ignorance of those pathways. It is no wonder that unit testing was so successful and yet integration testing has such problems.
To improve the quality of code that is handed over to customers for testing, your team leader or development manager should:
Work with the customer to define the set of tests that must be run successfully before an application is considered to be ready for production.
Establish a distinct testing group—either a devoted Quality Assurance organization or simply a bunch of developers who haven’t write any of the software to be tested.
This extra layer of testing, based on the customer’s own requirements and performed before the handoff to customers for their “sign off” test, will greatly improve code quality and customer confidence in the development team.
Example
I spend several days building a really slick application in Oracle Developer (or Visual Basic or Java or...). It allows users to manage data in a few different tables, request reports, and so on. I then devote most of a day to running the application through its paces. I click here, click there, enter good data, enter bad data, find a bunch of bugs, fix them, and finally hand it over to my main customer, Johanna. I feel confident in my application. I can no longer break it.
Imagine how crushed I feel (and I bet you can imagine it, because undoubtedly the same thing has happened to you) when Johanna sits down in front of the computer, starts up the application, and in no more than three clicks of the mouse causes an error window to pop up on the screen. The look she sends my way (“Why are you wasting my time?”) will stay with me for years.
There is no way for me to convince Johanna that I really, truly did spend hours testing the application. Why should she believe such a thing? She is then left to believe I am a totally incompetent tester.
Benefits
Quality of code handed to users for testing is higher, which means the end result moved to production is of correspondingly higher quality.
Customer confidence in the development organization remains high. This confidence—and the respect that comes with it—makes it easier for developers to negotiate with customers over the time-versus-quality dilemma so many of us face in software development.
Challenges
Many small development groups can’t afford (i.e., can’t convince management to spend the money) to staff a separate QA organization. At a minimum, you must make sure that customers have defined a clear set of tests. Then distribute the functional testing load to the developers so that they do not test their own code.
Resources
http://www.well.com/~vision/sqa.html : A gathering place for references related to the theory and practice of Software Quality Assurance. This site is growing to include information on Standards and Development Procedures, Product Evaluation and Process Monitoring, Configuration Management Monitoring, the role of SQA in the Product Development Cycle, and Automated Testing Tools.