The iterative nature of software development implies that the system must track revision histories and maintain version control of alternate RAD versions. The user’s response to a demonstration may require that the RAD fall back to a previous iteration for change, or the developer might wish to demonstrate several iteration alternatives for user comment (one of which, or portions of several being selected). Requirements may be added, changed, or deleted. Test goals must easily change to fit modified requirements. Ideally, the developing environment will capture this modification explicitly, along with the accompanying purpose of the modification. Any testing tools developed must take these modifications into account to test the proper version and to appropriately consider the requirements and purpose germane to that version for test development. The tool should also exploit change as a likely source of errors. A tool that helps testers compare changes from one iteration to the next, along with system dependencies potentially affected by changes, helps test development considerably.

The use of reusable components raises reliability concerns. Have the components been unit tested, and if so, to what degree? Have they been used in previous implementations, and if so, which ones? What testing has been conducted on the previous implementations as well as the individual components? The testing methodology must consider how information on past component testing can be recorded and referenced to determine what unit testing might still be needed and what integration testing strategies might best check the components in their instantiated context.

One necessary testing component is a set of test conditions. Requirements-based and functional testing base test conditions on some stated form of behavior or required performance standard, such as formal specifications or a requirements document. The development methodology does not provide a separate performance standard. The testing methodology must establish an objective standard of the intended behavior of the RAD under consideration. Every program must have an objective performance standard. The developing system must then, in some fashion, provide the tester and his or her tools with access to a system functionality description and system requirements to allow rapid, complete, and consistent derivation from the RAD. This access to functionality descriptions and requirements has the added benefit of helping develop scripts for demonstrations so that particular iteration changes and enhancements will be highlighted for the user’s comments.

The software development environment not only should capture requirements, assumptions, and design decisions, but, ideally, should map these into the RAD in a way useful to both rapid application development and testing. This mapping automatically provides a trace, documenting the RAD’s development. As the system grows, knowing why a particular design decision was made and being able to see where (and how) the RAD implements it will be difficult without automated support. The developing/testing paradigm must capture mappings from design or development decisions to the implementation. These mappings need to be rapidly revisable to quickly make the next RAD iteration.

The chapter provides a framework for iterative RAD testing. The most obvious approach to testing during RAD would be to treat each development iteration as one software life cycle. An advantage is that this keeps intact the methodology of testing familiar to most testers. The lack of a conventional specification effectively removes the information basis for test planning. Under current descriptions of the developing process, a specification would need to be generated, at least in part, before conventional techniques could be applied.

The process’s complexity is also compounded by the need to conduct a full cycle of testing for each iteration, even though the early iterations will almost never contain detailed or unchanging requirements. This would be inefficient and impractical testing.

An alternative test approach is to iterate test planning in parallel with the developing iterations. This will simplify testing and reduce overall testing costs when compared to the preceding approach. The initial test plan would consider only the basic system description contained in the initial RAD iteration. As RAD iterations proceed, the test plan would expand to incorporate the latest iteration modifications. One disadvantage is that this approach causes the test plan to follow the RAD process closely. The decisions in the RAD might unduly influence the test plan, causing important test conditions not to be explored. The possible disadvantage suggests that the unit and integration testing might be done iteratively, with acceptance testing occurring entirely on the final iteration of the development cycle.

By considering how the developing process closely follows the spiral process model that leads to a spiral iterative test planning process. Figure 16-2 illustrates this process.

Figure 16-2. Spiral test planning process.

The proposed RAD testing strategy, termed spiral testing, remains iterative and parallels the RAD process. Spiral testing characterizes the varying types of RAD iterations by tailoring the testing process to account for these differences. Spiral testing distinguishes between the initial few RAD testing iterations, subsequent iterations, and the final few iterations. The major distinction between the first few testing iterations and the subsequent ones is that the first iterations, for any but the smallest of systems, probably will have only test planning and structuring activities that establish priorities and areas of test emphasis.

The framework for intermediate testing activity and final acceptance testing, to include test derivation, is laid in the initial iterations. Unit and integration testing will likely be confined to subsequent and final testing iterations. Subsequent testing iterations will have less framework-related activity and more acceptance test oracle derivation activity. The major distinction between the subsequent and final iterations is that the final iterations are where developers return to their RAD to fix identified errors and testers conduct final integration and acceptance and regression testing. Figure 16-3 shows the separation of the groups of iterations for either the development or testing spiral. The following sections cover spiral testing in detail.

Figure 16-3. A “targeted” spiral.

There are strengths and weaknesses to using the RAD concept to build software. If the advantages outweigh the disadvantages, RAD should be used. However, some applications built under the RAD concept would have been better built under other software development models.

RAD development offers the following strengths:

The problems associated with using the RAD model for development, on the other hand, include the following:

Understanding the strengths and weaknesses of the RAD model, testers should assess whether RAD is a desirable model for a specific software system. Using this analysis, the IT organization can be prevented from building software using the RAD model when other developmental models would be more appropriate.

Testers should use Work Paper 16-1 to determine whether the RAD model is appropriate for their software system. If the testers are uncertain whether a specific item can be answered Yes, they should give a No response. If more than 25 percent of the items have a No response, the applicability of using the RAD model for this specific software system should be re-evaluated.

The first few iterations of the RAD serve varying purposes, depending on the particular software under design. When feasibility is not a consideration or when a detailed requirements document exists, the first development iterations establish the product’s design framework as a base upon which to RAD the remainder of the system. When feasibility must be investigated and/or when requirements are unknown, the first several development iterations seek to construct abstract RADs to see if an acceptable system can be designed. If the RAD is feasible, developers establish the major software requirements and design a development plan upon which to build the system during successive iterations, as in the first case mentioned previously. The first few development iterations will usually be devoted to establishing an overall RAD structural framework.

To mirror this process for test planning purposes, the initial test planning iterations consider the developing results and frame the testing process for the remainder of the project. This is the logical point for testers to determine the major critical portions of the RAD, and establish test priorities. As RADs establish major requirements, the test team begins to break these down into test goals, determining derived goals as well. As development continues, the testers can define the testing emphasis in greater detail, make needed test plan adjustments, and record test justifications.

It appears prudent (although not essential) throughout the development process for the test team to review user input and generate goals independently from the RAD team, so as to identify misstated requirements and to find missing requirements. This increases the quality of the RAD versions, thereby decreasing the number of iterations needed.

The initial iterations are where the test team will forecast the most important portions of the system to test. As the implementation hierarchy of the system takes shape, the testers establish test sections for path and integration testing. The long-term testing purpose is to build the framework for constructing the final acceptance-test oracle and to fit the intermediate testing activities into the overall development plan. The process will be manual for the most part, and this would be where initial testing tools and their databases/instrumentation would be initialized. The initial iteration phase would end at the RAD iteration in which the top-level requirements specification is established.

It is recommended that the documentation for each iteration of the RAD process be subject to the inspection process, as outlined in Part Three of this book, as well as complete the checklist in Work Paper 16-2.

Once the basic RAD framework is established, subsequent iterations commence in which developers enhance the RAD’s functionality and demonstrate it for user/designer review. In the typical case, additional requirements are identified and the design matures in parallel over multiple iterations. Both are validated in the review process. At some point, sufficient requirements are identified to establish the overall system design.

The subsequent testing iterations will be characterized by unit testing, integration testing, and continued requirements review to determine whether any requirements are missing. To complement the design process, the test team concurrently develops integration test plans as designers complete subsections of the system design. In addition, as reusable components are instantiated to provide required functionality, the test team or design team unit tests these modules (both approaches are acceptable in current practice) by consulting the test history of the instantiated modules and conducting additional unit testing appropriate to the developing system.

Should this additional testing be necessary, the test team updates the test history to reflect the additional testing and its results. This update could be as simple as appending a test script (although this could eventually cost a large amount of storage for a large components test library) or as complex as completely revising the test history to incorporate new test sets, assumptions tested, test case justifications, and additional test history details. The complex update may be needed to reduce a series of simple updates into a consistent and usefully compact synthesis. As performance aberrations are found during a given iteration’s tests, they are readdressed to the design team for correction prior to the next iteration demonstration.

As the design team RADs system components, the test team can commence integration test planning for those components. The integration testing process is the same at any hierarchical system level for integration testing. The test team needs to keep integration testing at various levels coordinated to maximize test efficiency. If a standard structure for integration test sets could be constructed, then it might be possible to develop tools to manipulate these to conduct increasing levels of integration testing as more components and system subsections are implemented. Currently, most of this process would be manual. Final integration testing cannot commence until the RAD implementation is complete.

Throughout testing, testers consult the RAD specification and all requirements to determine the correct responses to test data. Considerable information needed for test data selection will likely be found in the RAD specification language. Automated support to extract this information would be very helpful, but will depend on the developing language in question and on possessing the capability automatically to relate the criteria to selected test data and execute the test.

Throughout the iterations, the test methodology must remain responsive to change. Existing components and specifications may change or disappear between iterations, contingent with user/developer/tester input. Additionally, each new iteration adds increased functionality or furthers the completion of existing incomplete implementations. The test development process must capture all effects of change because additional testing or retesting of previously tested code may be required. This retesting raises the issue of “phase agreement” between the RAD spiral and the test-planning spiral.

An in-phase agreement would have formal iteration testing proceed at the completion of a development iteration and prior to iteration demonstration to the user. The advantage here is that the test team will have tested the system and the developers are not as likely to demonstrate a system that contains undiscovered or obvious bugs. Any problems encountered in testing are corrected prior to user review. User confidence is not enhanced when bugs that are not related to design issues are discovered during the demonstration. On the other hand, many iterations will usually be demonstrating requirements not yet validated, and it is wasteful to test requirements that have not been validated.

An out-of-phase agreement relies on the designers to test their RAD iteration sufficiently prior to demonstration (for their reputation, not for formal testing). The test team conducts formal testing for an iteration at the conclusion of the user demonstration. They modify the formal test plan, developed during the development iteration, by removing any planned testing of eliminated, changed, or superseded requirements and by adding additional testing of corrections and modifications resulting from the user’s review. Test planning proceeds in tandem with iterations development, but actual testing waits for the results of the user’s review. Once the testers obtain user comments, they may assume that the stated requirements at that point represent solid requirements for the purpose of testing. This assumption continues until a requirement is explicitly or implicitly changed or eliminated. The advantages of out-of-phase testing are a savings in testing conducted (because you test only reviewed requirements) and increased test responsiveness to user review. The disadvantages are the increased likelihood of missed requirements and the possibility of bugs in the demonstrated systems. Work Paper 16-3 should be used when subsequent iterations are inspected.

Once developers establish all requirements (usually in the latter iterations), the final few iterations of development are devoted to implementing the remaining functionality, followed by error correction. Therefore, the testers can devote their work to completing the test for acceptance testing, and to remaining unit testing and subsection integration testing.

The final test planning iterations commence with the completion of the operational RAD and prior to final user acceptance. As any final requirements are implemented or as system components are fine-tuned, tests are developed and conducted to cover these changes. Most important, the test team completes the acceptance test plan. Once the system is completely implemented and the acceptance test design is complete, the test team conducts the acceptance test. The test team checks differences in actual results from expected results and corrects the tests as appropriate while the design team corrects system faults. The cycle continues until the system successfully completes testing. If the design team is busier than the test team, the test team can use the available time to conduct additional testing or priority-superseded testing previously skipped. The result should be a sufficiently tested software system. Work Paper 16-4 should be used when the final iteration is inspected.