Depending on the product and testing objectives, designers have options ranging from lab testing, where real-life scenarios are simulated such that all participants can engage and assess the same interactions, to field testing, where user-product interaction is evaluated in real-life settings, but changing conditions make it harder to ensure situational consistency across all testers.
For those who need to approach usability testing with the rigor of scientific research, or because real-life testing is unsafe, such as early testing of new commercial or military airplanes--usability test labs provide an environment for testing under optimal controlled conditions--while the testing participant is performing simulated scenarios in an isolated environment, the session can be observed by designers and stakeholders through double sided glass or streaming, and recordings are used later for detailed analysis.
For example, the Center for Medical Simulation in Boston is a company that specializes in the use of "high-realism medical simulation" to:
Eye tracking usability testing, mentioned earlier, is a method of recording and analyzing the tester's eye movement, gaze, and focus throughout an interaction. Analysis of such testing helps design aircraft cockpits, and other types of high intensity interfaces. In such environments, the operator is required to make critical, sometimes life and death decisions, while their attention is divided across multiple instrumental data sources as well as external conditions. The test provides evidence to the efficiency with which the operator shifts their focus across needed instruments, and which elements are most looked at. The latter should be placed in more prominent positions if they are found to be in a less optimal place.
Performing eye tracking tests outside of a lab-based simulator and in the real world, involves wearing special glasses that record the objects the user looked at, the amount of time the eyes were focused on certain areas, the path the eyes followed, and so on.
The preceding picture shows a tester equipped with Tobii pro glasses (A). With these, it is possible to conduct the tests in a real environment, such as a store, in a car, and so on. The output of the date stream (B) makes it possible to generate heat-maps of the products the tester focused on and spent most time looking at.
Software and web-based applications lend themselves to remote testing using screen sharing and video conferencing technology. This technology has become pervasive, inexpensive, and reliable--just when travel, and especially cross-state or international-travel has become expensive and difficult.
Although the design can be modified after the release, as is often the case, there are some design problems which, if not fixed prior to the product's release, risk generating significant negative responses. This is one of the major risks companies worry about and yet, it is often the case that the budget and time allocated for testing is inadequate. As with other aspects of the design process, competitive pressure is on the business to release the product as soon as possible. Companies and designers are faced with choosing one of the following approaches to testing:
- Incremental testing: Frequent tests with real users throughout the design and development process, which provide continuous streams of validation data to the team. Some argue that this is the most effective approach, in terms of overall costs and impact on the final results.
- Milestone testing: At specific points in the design process, the work is taken for testing--often when it is already too late to make any significant changes before the start of production without missing target release dates.
- No testing: In this approach, the team is satisfied that internal quality control testing is sufficient, but the actual design itself is not tested with potential users. This approach is the riskiest, but probably also the most common.