17
Validating the Design
Abstract
Testing is an important part of any product's development process, and it's generally accepted that the sooner problems are found, the less it costs to fix them. Rough paper prototypes of the product design test the structure and user interface ideas before anything is committed to code. Paper prototypes, now a well-known, documented practice, support continuous iteration of the new product, keeping it true to the user and giving designers a data-based way of resolving disagreements. In prototyping sessions, users and designers redesign the mockup together to better fit the user's activities. In Contextual Design we use a combination of paper mockups, online mockups, and early versions of running code to get the right kind of feedback at each stage in the process. In this chapter we introduce how to build a mockup, run a paper prototype interview, interpret the data, and update the design for the next round of testing.
Keywords
Business analysis; Cool concepts; Design; Design thinking; Human–computer interaction (HCI); Human–machine interaction; Iterative testing; Marketing; Mobile design; Paper prototyping; Product design; Prototyping; Requirements gathering; System design; Usability; User experience; User research; User testing; User-centered design; UX
In Contextual Design, we use validation with paper prototypes to get user feedback quickly. Iterating with prototypes is a design tool, and testing early keeps the design on track. We like to get back into the field quickly after the vision. So keep the time from the beginning of consolidation to the first prototype interview short: 1–2 weeks for a small project or 3–4 weeks for something more complex. Even if you need to gather additional interviews to fill gaps in your knowledge, you can fit them into this timeframe. And that’s when you’re doing a full design process for the first time. Once the team has developed their basic product and UI structure, Contextual Design moves into a continuous process of extending and iterating the design with user feedback—a process which is likely to continue across several releases.
If you take longer than that, either you are working out more detail than you need or you are bogged down in arguing about the design. All design work after the vision is on the way to test. It is not on the way to implementation. Performing product concept and structure validation before further specification can help the team let go of the need for every detail to be finalized. The quicker you get back into the field, the less the team will get attached to any part of the design.
Testing with paper prototypes validates not only the structure and preliminary user interface of the product, but it also is another way to run scenarios of use through the product to make sure it will work for the user. But in this case these scenarios come straight
from the users—they use their own work or life activities and interact with the mocked-up product as though it was real. Then the designers can see where function and links need to be added and if the Interaction Patterns guide the user to the function and content they are looking for. In this way we not only test the product structure but we also give the users an experience with the proposed product that is immediately relevant to their life. They either get excited by this or not. This is the best way to find out early if the new product concept is worth building and get feedback to build a business case.
This iterative testing starts with rough mockups built using ordinary stationery supplies (Fig. 17.1). These prototypes are quick to build, focus the user on the product concept and structure, and invite the user to codesign. The original work on low-fidelity mockups was done at Aarhus University as part of their research into Participatory Design.1 Karen Holtzblatt adapted these methods to software in her early work on Contextual Design.2 Since then, the technique has been widely popularized.3,4 We’ll describe our approach in a minute, but as you read it, keep your focus on the primary value we see in paper prototyping: the ability to use the prototype to foster a design conversation, not a demo. It’s the interactivity of the prototyping session, where the user is placed in his or her own context, can interact with the prototype like it’s real, and can make changes in real time that gives prototype iterations their power.
The prototype is built so that it tests the most important issues of the design.5 If the design is to run on mobile devices, the prototype may be implemented as smartphone-sized cards to test whether it can be useful in that size and in real life contexts (Fig. 17.2). We validate these designs with people while they are on the go, not just in their homes or offices. If the product collects and presents information, as many task support products do, real content is developed and embedded in the prototype so that users can experience having such content available at the moment of need. This helps us test content tone, length, clarity, and structure. And it lets us test how the overall product works across platforms.
The interview is designed to mimic as closely as possible the context of real use. For example, when making new interfaces for automobiles, one team puts mockups on the dashboard of the users’ own car and played prerecorded audio from an iPod and mini speaker. In the test, they drove with users in their cars, imitating the verbal and visual messages of the new system. In this way the design team saw the user’s actual response in their car while driving. They discussed the user’s reaction with them and, codesigning, determined the best way to communicate to a driver. In the same way, mockups for mobile devices can be presented as cards on the target device itself, so designers can see how the device affects user interaction.
The design should be tested with three to six users in a round before redesigning based on the feedback. The number of users in a round depends on the scope of the project, but you never need more than six in a round. Multiple rounds of mockup interviews and iteration allow design and testing in increasing levels of detail. Over the course of the rounds of prototype interviews, the team moves from rough representations to wireframes with increasing detail, to online clickable prototypes—possibly with a visual design treatment so that can be tested as well.
Figure 17.1 A sample paper prototype. Parts of the UI are differentiated with sticky notes; information is handwritten so it can be added to or changed with users on the fly. Each trip activity is on a standard-sized Post-it so that they can be manipulated individually, and new ones can easily be added in the interview. The Interaction Pattern for this page is the Dashboard, which is also being tested.
The Contextual Design process for prototyping is: walk through the User Environment Design to ensure it’s consistent; design a wireframe-level UI using your Interaction Pattern that represents the User Environment Design and mock it up in paper; interview users using the paper mockup in their own life or work context; interpret those interviews in the design team; make changes to the User Environment Design and UI to respond to the issues; and repeat until the design stabilizes. Three rounds of testing is enough to validate the product concept, work out all the issues, finalize the product and UI structure, and build in usability. Add visual treatment in the third round or add a fourth round to test the aesthetic design and branding. In an Agile process, stories can be written and sprints started after the first round or two, with subsequent design iterations happening in parallel with development sprints.
Building a paper prototype
A primary requirement of the prototyping process is ease and speed of building. Remember that part of the goal is to make it easy to try out design options with users—if it’s too hard to build the prototype, people will be less willing to use them as design tools. Off-the-shelf stationary supplies, especially sticky notes in all their varieties, are the basic components of a paper prototype.
Rapid prototyping tools6 are so easy to use these days that it’s very tempting to build with them right from the start. Resist that temptation. Even if they were as fast as paper, they won’t support the same kind of codesign conversation you’ll get in paper. If you’re skeptical, try two or three in paper, and a few using your tool of choice. Then, you decide.
The key for a successful prototype is to put everything that might have to move during the interview on its own sticky note. This includes pull-down menus, accordion boxes, and the objects of a direct-manipulation interface. (Anything you can drag-and-drop should be a sticky note that you can literally drag-and-drop.) The interviewer will fill in the content of the interface with the user’s own data during the interview, so leave room for that. If your system is presenting content, you need to provide it written exactly as called for in the design. Print it and put it in the right place in the mockup. Take extra copies of UI elements, both to write the user’s content on and so they can try out different things over the course of the interview.
If the product combines hardware and software (Fig. 17.3), use other kinds of props in addition to the paper mockup. Pens make good bar-code scanners; pen boxes make good handheld devices; and stationary boxes make good tablets. Excellent low-fidelity devices can be built with foam core. Sticky notes can be stuck to the user’s own phone or tablet to test interfaces in the places and contexts of use for those devices.
The paper prototype represents the structure and the behavior of the proposed user interface. It’s rough and handwritten, but neat and legible—the user needs to be able to read it. If you need to go out before all focus areas have been designed, the ones you didn’t get to can be represented as a blank sticky note with a title bar—this gives enough structure to discuss the place with the user, should it be wanted. Organize the paper so that all the parts for a screen are together, with extra parts that appear on demand on a separate sheet. Put the screens in order of expected use. Make a copy of the final prototype and all its pieces for every user to be interviewed—you can copy the parts and cut them into pieces. You are pulling together a little interviewing packet complete with extra supplies for changes. Now you’re ready for an interview.
Running a prototype interview
A paper prototype interview is run by two people, one who runs the interview and moves the parts of the prototype around and a second who silently takes handwritten notes to track what is said and what is moved. Conducting the interview is very similar to a contextual interview in attitude but very different logistically. The mechanics of handling the paper prototype make it a little more complex to run. But like a contextual interview, the attitude is one of inquiry, probing into the reasons for the user’s actions and generating a sense of shared discovery, cointerpretation, and codesign. The same principles that guide Contextual Inquiry guide a prototyping interview:
Context: In a contextual interview, you stay grounded by staying close to the on-going actions and real past events of when the user was performing the target activity. You can’t do real “work” in a paper prototype, but you can stay grounded in real events. Either pretend to do the user’s activity of the day or replay a real past event. You can also alternate between doing a real task and replaying it in the prototype:
User (U): Well you know, I’d never use your product. I run across fun destinations when I’m just browsing around the web. I wouldn’t use a special tool.
Designer (D): [Knows that their design integrates with the user’s regular browser, but doesn’t need to say so yet.] So show me. When was the last time you ran across an interesting site?
U: We’re planning a trip to Paris, so I’m keeping an eye out for interesting things to do. Here’s one site I like a lot because they talk about nontouristy things to do. See—here’s riding bicycles in the Bois du Boulogne. [Clicks link]
U: It’s outdoors, it’s not typically touristy, and we all like to bike.
D: So, let’s say this is your browser. [Puts the page representing the browser window in front of the user. Page has the plug-in widget for capturing activities.]
U: Okay. What’s this? [Points to the widget.]
D: Why not try it and see?
U: [Touches the widget. Designer puts the sticky note representing the dialogue to capture an activity on the browser window.] You mean I can save the page right from here? Cool! [User touches the button for saving the page.]
U: What do I do now?
D: What do you want to do? Keep browsing around?
U: Really, I want to share this with my husband right away. If he’s not up for it, there’s no point in even saving the idea.
D: Okay, we’ll let you do that. [Writes a new link, “Share…” onto the sticky note.] How about that?
U: That’s good. What happens when I touch it?
D: Hmm… Maybe this. [Pulls a “trip activity” sticky note off the Idea Collector part of the prototype, puts it on a larger sticky note, writes “Share Trip Idea” across the top, and puts them in front of the user.]
U: That’s good. So they can see what the idea is? Nice. But I want to be able to write a note too…
The designer writes new data into the prototype to show how the interface would look with the user’s own data in it. This keeps users interacting with the prototype, either touching and changing it themselves or telling the designer how to manipulate it. Don’t let users drift into generalities—if they start talking about what they would like in a product, pursue a real story to see how the changes would play out. As they act out the story, invent fixes to the product to support them better. One design team working on a portable device well before smartphones were invented drove around with their user. When she bought gas, she said, “And now, I pay for it with this thing”—and she pretended to plug it into the pump. Having the device in her hand, it was easy to invent new uses for it.
Partnership: The partnership between user and designer is around codesign of the prototype. As the user interacts with the prototype, both user and designer will discover problems. When the user raises problems or suggests different ways to do things, the designer modifies the prototype to represent the suggestion. The designer also gives the user design options by suggesting several alternative solutions to a problem they’ve run into.
There will often be points where the user’s expectations don’t match what the designer intended: “So clicking on the widget will bring up user reviews of this activity, right?” In such cases, always pursue the user’s interpretation first: “Right, how should that look?” Start codesigning this new possibility immediately. You’re not committed to the design you and the user come up with, but by exploring it you can find out what they are thinking. You may discover a whole new issue or approach that you hadn’t thought of before. You’ll take the design you work out back to the team and integrate it properly later—or at least incorporate the ideas underlying the changes. In the interview, once you’ve explored this other avenue with your user and come to a natural stopping point, you can return to the prototype you designed: “That was interesting. But remember back here, when you first saw these photos? Suppose I told you it just captured the idea without interrupting your browsing?” This is also a good way to handle the user’s design ideas. If they are limited by their experience or skills—if their idea is just what they are used to on current websites or apps—pursue their idea until you see what they are trying to get at. Then you can draw on your wider range of options to come up with cleaner or more inventive solutions. In this way, you’ll see both what the user had in mind and, when you share them, his reaction to your ideas. You’ll also give the users more technology ideas they can incorporate and apply themselves.
Interpretation: When the user reacts to some aspect of the prototype or to the designer’s ideas, the goal is to find out what they expected and why the prototype or suggestion doesn’t match. It’s okay to discuss their ideas—it’s important to understand what they want and why, not just the specific idea they propose. Remember, just because they suggest a design change or idea doesn’t mean you have to implement it—it is just another way of getting information about their intent.
In every case, you’re looking to understand the structure of the user’s practice and whether it matches the prototype, but you’ll be talking in the language of the UI. So in this example, designer and user talk about the browser widget and the dialogue it pops up. But the solution they settle on is the one that matches the practice. The data the designer will take home is structural—do the places and functions really support the user’s intent? Do the Interaction Patterns help the user see what is presented and know how to act in each place? The particular presentation might work, or might be replaced by a better mechanism that the team invents once feedback is collected. As long as the user’s intent is met, the UI designer is free to redesign the structure and details of the interface.
It’s important that you stay open to the user’s reaction (verbal and nonverbal), and that you be willing to respond by changing the prototype promptly. One designer took out a prototype with two alternative interfaces, one of which (her favorite) merged two focus areas in the User Environment design and was based on a calendar. When that one was placed in front of the first user, she visibly recoiled. Her reaction was clear even before she said, “Oh no, I don’t want that—that’s much too complicated.”
Focus: As we discussed above, the User Environment Design represents the team’s claim that this product will improve the users’ practice. The focus of a prototyping interview is to test that claim, and fix the product when it’s wrong.
Keeping to this focus is hard, because it’s easier for people to assimilate changes and see them as a minor adjustment than to recognize a challenge to the basic structure or assumptions of the product. It’s important for the designer to be looking not for validation, but for the ways in which the product fails. Taking this attitude makes it more likely that designers will recognize a fundamental challenge.
The User Environment Design gives designers a way to listen, which also makes it easier to break existing assumptions. With the structure of the User Environment Design supporting them, designers can tell whether a suggested change affects only the UI, the Interaction Pattern, or whether it’s really challenging the structure of the product. When the User Environment Design was created and when the prototype was reviewed, designers identified specific tests to check for during the interview (we’ll discuss this more below). Where the team considered alternative designs, the prototype tests the chosen option; if the user has problems with it, the designer can introduce the alternative design on the fly and see if it fares better.
Finally, focus keeps the conversation on the right level of design. Early in the process the prototypes test the structure of the User Environment Design and the Interaction Pattern, not the UI details. If the user suggests changes to low-level UI detail—a new icon, a different word—the designer just writes them in. They don’t need to be discussed—they aren’t in the focus—but the user does need to be heard, so write it down. Later, when the prototypes are intended to test the specific Interaction Pattern and UI detail, the designer will discuss and suggest alternative UI mechanisms. The same is true if the entire focus of the project is to clean up an existing product’s UI—the prototyping interviews will focus on UI issues from the beginning.
The structure of an interview
Interviewing around a paper prototype has very much the same structure as a normal contextual interview. The difference is that after the initial discussion, you move to working with the prototype. The interview goes as follows:
Setup
Prototype interviews, like any Contextual Inquiry, need to be set up in advance so that everyone knows what to expect. Users can be people the team has talked to already, or entirely new users—new users have the advantage of expanding your user pool. In this way, some large projects have worked with 40–60 users over all the iterations they did in the course of the project. Interview three to six users with a prototype, then review the feedback from them and redesign the prototype before going out again.
It’s especially important to make sure a prototype interview is set up with people who actually do the activities the prototype supports. The user needs to have current or recent examples of doing the activity they can replay in the product, or there’s no way to test the prototype with them. In setting up the interviews, find out what the users are doing and make sure the activities you care about are covered.
And set the expectation that you’ll run the interview in the location where they do the activity—work, home, or on the go. The context and information or artifacts in that location will stimulate a more realistic use of the prototype.
For the team, the designers who will run the interviews need to be familiar with the User Environment Design and the paper prototypes. Review the prototype as a team and identify additional tests—issues which the prototype will test because of the way the prototype was designed. Perhaps the designer put lots of buttons and other interface components on the screen—then you’ll find out if the user is prone to overwhelm. Perhaps the designer added a strong visual element that separates what should be one focus area in two—then you’ll test whether dividing the focus areas. Whatever the issues are, note them along with the design choices you decided to test in developing the User Environment Design. These will refine your focus for the interviews.
Introduction
Start by introducing yourself and the focus of your design, including the kind of activities the design supports. It’s not necessary to describe the design itself at this point. You just want to start the user thinking about the kind of activity you’ll want him to do.
Then find out about the user, how the target activity is part of their work or life, and the particular relevant tasks they have to do or have done recently. At this point you’re looking for a hook to get you into the prototype. You’re looking for all the different situations, current or in the recent past, which your product would support. You may not find one; it’s possible that this person simply isn’t doing the activity you support right now. But that’s rare if the interviews are well set up. Usually, you’ll find a couple of situations which are good candidates for recreating or doing for the first time in the prototype.
Transition
Once you’ve found a set of appropriate situations to recreate, choose one to start with and transition to the prototype interview. Bring out the prototype and introduce it. Give a brief summary of the screen they start with: “Here’s an app that lets you collect ideas for a vacation trip.” Do not do a whole walk-through of the prototype. As you introduce it, write in the specific data for the user—if they’re in the middle of planning for a trip then write in places they are considering. If not, write in places from their last trip—or leave it blank and start planning a new trip. In any case the UI should show what the user would see at that point in the process.
The amount of discussion needed to introduce the product depends on how much change you’re introducing to the practice—if it’s small, you can go right into the prototype; if large, you’ll have to introduce your approach. “This product organizes your travel planning by giving you a place to collect ideas and then turn them into a concrete plan for the trip.” That would be sufficient to introduce the user to our Travel Concierge.
The interview
Once you have the prototype out and ready, move the user into interacting with it. If you’re reproducing a recent event, suggest that he replay his actions in the prototype and you’ll play computer, making the product work like it should. Or get him to start interacting with the prototype by inviting him to explore, describing what he sees and what he thinks it will do. You might run the real task in parallel in the old or a competitive product—then switch back to the prototype to show how the new product will work. If the product is supposed to show data or content, using the content from doing the real task.
Give the user a pen so he can modify parts of the prototype as you go. Some users will dive right into doing their activity; others will want to poke around and explore the different parts of the product. Let them follow whichever style is natural for them.
If the user asks for an explanation of some part of the product, you can give them a one- or two-sentence description. This is an important place to listen for the “no.” If you get a blank stare, and have to keep elaborating on the explanation in hopes that they will get it, you have a concept that doesn’t work. If your user can’t figure out what an “idea collector” is, or can’t understand how they might use it to capture ideas while browsing, it’s too big a mismatch with their current practice to be useful. Using the product will require huge amounts of retraining. Don’t try to force it to work—start looking for alternatives.
Always run prototype interviews in pairs; it’s too hard to try and manage the prototype, interact with the user, and keep notes at the same time. The notes of a prototype interview are critical to reconstructing it with the team later—it will be hard to recover the sequence of events from just the prototype, and an audiotape misses too much. It usually works best to assign one person to be notetaker, who writes down what is happening and does not interact with the user. The interviewer runs the interview and manipulates the prototype. It’s usually not necessary to videotape the interview. Video can be valuable if you are communicating back to a design team that is not going on their own user interviews and doesn’t really understand them. Video can also be critical if you are looking at problems in the detailed interaction with the UI. Otherwise, we’ve found the extra effort of videotaping gets in the way of rapid and frequent prototyping.
While running the interview, if you’re replaying a past event, keep referring to that event to keep the interview grounded. Ask how the user how the product should respond, and when she says something you don’t expect, design on the fly, extending the design you have and pulling in parts from other focus areas in the prototype when they’re useful.
In our example above, the user suggested something the designers hadn’t thought of—sharing an activity idea immediately before even saving it to the Idea Collector. So the designer follows the idea by adapting and repurposing elements of the design. The design already had a way to summarize an activity, so the designer reused it. User and designer pursue the direction the user proposed, sharing the idea. This is a challenge to the User Environment Design and to the flow of the activity, but that’s for later—after the interview is being interpreted the team can think about User Environment Design implications and redesign as necessary. The designer can also return to the question later in the interview, if that makes sense:
D: So now you’ve seen this whole Idea Collector piece…
U: Yeah, it’s good. I like that.
D: But look, remember how we started with sharing the bicycling idea with your husband? Let’s say you’ve done that, but he hasn’t responded. That shouldn’t show up here yet, right?
U: No, I’m going to want to see it. I want to know that I suggested it and he hasn’t responded.
D: So where should that go?
U: Let’s see… probably here in the “Under Consideration” section. We are considering it, after all, until he shoots it down.
The user returned to the original design idea when she added a new intent. Once she had a chance to work through using the product in a pretend situation, she could look back, see the implications of her original suggestion, and provide further feedback.
Wrap-up
The final wrap-up of a prototype interview starts with a simple summary of the key points that came up during the interview. Summarize the points, and if it’s useful summarize any parts of the prototype you didn’t get to, for a quick reaction. But this won’t be contextual data, so don’t spend a lot of time on it.
Finally, check the emotional response to the product and the sales point. Ask, “Do you like this product? Will you buy or recommend it? How much will you pay for it?” You’re not looking for a real, committed figure here. You’re looking for a sense of how valuable and exciting they think the product really is. And you can uncover their expectations and threshold figures for cost. You can also ask them for the top three features they think are the most important—now that they used the product they can tell you from experience. Then say, “If the team can only implement one which should it be?” Now the user is helping you prioritize. At the end of the interview, you’ll get a response based on experience with the product, influenced by excitement generated by the interview. In this way, the response is better grounded than you can get from a focus group or demo.
This is the general pattern of a prototype interview. If you’re tracking an ongoing task rather than replaying an old one, the user will alternate between doing some real tasks and then redoing it in the prototype; if you’re replaying past activity you’ll be repeating all steps of the activity in the mockup. Just like in a retrospective field interview, look for any artifacts (like receipts of air tickets or hotel reservations) to help the user remember what they were doing. Designer and user discuss the prototype, using real interactions relevant to activities that matter to the user to drive the conversation. From time to time they invent some alternative, elaborate it for a while, then come back to the prototype as designed. The designer uses her knowledge of the User Environment Design, Interaction Pattern, and technology to drive the interview—if you can recognize when the user is challenging some aspect of the overall design you can probe for details immediately, instead of having to wait for another interview.
Later in the design process, when you trust the structure of the product represented in the prototype, concentrate more on the specific Interaction Pattern and UI elements and on enforcing the limits of a real product. When the user tries to do something the design doesn’t allow, instead of taking it as an opportunity for codesign, act like a real CPU: beep at him or her. See if he or she can figure out how to make the product work given the limitations you’re building in.
Exercise different parts of the prototype by switching activities. Follow the task the user is doing or recreating until it’s done or has moved beyond the scope covered by the prototype. Then choose another situation to follow that will exercise any parts of the prototype that haven’t been touched yet. Usually, you won’t get to all the parts of the prototype in a single interview and that’s okay; end the interview after 2 hours and save the rest of the prototype for the next user.
Going online
The first two rounds of prototyping should be in paper and in person, for all the reasons we listed above. But after the basic structure has been stabilized you’ll start to be interested in details of the interaction and UI that are hard to reveal in paper. With modern prototyping tools—and modern tools for building real UIs—it soon gets simpler to model these interactions in working code than in paper. By this point you’ve seen the users in their own contexts many times, so you have a good understanding of it—and the issues you’re looking at depend less on context anyway. This is a good time to start testing your visual design—if you want to test more than one look you can do two versions and get feedback on the look as well.
You may use your online prototype in person or, if it is a web-based product, you may run a remote interview. At this point your product structure, Interaction Patterns, and basic content and function are set. So the focus of these interviews is more on visual design and low-level user interface elements. For elements of the design that run on mobile devices, you’ll do best to run at least some of the interviews in person with running prototypes on the device. That way you’ll get a better feel for how the user interacts not just with the screen, but with the whole device. You’ll also be able to better see device-specific issues—use of screen real estate, touch versus mouse, swiping, and all the rest.
The prototype you test may range from pure images to working code, and you’ll adapt the interview accordingly. At the beginning, you may have UI images in Adobe Illustrator or a similar product, and you just screenshare with the user, showing the UI by turning layers on or off. You may use a prototyping tool that implements some of the interactivity, in which case you might prefer to give the user control and watch as they use the prototype. Especially in an Agile context, you may have working code which can access real content, in which case you’ll definitely want to let them run the interaction. In any case, tailor the prototype to your user’s interests if you can—if you found out when setting up the interview that they are interested in a Caribbean vacation, don’t show them a prototype full of European cathedrals.
An in-person interview using an online prototype is run like a paper prototype session. For a remote interview, use a screen-sharing session so you can both see the same thing. It’s possible but harder to host the prototype on a web server and look at it together without sharing screens—only do that if screen sharing is impossible.
For the introduction, get the user to tell you what they’re doing and look for a current or recent task that you can replay in your prototype. You usually won’t be able to actually change the content, but you’ll continually refer back to their actual tasks as examples. Get the user to bring up and show examples of what they do.
When you’re ready for the transition, bring up your prototype. Let them tell you what it does and what they see. If they want to explore, either give them control or tell them you are their cursor and have them tell you what to click on.
Then follow a task. As each screen comes up, let them react to it and describe how they would use it. As necessary make the connection to their data and your prototype by saying things such as, “The main area there where you’re seeing beaches would be full of snorkeling tours, since that’s what you’re looking for right now.”
Since your focus is on the UI and visual design, now probe for responses to layout, color, and imagery. Listen for the emotional reaction caused by UI choices: “Sounds like you’re finding this screen a bit stressful or confusing. Do you think all the red is making that worse?” Listen for places where the interactivity of the UI is causing confusion—controls that only appear on hover, accordion folds that aren’t obviously able to open, trays that are nearly invisible when shut. Do not jump in quickly with explanations, or you’ll never find out about these issues.
Wrap up as for any other prototype interview.
The Interpretation Session
The last part of a prototype interview process is the same as for a contextual interview: the interviewers bring the data—their own and the notetaker’s handwritten notes and the marked up paper prototype—back to a design team and replay the interview for them so that everyone can see what happened and offer their different perspectives. This Interpretation Session is focused narrowly on identifying the issues raised by the interview.
Issues are captured as notes by a notetaker, but in general we don’t build an affinity from these notes. Most of them are about specific issues in the interface, so a general technique like an Affinity isn’t needed. Instead, capture them in a spreadsheet, associating each note with the focus area and section it relates to. See Fig. 17.4.
Figure 17.4 Notes captured during a paper prototype Interpretation Session. The columns show: which round of interviews the note came from; user; the note itself (“V” means it’s a validation of the design); whether the note introduces new practice, or addresses the User Environment Design or UI; the focus area affected; the section or component of the Interaction Pattern affected, if any; and how the issue will be resolved.
Most of the data from an early prototype interviews will be structural issues for the User Environment Design. Capture these and tag them with the Focus Area they relate to. Do the same for reactions to content. You may also get feedback on the Interaction Pattern used for layout and lower-level detail of the user interface, even though this is not the primary focus of the first round of testing—capture them and tag them with the User Environment Design focus area and section. They will be dealt with after the User Environment Design structural issues are refined. Any issue that has to do with layout, presentation, graphic elements, interactivity, or wording is a UI issue. There will also be some issues, which describe aspects of the user’s practice or raise new issues that weren’t seen before but aren’t directly related to the prototype. Flag these with “UP” for user practice, but don’t try to resolve them in the meeting. Afterward, you can add them to the affinity or update an Experience Model as necessary to capture the new practice elements. If there’s any disagreement about where an issue goes, move it upstream—put it on the User Environment Design in preference to the UI, and flag it as user practice in preference to either. This whole session is good practice in separating conversations, as each conversation has its own model and its own place on the wall.
Throughout the Interpretation Session, the primary task is to see behind the user’s reaction to the UI to understand the underlying issue. If the user was overwhelmed, was that because the focus area wasn’t clean, the design didn’t match his approach, or was the UI for that part of the product unnecessarily complex? Examine the user’s actions and words to understand what his reaction meant to the design. After multiple iterations, the product structure stabilizes and the interviews will focus more on the Interaction Pattern and UI and finally, visual design. In these Interpretation Sessions you will see User Environment Design issues drop off and an increase in feedback on the layer of the design being tested.
Design iteration
When a design has been tested with three to four users, it’s time to iterate. The issues raised by the users are grouped in the Interpretation Session so related issues can be addressed together. Changes to Contextual Design Models may affect the User Environment Design, and changes to the User Environment Design may affect the Interaction Pattern and user interface detail. No one wants to spend hours on some aspect of the user interface only to discover later that a change to the User Environment Design obviates that whole part of the interface. So the first issues to deal with are those related to the models—look those over and decide whether any of the new practice poses a sufficiently serious challenge to your design that you should do new storyboards. (If you did your initial research well, this should be rare.)
For the models, collect the issues from all users for each type of model. Organize them to see what they imply for the model. Extend it with any new aspects of user practice: new activities in the Day in the Life, new or expanded identities, new relationships, or collaboration strategies. If this new data affects the focus of the design, you’ll deal with it as part of addressing the User Environment Design; otherwise, it becomes part of your permanent representation of your user population.
Then turn to the User Environment Design. First consider whether any of the model changes affect the design, and if so identify which parts they affect. Then look across the data from all the users and ask what the primary structural issues are. Look for ways to redesign the overall product to address these issues. Then start working on sets of issues part by part, starting with the parts that are involved in the most important and far-reaching issues. Collect the issues across all users for each of those parts, and consider how to redesign them to address the issues and the new information from the models. Use quick storyboards, if necessary, to help you think through particular activities in the changed product. Go on from section to section of the User Environment Design until you’ve addressed all the issues you need to. You may decide that some issues are at too low a level of detail to bother with yet, or affect some part of the product which is too peripheral. There’s no point in spending a lot of time to get a part of the design right if it’s only going to be cut later. Work to get the core product stable, then prioritize what to ship, and then do the details.
If you build the first prototype on the core storyboards only, you may have additional storyboards to roll into the product for your second round. Do so at this time and restructure the User Environment Design so that any new function, content, and changes to the Interaction Patterns are incorporated. Restructuring the product tends to pull apart the design, so finish with a User Environment Design Walk-through to pull it back together again. Check the Interaction Pattern for implied changes too. Clean up the loose ends and make sure the design is reasonably clean.
Now, roll the User Environment Design changes forward into a redesign of the Interaction Patterns, including any changes needed based on user feedback at that level. First look for broad issues that affect the Interaction Pattern architecture as it was designed. Did the base structure work? Did your Interaction Patterns guide users through each place? Did you achieve overall consistency? If you need to change the base metaphor or redesign Interaction Patterns, do those first, before addressing any of the lower-level UI issues.
Finally, collect the lower-level user interface feedback and redesign all aspects of the prototype. Review the new design together, and mock the whole thing up in paper. Do a second round of testing, review, and redesign. Your third round of testing can have a more detailed UI and, if you choose, a visual design. At this point you may put the prototype online. This round may be run in parallel with your first development sprint in an Agile process.
Completing the design
This is the iterative, user-centered process of Contextual Design. As you expand your design to address more and more of your vision, the product will change and flex to accommodate new issues. The core design may be quite stable, but when you move to address a new area of practice, you’ll collect more user data—though it’s likely to be tightly focused on just the aspect you want to address. You may set up interviews to capture data for an activity you were never able to observe until this point. Or you may discover the opportunity to get this data in the middle of a prototype interview. If it’s important, set the prototype aside for a bit and do a regular Contextual Interview to get the data you need. Then return the interview to the prototype. Later, in the Interpretation Session, move to capturing Contextual Inquiry-style notes when you hit that part of the interview. You may choose to add them to your Affinity or consolidated models later.
Throughout these iterations, you’re always using the prototype to drive user visits and keep the team grounded in real user data. Returning to the user every 10 days to 2 weeks keeps the team focused and moving forward; in our experience lack of regular contact with users is a primary reason teams lose focus and break into arguments with each other. A prototyping approach to design keeps you going.
This iterative design process continues until the team is sure it has a workable design. Usually after two to three iterations of a part of the User Environment Design with users, that part is pretty stable. The number of structural issues fall off and UI issues start to predominate. This is your signal that the structure is pretty much right. The same thing happens with the UI architecture—after a round or two, the basic set of Interaction Patterns should be stable. With the product and layout structure stabilized you can concentrate on UI details. And if you are presenting content, you keep iterating its structure and tone. With each iteration you can extend the prototype to test the structure of another part of the product while getting detailed feedback on more stable parts as well. The part that has stabilized can also be moved simultaneously to implementation design and code. Prioritize what parts of the product to deliver, as we describe in Chapter 18, and build a shipping User Environment Design and set of Interaction Patterns for the next release. This becomes your working requirements specification.
When you move through a design in this way you can be confident that you’ve understood the requirements and have designed the appropriate product structure. Development of the product can proceed through the implementation and testing of running code in much the same way that we’ve tested prototypes. This maintains the user contact while implementation progresses. The iterative prototyping process merges with an iterative implementation process that coordinates all the parts of the team to deliver on the vision.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.