Use Your Product

It may sound trite, but the best way to learn about your product is to use it (if it already exists). In the case of a travel website, you should search for a hotel and flight, make a reservation, cancel your reservation, ask for help. Stretch the product to its limits.

Networking

You are surrounded by people who know about your product and domain, you just have to get to know them. Start by finding out whether your company has a usability group (that is if you are not a part of such a group). Meet the people who support your product and read their usability reports. If you are unable to watch a live activity, ask to view tapes of previous activities for your product and related products. What previous usability issues were encountered? What user requirements have been documented so far?

You should also meet the content writers for the company. These are the folks who create the user manuals and online help (for websites and web applications). What is difficult to document – either because it is difficult to articulate or because the product is so complicated to explain?

If your company offers training courses, attend classes. Speak with the folks who teach the courses. What is hard to teach? What types of question are users asking? What tips (not documented) are the instructors offering?

Customer Support Comments

If you are working on a product that already has an existing version and your company has a Help Desk or Customer Support Group, you can learn a great deal about your product by visiting that department.

People rarely call or e-mail in compliments to tell a company how wonderful their product is, so those calls pertain to issues that you should become familiar with. If you can access logs of customer questions, problems, and complaints over time, you will see trends of difficulties users are encountering. Perhaps the user’s manual or Help isn’t helpful enough. Perhaps the product has a bug that wasn’t captured during QA. Or perhaps users do not like or cannot figure out a “feature” the product team thought every user had to have. This can give you a sense of where you will need to focus your efforts to improve the product.

Users may not be able to accurately describe the problem they are having or may misdiagnose the cause. Likewise, customer support may not have experience with the product under consideration. Although this should never be the case, it often is. If the support staff are not very familiar with the product in question, they may also misdiagnose the cause of the customer’s problem or may not capture the issue correctly. This means that, once you have a log of customer feedback, you may need to conduct interviews or field studies with users to get a full understanding of the issues.

Log Files

If you are responsible for the usability of a website, web server log files may provide an interesting perspective for you. When a file is retrieved from a website, server software keeps a record of it. The server stores this information in the form of text files.

The information contained in a log file varies but will typically include: the source of a request, the file requested, the date and time of the request, the content type and length of the transferred file, the referring page, the user’s browser and platform, and error messages. Figure 2.1 shows a sample of a server log file.

Programs are available that can analyze log files and produce usability data. However, you will likely need to work with your organization’s IT department to have useful information captured in your server log file in order to conduct usability analysis.

Information that can be captured includes:

There are some issues or limitations that you should be aware of when interpreting the data you collect from log files.

You can work with your IT department to address each of these issues, but it takes time and knowledge of what you need versus what can be captured. External companies like WebTrends (www.netiq.com/webtrends/default.asp) can be hired to capture web usage data. These companies are a great source of basic usage data once you have a website running (e.g., number of hits per hour or per day, what page the user came from, how the user got here, time spent on the home page, number of excursions to other pages, time spent on each page, ads clicked on). They can collect data on almost any user action you want to keep track of.

Examining the time users spend per page is a more meaningful measure than the number of hits per page. When analyzing time data in a log file, it is best to use median rather than average times because the median is less susceptible to outliers.

In addition to capturing information about user motivation, you can learn about visitor profiles and activity from log files. However, it is best to study log files over a long period to discover data that compliments or spurs studies in greater depth. For example, you can identify areas of a site for further testing.

Your Marketing Department

Frequently, your company’s marketing department will conduct focus groups or competitive analysis to determine the need for a product and how best to promote and place it. Although this information is meant to drive sales, it is an excellent source to learn about the product, as well as potential customers and competitors.

Unfortunately, you do not always learn a lot about end users. Marketing information should not be confused with user requirements. The data from the marketing department can reflect the end users’ needs, but not always. Marketing collects information about the value and perceived worth of a product, whereas usability professionals collect information about how something is used and how the product’s worth is realized.

Additionally, the information you collect from the marketing department is only part of the information needed when creating a user profile. It is often missing contextual information about circumstances and environment that can affect a user’s decision to use and how to use your product. This is often the case for products that are to be used by corporations rather than an individual (e.g., Human Resources applications versus a website designed to sell books to the public). In the case of corporate products, the marketing department is typically interested in the business needs of the marketplace (i.e., companies that could be potential buyers) rather than the end users. Regardless, this information can be very helpful to you when you are starting out.

When you contact the marketing department, you may want to ask them questions such as:

Early Adopter or Partner Feedback

Frequently, companies will align themselves with a small number of customers in the early stages of development. These customers may play an active role in the design of the product and become partners in the process. They may implement early versions of the product on a small scale in their own companies. This relationship is beneficial to all parties. The customers get to help design the product to meet the needs of their own companies. On the other side, the product team gets early feedback and the company can ask for references when selling the product to others.

The feedback early adopters provide can be enlightening because the product can be implemented and used in a real-world setting (as opposed to testing in the lab). You can leverage these existing relationships to learn about the product space and some of the product’s known issues. However, when you are ready to move on to the next stage and collect user requirements, be wary of basing all of your user requirements on the few existing customers. You should obtain user requirements from a variety of users (refer to “Learn About Your Users” section, page 41).

Competitors

You can learn a lot from your competitors. A competitive analysis can be an effective way to gain an advantage over your competitors. It is beneficial to conduct a competitive analysis when you are creating an entirely new product or simply entering a new product domain. It can also be a great strategy if your product is failing, but your competitor’s product is thriving. It is wise to periodically check out your competition to see where you stand with the rest of the pack. Some companies have product analysts whose primary job is to keep tabs on the competitors and the market. Get to know this person and learn everything he or she knows. If the product does not have a product analyst, this is something that you can do.

Do not limit yourself to direct competitors. You should also examine surrogate products. These products may or may not compete directly with your product but they have similar features to your product and should be studied to learn about your strengths and weaknesses. For example, if you are adding a shopping cart to your travel website, check out companies that have shopping carts, even if they don’t compete with you (e.g., online book stores). Some people make the mistake of thinking their product or service is so innovative that no one else does it; therefore, they skip competitive analysis. No product is so revolutionary that there isn’t someone out there to learn from.

Traditional competitive analysis focuses more on cost, buying trends, and advertising. A usability competitive analysis is more concerned with the user experience (user interface, features, user satisfaction, overall usability). The goals of both types of competitive analysis are to learn from your competitors and snag a strategic advantage. Below we will concentrate on conducting a usability competitive analysis.

To identify your competitors, speak with the product team, sales or marketing department, conduct a web search, read trade magazines, and conduct user surveys, interviews, or focus groups. Market analysts and researchers – such as CNet, ZDNet, Gartner, Anderson, Forrester, and IDC – can be a great way to collect information about your product’s market space and competitors. These companies identify and analyze emerging trends in products and their impact on business.

Keep in mind primary competitors as well as secondary competitors. A secondary competitor can be a smaller company with a less threatening product offering, one having only a few features in common with your product, or one competing indirectly with your product. For example, the bricks and mortar travel agency does not compete directly with your online travel company but it is an alternative that should not be ignored.

Once you have identified your competitors, you should ascertain their:

In gathering this information, it is extremely helpful to access some of the competitors’ products. If a product can be bought off the shelf, or is an easily accessible website, this should not be a problem. However, some products or services are sold only directly to companies. Most major software companies state in their licensing agreement that you are not allowed to conduct competitive tests against their product. They also state that you cannot show the installation of the product (or the product in use) to a competitor company.

If you are able to evaluate the competitor product yourself, you should identify a set of core tasks with which to compare your product (if you have one at this stage) and the competitor’s. This is particularly important if you plan to include functionality from the other product, because you may learn that a function doesn’t work well. Take numerous screenshots or photos and record your interaction as you conduct the evaluation.

Whether or not you are able to access the product yourself, surveys (Chapter 8, page 312), interviews (Chapter 7, page 246), focus groups (Chapter 12, page 514), and wants and needs analyses (Chapter 9, page 370) will be valuable ways to learn about users’ perceptions of the product. By conducting these kinds of activities with users of your competitor’s products, you can learn about the strengths, weaknesses, and key features of these products. In a competitive analysis, the majority of your effort should be spent mining the competitor’s product for ideas (e.g., functionality, user interface style, widgets, task structure, terminology).

There are a number of companies available to measure the usability or user satisfaction of a website (e.g., Vividence, Biz Rate, OpinionLab). They can do this for your site or for your competitor’s site. These companies send actual or target customers to any website (i.e., yours or a competitor’s) and then collect qualitative, quantitative, and behavioral data as users pursue actual tasks in a natural environment, such as their own homes and offices. Most companies allow clients to easily analyze both the quantitative and qualitative data gathered during an evaluation. This approach can be quite beneficial, but it is often expensive and requires that users can easily access the website. If the web-based product must be purchased or is behind a firewall, you will have to provide the users with access. In the case of a competitor that sells licensed software, this option is not possible.

As you conduct your competitive analysis, it is helpful to create a grid comparing your product against the competitor’s (see Table 2.1). List the key features, design strengths and weaknesses, usability scores or issues … anything you can learn. Tracking this information over time will show you how your product compares and how the market may be changing.

Finding Information to Build Your User Profile

Before you can conduct a user requirements activity, you need users. It is vital to get the right users, otherwise the data you collect are not only worthless, they can actually harm your product, your credibility, and the credibility of usability activities. But who are your users? What are their goals?

You should begin by developing a user profile. This is a detailed description of your users’ attributes (job title, experience, level of education, key tasks, age range, etc.).

For example, the typical users might be between 18 and 35 years of age, have job titles like “Travel Specialist,” “Travel Agent,” or “Travel Assistant,” and work for travel agencies with fewer than 50 employees.

Creating a user profile is an iterative process. You will likely have some idea of who your users are at first, but this will probably not be detailed and may even be just a guess. But it is a place to start. The example above is just our first, best guess of who our travel agent user might be. You can capture the initial information to build your user profile from:

Once you have an initial stake in the ground, you can develop a phone screener and begin recruiting for your user requirements activity (refer to Chapter 5, Preparing for Your User Requirements Activity, “Recruiting Participants” section, page 156). As you conduct your activities and learn more about the end users, you should come back to the user profile and update it. Your first guess may have been slightly off left. At the very least it probably is not as detailed as it could be.

Understanding the Types of Users

You need to define what you mean by “user.” Most people consider the individuals who will interact directly with the product as their users, but you may need to consider other individuals as well:

Try to categorize your users into one of three categories: primary, secondary, and tertiary. Primary users are those individuals who work regularly or directly with the product. Secondary users will use the product infrequently or through an intermediary. Tertiary users are those who are affected by the system or the purchasing decision-makers. All of these individuals have an interest in the product. This does not mean that you have to conduct user requirements activities with the secondary and tertiary users, but you should at least know who they are. If the tertiary decision-makers do not purchase your product, the primary users will never use it. If the secondary system administrators cannot figure out how to customize and implement the product, the primary users will have a painful experience with it.

It is also important to realize that a single user may have several roles and sometimes these roles can have contradictory needs. For example, many online auction users are both buyers and sellers. Buyers want to pay the least they can while sellers want to get as much as they can, and a single auction site must support both these contradictory roles without harming either. Additionally, the product should behave similarly for both roles – users should not have to learn a different interaction model, navigation, terminology, etc. based on their role. Only the information presented and some of the functions available should be different.

Creating a User Profile

There are several characteristics you need to consider in order to develop a thorough user profile. We provide an ideal list of characteristics below, but you will likely not have access to all of this information. As you do further research and conduct additional user requirements activities, you will fill in these blanks, but you may never find the answers to some of the questions. Ideally, you should determine not only the typical or most frequent level for each of the characteristics, but also the range and the percentage of users who fall along that range. As a final note, some of the characteristics on page 46 are more important than others in regards to your product and situation. Prioritize the characteristics and spend the majority of resources capturing information on those key characteristics for your product. For example, if a human resources administrator enters in the wrong social security number into a financial application, you might not get paid. This is terrible, but if a medical professional enters in the wrong social security number in an electronic chart, you might get the wrong medication. This is much more serious so it is important to understand not only the tasks a user does but the consequences of a possible error. Figure 2.3 shows a sample user profile.

Once you determine the range of responses for each of the characteristics and the percentage of users along that range, you will want to categorize your users into groups based on their similarities. Some groupings you may use are:

You can use an affinity diagram to organize the characteristics into groups(see Appendix F, page 714). The groups should be significantly different from each other in order to justify them as different user types. As with many things, this is more of an art than a science and there are rarely clearly marked boundaries that put every user in one group or another. Having multiple stakeholders take part in the affinity diagram exercise can help when creating these groups and also assures stakeholder buy-in from the very beginning (refer to Chapter 1, Introduction to User Requirements, “Getting Stakeholder Buy-in for Your Activity” section page 14).

Now that you have a handle on your user population, you can develop personas, scenarios, and a recruitment screener (refer to Chapter 5, Preparing for Your User Requirements Activity, “Recruiting Participants” section, page 156). As you collect user requirements over time, you can add the information into your personas and scenarios to make them more robust and realistic. Composing a persona or scenario may even point out areas where you need more information and help you identify user requirement activities you need to conduct.

Benefits of Personas

Personas take a user profile and then fill in details to create a “typical” user. A persona is simply a fictional individual created to describe a specific user. Since you cannot possibly speak with every end user, you must create a model that can represent those end users.

There are many benefits to using personas. Because it can be difficult to feel connected to an abstract description of something, personas give your users life and help team members feel connected to them. They also get everyone on the same page by encouraging all team members to think about the same persona, instead of each individual working towards his or her own vision of who the end user is. Trying to build a product for the generic “user” is like trying to hit a moving target. Without a specific target to focus on, “the user” can change from the expert to the novice to your grandmother, all in the midst of a single conversation. Designing for a small set of personas will assure greater success of hitting that target. A persona can be used in meetings as a discussion tool (e.g., “Mary would never use that feature”), in cognitive walkthroughs, storyboarding, role-playing, and other usability activities. Finally, personas can also help new team members quickly learn who the end user is. You should create at least one persona per user type (e.g., one for the travel agent, one for the travel customer).

Things To Be Aware of When Creating Personas

You may want to develop multiple personas for each user type. This will help you cover the range of characteristics for each user type. For example, if one of your user types is a “novice travel agent,” you may want to create multiple “novice” personas: one at a small company, one at a large company, one who received formal training, one who was self-taught, etc. By limiting your vision to just one persona, you may end up filtering out valuable data from end users who do not match that one profile. For example, if we did not create a persona for the self-taught travel agent, team members might assume all travel agents receive formal training and make all their design decisions based on that fact. Having multiple personas for each user type will prevent people from building the product around a single user and help develop a product that works for all of your users. However, you should keep the set of personas manageable. It is a balancing act. If you have too many personas to represent one user type, they will simply blur together in everyone’s mind and diminish their benefits. You want your personas to be memorable. Three primary personas is a common recommendation.

You must also make sure that the personas you devise are specific to the product or feature you are developing. As we mentioned above, not all users use all parts of a product or system; therefore, it is unrealistic to assume that the same persona will work for all parts of your product. In the second case study at the end of the chapter, the authors warn about the dangers of persona reuse.

As a final note, we want to stress that personas should never replace conducting usability activities with your end users. Personas should be based on the data from usability activities. In most cases you cannot have users present with you all the time, so personas act as placeholders in cases where you cannot speak with your end users. They allow the users’ voice to be heard even when they are not physically present. Of course you should still conduct user requirements activities wherever possible.

Creating a Persona

There are several components to a persona. You can add as much detail to each of these areas as you have, but you may not be able to fill in all areas at first. The details will come from the information in your user profile. Just as developing a user profile is an iterative process, so is persona development. As you conduct user requirements activities, you should take what you learn to validate and beef up your personas. When creating a persona, it should be fictional but describe attributes from real users. Provide details and maintain authenticity. The list below is an idealized list – you may not have all the information below. Fill in what you can.

Just as there are several types of user profiles, there are several types of personas: primary, secondary, tertiary, and the anti-users (or negative users). The primary, secondary, and tertiary users have been described. An anti-user is one who would not buy or use your product in any way. You want to keep this persona in mind to warn you when you are getting off track. For example, if you are designing a product for an expert user but find more and more instruction text, tutorials, and help dialogs creeping into the product, you should check your anti-user persona (a novice user in need of a “walk up and use” product) to see whether this product would now work for him/her. If so, you are on the wrong track. You want to be sure that you are designing for the primary user while considering the secondary and tertiary users. You should be aware of the anti-users but do not design for them. Figure 2.4 shows a persona for a travel agent.

Benefits of a Scenario

Scenarios are another way to bring your users to life during product development. They can be used to test a system during early evaluation. Is this a system that meets your users’ needs? Does it satisfy the goals and fit in the user’s workflow? You can also use scenarios to create “day-in-the-life” videos. These are useful artifacts for focus groups (refer to Chapter 12 page 514).

Things To Be Aware of When Creating Scenarios

Scenario development can be time-consuming. It is not necessary to create a library of scenarios that cover every conceivable task or situation the end users might encounter. Focus on developing scenarios for the primary tasks users will encounter, and then, if there is time, move to secondary tasks. Never let user profiles, personas, or scenarios replace usability activities with actual users. You need data from real users to build your product and to keep your profiles, personas, and scenarios fresh. People change over time. Their needs, expectations, desires, and skills are not permanent – so your scenarios shouldn’t be either.

Creating Scenarios

Scenarios normally include descriptions about:

You may also want to include exceptions. What are some of the rare events that happen? (Remember, frequency does not equate to importance!) By understanding the extreme or infrequent situations users encounter, you may identify situations where your product would be obsolete or even problematic. You could also identify key features that would benefit your end users.

Using the list of tasks in the user profile and/or persona, choose the critical tasks and begin creating scenarios with your stakeholders. In one scenario, describe the ideal way the persona might complete a given task. In another scenario, describe a problem (or problems) the persona might encounter while completing this task and how the persona would react. Continue building a set of scenarios for each of your personas until you feel you have covered the functionality of your product and the tasks/situations users encounter. As with user profiles and personas, you should use the information from user requirements activities to validate your scenarios and add more information to them.

Scenarios should not describe individual widgets. For example, you should avoid things like “… and then Mary selected her preferred hotel from the droplist” or “Mary scrolled to the bottom of the page and clicked the ‘Submit’ button.” Instead, you should describe the basic actions, like “Mary selected her preferred airline” or “Mary submitted the information.” Below is an example of a very simple scenario:

Scenarios can be more sophisticated depending on the information you have collected. Often they will start out small – based on the information you were able to collect initially – and then expand to give more detail as you gather data from user requirements activities.

One scenario proponent suggests there are five types of scenario components (McInerney 2003). To help you cover a large portion of the users, tasks, functionality, and situations, make sure that your set of scenarios covers these five components. All of the scenario elements we listed above (e.g., the individual, task, desired outcome, envisioned features) will still be included in the scenarios, but also make sure that your set of scenarios includes the five components or topics below:

Each individual scenario does not have to include all five components, but the set of scenarios for a persona should. In other words, do not select one type of topic and ignore the other four. One partial set of scenarios might be as set out in Figure 2.5 – this would result in six separate scenarios. When you are first creating your user profile you may be able to develop only a simple scenario, but with further research it should expand into more detailed scenarios.

To make scenarios more consistent between each other and more complete, a template is recommended for each scenario. One possible template is provided below (McInerney 2003). The sections include:

Figure 2.6 is a detailed scenario using the template described above.

 Integrate the design with the newly redesigned business school website

 Make information and resources easier to find

 Achieve a design that reflects the best of practice in online library services.

 Identify the competitors

 Inventory and rate each competitor

 Derive design recommendations

 Expand the competitor list

 Revisit competitors throughout the design process.

Identifying Competitors

Kresge staff provided an initial list of five competitors that exhibited characteristics they found desirable. We then added five more that represented the libraries of other well-known business schools. After our primary analysis we found a large online index of business school library sites around the world (www.lib.berkeley.edu/BUSI/businessLibs.html). Using this list, we opened windows onto every listed site and did a cursory review of each, saving a few standouts for deeper analysis. Few of these sites are really considered “competitors,” for the simple reason that many of these institutions work collaboratively or target different audiences. The choice of comparison sites was really to find any that had useful ideas to draw from. The ten primary sites we reviewed for our analysis were:

An additional international site we located later that was helpful was Macquarie University Library, at www.lib.mq.edu.au.

Evaluation Criteria

We defined a set of evaluation criteria that would help us establish the best of practice elements for online library services (Table 2.3).

One of our main areas of focus was the organization and presentation of each library’s electronic resources (the searchable databases). We broke down our analysis of this section into the following subsections:

Furthermore, we evaluated each site within the following areas according to a rating scale ranging from 1 to 5 (poor to excellent):

Below is a summary of observations we made of the other business libraries to illustrate the types of issues we looked at and ideas those sites generated. These observations provide a sense of conventions and design solutions used on library sites. In the requirements phase, the features that are observed are used to suggest features that may be requirements for our own design. In addition, the lack of features suggests valuable opportunities for distinguishing our own design solution. At later points in the design, specific design approaches suggest common design solutions, organizational schemes, and terminology to use. The benchmarks provide a qualitative way to assess whether proposed feature sets and designs would compare favorably against competitors.

Usability and Accessibility

The majority of the sites showed little attention to web usability and accessibility issues. For example, only six of the ten sites included alternate text for images, and none of the sites provided back-up functionality for JavaScript in case a user has their scripting turned off in their browser preferences. Other items lacking from the majority of the sites included missing meta tags (hidden meta data that provides keywords and descriptions of each page – a surprise to us that library websites would lack this type of markup), and a lack of breadcrumb links (which indicate a page’s position within the overall website).

Navigation Categories

Almost all of the sites reviewed included an audience-oriented approach to organization, clearly indicating the resources and services that are available to each of their audiences (e.g., faculty, students, alumni). The most common top-level navigation items included Services, Resources, Help, and About Us. Other common options were Catalogs, Course Information, Career Information, and a News or Featured Items section.

Categorization of Online Databases/Electronic Resources

Library websites typically contain anywhere from a couple dozen searchable databases to well over one hundred. These databases, often called “Electronic Resources,” allow people to search library catalogs, journal articles, corporate financial data, and a wide variety of other data from an extremely diverse group of information providers, each with unique searching interfaces.

The categorization schemes for the databases were quite similar for each site. All sites included the alphabetical A through Z listing of resources. The other most common approaches included organization by resource type (e.g., books versus journals) and by subject. Interestingly, fewer than half of the sites evaluated included the ability to search the databases by keyword. Only one site, MIT, allowed users to search for databases by provider.

A variety of resource types were used to categorize electronic resources. While ideally it is the decision of the library staff to determine the level of specification in their classification system, articles, journals, and newspaper articles were the most common resource type categorizations. Others of note included electronic books, statistical sources, biographical sources, dictionaries and thesauri, directories, encyclopedias, government, market research, reference sources, research guides, search engines, and working papers.

While some of the library sites in this analysis covered more than business-related areas, only subjects broadly related to business were analyzed. The most common business subjects included economics, psychology, management, law, finance, marketing, and social sciences.

Each individual entry listing for an electronic resource includes descriptive information to help users determine whether it is the resource to meet their needs. The more extensive the information, the better equipped users are to make an informed decision. All of the sites included the title, URL, a description, and access information with each entry. (Owing to the heterogeneity of these resources, many have different access privileges, serving different subsets of the user population.) Other common fields included the dates of coverage, help and research guides, the type of resource or format, the resource provider or publisher, related subjects, and the frequency of publication.

MIT

The MIT Libraries’ website received the highest ratings of all for its clear, simple, and easy-to-use approach. There are no extraneous elements to clutter the pages. The focus is on presenting information in a clear and consistent manner that allows for targeted searches as well as browsing. The site nicely categorizes the different research resources (Figure 2.8 on page 68 shows an organization of databases by type or format of the resource) and provides a variety of help options, explaining for instance how to conduct research online and all other means of getting help, through face-to-face assistance, phone, e-mail, chat, and so forth (Figure 2.9 on page 68). Furthermore, the site is organized in a manner that allows for a variety of information-seeking preferences and approaches.

Northwestern

The Northwestern, UCLA, and Penn websites all include excellent presentation of their electronic resources and excellent help instruction. Northwestern allows browsing electronic sources by type, field of study, and/or title along with plenty of explanatory text (Figure 2.10). The site also includes a well-organized help section.

UCLA

UCLA Library offers electronic resources organized by online materials, books, articles, research guides by subject area, and help with research. Furthermore, they include embedded help icons with each section that include a wealth of useful information (Figure 2.11).

Their interface of multiple drop-down menus seems to be potentially confusing, but it suggests an alternative we might not have otherwise considered that manages the complexity of a large number of options. In this spirit, competitive analysis is primarily an idea-generation exercise – we don’t necessarily conclude that the competitors have designed a “good” or “correct” interface, and our own design cannot be assumed to be well designed just because it imitates another. Competitive analysis does not remove the need for evaluating and refining our own designs, preferably through user testing and related techniques.

Penn

The Penn Library site offers a variety of search and browse options for electronic sources (Figure 2.12), including search by title, select by interest or type, browse by alphabetical listing, browse by area of discipline, search e-resources, and search the catalogs. Plus, they offer help sections for using their e-resources: an online research tutorial, research guides by subject and topic, and instruction on good starting points.

The Penn site provides abstracts for each resource that include the ability to view more information about the resource – including other names for the resource, the URL for the resource, who is allowed to use the resource, a description of the resource, and topics or subjects to which the resource is relevant (Figure 2.13).

This detailed list of specific information features is one of the most valuable aspects to a competitive analysis, as it ensures a more complete consideration of possible information we can provide for our own users. While we may not duplicate all the components of a competitor’s product, by creating an inventory of all possibilities, we can prioritize and select the ones appropriate to us.

General Site Features

Usability and Accessibility

Navigation

Electronic Resources

Note that these recommendations are derived from a variety of aspects of our evaluation. The “general site features” reflect a set of common practices observed online. The “usability” and “navigation” recommendations are grounded in strengths and weaknesses we observed on other sites. Finally, our approach of doing an inventory of other sites was particularly informative in the structure of the online databases/electronic resources, where we can list suggested fields to structure the information.

 The characters were not believable; either they were obviously designed by committee (not based on data) or, if data were used to define them, the relationship to the data was not clear. This led to a certain lack of use by some product team members, and even total objection to the method outright.

 The characters were not communicated well. Often the main communication method was a resumé-like document – blown up to poster size and posted around the hallways – that was not easily understood or remembered. The result here was that most people on the product team didn’t know much about the personas. Thus there was really no sense of shared understanding or language.

 There were no explicit uses of the personas beyond discussion in design meetings. In particular, there was typically nothing that spoke to all disciplines or was applicable during all stages of the development cycle. As a result, many people on the team, including those that understood the personas and wanted to use them, were not able to deeply utilize them in their work efforts.

 Persona projects were often grass-roots efforts with little or no high-level support (such as people resources for creating and promoting personas, budget for posters or other materials to make the personas visible, or a mandate from team leaders to use the personas). Thus, in many cases, persona use was limited to a select few and typically died out over time.

 Minus 1 – the persona is confused, annoyed, or in some way harmed by the feature

 Zero – the persona doesn’t care about the feature one way or the other

 Plus 1 – the feature provides some value to the persona

 Plus 2 – the persona loves this feature or the feature does something wonderful for the persona even if the person doesn’t realize it.

Benefits of Personas

It is clear to us that personas can create a strong focus on users and their work contexts. Though we have not tried to rigorously measure the impact of our personas, there are several indications that the method has had an impact.

The act of creating personas has helped us make our assumptions about the target audience more explicit. Once created, the personas have helped make our decision-making criteria for the product equally explicit – Why are we building this feature? Why are we building it like this? Without personas, our development teams routinely made decisions about features and implementation without recognizing or communicating their underlying assumptions about who would use the product and how it would be used. The “feature-persona weighted priority matrix” described earlier is a good example of how the personas have helped clarify our decision-making. Using that tool regularly results in “pet” or favored features (or even seemingly important features) being pushed down in the priority list. When this happens, our team members have to be very explicit with their reasoning to get a feature back in the plan.

Personas are a medium for communication; a conduit for information about users and work settings derived from ethnography, market research, usability studies, interviews, observations, and so on. Once a set of personas is familiar to a team, a new finding can be instantly communicated: “Sondra cannot use the search tool on your web page.” This communication style has an immediacy lacking from the finding “A subset of participants in the usability study had problems with the search tool” – especially for team members who now, for all intents and purposes, see Sondra (the persona) as a real person. We have found this to be extremely powerful for communicating results and furthering our teammates’ understanding of the personas.

Finally, personas focus attention on a specific target audience. The method helps establish who is and consequently who is not being designed for. Personas explicitly do not cover every conceivable user; they help focus sequentially on different kinds of users. For example, a quality assurance engineer can test a product one day focusing on Patrick scenarios and another day focusing on Abby scenarios. As stated in the previous section, this works for testers and other product team members, in “bug bashes” for example. An experienced tester reported feeling that he was identifying “the right kind” of problems in drawing on knowledge of a persona in guiding his test scripts and activities.

Risks of Personas

Getting the right persona or set of personas is a huge challenge. Cooper argues that designing for any one external person is better than trying to design vaguely for everyone or specifically for oneself. This may be true, and it does feel as though settling on a small set of personas provides some insurance; but it also seems clear that personas should be developed for a particular effort. In making choices it becomes clear that the choices have consequences. For example, they will guide participant selection for future studies and could be used to filter out data from sources not matching one of the persona profiles.

Related to this is the temptation of persona reuse. After the investment in developing personas and acquainting people with them, it may be difficult to avoid over-extending their use when it would be better to disband one cast of characters and recruit another one. It can be good or bad when our partner teams adopt or adapt our personas. Different teams and products have different goals, so the personas are stretched a bit. So far, the stretching around our company has been modest and closely tied to data (because our target customers do indeed overlap), but it is a concern.

In addition, marketing and product development have different needs that require different persona attributes, and sometimes different target audiences. Marketing is generally interested in buyer behavior and customers; product development is interested in end-users. We’ve had some success in collaborating here, but there are rough edges.

Finally, we have seen a certain level of “persona mania” within our organization and others. Personas can be overused. At worst, they could replace other user-lefted methods, ongoing data collection, or product evaluation. Readers of this case study might conclude that we have gone overboard regarding our persona effort. It is important to note that the things we have described here have happened over several years. Not everything was created up front. There are periods of persona focus and periods where we do many other types of UCD-oriented work. We know that personas are not a panacea. We believe personas should augment and enhance – augment existing design processes and enhance user focus. We have found that personas can enhance user testing and other evaluation methods, field research, scenario generation, design exploration, and solution brainstorming.

Persona use does require decision-making. It isn’t a science. If not used appropriately, any powerful tool can take one down the wrong path, as in lying with statistics or using unrepresentative video examples. Personas are one such powerful tool. It is up to all of us together to develop effective and appropriate ways to use them.

Acknowledgments. Many people contributed to the personas effort described in this case study. We thank Kelly Fisher, Manish Sharma, Dave Ciuba, Kris Tibbetts, Lisa Olfert, Louise Kapustka, Leslie Scott, Scott Ottaway, Steve Scallen, Stacy Lewis, Karen Carncross, Colleen Dunham, Kent Sullivan, Shannon Kallin, Nelle Steele, Craig Hally, Jenny Lam, Greg Melander, Mark Ligameri, Heesung Koo, Dan Black, Neil Powell, and Hillel Cooperman. We also thank Jonathan Grudin for encouraging us to publish our experiences with personas as well as reflecting with us on the method.