CHAPTER 1

Defining the Objective

Before you can begin writing effective survey questions, you must understand what the survey is trying to accomplish. This can be done by identifying the objective or objectives of the survey. These objectives are broken down into possible variables and variations and further analyzed. This information is then used to write the survey questions. The objectives will be the guide to creating an effective survey and effective questions. You should be able to trace each question in the survey back to a single objective.

Setting Objectives

The objective of the survey may be taken directly from project documentation, such as the business case or problem definition. But if the survey is one of many elicitation activities, you may have to put more effort into determining the objectives for the survey in particular. When setting objectives for the survey, ask:

What is the overall goal of the survey?

What question(s) must be answered?

What will be done with the data collected?

What decisions could be affected based on the information collected?

How will data be reported, and who will see the results?

How can we determine whether the right information has been collected?

Who will act on the data?

Start with one or more general objectives, and refine them as you go along. For example, if the general objective is to evaluate the new system that product specialists have recently begun using, related specific objectives might include:

Understanding how the system is being used

Determining user satisfaction with the new system

Identifying changes that have resulted from implementing the new system.

Each concept or idea should be stated as a single objective. If the word “and” is used when stating an objective, it probably consists of two objectives. The more focused and clear the objectives are, the more likely the survey will gather the information needed.

Determining the Variables and Variations

Next, for each single objective, identify the specific information that must be collected. Determine the variables for each objective. Let’s say we are looking at satisfaction with the new system. For each variable identified, such as overall satisfaction, dig deeper. Can overall satisfaction be assessed, and then broken down into specific components, such as satisfaction with available data, system response time, and the usefulness of data? Ask what other variables might help explain why some users are satisfied and others are dissatisfied with the system.

The challenge is to think of all possible variables that may help collect the correct information. For example, user satisfaction may be affected by the training users received. Was it enough training? Was the training in the right format? What do the users think about their jobs? How does the system help them do their jobs? Does the new system make tasks easier than they used to be? Each variable and its components or variations will be used to develop questions.

Some variations may require further analysis and breaking down. For example, if the survey is looking at how users view their jobs now that the new system has been implemented, for the variation “view their jobs,” consider asking questions that address:

How much users valued the interaction they had with other departments before the system was implemented.

How concerned users are about maintaining existing relationships with other departments. In other words, if the system will conduct transactions that previously were carried out through conversations among departments, this might result in less interaction among the departments.

Whether they feel the system has increased the professionalism of their jobs. If the users were previously in a paper-based manual environment (outdated) and now are operating in an environment with technology, do they feel more professional?

This analysis and breakdown can lead to more questions. For example, you might ask whether the new features being proposed for the new system would enhance or constrain interdepartmental interaction and relationships and professionalism.

Mind mapping tools (Figure 1-1) are a great way to quickly trace your objective to its variables and variations.

Once all variables and variations for each objective have been identified, determine which ones will be used to write questions. Depending on the length of the survey, all or only some variables and variations may be used. This process of breaking down objectives into smaller variables and variations on which all questions are based allows each question to be traced back to a single objective.

Once all objectives have been finalized, look at each objective and identify the target audience for questions about that objective. You may ask all the participants about all the objectives or present subsets of questions to different segments of participants.

FIGURE 1-1: Example Mind Map

Depending on the type and size of the initiative, additional preparation may be warranted before proceeding with the survey. This additional work could include identifying participants, interviewing or having informal discussions with participants, analyzing relevant literature (in trade magazines, for example), or researching similar product features on the Internet.