Overview of Choice Designs
Discrete choice experiments support the process of designing a product. They help prioritize product features for a company’s market so that the company can design a product that people want to buy. The Choice Design platform creates experiments using factors that are attributes of a product. Selecting the attributes to be studied and their values is of critical importance. You must include all attributes that are likely to influence a consumer’s decision to buy the product. For more information and guidelines for designing a choice experiment, see Sall (2008).
Choice Design Terminology
The following terminology is associated with choice designs:
• An attribute is a feature of a product.
• A profile is a specification of product attributes.
• A choice set is a collection of profiles.
• A survey is a collection of choice sets.
• A partial profile is a profile in a choice design where only a specified number of attributes are varied within each choice set. The remaining attributes are not varied.
In a discrete choice experiment, respondents are presented with a survey containing several choice sets. Choice sets usually contain only a small number of profiles to facilitate the decision process. Within each choice set, each respondent specifies which of the profiles he or she prefers. For example, attributes for a laptop experiment might include speed, storage, screen size, battery life, and price. Different combinations of these attributes comprise product profiles. A choice set might consist of two profiles. From each choice set, a respondent chooses the profile that he or she prefers.
In cases where many attributes are involved, you can construct surveys where each choice set contains partial profiles. In a choice set with partial profiles, only a specified number of attributes are varied and the remaining attributes are held constant. This reduces the complexity of the choice task.
Bayesian D-Optimality
Because discrete choice models are nonlinear in their parameters, the efficiency of a choice design depends on the unknown parameters. The Choice Design platform uses a Bayesian approach, optimizing the design over a prior distribution of likely parameter values that you specify. The Bayesian D-optimality criterion is the expected logarithm of the determinant of the information matrix, taken with respect to the prior distribution. The Choice Design platform maximizes this expectation with respect to the prior probability distribution. For details, see “Bayesian D-Optimality and Design Construction” and Kessels et al. (2011).
You can also generate the following types of designs:
• Utility-neutral designs - In a utility-neutral design, all choices within a choice set are equally probable. The prior mean is set to 0.
• Local D-optimal designs - A local D-optimal design takes into account the prior on the mean, but does not include any information from a prior covariance matrix.
For more information about utility neutral and local D-optimal designs, see “Utility-Neutral and Local D-Optimal Designs”.
Example of a Choice Design
About the Experiment
In this example, a coffee shop is interested in making an ideal cup of coffee to satisfy the majority of its customers. The manager has asked you to determine which factors affect customer preferences. Specifically, you need to determine which settings of the following factors (attributes) result in an ideal cup of coffee:
• Grind size (medium or coarse)
• Temperature (195°, 200°, 205°)
• Brewing time (3 minutes, 3.5 minutes, or 4 minutes)
• Charge (1.6 grams/ounce, 2 grams/ounce, or 2.4 grams/ounce)
Each combination of factor levels is a profile. Trying to obtain information about preferences by having every respondent sample every possible profile is not practical. However, you can ask a respondent to select a preferred profile from a choice set consisting of a small number of profiles.
In this example, you design an experiment where each respondent indicates his or her preference in several choice sets. Your design will have the following structure:
• ten respondents
• twelve choice sets
• two profiles per choice set
• one survey per respondent containing all 12 choice sets
The experiment results in 12 responses per respondent. Analysis of these preferences can be used to draw conclusions about how to make a cup of coffee that pleases most customers.
Create the Design
You can enter factors either manually or automatically using a preexisting table that contains the factors and settings. In this example, for convenience, you use a preexisting table. But, if you are designing a new experiment, you must first enter the factors manually. For details on entering factors manually, see “Attributes”.
1. Select DOE > Consumer Studies > Choice Design.
2. Select Help > Sample Data Library and open Design Experiment/Coffee Choice Factors.jmp.
3. Click the Choice Design red triangle and select Load Factors.
Figure 18.2 Choice Design Window with Attributes Defined
4. Click Continue.
5. Open the DOE Model Controls outline.
In this example, you are only interested in a model that contains the main effects of your four factors. However, if you wanted your design to be capable of estimating additional effects, you would add them in this outline.
6. In the Design Generation panel:
‒ Keep the Number of attributes that can change within a choice set at 4.
‒ Keep the Number of profiles per choice set at 2.
‒ Type 12 for the Number of choice sets per survey.
In this example, the respondents evaluate 12 choice sets.
‒ Keep the Number of surveys at 1.
‒ Type 10 for the Expected number of respondents per survey.
In this example, there are ten respondents.
Figure 18.3 Completed Design Generation Panel
Note: Setting the Random Seed in step 7 reproduces the exact results shown in this example. In constructing a design on your own, this step is not necessary.
7. (Optional) Click the Choice Design red triangle and select Set Random Seed. Type 12345 and click OK.
8. Click Make Design.
There are 12 choice sets, each consisting of two coffee profiles.
9. Select Output separate tables for profiles and responses.
This places the descriptions of the choice sets in a data table called Choice Profiles. A second data table (called Choice Runs) is constructed to facilitate entry of the response information.
10. Click Make Table.
The Choice Profiles table shows the 12 choice sets, each consisting of two profiles. The Choice Runs table enables you to record results in the column Response Indicator. Enter 1 for the preferred profile and 0 for the other profile. Alternatively, if the respondent has no preference, enter 0 for both profiles or leave both missing.
The Choice script in the Choice Profiles table facilitates analysis of experimental results. It opens a completed launch window for a Choice Model. For information about Choice Models, see the Choice Models chapter in the Consumer Research book.
Example of a Choice Design with Analysis
In this example, a computer manufacturer is interested in manufacturing a new laptop and wants information about customer preferences before beginning an expensive development process. The manufacturer decides to construct a design consisting of two sets of profiles that will be administered to ten respondents. The goal of the choice design is to understand how potential laptop purchasers view the advantages of a collection of four attributes:
• size of hard drive disk (40 GB or 80 GB)
• speed of processor (1.5 GHz or 2.0 GHz)
• battery life (4 Hrs or 6 Hrs)
• cost of computer ($1000, $1200 or $1500)
To construct the design for the ten respondents, you first conduct a small pilot study using only one respondent. Then you analyze the results and use the parameter estimates as prior information in designing the final study for the ten respondents. Here are the steps:
Create a Choice Design for a Pilot Study
In this section, you construct a choice design for a one-respondent study.
Define Factors and Levels
In this example, you load the factors from an existing table. When designing a new experiment on your own, enter the factors manually.
1. Select DOE > Consumer Studies > Choice Design.
2. Select Help > Sample Data Library and open Design Experiment/Laptop Factors.jmp.
3. Click the Choice Design red triangle and select Load Factors.
Figure 18.4 Choice Design Window with Attributes Defined
Create the Design
1. Click Continue.
This pilot survey will be given to a single respondent. The default values in the DOE Model Controls, Prior Specification, and Design Generation panels are appropriate as is.
2. Click Make Design.
Note: Because choice designs are constructed using a random starting design, your design will likely differ from the one in Figure 18.5.
Figure 18.5 Pilot Design
The single survey contains eight choice sets, each consisting of two laptop profiles.
3. Verify that the Combine profiles and responses in one table option is selected.
This places the choice sets and the survey results in the same table.
4. Click Make Table.
This survey was designed assuming no prior information. For this reason, some choice sets might not elicit useful information. The plan is to obtain survey results from the single respondent, analyze the results, and then use the results from the pilot survey as prior information in designing the final survey.
Analyze the Pilot Study Data
Now that the pilot survey design is complete, it is administered to single respondent. The respondent chooses one profile from each set, entering 1 for the chosen profile and 0 for the rejected profile. You will analyze the results using the Choice platform.
Note: For details about the Choice platform, see the Choice Models chapter the Consumer Research book.
1. Select Help > Sample Data Library and open Design Experiment/Laptop Design.jmp.
2. Run the Choice script.
Figure 18.6 Choice Model Launch Window
The only grouping variable is Choice Set because there is a single survey and a single respondent.
3. Click Run Model.
Figure 18.7 Parameter Estimates for Pilot Survey
To construct the final choice design that you will give to ten respondents, you need prior means and variances for the parameter estimates. The analysis in Figure 18.7 gives estimates of the parameter means (Estimate) and estimates of their standard errors (Std Error). You will treat the standard errors as prior estimates of the standard deviations. Next, to calculate estimates of the variances of the attributes, construct a JMP table and square the standard errors.
4. Right-click in the Parameter Estimates report and select Make into Data Table.
5. In the new data table, right-click in the Std Error column header and select New Formula Column > Transform > Square.
A column called Std Error^2 is added to the data table. Its values will serve as your estimates of the prior variance for the choice model parameters.
Figure 18.8 Untitled Data Table with Variance Estimates in Last Column
Note: Do not close the Untitled data table at this point.
Design the Final Choice Experiment Using Prior Information
In this section, you use the prior information obtained from the pilot laptop study to construct a final design. The final design will be administered to a set of ten participants.
1. Select Help > Sample Data Library and open Design Experiment/Laptop Factors.jmp.
2. Select DOE > Consumer Studies > Choice Design.
3. Click the Choice Design red triangle and select Load Factors.
4. Click Continue.
5. From the Untitled table, enter the values in the Estimate column into the Prior Mean outline in the Choice Design window, as shown in Figure 18.9.
You can copy and paste the entire column from the Untitled table, then click in the Disk Size text box under Prior Mean in the Prior Mean outline, right-click, and select Paste.
6. From the Untitled table, enter the values in the Std Error^2 column into the diagonal entries in the Prior Variance outline in the Choice Design window, as shown in Figure 18.9. Enter these one-by-one, rounded to three decimal places.
Figure 18.9 Prior Mean and Variance Information from Pilot Study
7. In the Design Generation panel, enter 2 for the Number of surveys and five for the Expected number of respondents per survey.
This gives instruments for a total of 10 respondents and allows for two different sets of profiles.
8. Click Make Design.
9. Click Make Table.
The design table has 160 rows. There are 16 rows for each of the ten study respondents. Each respondent has 8 choice sets, each with 2 profiles. There are two surveys, each given to 5 respondents. The 160 rows result from the following calculation: 2 profiles * 8 choice sets * 2 surveys * 5 respondents = 160 rows.
The final design is now ready to be administered to the 10 respondents.
Run the Design and Analyze the Results
In this section, you analyze the results obtained when you obtain results from the final design. In particular, you want to know how changing the price or other characteristics of a laptop affects its desirability as perceived by potential buyers. This desirability is called the utility value of the laptop attributes.
Determine Significant Attributes
1. Select Help > Sample Data Library and open Design Experiment/Laptop Results.jmp.
2. Run the Choice script.
Figure 18.10 Choice Model Launch Window
There are three grouping variables, Respondent, Survey, and Choice Set, because there are multiple surveys and respondents.
3. Click Run Model.
Figure 18.11 Initial Analysis of the Final Laptop Design
The Effect Summary and Likelihood Ratio Tests outlines indicate that Disk Size, Speed, and Price are significant at the 0.05 level, and that Battery Life is marginally significant.
Find Unit Cost and Trade Off Costs
Next, use the Profiler to see the utility value and how it changes as the laptop attributes change.
1. Click the Choice Model red triangle and select Utility Profiler.
Figure 18.12 Utility Profiler at Price = $1000
When each attribute value is set to its lowest value, the Utility value is –0.3406. The first thing that you want to do is determine the unit utility cost.
2. Move the slider for Price to $1,500.
Figure 18.13 Utility Profiler at Price = $1500
When Price changes from $1,000 to $1,500, the Utility changes from –0.3406 to –2.3303. That is, raising the price of a laptop by $500.00 lowers the utility (or desirability) approximately 2 units. Therefore, you can estimate the unit utility cost to be approximately $250.00.
With this unit utility cost estimate, you can now vary the other attributes, note the change in utility, and find an approximate monetary value associated with attribute changes. For example, the most significant attribute is Speed (see Figure 18.11).
3. In the Utility Profiler, set Price back to $1,000, its lowest value, and change Speed to 2.0 GHz, its higher value.
Figure 18.14 Utility Value of Higher Speed
The Utility value changes from the original value shown in Figure 18.12 of –0.3406 to 0.9886, for a total change of 1.3292 units. Using the utility cost estimate or $250.00, the increase in price for a 2.0 GHz laptop over a 1.5 GHz laptop can be computed to be 1.3292*$250.00 = $332.30. This is the dollar value that the Choice study indicates that the manufacturer can use as a basis for pricing this laptop attribute. You can make similar calculations for the other attributes.
Choice Design Window
The Choice Design window walks you through the steps to construct a choice design for modeling attribute preferences. You can specify an assumed model and prior information.
The Choice Design window updates as you work through the design steps. The outlines, separated by buttons that update the outlines, follow the flow in Figure 18.15.
Figure 18.15 Choice Design Flow
The following sections describe the steps in creating a choice design:
• “Model”
• “Design”
Attributes
Attributes in a choice design can only be categorical.
Tip: When you have completed the Attributes outline, consider selecting Save Factors from the red triangle menu. This saves the attribute names, roles, and levels in a data table that you can later reload and reuse.
Figure 18.16 Attributes Outline
The Attributes outline contains the following buttons:
Add Factor
Adds an attribute with the selected number of levels.
Remove
Removes the selected attributes.
Add N Factors
Adds multiple attributes. Enter the number of attributes to add, click Add Factor, and then select the number of levels. Repeat Add N Factors to add multiple attributes with different numbers of levels.
The Attributes outline contains the following columns:
Name
Name of the attribute. Attributes are given default names of X1, X2, and so on. To change a name, double-click it and type the desired name.
Role
Specifies the Design Role of the attribute as Categorical.
Attribute Levels
The attribute name or description. To insert Attribute Levels, click on the default levels and type the desired names.
Editing the Attributes Outline
• To edit the Name of an attribute, double-click the attribute name.
• To edit an Attribute Level, click the level.
Attribute Column Properties
For each attribute, JMP saves the Value Ordering column property to the data tables constructed by the Choice platform. The Value Ordering column property specifies that levels appear in reports using the ordering specified in the Attributes outline. For details, see “Value Ordering” in the “Column Properties” appendix.
Model
The model outline consists of two parts:
• You can specify your assumed model, which contains all the effects that you want to estimate. See “DOE Model Controls”.
• You can specify prior knowledge about the attribute levels, which can result in a better design. See “Prior Specification”.
DOE Model Controls
Specify your assumed model in the DOE Model Controls outline. All main effects are included by default. Click the Interactions button to add all two-way interactions.
Figure 18.17 DOE Model Controls Outline
When you construct your design table, JMP saves a Choice script to the data table. The Choice script contains the main effects shown in the DOE Model Controls outline.
The DOE Model Controls outline contains the following buttons:
Main Effects
Adds main effects for all attributes in the model.
Interactions
Adds all second-order interactions. If you do not want to include all of the interactions, select the interactions that you want to remove and click Remove Term.
Remove Term
Removes selected effects.
Prior Specification
Enter specifications for a multivariate normal prior distribution on the model parameters. Enter the prior distribution’s mean in the Prior Mean outline and its covariance matrix in the Prior Variance outline.
Figure 18.18 Prior Specification Outline
You can ignore the prior specifications using these options:
Ignore prior specifications. Generate the Utility Neutral design.
Sets the prior means to 0 and generates a locally D-optimal design. This design is called a utility-neutral design. For more information, see Huber and Zwerina (1996).
Ignore prior specifications. Generate the local design for the prior mean.
Generates a locally D-optimal design. The local design takes into account the prior on the mean but ignores the covariance matrix. For more information, see Huber and Zwerina (1996).
Design Generation
Enter specifications that define the structure for your design in the Design Generation panel.
Figure 18.19 Design Generation Panel for Coffee Example
Note: Figure 18.19 is taken from the coffee example. See “Example of a Choice Design”.
Enter a number for each of the following items:
Number of attributes that can change within a choice set
Enter a number less than or equal to the total number of attributes. This is often set to the total number of attributes. However, if you are comparing many attributes and want to simplify the selection process for respondents, set this to a number that is smaller than the total number of attributes.
Number of profiles per choice set
Enter the number of profiles that a respondent must choose from in stating a preference.
Number of choice sets per survey
Enter the number of preferences that you want to obtain from each respondent.
Number of surveys
Enter the number of distinct collections of choice sets. This is useful if you want to administer surveys to multiple respondents.
Expected number of respondents per survey
Enter the total number of respondents divided by the number of surveys.
Make Design
Once you have completed the Design Generation outline, click Make Design to generate the design. The design appears in the Design outline.
Design
The Design outline shows the runs for a design that is optimal, given your selections. Review the design to ensure that it meets your needs.
Figure 18.20 Design Outline for Coffee Example
Note: Figure 18.20 is taken from the coffee example. See“Example of a Choice Design”.
Note: The algorithm for finding an optimal design is based on a random starting design. Because of this, the design you obtain is not unique. The design algorithm will generate different designs when you click the Back and Make Design buttons repeatedly.
Output Options
Select one of the following output options:
Output separate tables for profiles and responses
Displays two data tables:
‒ The Choice Profiles table lists the profiles in each row, identified by Survey and Choice Set columns. Within a choice set, the profiles are identified by Choice ID. This table is useful for constructing the survey instruments.
‒ The Choice Runs table provides an empty Response column where you can enter respondent preferences. Each row corresponds to a single choice set. The rows are sorted by Respondent, Survey, and Choice Set. The choice set IDs are given in the next columns, followed by the Response column. Enter the choice set ID for the respondent’s preference in the Response column.
Combine profiles and responses in one table
Provides a single Choice Profiles table with an empty Response Indicator column where you can enter respondent preferences. Each row corresponds to a single profile. The table is sorted by Respondent, Survey, and Choice Set. Enter the value 1 (or another nonzero numerical indicator) for the respondent’s preferred profile and a 0 indicator for the other profiles in that choice set.
Note: The values you enter in the Response Indicator column must be numerical.
Make Table
Click Make Table to construct the table or tables that you selected in “Output Options”. In the table panel of the Choice Profiles table, there is a Choice script. Run the script and then click Run Model to analyze your experimental results.
Caution: Only main effects are added by the Choice script. Add interactions manually.
For information about the Choice Model report, see the Choice Models chapter in the Consumer Research book.
Choice Design Options
Save Factors
Saves the information in the Factors panel to a new data table. Each factor’s column contains its levels. Other information is stored as column properties. You can then quickly load the factors and their associated information into most DOE windows.
Note: It is possible to create a factors table by entering data into an empty table, but remember to assign each column an appropriate Design Role. Do this by right-clicking on the column name in the data grid and selecting Column Properties > Design Role. In the Design Role area, select the appropriate role.
Load Factors
Loads factors that you have saved using the Save Factors option.
Set Random Seed
Sets the random seed that JMP uses to control certain actions that have a random component. These actions include:
‒ simulating responses using the Simulate Responses option
‒ randomizing Run Order for design construction
‒ selecting a starting design for designs based on random starts
To reproduce a design or simulated responses, enter the random seed that generated them. For designs using random starts, set the seed before clicking Make Design. To control simulated responses or run order, set the seed before clicking Make Table.
Note: The random seed associated with a design is included in the DOE Dialog script that is saved to the design data table.
Number of Starts
Enables you to specify the number of random starts used in constructing the design. See “Bayesian D-Optimality and Design Construction”.
Advanced Options
None available.
Save Script to Script Window
Creates the script for the design that you specified in the Choice Design window and saves it in an open script window.
Technical Details
Bayesian D-Optimality and Design Construction
The Bayesian D-optimality criterion is the expected logarithm of the determinant of the information matrix of the maximum likelihood of the parameter estimators in the multinomial logit model, taken with respect to the prior distribution. The Choice Design platform maximizes this expectation with respect to a sample of parameter vectors that represents the prior probability distribution. For details, see Kessels et al. (2011).
For partial profile designs, JMP uses a two-stage design algorithm:
1. The constant attributes in each choice set are determined using an attribute balance approach.
2. The levels of the non-constant attributes are determined using Bayesian D-optimality.
Attribute balance means that the algorithm attempts to balance the number of times each attribute is held constant in the entire design. If two or more attributes are held constant, the algorithm attempts to balance the occurrence of pairs of attributes held constant in the design.
The levels of the non-constant attributes are determined to optimize the Bayesian D-optimal criterion. A random starting design is found. Then levels of the non-constant attributes are generated using a coordinate-exchange algorithm and evaluated until the Bayesian D-optimality criterion is optimized. The calculations, which involve integration with respect to a multivariate normal prior, use the quadrature method described in Gotwalt et al. (2009).
Note: The Bayesian D-optimality criterion can result in choice sets where some non-constant attributes have identical levels. This situation occurs when varying the non-constant levels within a profile would result in uninformative choice sets where all profiles have very high or very low probabilities.
Utility-Neutral and Local D-Optimal Designs
You can use the Choice Design platform to generate a utility-neutral design by setting prior means to 0. In a utility-neutral design, all choices within a choice set are equally probable. For more information, see Huber and Zwerina (1996).
You can also generate a local D-optimal design. The local design takes into account the prior of the mean, but does not include any information from a prior covariance matrix. For more information, see Huber and Zwerina (1996).
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.