4
Utility in action

The von Neumann and Morgenstern (NM) theory is an axiomatization of the expected utility principle, but its logic also provides the basis for measuring an individual decision maker’s utilities for outcomes. In this chapter, we discuss how NM utility theory is typically applied to utility elicitation in practical decision problems. In Section 4.1 we discuss a general utility elicitation approach, and then review basic applications in economics and medicine.

In Section 4.2, we apply NM utility theory to situations where the set of rewards consists of alternative sums of money. An important question in this scenario is how attitudes towards future uncertainties change as current wealth changes. The key concept in this regard is risk aversion. We review some intuitive results which describe risk aversion mathematically and characterize properties of utility functions for money. Much of our discussion is based on Keeney et al. (1976) and Kreps (1988).

In Section 4.3, we discuss how the lottery approach can be used to elicit patients’ preferences when the outcomes are related to future health. The main issue we will consider is the trade-off between length and quality of life. We also introduce the concept of a quality-adjusted life year (QALY), which is defined as the period of time in good health that a patient says is equivalent to a year in ill health, and commonly used in medical decision-making applications. We discuss its relationship with utility theory, and illustrate both with a simple example. A seminal application of this methodology in medicine is McNeil et al. (1981).

Featured articles:

Pratt, J. (1964). Risk aversion in the small and in the large, Econometrica 32: 122–136.

Torrance, G., Thomas, W. and Sackett, D. (1972). A utility maximization model for evaluation of health care programs, Health Services Research 7: 118–133.

Useful general readings are Kreps (1988) for the economics; Pliskin et al. (1980) and Chapman and Sonnenberg (2000) for the medicine.

4.1 The “standard gamble”

When we presented the proof of the NM representation theorem we encountered one step where the utility of each outcome was constructed by identifying the value u that would make receiving that outcome for sure indifferent to a lottery giving the best outcome with probability u and the worse with probability 1 – u. This step is also the essence of a widely used approach to utility elicitation called “standard gamble.”

As a refresher, recall that in the NM theory Z is the set of rewards. The set of actions (or lotteries, or gambles) is the set probability distributions on Z. It is called A and its typical elements are things like a, a′, a″. In particular, χ_z denotes the degenerate lottery with mass 1 at reward z. We also define u : Z → ℜ as the decision maker’s utility function. Given a utility function u, the expected utility of lottery a is

Preferences are described by the binary relation ≻, satisfying the NM axioms, so that a ≻ a′ if and only if

When using the standard gamble approach to utility elicitation, the decision maker lists all outcomes that can occur and ranks them in order of preference. If we avoid the boring case in which all outcomes are valued equally by the decision maker, the weak ordering assumption allows us to identify a worst outcome z₀ and a best outcome z⁰. For example, in assessing the utility of health states, “death” is often chosen as the worse outcome and “full health” as the best, although in some problems there are health outcomes that could be ranked worse than death (Torrance 1986). Worst and best outcomes need not be unique. Because all utility functions that are positive linear transformations of the same utility function lead to the same preferences, we can arbitrarily set u(z₀) = 0 and u(z⁰) = 1, which results in a convenient and interpretable utility scale.

The decision maker’s utility for each intermediate outcome z can be inferred by eliciting the value of u such that he or she is indifferent between two actions:

a :	outcome z for certain;
a′ :	outcome z0 with probability 1 – u, or
	outcome z0 with probability u.

Another way of writing action a′ is uχ_z⁰ + (1 – u)χ_z0. The existence of a value of u reaching indifference is implied by the Archimedean and independence properties of the decision maker’s preferences, as we have seen from Lemma 1. As you can check, the expected utility of both actions is u, and therefore u(z) = u.

4.2 Utility of money

4.2.1 Certainty equivalents

After a long digression we are ready to go back to the notion, introduced by Bernoulli, that decisions about lotteries should be made by considering the moral expectation, that is the expectation of the value of money to the decision maker. We are now ready to explore in more detail how utility functions for money typically look.

Say you are about to ship home a valuable rug you just bought in Samarkand for $9000. The probability that it will be lost during transport is 3% according to your intelligence in Uzbekistan. At which price would you be indifferent between buying the insurance or taking the risk? This price defines your certainty equivalent of the lottery defined by shipping without insurance.

Formally, assume that the outcome space Z is an open interval, that is Z = (z₀, z⁰) ⊆ ℜ. Then, using the same notation as in Chapter 3, a certainty equivalent is any reward z ∊ ℜ that makes you indifferent between that reward for sure, or choosing action a.

Definition 4.1 (Certainty equivalent) A certainty equivalent of lottery a is any amount z* such that

or equivalently,

A certainty equivalent is also referred to as “cash equivalent” and “selling price” (or “asking price”) of a lottery.

4.2.2 Risk aversion

Meanwhile, in Samarkand, you calculate the expected monetary loss from the uninsured shipment, that is $9000 times 0.03 or $270. Would you be willing to pay more to buy insurance? If you do, you qualify as a risk-averse individual.

If you define

as the expected reward under lottery a, then:

Analogously, strict risk seeking and risk neutrality can be defined in terms of convexity and linearity of u, respectively, as illustrated in Figure 4.1.

Certainty equivalents exist and are unique under relatively mild conditions:

Figure 4.1 Utility functions corresponding to different risk behaviors.

Proposition 4.2

1. If u is strictly increasing, then z* is unique.

2. If u is continuous, then there is at least one certainty equivalent.

3. If u is concave, then there is at least one certainty equivalent.

For the remainder of this chapter, we will assume that u is a concave strictly increasing utility function, so that the certainty equivalent is

Back in Samarkand you have done some thinking and concluded that you would be willing to pay up to $350 to insure the rug. The extra $80 amount you are willing to pay, on top of the expected value of $270, is called the insurance premium. If insurers did not ask for this extra amount they would need a day job. The risk premium is the flip side of the insurance premium:

Definition 4.3 (Risk premium) The risk premium associated with a lottery a is the difference between the lottery’s expected value and its certainty equivalent:

Figure 4.2 shows this. The negative of the risk premium, or –RP(a), is the insurance premium. This works out in our example too. You have bought the rug already, so we are talking about negative sums of money at this point.

Figure 4.2 Risk premium.

4.2.3 A measure of risk aversion

We discuss next how the decision maker’s behavior changes when his or her initial wealth changes proportionally. Let ω be the decision maker’s wealth prior to the gamble and let a + ω be a shorthand notation to represent the final wealth position. Then the decision maker seeks to maximize

For example, suppose that the choices are: (i) gamble represented by a, or (ii) sure payment in the amount z. Then, we can represent his or her final wealth as a + ω or z + ω. The choice will depend on whether U(a + ω) is greater, equal, or lower than u(z + ω).

Definition 4.4 (Decreasing risk aversion) A utility function u is said to be decreasingly absolute risk averse if for all a ∊ A, z ∊, ω, ω′ ∊ ℜ, such that a + ω, a + ω′, a + z, and ω′ + z all lie in Z and ω′ > ω,

The idea behind this definition is that, for instance, if you are willing to take the lottery over the sure amount when your wealth is 10 000 dollars, you would also take the lottery if you had a larger wealth. The richer you are, the less risk averse you become.

Another way to define decreasing risk aversion is to check whether the risk premium goes down with wealth:

Proposition 4.3 A utility function u is decreasingly risk averse if and only if for all ain A, the function ω → RP(a + ω) is nonincreasing in ω.

A third way to think about decreasing risk aversion is to look at the derivatives of the utility function:

Theorem 4.1 A utility function u is decreasingly risk averse if and only if the function

is nonincreasing in z.

While we do not provide a complete proof (see Pratt 1964), the following two results do a large share of the work:

Proposition 4.4 If u₁ and u₂ are such that λ₁(z) ≥ λ₂(z) for all z in Z, then u₁(z) = f (u₂(z)) for some concave, strictly increasing function f from the range of u₂ to ℜ.

Proposition 4.5 A decision maker, with utility function u₁, is at least as risk averse as another decision maker, with utility function u₂ if and only if, for all z in Z, λ₁(z) ≥ λ₂(z).

The function λ(z), defined in Theorem 4.1, is known as the Arrow–Pratt measure of risk aversion, or local risk aversion at z. Pratt writes:

The risk-aversion function captures the information on preferences in the following sense

Theorem 4.2 Two utility functions u₁ and u₂ have the same preference ranking for any two lotteries if and only if they have the same risk-aversion function.

Proof: If u₁ and u₂ have the same preference ranking for any two lotteries, they are affine transformations of one another. That is, u₁(z) = a + bu₂(z). Therefore, and , so

Conversely, λ(z) = −(d/dz) (log u′(z)) and then

∫ –λ(z)dz = log u′(z) + c

and exp(– ∫ λ(z)dz) = e^cu′(z), which implies

Since e^c and d are constants, λ(z) specifies u(z) up to positive linear transformations.

With regard to this result, Pratt comments that:

4.3 Utility functions for medical decisions

4.3.1 Length and quality of life

Decision theory is used in medicine in two scenarios: one is the evaluation of policy decisions that affect groups of individuals, typically carried out by cost-effectiveness, cost–utility or similar analyses. The other is decision making for individuals facing complicated choices regarding their health. Though the two scenarios are very different from a decision-theoretic standpoint, in both cases the foundation is a measurement of utility for future health outcome. The reason why utility plays a key role is that simpler measures of outcome, like duration of life (or in medical jargon survival), fail to capture critical trade-offs. In this regard, McNeil et al. (1981) observe:

Utility elicitation in medical decision making is complex, and the literature is extensive (Naglie et al. 1997, Chapman & Sonnenberg 2000). Here we first illustrate the application of the NM theory, then present an alternative approach based on time trade-offs rather than probability trade-offs, and lastly discuss conditions for the two to be equivalent.

4.3.2 Standard gamble for health states

In Section 4.1 we described a general procedure for utility elicitation. We discuss next a mathematically equivalent example of utility elicitation using the standard gamble approach in a medical example.

4.3.3 The time trade-off methods

In the standard gamble, the trade-off is between an intermediate option for sure and two extreme options with given probabilities. The time trade-off method (Torrance 1971, Torrance et al. 1972) plays a similar game with time instead of probability. It is typically used to assess a patient’s attitude towards the number of years in ill health he or she is willing to give up in order to live in good health for a shorter number of years. The time in good health equivalent to a year of ill health is called the quality-adjusted life year (QALY). To implement this method we first pick an arbitrary period of time t in a particular condition, say chronic pain. Then, we find the amount of time in good health the patient considers equivalent to the arbitrary period of time in the condition. For applications in the analysis of clinical trials results, see Glasziou et al. (1990).

Figure 4.3 Standard gamble for assessing utility for chronic pain.

Figure 4.4 Time trade-off for assessing surgical treatment of chronic pain. Figure based on Torrance et al. (1972).

4.3.4 Relation between QALYs and utilities

Weinstein et al. (1980) identified conditions under which the time trade-off method and the standard gamble method are equivalent. The first condition is that the utility of health states must be such that there is independence between length of life and quality of life. In other words, the trade-offs established on one dimension do not depend on the levels of the other dimension. The second condition is that the utility of health states must be such that trade-offs are proportional, in the following sense: if a person is indifferent between x years of life in good health and x′ years of life in chronic pain, then he or she must also be indifferent between spending qx years of life in excellent health and qx′ years in chronic pain. The third condition is that the individual must be risk neutral with respect to years of life. This means that the patient’s utility for living the next year is the same as for living each subsequent year (that is, his or her marginal utility is constant). The last assumption is not generally considered very realistic. Moreover, that the time trade-off and standard gamble could lead to empirically different results even when the decision maker is close to risk neutrality, depending on how different aspects of utilities, such as desirability of states, and time preferences are empirically captured by these methods.

Figure 4.5 Relationship between the standard gamble and time trade-off methods.

If all these conditions hold, we can establish a correspondence between the quality adjustment factor q elicited using the time trade-off method and the utility α. We wish to establish that, for a given health outcome z, held constant over a period of t years, αt = qt no matter what t is. Figure 4.5 outlines the logic. Using the first condition we can establish trade-offs for length of life and quality of life independently. So, let us first consider the trade-offs for quality of life. Using the standard gamble approach, we find α such that a lottery with probability α for 1 unit of time in z⁰ and (1 – α) in z₀ is equivalent to a lottery which yields an intermediate outcome z for 1 unit of time, for sure. Thus, the utility of outcome z for 1 unit of time is α. Using the risk-neutrality condition, the expected utility of t units of time in z is t × α. Next, let us consider the time trade-off method. From the proportional trade-offs condition, if a person is indifferent between 1 unit of time in outcome z and q units of time in z⁰, he or she is also indifferent between t units of time in outcome z and qt units of time in z⁰. So if these three conditions are true, both methods lead to the same evaluation for living t years in health outcome z.

4.3.5 Utilities for time in ill health

The logic outlined in the previous section can also be used to elicit patient’s utilities for a period of time in ill health in a two-step procedure. In the first step the standard gamble strategy is used to elicit the utility of a certain number of years in good health. In the second step the time trade-off is used to elicit patient’s preferences between living a shorter time in good health and a longer time in ill health. Alternatively, one could assess patient’s utility considering length and quality of life together via lotteries such as those in the NM theory. However, it is usually more difficult to think about length and quality of life at the same time.

Example 4.4 We will illustrate this using an example based on a patient currently suffering for some chronic condition; that is, whose current health state z is stable over time and less desirable than full health. The numbers in the example are from Sox et al. (1988). Using the standard gamble approach, we can elicit patient utilities for different lengths of years in perfect health as shown by Table 4.1. This table requires three separate applications of the standard gamble: one for each of the three intermediate lengths of life. Panel (a) of Figure 4.6 shows an interpolation of the resulting mapping between life length and utility, represented by the solid line. The dashed diagonal line represents a person with constant marginal utility. The patient considers a 50/50 gamble between 0 (immediate death) and 25 years (full life expectancy) equivalent to a sure survival of 7 years (represented by a triangle in the figure). Thus, the patient would be willing to “trade off” the difference between the average life expectancy and the certainty equivalent, that is 12.5 – 7 = 5.5 years to avoid the risk. This is a complicated choice to think about in the abstract, but it is not unrealistic for patients facing surgeries that involve a serious risk of death.

Table 4.1 Patient’s utility function for the length of a healthy life.

Years of perfect health	Utility
0	0.00
3	0.25
7	0.50
12.5	0.75
25	1.00

Figure 4.6 Trade-off curves for the elicitation of utilities for time in ill health.

Table 4.2 Equivalent years of life and patient’s utility function for various lengths of life in ill health.

Years of disabled life	Equivalent years of perfect health	Utility
5	5	0.38
10	9	0.59
15	13	0.76
20	17	0.84
25	21	0.92

Next, we elicit how long a period of life in full health the patient considered to be equivalent to life in his or her current state. Responses are reported in the first two columns of Table 4.2 and shown in Figure 4.6(b). The solid line represents the relation for a person for whom full health and current health are valued the same. The dashed line represents the relation for a person for whom the current health condition represents an important loss of quality of life. The patient would “trade off” some years in his or her current health for a better health state.

Finally, using both tables (and parts (a) and (b) of Figure 4.6) we can derive the patient’s utility for different durations of years of life with the current chronic condition. This is listed in the last column of Table 4.2 and shown in Figure 4.6(c), where the solid line is the utility for being in full health throughout one’s life, while the dashed line is the utility for the same duration with the current chronic condition. The derivation is straightforward. Consider the second row of Table 4.2: 10 years in the current health state are equivalent to 9 years in perfect health. Using Table 4.1, we find the utility using a linear interpolation. Let u_x denote the utility for x years of healthy life which gives

Similar calculations give the remaining utility values shown in Table 4.2 and Figure 4.6(c).

4.3.6 Difficulties in assessing utility

The elicitation of reliable utilities is one of the major obstacles in the application of decision analysis to medical problems. Research on descriptive decision making compares preferences revealed by empirical observations of decisions to utilities ascertained by elicitation methods. The conclusion is generally that the assessed utility function may not reflect the patient’s true preferences. The difficulties arise from the fact that in many circumstances, important components of the decision problem may not be reliably quantified. Factors such as attitudes in coping with disease and death, ethical or religious beliefs, etc., can lead to preferences that are more complex than what we can describe within the axiomatization of expected utility theory.

Even if that is not the case, cognitive aspects may challenge utility assessment. Three categories of difficulties are often cited: framing effects, problems with small probability, and changes in utilities over time. Framing effects are artifacts and inconsistencies that arise from how questions are formulated during elicitation, or on the description of the scenarios for the outcomes. For example, “the framing of benefit (or risk) in relative versus absolute terms may have a major influence on patient preferences” (Malenka et al. 1993).

Very small or very high probability is a challenge even for decision makers who may have an intuitive understanding of the concept of probability. Yet these are pervasive in medicine: for example, all major surgeries involve a chance of seriously adverse outcomes; all vaccination decisions are based on trading off short-term symptoms for sure against a serious illness with very small probability. In such cases, a sensitivity analysis can be used to evaluate how changes in the utilities would affect the ultimate decision making. A related alternative is inverse decision theory (Swartz et al. 2006, Davies et al. 2007). This is applicable to cases where a small number of actions are available, and is based on partitioning the utility and probability inputs into sets, each collecting all the inputs that lead to the same decision. For example, in Section 4.3.2 we would determine values of u for which surgery is the best strategy. The expected utility of no surgery is u, while for surgery it is 0.04 × 0 + 0.16 × u + 0.8 × 1. Thus, the optimal strategy is surgery whenever u < 0.95. A more general version would partition the probabilities as well.

The patient’s attitudes towards length of life may change with age, or life expectancy. Should a patient anticipate these changes and decide now based on the prediction of what his or her utility are expected to become? This sounds rational but prohibitive to implement. The default approach is that the patient’s current preferences should be the basis for making decisions.

Lastly, utilities vary widely across individuals, partly because their preferences do and partly as a result of challenges in measurement. For certain health states, such as the consequences of a major stroke, patients distribute across the whole range—with some patients ranking the health state worse than death, and others considering it close to full health (Samsa et al. 1988). This makes it difficult to use replication across individuals to increase the precision of estimates.

For an extended review of these issues see also Chapman & Elstein (2000).

4.4 Exercises

Problem 4.1 (Kreps 1988, problem 7) Suppose a decision maker has constant absolute risk aversion over the range –$100 to $1000. We ask her for her certainty equivalent for gamble with prizes $0 and $1000, each with probability one-half, and she says that her certainty equivalent for this gamble is $488. What, then, should she choose, if faced with the choice of:

Solution

Since the decision maker has constant absolute risk aversion over the range –$100 to $1000, we have

We know that the certainty equivalent for a 50/50 gamble with prizes $0 and $1000 is $488. Therefore,

Suppose u(0) = 0, u(1000) = 1, and consider equations (4.21) and (4.22). We have

0 =	−ae−λ0 + b
1 =	−ae−λ1000 + b
1/2 =	−ae−λ488 + b.

This system implies that a = b = 10.9207 and λ = 0.000 096 036 9. Therefore, we have

From equation (4.23) we obtain

u(–100) =	−0.105 384
u(300) =	0.310 148
u(385) =	0.396 411
u(530) =	0.541 949.

The expected utilities associated with gambles a, a′, and a″ are

Based on these values we conclude that the decision maker should choose gamble a′ since it has the maximum expected utility.

Problem 4.2 Find the following:

1. A practical decision problem where it is reasonable to assume that there are only four relevant outcomes z₁, ..., z₄.

2. A friend willing to waste 15 minutes.

3. Your friend’s utilities u(z₁), ..., u(z₄).

Here’s the catch. You can only ask your friend questions about preferences for von Neumann–Morgenstern lotteries. You can assume that your friend believes that Axioms NM1, NM2, and NM3 are reasonable.

Problem 4.3 Prove part (b) of Proposition 4.2.

Problem 4.4 Was Bernoulli risk averse? Decreasingly risk averse?

Problem 4.5 (Lindley 1985) A doctor has the task of deciding whether or not to carry out a dangerous operation on a person suspected of suffering from a disease. If he has the disease and does operate, the chance of recovery is only 50%; without an operation the similar chance is only 1 in 20. On the other hand if he does not have the disease and the operation is performed there is 1 chance in 5 of his dying as a result of the operation, whereas there is no chance of death without the operation. Advise the doctor (you may assume there are always only two possibilities, death or recovery).

Problem 4.6 You are eliciting someone’s utility for money. You know this person has constant risk aversion in the range $0 to $1000. You propose gambles of the form $0 with probability p and $1000 with probability 1 – p for the following four values of p: 1/10, 1/3, 2/3, and 9/10. You get the following certainty equivalents: 0.25, 0.60, 0.85, and 0.93. Verify that these are not consistent with constant risk aversion. Assuming that the discrepancies are due to difficulty in the exact elicitation on certainty equivalents, rounding, etc., find a utility function with constant risk aversion that closely approximates the elicited certainty equivalents. Justify briefly the method you use for choosing the approximation.

Problem 4.7 (From French 1988) An investor has $1000 to invest in two types of shares. If he invests $a in share A, he will invest $(1000 – a) in share B. An investment in share A has a 0.7 chance of doubling value and a 0.3 chance of being lost altogether. An investment in share B has a 0.6 chance of doubling value and a 0.4 chance of being lost altogether. Outcomes of the two investments are statistically independent. Determine the optimal value of a if the utility function is u(z) = log(z + 3000).

Problem 4.8 An interesting special case of Example 4.1 happens when Z = {−k, k}, k > 0. As in that example, assume that ω is the decision maker’s initial wealth. The probability premium p(ω, k) of a is defined as the difference a(k)–a(–k) which makes the decision maker indifferent between the status “quo” and a risk z in {−k, k}. Prove that λ(ω) is twice the probability premium per unit risked for small risks (Pratt 1964).

Problem 4.9 The standard gamble technique has to be slightly modified for chronic states considered worse than death as shown in Figure 4.7. Show that the utility of the chronic state is u(z) = −α/(1 – α). Moreover, show that the quality-adjusted life year is given by q = x/(x – t).

Problem 4.10 A machine can be functioning (F), in repair (R), or dead (D). Under the normal course of operations, the probability of making a transition between any of the three states, in a day, is given by this table:

Figure 4.7 Modified standard gamble for chronic states considered worse than death.

These probabilities only depend on where the machine is today, and not on its past history of repairs, age, etc. This is obviously an unreasonable assumption; if you want to relax it, be our guest.

When the machine functions, the daily income is $1000. When it is in repair, the daily cost is $150. There is no income after the machine is dead. Suppose you can put in place a regimen that gives a lower daily income ($900), but also decreases the probability that the machine needs repair or dies. Specifically, the transition table under the new regimen is

New regimen or old? Assume the decision maker is risk neutral and that the machine is functioning today. You can use an analytic approach or a simulation.

Problem 4.11 Describe how you would approach the problem of estimating the cost-effectiveness of regulating emission of pollutants into the environment. Pick a real or hypothetical pollutant/regulation combination, and briefly describe data collection, modeling, and utility elicitation.