PHIL 109 · Introduction to Formal Reasoning and Decision Making

8 Decisions under risk

As we saw in the previous chapter, a decision counts as a decision under risk when the agent cannot eliminate all but one of the states in the relevant partition, but can assign probabilities to those states. This is the case that our work on probability in Part II has prepared us for, and it is the case in which decision theory has the most to say.

8.1 Cardinal utilities

Remember the decision we discussed in relation to the minimax regret norm (Table 7.17):

The minimax regret norm told us to go with option $A_{1}$, because it has a regret value of 99, whereas option $A_{2}$ has a regret value of 100. However, that verdict seemed wrong: most of us are willing to let go of a single possibility of winning 100 in exchange for 99 possibilities of winning 99.

Recall that the friend of minimax regret had a reply to this worry, however. For minimax regret is advertised as a norm for decisions under ignorance. That means that we have no idea how probable each state is. The fact that there are 99 states with an outcome of 99 for option $A_{2}$ and only 1 with an outcome of 100 for option $A_{1}$ means very little if we have no idea of how probable each state is: it may be, for all we know, that state $S_{1}$ has probability 1 of occurring, whereas the other 99 have no chance at all.

But let us now treat it as a case of decision under risk. Let us suppose, in particular, that we have equal credence for each state:

So now that we think each state is equally likely, it is obvious that the minimax regret norm gets it wrong, right? No. If the utilities in question (the 100 for $(A_{1} \land S_{1})$ and the 99 for each of the $(A_{2} \land S_{i})$ from $S_{2}$ to $S_{100}$) are ordinal utilities, then we still cannot reach the conclusion that $A_{2}$ is the rational choice. For remember that utilities measured in an ordinal scale only tell us the order of the outcomes, and nothing else. In particular, the fact that there is only a difference of 1 between 100 and 99 means absolutely nothing for utilities measured in an ordinal scale. In fact, the very same information is transmitted by the following table:

With these numbers, $A_{1}$ even has the higher expected value ($0.01 \times 100 = 1$, against $0.99 \times 1 = 0.99$ for $A_{2}$). So, just assigning probabilities to the states is not enough to justify the intuitive thought that $A_{2}$ is the rational action instead of $A_{1}$. What we need in order to fully justify that thought is a more fine-grained way of measuring our preferences.

What we are missing when we measure our preferences using a merely ordinal utility scale is by how much we prefer one outcome to another. In addition to ordinal scales there are two other kinds of scales that are of interest to us: interval scales and ratio scales.

Interval scales can represent not just order but also the relative strength of our preferences. Thus, suppose that I prefer mate ($M$) to coffee ($C$) to tea ($T$) to coke ($O$), and that I represent my preferences with the following line:

If I measure my preferences using an ordinal scale, I need only to use numbers in ascending order from left to right. But if I want to measure my preferences using an interval scale, I must make sure that the relative lengths of the intervals are preserved as well. Thus, I could not assign 1 to $O$, 2 to $T$, 3 to $C$ and 4 to $M$, for that would misrepresent the difference between the intervals. In general, an interval scale will be able to represent the differences in utilities, in the sense that, for any four outcomes $\varphi, \psi, \chi$, and $\upsilon$, the preference interval between $\varphi$ and $\psi$ is greater than or equal to that between $\chi$ and $\upsilon$ if and only if $|u(\varphi)-u(\psi)| \geq |u(\chi) - u(\upsilon)|$.

Recall that two ordinal scales are equivalent just in case they are ordinal transformations of each other. Interval scales are equivalent just in case they are positive linear transformations of each other:

Interval scales are only one type of cardinal scales. There are also ratio scales, which we use to measure weights and lengths, for instance, where there are natural zero points. For our purposes, we will need just interval scales. So, from now on, when we talk about cardinal scales you can assume that we mean interval scales, and in particular that any two cardinal scales that are positive linear transformations of each other count as equivalent.

We know that ordinal utilities come simply from your pairwise preferences among outcomes. Where do cardinal utilities come from? Before answering that question, let us see what we can do once we do have cardinal utilities to work with.

8.2 Expected value

The notion of expectation, in mathematics, is very general. Whenever we have a partition, a probability distribution over it, and a variable that takes values in each state of the partition, we have an expectation for that variable:

If you haven’t seen the $\sum$ symbol before, here is what it means. $\sum$ is the capital Greek letter “sigma,” and it is just shorthand for “add up.” An expression like $\sum_{i=1}^{n} \Prb(S_{i}) \cdot X(S_{i})$ tells you three things: what to add up (the general term, $\Prb(S_{i}) \cdot X(S_{i})$), where to start ($i = 1$), and where to stop ($i = n$). To evaluate it, you plug in each value of $i$ from $1$ to $n$, one at a time, into the general term, and add up all the results: $$\sum_{i=1}^{n} \Prb(S_{i}) \cdot X(S_{i}) = \Prb(S_{1}) \cdot X(S_{1}) + \Prb(S_{2}) \cdot X(S_{2}) + \cdots + \Prb(S_{n}) \cdot X(S_{n})$$ So the expectation of $X$ is nothing more than a weighted sum: you take the value $X$ takes in each state, weight it by how probable that state is, and add up the results across all the states in the partition.

Let us see some examples. Suppose, for instance, that you are thinking about how many slices of pizza you will eat for dinner. You think there is a small chance, say 10%, that you will eat just one slice, 30% chance that you will eat two slices, 50% chance that you will eat three slices, and 10% chance that you will eat four slices (you think there’s no chance you will eat nothing or more than four slices). Then the expected value for the number of pizza slices that you will have for dinner is: $\frac{1}{10} \times 1 + \frac{3}{10} \times 2 + \frac{1}{2} \times 3 + \frac{1}{10} \times 4 = 2$ and $\frac{3}{5}$ slices. Or suppose that you are rolling two dice, and that you will get paid an amount of dollars equal to the sum of the faces of the dice that come up. In that case, your expected gain is $\frac{1}{36} \times 2 + \frac{2}{36} \times 3 + \frac{3}{36} \times 4 + \frac{4}{36} \times 5 + \frac{5}{36} \times 6 + \frac{6}{36} \times 7 + \frac{5}{36} \times 8 + \frac{4}{36} \times 9 + \frac{3}{36} \times 10 + \frac{2}{36} \times 11 + \frac{1}{36} \times 12 = 7$ dollars.

Notice something interesting about these examples: the expected value of a variable is not, in general, the value you expect to get. Suppose that you always finish a slice of pizza once you start it. Then you do not expect to eat 2 and $\frac{3}{5}$ slices, but that is the expected number of pizza slices you will eat in the technical sense of “expected.” Similarly, you do not really expect to get paid 7 dollars in the dice game (there is only a $\frac{6}{36}$ probability of that happening), and yet that is the expected monetary value of playing that game. If the expected value is not the value you expect to obtain, then what is it?

One answer to that question is that the expected value is the average value you should expect to obtain if you do the same thing over and over. For instance, if you play the dice game a number of times, on average you will get 7 dollars per game, and so the higher the number $n$ of games you play, the closer your average gain per game will be to 7.¹ Expected value is, after all, a weighted average. But suppose you will play the game just once, and so this line of thought doesn’t apply. Does expected value have no meaning though? Arguably, it still does. You can think of expected value as a way of balancing risks: sure, you may be unlucky to win only 2 dollars, but you may also get lucky and win 12. Most likely, though, you’ll win something in between. Here a weighted average does not seem such a bad idea.

7 dollars is the expected monetary value of our dice game. Likewise, the expected monetary value of a ticket in the “SuperEnalotto” we talked about in Chapter 6 is, if we suppose the jackpot stands at 100,000,000 euros and we set aside the smaller secondary prizes: $\frac{1}{622{,}614{,}630} \times 100{,}000{,}000 + \frac{622{,}614{,}629}{622{,}614{,}630} \times 0 \approx 0.16$ euros—a small fraction of the 1 euro it costs to buy the ticket.²

Does that mean that anyone who buys a ticket in the SuperEnalotto is simply irrational? Not necessarily. Suppose, for instance, that one needs millions of euros in order to pay ransom for the kidnapping of one’s pet, and that one simply does not have that much money. If you care a lot about your pet, you might think it’s worth paying 1 euro for the minuscule possibility of saving your pet. Or perhaps some people simply enjoy the thrill of playing the lottery. In both of these cases, the utility associated with buying a ticket cannot simply be equated with its expected monetary value.

8.3 Where do cardinal utilities come from?

Recall that ordinal utilities come simply from your preference order among outcomes. Where do cardinal utilities come from? Not from the expected monetary value of the outcomes, as we just saw. But then, from where?

Suppose that you like coffee more than tea, and tea more than coke. So far, that just gives us a ranking. But if you can answer some additional questions, we can transform that ranking into an interval scale. The kinds of questions you have to answer have the following form: what do you prefer, tea outright, or a lottery ticket with 50% chance of winning a coffee and 50% chance of winning a coke? Suppose you answer that you prefer the lottery. In that case, the difference between your utility for coffee and your utility for tea is larger than the difference between your utility for tea and your utility for coke: your preference for coffee over tea is stronger than your preference for tea over coke. If, on the other hand, you prefer tea outright to the lottery, then that means that the difference between your utilities for tea and coke is larger than the difference between your utilities for coffee and tea: your preference for tea over coke is stronger than your preference for coffee over tea. In each of these cases, we still don’t know how strong the difference in preferences is. For that, we need to ask you about lotteries with other chances for coffee and coke, until we hit one lottery such that you are indifferent between that lottery and tea outright. Finally, if you answer that you are indifferent between tea and the 50-50 lottery, then that means that the difference between your utility for coffee and your utility for tea is exactly the same as the difference between your utility for tea and your utility for coke: tea sits right in the middle between coffee and coke for you.

What we did for tea we can do for every other outcome. To fix the scale we also need two anchor points, which we are free to stipulate: say, utility 0 for coke and utility 1 for coffee, so that indifference between tea and the 50-50 lottery gives tea a utility of $0.5$. In that way, we can construct a cardinal utility scale for you.

This method assumes that you value a lottery by weighting the utilities of its prizes by their probabilities; otherwise we could not tell whether you prefer tea outright because the difference between tea and coke is large, or merely because you dislike risk. Von Neumann and Morgenstern showed that this assumption holds when your preferences over lotteries meet certain plausible conditions, which we will not study here.

8.4 Maximization of expected utility

Now that we have utilities measured in a cardinal scale, we can make sense of the expected utility of a choice $\varphi$:

And, with all that preamble out of the way, we are now in a position to define perhaps the most famous decision norm of them all, maximization of expected utility:

Notice that the maximization of expected utility norm allows for ties.

Now that we have to represent probabilities of states in addition to utilities for outcomes, we need to modify our decision matrices to include this extra information. Let us go back to our park/movies example from last chapter, and this time let us assume that we have a 0.7 credence that it will rain and the corresponding 0.3 credence that it will not rain. The matrix for this example is the following:

Try it in the Decisions Lab Step through expected utility on this matrix.

Let us now find out the expected utility for each option:

$$EU(P) = 1 \times \frac{7}{10} + 4 \times \frac{3}{10} = 1.9$$

$$EU(M) = 3 \times \frac{7}{10} + 2 \times \frac{3}{10} = 2.7$$

The result is that going to the movies is the choice that maximizes expected utility, and thus the rational choice according to the MEU norm.

We should notice an interesting fact about the interaction between the dominance norm and the MEU norm. Even though we talked about dominance in connection with decisions under ignorance, dominated choices will never maximize expected utility. This is because if a choice is dominated then its expected utility is guaranteed to be less than that of the choice that dominates it. Moreover, if there is just one non-dominated choice, then it will maximize expected utility. In that sense, the dominance norm and the MEU norm cannot enter into conflict. And this means that if in a decision under risk just one choice is non-dominated, then we do not need to bother with calculating expected utility. Consider, for instance, what happens when we modify the example so that going to the park dominates going to the movies:

Try it in the Decisions Lab Step through expected utility on this matrix.

It is easy to see why if there is a unique dominant choice then it will maximize expected utility: because in the calculation of expected utility, the dominant choice has larger utilities no matter what state occurs, and the utilities are weighted by the same probability for each option:³

$$EU(P) = 5 \times \frac{7}{10} + 4 \times \frac{3}{10} = 4.7$$

$$EU(M) = 3 \times \frac{7}{10} + 2 \times \frac{3}{10} = 2.7$$

8.5 When states depend on choices: maximization of conditional expected utility

The fact that dominance and the MEU norm never conflict gives rise to a problem. A lazy student has two options: study for the class, or not study. There are two states the world could be in: either he passes the class or he doesn’t. He prefers passing to not passing, but, for every state, he prefers not studying to studying. His utilities can be whatever you want compatible with this. Let us assume they are as follows:

Dictionary:
$S$ = The student studies.
$P$ = The student passes the class.
Utilities:
$u(\neg S \land P) = 7$
$u(\neg S \land \neg P) = 2$
$u(S \land P) = 6$
$u(S \land \neg P) = 1$

To complete the decision matrix we would have to know what credence the student has that he will pass the class. But, on second thought, it doesn’t really matter, for not studying dominates studying for this student. Whatever probability $n$ he assigns to passing, his matrix will look like this:

Notice that the expected utilities are as follows:

$$EU(S) = 6n + 1(1-n) = 5n +1$$ $$EU(\neg S) = 7n + 2(1-n) = 5n +2$$

Thus, as we know, dominance and the MEU norm give the same result: the student should not study.

But this is obviously the wrong result, for the model of the decision is missing one crucial fact: studying will increase the probability that the student will pass, whereas not studying will decrease it. So far, we have been dealing with the unconditional probabilities of the states, which means that we assign any outcome under a given state the same probability. In this case, passing the class has the same probability no matter whether the student studies or not. Dealing with these unconditional probabilities is fine when the probability of the states does not vary with the choices the agent makes. For instance, going to the movies or going to the park does not affect the probability that it will rain, and so for that kind of decision problem it is ok to assign unconditional probabilities to the states. But for the kind of problem where the choices do affect the probability of the states, as is the case with the lazy student, we need to consider, not the unconditional probability that the state will occur, but rather the conditional probability that the state will occur given that a certain action is chosen.

When dealing with unconditional probabilities it is fine to have them just under the label for the state, because the same probability can be used for the whole column, but when dealing with conditional probabilities we need to consider potentially different conditional probabilities for different outcomes under the same column, and so we will write the probabilities in the cells. The utilities will go in the upper-left corner of the cell and the probabilities in the lower right. Thus, let us assume that the conditional probability that the student will pass given that he studies is 0.7, whereas the conditional probability that the student will pass given that he doesn’t study is 0.1.⁴ Under these assumptions, the matrix for this decision problem looks like this:

Try it in the Decisions Lab Step through expected utility on this matrix.

Now, how do we calculate the expected utility of each choice? Well, instead of using the unconditional probability of the state to multiply the utility of each outcome, we use the conditional probability of the state given the choice:

Corresponding to this definition of conditional expected utility, we have the following maximization of conditional expected utility norm:

Applied to the lazy student case, the MCEU norm gives the intuitively right result that the student should study:

$$CEU(S) = 0.7 \times 6 + 0.3 \times 1 = 4.2 + 0.3 = 4.5$$ $$CEU(\neg S) = 0.1 \times 7 + 0.9 \times 2 = 0.7 + 1.8 = 2.5$$

Since $CEU(S) = 4.5 > 2.5 = CEU(\neg S)$, by MCEU, the student should study.

So now that we have two different norms for decisions under risk, MEU and MCEU, which one do we apply when? The straightforward answer is that we should apply MEU when the states are probabilistically independent of the choices, and MCEU otherwise. As we will see in the next chapter, however, this simple answer gives rise to some complicated issues.

Exercises

For each of the decision problems below, write down the corresponding decision matrix and then apply MEU or MCEU as appropriate.

1. A basketball player must decide whether to practice free throws every day before a big game ($T$) or skip practice ($\neg T$). The relevant states are whether he makes the game-winning free throw ($W$) or misses it ($\neg W$). Practicing raises the conditional probability of making the shot: $\Prb(W \given T) = 0.8$ and $\Prb(W \given \neg T) = 0.5$. His utilities are $u(T \land W) = 10$, $u(T \land \neg W) = 3$, $u(\neg T \land W) = 10$, and $u(\neg T \land \neg W) = 4$.

   Show solution

   MCEU: $T: 8.6$ vs. $\lnot T: 7$. Choice: practice.

2. An investor is deciding whether to put her savings into stock $A$ or stock $B$. The market will either go up ($M$) or down ($\neg M$) next year. She assigns a credence of $0.55$ to the market going up. Her utilities are $u(A \land M) = 8$, $u(A \land \neg M) = 2$, $u(B \land M) = 5$, and $u(B \land \neg M) = 4$.

   Show solution

   MEU: $A: 5.3$ vs. $B: 4.55$. Choice: $A$.

3. A gambler must choose between betting on red ($A_R$) or betting on black ($A_B$) on a single spin of a roulette wheel. The relevant states are that the wheel lands red ($R$) or lands black ($B$), and she believes the wheel has a $0.52$ chance of landing red and a $0.48$ chance of landing black. If she bets red and it lands red, she wins $\$10$; if she bets red and it lands black, she loses $\$5$. If she bets black and it lands black, she wins $\$8$; if she bets black and it lands red, she loses $\$4$. (Assume her utilities equal the dollar amount won.)

   Show solution

   MEU: red $2.8$ vs. black $1.76$. Choice: red.

4. A woman is deciding whether to exercise regularly ($E$) or remain sedentary ($\neg E$) this year. The relevant states are whether she gets seriously ill ($I$) or stays healthy ($\neg I$). Exercising lowers the conditional probability of illness: $\Prb(I \given E) = 0.1$ and $\Prb(I \given \neg E) = 0.3$. Her utilities are $u(E \land I) = 2$, $u(E \land \neg I) = 8$, $u(\neg E \land I) = 1$, and $u(\neg E \land \neg I) = 9$.

   Show solution

   MCEU: $E: 7.4$ vs. $\lnot E: 6.6$. Choice: exercise.

5. You are deciding whether to bring an umbrella ($U$) or leave it at home ($\neg U$) on your way to work. There are two possible states: it rains ($R$) or it doesn’t ($\neg R$). You assign a credence of $0.4$ to rain. Your utilities are $u(U \land R) = 5$, $u(U \land \neg R) = 3$, $u(\neg U \land R) = 1$, and $u(\neg U \land \neg R) = 6$.

   Show solution

   MEU: $U: 3.8$ vs. $\lnot U: 4.0$. Choice: leave the umbrella.

6. A driver must decide whether to put her phone away while driving ($C$) or keep glancing at it ($\neg C$). The relevant states are whether she gets into a serious accident ($A$) or not ($\neg A$). Putting the phone away lowers the conditional probability of an accident: $\Prb(A \given C) = 0.02$ and $\Prb(A \given \neg C) = 0.15$. Her utilities are $u(C \land A) = 1$, $u(C \land \neg A) = 9$, $u(\neg C \land A) = 0$, and $u(\neg C \land \neg A) = 10$.

   Show solution

   MCEU: $C: 8.84$ vs. $\lnot C: 8.5$. Choice: put the phone away.

7. A person is deciding whether to get a flu vaccine ($V$) or skip it ($\neg V$) this winter. The relevant states are whether she catches the flu ($F$) or not ($\neg F$). Getting vaccinated lowers the conditional probability of catching the flu: $\Prb(F \given V) = 0.05$ and $\Prb(F \given \neg V) = 0.25$. Her utilities are $u(V \land F) = 4$, $u(V \land \neg F) = 9$, $u(\neg V \land F) = 2$, and $u(\neg V \land \neg F) = 10$.

   Show solution

   MCEU: $V: 8.75$ vs. $\lnot V: 8.0$. Choice: vaccinate.

8. A salesperson is deciding whether to spend an extra hour each day cold-calling potential clients ($H$) or use that hour for something else ($\neg H$). The relevant states are whether she closes a major deal this month ($D$) or not ($\neg D$). Cold-calling raises the conditional probability of closing a deal: $\Prb(D \given H) = 0.4$ and $\Prb(D \given \neg H) = 0.1$. Her utilities are $u(H \land D) = 10$, $u(H \land \neg D) = 4$, $u(\neg H \land D) = 12$, and $u(\neg H \land \neg D) = 5$.

   Show solution

   MCEU: $H: 6.4$ vs. $\lnot H: 5.7$. Choice: cold-call.