PHIL 109 · Introduction to Formal Reasoning and Decision Making
7 Decisions under ignorance
This chapter sets out the mathematical model of a decision (choices, states, outcomes, and utilities), represents decisions with matrices, and shows how ordinal utilities can be read off an agent’s preferences when those preferences form a weak order. It then turns to choice under ignorance, where we cannot assign probabilities to the states, and compares three norms: dominance, maximin (with its refinement, leximin), and minimax regret.
In Part I we learned about propositional logic and how it helps us analyze arguments and determine their tautological validity. In Part II we learned about probability, and how it helps us analyze probabilistic arguments. In this part, we’ll apply everything we learned in the previous two, plus some new techniques, to analyze how to make rational decisions. We start by considering those decisions that involve only one agent as a decision-maker, and then we move on to interactive decision theory—also known as game theory.
We start by setting out the main elements in our mathematical model of decisions. They are: choices, states, outcomes, and utilities. Choices are the options that the agent has: go to college, work on the family farm, or form a rock band, for instance. States are ways the world can be that affect what outcome your choice will have: whether the economy will tank in the next five years or not, for example. Outcomes are, precisely, a function of your choice on the one hand and the state of the world on the other: for instance, if you decide to work on the family farm and the economy tanks, the outcome is not as good as if it did not. Finally, utilities are a measure of your preferences over outcomes.
7.1 Representing decisions
The standard way of representing decisions is by using a matrix. For a decision with choices and states, we use an by matrix where the choices label the rows, the states label the columns, and the cells represent outcomes:
| states | |||||
|---|---|---|---|---|---|
| choices | |||||
In general, we will write the utilities associated with the outcomes in each cell instead:
| states | |||||
|---|---|---|---|---|---|
| choices | |||||
Suppose that we have a choice between going to the park or going to the movies. Those are our choices. It could rain or not. Those are the states. Pairing a choice with a state gives us an outcome, and so there are four of them: going to the park while it rains, going to the park while it doesn’t rain, going to the movies while it rains, and going to the movies while it doesn’t rain. Let us assume that your utilities for those four outcomes are 1, 4, 3, and 2 respectively (we will study where those utilities come from later in this chapter). A rained-out afternoon at the park is the worst thing that could happen to you, and a sunny one is the best; the movies are somewhere in the middle either way, and marginally better when it rains, since then at least you are not squandering good weather indoors. Writing for going to the park, for going to the movies, and for its raining, we can represent this decision problem with the following matrix:
Not just any list of propositions will do for the columns of such a matrix. Suppose we had taken our states to be it rains and it snows. Then a clear day is a way things might turn out that no column covers, and the matrix simply says nothing about what would happen if it came about—our list of outcomes would be incomplete. Suppose instead that we had taken our states to be it rains and it is cold. Now a cold, rainy day falls under both columns at once, and the cells disagree with each other about what its outcome is. To rule out both defects, we require that the states partition the possibility space:
Definition 7.1. Partition: A set of propositions is a partition of the possibility space if and only if exactly one of them is true in each possible world. That is to say, they must be both:
Mutually exclusive: no two of them are true in the same possible world (see Definition 4.1); and
Jointly exhaustive: there is no possible world in which none of them is true.
Mutual exclusivity is what rules out the second bad case, and joint exhaustiveness what rules out the first. Together they guarantee that, however things turn out, exactly one column describes the situation you are actually in—which is what entitles us to read each row as a complete specification of what a choice would get you.
Our original example passes the test, and does so for a reason that will often be useful: a proposition and its negation always form a partition, since as a matter of logic and cannot both be true and cannot both be false. The rows of a truth-table are a partition as well: exactly one row describes any given possible world, which is precisely why the probabilities we assign to the rows of a stochastic truth-table have to sum to 1.
7.2 Where do ordinal utilities come from?
Where did the utility numbers come from? The answer is: from your preferences among outcomes. Utilities are a scale we use to represent your preferences. We will consider two different kinds of utility scales, ordinal and cardinal. We start with ordinal scales in this chapter, and turn to cardinal scales in Chapter 8.
We will use the symbol “” to represent preference relations— means that the agent prefers to . If an agent is indifferent between and , we will write . Recall that your most preferred option is a sunny day at the park, which you prefer even to the movies when it rains; that you prefer being at the movies when it rains to being at the movies when it doesn’t; and that you prefer being at the movies when it doesn’t rain even to being at the park when it rains. We can therefore represent these preferences as follows:
Notice that we have not stated all of your preferences here, because we have not stated that you prefer being at the park when it doesn’t rain to being at the movies when it doesn’t rain. You might think that goes without saying, given that you prefer being at the park when it doesn’t rain to being at the movies when it rains, and you prefer being at the movies when it rains to being at the movies when it doesn’t rain. That is to say, you might think it goes without saying that your preferences should be transitive. If so, decision theorists agree with you. In fact, decision theory presupposes that your preferences satisfy several global constraints:
Definition 7.2. Asymmetric: An agent’s preferences are asymmetric if and only if, for any two outcomes and :
If , then not .
If , then not .
If , then not and also not .
Definition 7.3. Transitive: An agent’s preferences are transitive if and only if, for any three outcomes , and :
If and , then .
If and , then .
If and , then .
If and , then .
Definition 7.4. Connected: An agent’s preferences are connected if and only if, for any two outcomes and , at least one of the following holds:
.
.
.
Notice that Definition 7.2 and Definition 7.4 work together: connectedness guarantees that at least one of the three relations holds between any two outcomes, and asymmetry guarantees that at most one of them does. Taken together, then, they tell us that for any two outcomes exactly one of , , and is true—an agent whose preferences satisfy both is never undecided between two outcomes, and never prefers each of two outcomes to the other.
Notice also the crucial distinction between being indifferent between two outcomes, on the one hand, and having no preference relations whatsoever between them, not even indifference. If you are indifferent between two outcomes, that means that you are completely willing to replace one with the other. Right now, for instance, I am indifferent between having coffee and tea—I could go for either one. On the other hand, some people have no preference relations between The Rolling Stones and Mozart. It’s not that they are indifferent between them—it’s rather that they think that they each have different properties that make them incommensurable—they just cannot compare one to the other. For better or worse, Definition 7.4 prohibits these incommensurable outcomes. Normal humans also sometimes have intransitive preferences. Suppose you add a single grain of sugar to successive cups of coffee. Normal humans will be indifferent between neighboring cups of coffee, but will definitely have a preference between the first and the last.
What does the fact that we violate some of these conditions mean for the theory of decision that presupposes that we don’t? We won’t deal with this issue here at all, except to make the following two points: first, decision theory is at least partly normative—it tells us not how our preferences actually are ordered, but how they should be ordered if we are to make rational decisions; second, there is some comfort in the fact that we can make rational decisions even when we don’t satisfy all the structural conditions on preferences—for instance, even if your preferences are not connected among all the possible choices for dessert, but you know you can have your first choice, the lack of connectedness doesn’t matter for that particular choice.
If an agent’s preferences do satisfy all the structural conditions on preferences, we will say that their preferences form a weak order:
Definition 7.5. Weak order: An agent’s preferences form a weak order if and only if they are connected, asymmetric, and transitive.
If your preferences form a weak order, then they can be represented by a ranking. The agent’s preferences in the movies/park case, for example, can be represented in the following ranking (ordered from best to worst top to bottom):
Of course, the structural definitions allow ties, but in this case the agent is not indifferent between any of the outcomes.
Once we have a ranking (with ties allowed), we can assign a number to each outcome reflecting its place in the ranking, with higher numbers assigned to higher-ranked outcomes. For instance, we could have the following numbers assigned to the outcomes for the movies/park case:
Other than assigning higher numbers to higher-ranked outcomes, we are free to assign whatever number we wish. For instance, this other assignment is just as good as the previous one:
- 2000
- 1395
- 2
- -12
We call the number assigned to an outcome its utility, and we write it . If the utility function satisfies the requirement just laid out, we say that it is an ordinal utility function:
Definition 7.6. Ordinal utility function: A utility function is ordinal if and only if, for any two outcomes and :
- a if and only if ; and
- b if and only if .
The numbers in an ordinal utility function represent nothing more and nothing less than the order of the corresponding outcome in the ranking of the agent’s preferences. If is a correct ordinal representation of an agent’s preferences, then any ordinal transformation of will also be a correct ordinal representation of those preferences:
Definition 7.7. Ordinal transformation: is an ordinal transformation of if and only if, for every two outcomes and : if and only if
7.3 Choice under ignorance
Decision theorists usually distinguish between three different kinds of decisions: decisions under certainty, decisions under ignorance, and decisions under risk. A decision counts as a decision under certainty when the agent is certain which state in the partition obtains. Decision under certainty need not be computationally easy (the mathematical theory of linear programming has been fruitfully applied to this topic), but theoretically does not give rise to very interesting topics, and therefore we shall not dwell on it any more.
When agents cannot eliminate all but one of the states in the relevant partition, but can assign probabilities to those states, we are dealing with decisions under risk. We will deal with those kinds of decisions in the next chapter. Finally, when subjects cannot even assign meaningful probabilities to the states of the relevant partition, we are dealing with decisions under ignorance. That is what we will be dealing with in the rest of this chapter.
7.3.1 Dominance
Suppose that, in the movies/park example, we have no idea whether it will rain or not. Having no idea is very different from being 50/50 on whether it will rain or not—the latter would be a decision under risk, whereas if we genuinely have no idea then we are facing a decision under ignorance. You might think that there is not much that can be said about how to go about making decisions when we are completely ignorant regarding the probabilities of the different possible states, but you’d be wrong. Consider, for instance, a slight variation of our movies/park example, one where your ranking (together with a possible ordinal utility function) is instead as follows:
The matrix corresponding to that ranking is then the following:
Try it in the Decisions Lab Step through dominance on this matrix.
Something remarkable about this ranking of outcomes is that, no matter whether it rains or not, you prefer to go to the park than to go to the movies. So, even if you are completely ignorant of whether it is going to rain or not, you should go to the park: you will then get your preferred outcome no matter what state the world is in.
When that happens—i.e., when one choice results in a better outcome than all the other ones no matter what state obtains—we say that the choice in question dominates the other. We will define both a strong and a weak form of dominance:
Definition 7.8. Strong dominance: Option strongly dominates option if and only if, for every state ,
Definition 7.9. Weak dominance: Option weakly dominates option if and only if, for every state , and for at least one state ,
And, on the basis of those definitions, we will propose our first norm for decisions:
Norm 7.10. Dominance norm: dominated options are irrational
Notice that the dominance norm doesn’t tell us which option is rational, it only tells us which are irrational—namely, the dominated ones. In the special case where there are only two options, if one of them is dominated then the Dominance norm entails that the other one is the only rationally permissible one. But for cases with more than two options, the fact that one is dominated does not immediately tell you which option is rationally required.
If for two given choices neither one of them is dominated by the other, there are still two other possibilities: it may be that they have the same utility for every state (in which case we’ll say that they are equivalent), or it may be that one of them has a higher utility for at least one state, and the other one has a higher utility for at least one other state (in which case we’ll say that they are crossing):
Definition 7.11. Equivalent: Options and are equivalent if and only if, for every state ,
Definition 7.12. Crossing: Options and are crossing if and only if there is some state such that and some state such that
To see the dominance norm at work, consider the following three decision problems, each involving three choices and three states.
Try it in the Decisions Lab Step through dominance on this matrix.
We start looking for choices that dominate others by going through them pairwise—that is, we consider each possible pair of choices, ignoring order:
- :
-
weakly dominates : going through each state in turn, (since ), (since ), and (since ).
- :
-
strongly dominates : (since ), (since ), and (since ).
- :
-
weakly dominates : (since ), (since ), and (since ).
The fact that dominates does not mean that we should pick , since is itself dominated by . Applying the dominance norm is a matter of ruling out choices until, hopefully, only one is left. Once we see that is dominated, we can cross it off as something we would never want to do; once we see that is also dominated, by , we cross it off too. That leaves us with as the only remaining choice: dominance reasoning has identified as the single, or unique, solution to the decision problem. This is the best case for dominance reasoning, but things do not always work out so cleanly.
Try it in the Decisions Lab Step through dominance on this matrix.
In Table 7.6, weakly dominates : (since ), while (since ) and (since ). We can immediately cross off , which is convenient, since it means we do not even need to check how relates to . We still need to compare and , however, and what we find is that neither dominates the other: their utilities agree at every state, so by Definition 7.11 they are equivalent. Dominance reasoning cannot, then, deliver a unique solution to this decision problem—all it can tell us is that we should not pick . That is not so bad in this case, though, since it is clear that we should pick either or , and it does not matter which.
Try it in the Decisions Lab Step through dominance on this matrix.
It is harder to be so sanguine about Table 7.7. Looking at alone tells us that does not dominate either of the other choices, since it has the lowest utility at that state; and looking at alone tells us that cannot be dominated by either of the other choices, since it has the highest utility there. , however, is strongly dominated by : (since ), (since ), and (since ); so we can safely cross it off. That leaves us with and , and comparing them shows that (since ) while (since ): by Definition 7.12, and are crossing. All that dominance reasoning can tell us here is that we should not pick ; it is silent about how to choose between and , and unlike in Table 7.6 we cannot reassure ourselves that this silence is harmless, since and are not equivalent—it plainly matters which of the two we end up choosing.
7.3.2 Maximin
Whereas the dominance rule only tells us which choices are irrational, there are other rules for decision under ignorance which tell us which choices are rational. One of them, the maximin norm, tells us to maximize the minimum utility. To do this, we have to first figure out, for each choice, what is the minimum utility it can give us. Then we see which choice has the maximum of these minima. Of course, there may be ties. In that case, we can once again apply the maximin norm to the remaining utilities (eliminating first the minima from the previous steps). If we write for the minimum utility that option yields across all states —that is, ’s worst-case utility—then the two norms are the following:
Norm 7.13. Maximin norm: Option is (maximin-)rational if and only if for every other option .
Norm 7.14. Leximin norm: Let be the utilities of across all states, arranged from lowest to highest. Option is leximin-rational if and only if, for every other option , either or there is some such that and for every .
To see the maximin norm at work, consider the following decision problem, involving four options and four states. For each option we star its minimum utility, and we double-star the option whose minimum is itself the maximum among all the minima.
Try it in the Decisions Lab Step through maximin and leximin on this matrix.
Going through Table 7.8 choice by choice, the minimum utility of is , the minimum utility of is , the minimum utility of is , and the minimum utility of is . Of those four minima, is the largest, and it belongs to alone: no other option has a worst case that good. So, by Norm 7.13, is the uniquely maximin-rational choice.
Sometimes, though, maximin does not settle on a unique choice, because two or more choices are tied for the best worst case. When that happens, we turn to leximin, which, as we saw, tries to break the tie by looking at the second-worst utility of each of the tied choices (and, if they are tied there too, the third-worst, and so on). But even leximin can fail to settle on a unique choice, as the following decision problem, involving four choices and five states, illustrates.
Try it in the Decisions Lab Step through maximin and leximin on this matrix.
In Table 7.9, every choice has the same minimum utility, , so maximin alone cannot distinguish between them. To apply leximin, we arrange each choice’s utilities from lowest to highest: ’s become , ’s become , ’s become , and ’s become . Comparing these term by term, all four choices agree on the lowest utility () and on the second-lowest (), so the tie is not yet broken. At the third-lowest utility, however, (with ) and (with ) both fall below and (which each have there), so by Norm 7.14 both and are eliminated. That leaves and , and, as it turns out, their utilities sorted from lowest to highest are exactly the same: in both cases. So leximin narrows the field down to and , but—since their sorted utilities agree at every position—it cannot tell us which of the two to choose.
7.3.3 Minimax regret
Consider what happens when we try to apply maximin to the following table:
Strictly speaking, we cannot apply any decision rule to Table 7.10 because it has amounts of money instead of utilities in the cells. But for anyone whose preferences over money are more or less normal, the corresponding utilities will have the result that the maximin norm tells us to go with because it has the maximum minimal utility possible (50 cents more than ). That is to say, the maximin rule is telling us to forgo an opportunity to gain $10,000 in order to save $0.50. Many would find that ridiculous. Those who do think that the regret you would feel if obtains and you miss out on $10,000 is much greater than the relief you would feel if obtains and you gain $0.50.
Let us formalize this view a bit. Let us define to be the highest utility that any of the available options yields in state —i.e., the largest number in that state’s column: . Then the regret associated with is the result of subtracting that value from the maximum for that state: .
This suggests a new norm: instead of maximizing the minimum utility, minimize the maximum regret.
Norm 7.15. Minimax regret norm: Option is minimax-regret-rational if and only if for every other option .
Returning to Table 7.10: and . So ’s regret is $0 in (since it achieves the maximum there) and $9,998.25 in , for a maximum regret of $9,998.25; while ’s regret is $0.50 in and $0 in , for a maximum regret of only $0.50. By Norm 7.15, then, minimax regret recommends —the reverse of what maximin recommended, and, to many people, the more sensible verdict.
To see how a regret table is built from a utility table, and how minimax regret is applied to it, consider the following decision problem, involving four choices and three states.
Try it in the Decisions Lab Step through minimax regret on this matrix.
We build the regret table one state at a time. In , the highest utility anyone achieves is (by ), so ; in , the highest is (by ), so ; and in , the highest is (by ), so . Subtracting each choice’s utility from the relevant gives us the regret table below, where we star each choice’s maximum regret and double-star the choice whose maximum regret is itself the minimum among all the maxima.
Going through Table 7.12 choice by choice, the maximum regret of is , the maximum regret of is , the maximum regret of is , and the maximum regret of is . Of those four maxima, is the smallest, and it belongs to alone. So, by Norm 7.15, is the uniquely minimax-regret-rational choice.
Now, maximin (and leximin) have a notable virtue that minimax regret lacks: they only need to know how the outcomes are ranked, not by how much one outcome is better than another. If we transform every utility in a decision problem by any function that preserves the original ranking of the outcomes—an ordinal transformation—maximin and leximin will always recommend the very same choice, since they only ever compare utilities using , , or , and no ordinal transformation can change the truth value of any of those comparisons. Minimax regret, however, essentially involves subtraction, and ordinal transformations do not, in general, preserve differences between numbers. So minimax regret’s recommendation can change under a transformation that leaves maximin’s recommendation exactly where it was.
To see this, consider what happens if we square every utility in Table 7.11—a legitimate ordinal transformation, since squaring is strictly increasing on positive numbers, and hence preserves the ranking of every outcome.
Try it in the Decisions Lab Step through minimax regret on this matrix.
The minimum utility of is now , of is , of is , and of is —so, just as before, has the largest minimum, and maximin still uniquely recommends . Minimax regret, however, tells a different story. Now , , and , which gives us the following regret table.
The maximum regrets are now for , for , for , and for —so minimax regret now uniquely recommends , the very choice it had previously ranked worst but one. Simply squaring the utilities—without changing how any outcome is ranked relative to any other—was enough to flip minimax regret’s recommendation from to , even though maximin’s recommendation did not budge. This shows that minimax regret, unlike maximin and leximin, is not a purely ordinal norm: it presupposes that differences in utility are themselves meaningful, and can be compared to one another.
Minimax regret has a second, and arguably more troubling, defect: it can violate a principle we might call independence of irrelevant alternatives—roughly, the idea that whether or is recommended should not depend on which other choices happen to also be on the table, so long as neither of those other choices is itself the one recommended. To see minimax regret violate this principle, suppose our decision problem initially involves just two choices, and , and two states.
Try it in the Decisions Lab Step through minimax regret on this matrix.
Here and , so ’s regret is , for a maximum of , while ’s regret is , for a maximum of . By Norm 7.15, minimax regret recommends . Now suppose a third choice, , becomes available—one that does very well in and very poorly in .
Try it in the Decisions Lab Step through minimax regret on this matrix.
Adding raises from to , which increases everyone’s regret in . ’s regret is now , for a maximum of ; ’s regret is now , for a maximum of ; and ’s own regret is , for a maximum of . Minimax regret now recommends , not —even though , the newcomer, is not itself recommended, and nothing about ’s or ’s own prospects changed at all. It would not be strange if adding to the menu sometimes changed the recommendation to itself—that would just mean turned out to be a good choice. What is strange is that merely adding can flip the verdict between the two choices that were already on the table, even though never had a chance of being recommended in the first place.
Finally, minimax regret can recommend choices that seem straightforwardly wrong. Consider a decision problem with a hundred states, and two choices: , which yields in and nothing at all in every other state, and , which yields nothing in but in every one of the other ninety-nine states.
Intuitively, looks like the obviously better choice: it guarantees a solidly good outcome in ninety-nine out of a hundred states, while only pays off in a single state, and leaves us with nothing in all the rest. But minimax regret disagrees. and for every . So ’s regret is in and in every other state, for a maximum regret of ; while ’s regret is in and everywhere else, for a maximum regret of . Since , minimax regret recommends —the choice that, intuitively, we should avoid.
There is, however, a standard reply to this last objection: given that we are dealing with a decision under ignorance, we have no idea about the relative probabilities of the 100 states. For all we know, has a 99.9999% chance of obtaining, while the remaining 99 states have almost 0 chance each. If, on the other hand, we do have at least some idea of the probabilities of the states, then we are dealing with a decision under risk, which is the topic of our next chapter.
Exercises
What happens to an ordinal scale when it is transformed by multiplying each number by ? By 0? Are these acceptable transformations of the original scale? Why or why not?
Show solution
Multiplying by reverses the order of the outcomes, and multiplying by makes every outcome tie, so neither transformation preserves the ordinal ranking. Neither is acceptable.
Find the maximin act for the tables below. Use leximin to break ties.
Practice in the Decisions Lab Work out maximin and leximin yourself, and the lab checks your answer.
Practice in the Decisions Lab Work out maximin and leximin yourself, and the lab checks your answer.
Show solution
(a) Maximin: the row minima are , , , so choose . (b) The acts tie on their minima, so apply leximin: the sorted utility vectors are , , , and is lexicographically best.
Form the regret tables and find the minimax regret acts for the tables below.
Practice in the Decisions Lab Work out minimax regret yourself, and the lab checks your answer.
Practice in the Decisions Lab Work out minimax regret yourself, and the lab checks your answer.
Show solution
(a) Regrets: , max ; , max ; , max . Minimax regret chooses . (b) Regrets: , max ; , max ; , max . Minimax regret chooses .