Rational heuristics for one-shot games.

RATIONAL HEURISTICS FOR ONE-SHOT
GAMES
Frederick CallawayyThomas L. Grithsz
Gustav Karreskog Rehbinderx
May 9, 2023
Abstract
We present a theory of human behavior in one-shot interactions based on
the assumption that people use heuristics that optimally trade o the expected
payo and the cognitive cost. The theory predicts that people's behavior depends
on their past experience; specically, they make choices using heuristics that
performed well in previously played games. We conrm this prediction in a
large, preregistered experiment. The rational heuristics model provides a strong
quantitative account of participant behavior, and outperforms existing models.
More broadly, our results suggest that synthesizing heuristic and optimal models
is a powerful tool for understanding and predicting economic decisions.
Keywords: Bounded rationality, Experiments, Cognitive cost, Strategic
thinking, Game theory, One-shot games, Heuristics
JEL classication: C72, C90, D83, D01
We thank Drew Fudenberg, Alice Hallman, Benjamin Mandl, Erik Mohlin, Isak Trygg Kupersmidt,
J orgen Weibull, Peter Wikman, and seminar participants at SSE, Uppsala University, UCL, NHH, and
Princeton, for helpful comments and insights. This work was supported by the Templeton Foundation,
the NOMIS Foundation, the Jan Wallander and Tom Hedelius Foundation, and the Knut and Alice
Wallenberg Research Foundation.
yDepartment of Psychology, Princeton University, Princeton, NJ 08540; fredcallaway@princeton.edu
zDepartment of Psychology, Princeton University, Princeton, NJ 08540; tomg@princeton.edu
xDepartment of Economics, Uppsala University, Kyrkog ardsgatan 10, 751 20, Uppsala, Sweden;
gustav.karreskog@nek.uu.se
1 Introduction
A key assumption underlying classical economic theory is that people behave optimally
in order to maximize their subjective expected utility (Savage, 1954). However, a
large body of work in behavioral economics shows that human behavior systematically
deviates from this rational benchmark in many settings (Dhami, 2016). This suggests
that we can improve our understanding of economic behavior by incorporating more
realistic behavioral components into our models. While many of these deviations
are indeed systematic and show up in multiple studies, the estimated biases vary
considerably across studies and contexts. Apparent biases change or even disappear if
participants have opportunities to learn or if the details of the decision task change.
For example, this is the case with the endowment eect (Tun cel and Hammitt, 2014),
loss aversion (Ert and Erev, 2013), numerosity underestimation (Izard and Dehaene,
2008), and present bias (Imai et al., 2020).
In order to incorporate behavioral eects into theories with broader applications|
without having to run new experiments for each specic setting|we need a theory that
can account for this variation. That is, we need a theory that can help us understand
why|and predict when|people deviate from the rational benchmark. In this paper,
we propose such a theory based on the idea that people use simple decision procedures,
orheuristics , that are optimized to the environment to make the best possible use of
their limited cognitive resources and thereby maximize utility. This allows us to predict
behavior by analyzing which heuristics perform well in which environments. This paper
presents an explicit instantiation of this theory tailored to one-shot games and tests it
experimentally.
In situations where people play the same game multiple times against dierent
opponents, and hence there is an opportunity to learn, both theoretical and experi-
mental work suggests that Nash equilibria can often yield sensible long-run predictions
(Fudenberg et al., 1998; Camerer, 2003). However, in experimental studies of one-shot
games where players don't have experience with the particular game at hand, people
seldom follow the theoretical prediction of Nash equilibrium play (see Crawford et al.,
2013, for an overview). Consequently, we need an alternative theory for strategic
interactions that happen only once (or infrequently).
The most common theories of behavior in one-shot games in the literature assume
that players perform some kind of iterated reasoning to form beliefs about the other
player's action and then select the best action in response. This includes level-k (Nagel,
1995; Stahl and Wilson, 1994, 1995), cognitive hierarchy (Camerer et al., 2004), and
1
noisy introspection models (Goeree and Holt, 2004). In such models, participants
are characterized by dierent levels of reasoning. Level-0 reasoners behave naively by
playing a uniformly random strategy. Level-1 reasoners best respond to level-0 behavior,
while higher-level reasoners best respond to behavior based on lower-level reasoning. In
meta-analyses such as Crawford et al. (2013), Wright and Leyton-Brown (2017), and
Fudenberg and Liang (2019), variations of these iterated reasoning models best explain
human behavior.
All iterated reasoning models assume the basic structure of belief formation and
best responding to those beliefs. However, such a belief-formation and best-response
process is often inconsistent with empirical evidence. For example, Costa-Gomes and
Weizs acker (2008) found that participants who were asked to state their beliefs about
how the opponent would play, often failed to play a best response to those beliefs.
Moreover, eye-tracking studies have revealed that the order in which participants attend
to payos in visually presented normal-form games is inconsistent with a belief-formation
and best-response process (Polonio et al., 2015; Devetag et al., 2016; Stewart et al.,
2016). Furthermore, the estimated parameters of iterated reasoning models often vary
considerably across dierent data sets (Wright and Leyton-Brown, 2017), behavior
depends on aspects of the game that these models do not take into account (Bardsley
et al., 2010; Heap et al., 2014), and there is evidence that games played previously
have an eect on behavior, which the above static models fail to capture (Mengel and
Sciubba, 2014; Peysakhovich and Rand, 2016).
In this paper, we present a theory of human behavior in one-shot games based on
the rational use of heuristics (Lieder and Griths, 2017, 2020). That is, we assume
that people use simple cognitive strategies that
exibly and selectively process payo
information to take good decisions with minimal cognitive eort. Concretely, we
assume that people use heuristics that maximize expected payo minus cognitive cost.
Importantly, this optimization happens at the level of the environment; although people
might not choose the best action in a given game, they will learn which heuristics
generally work well (cf. procedural rationality in Simon, 1976).
Thus, our approach combines two perspectives on human decision-making, embracing
both the notion that human behavior is adaptive in a way that can be described as
optimizing and the notion that people use simple strategies that are eective for the
problems they actually need to solve. The key assumption of this resource-rational
analysis approach is that people use cognitive strategies that make optimal use of their
limited computational resources (Lieder and Griths, 2020; Griths et al., 2015; cf.
Howes et al., 2009; Lewis et al., 2014; Gershman et al., 2015).
2
It is instructive to compare resource-rational analysis with two other approaches
to explaining observed deviations from perfectly rational behavior: the information-
theoretic and ecological rationality approaches. Like information-theoretic approaches
such as rational inattention (Mat ejka and McKay, 2015; Sims, 1998; Caplin and Dean,
2013; Hebert and Woodford, 2019; Steiner et al., 2017), the resource-rational approach
assumes that the costs and benets of information processing are optimally traded o.
However, while information-theoretic approaches typically assume domain-general cost
functions (e.g., based on entropy reduction), the resource-rational approach typically
makes stronger assumptions about the specic computational processes and costs that
are likely to be involved in a given domain. In this way, the resource-rational approach
is more similar to the ecological rationality approach, a framework based on the idea
that people use computationally frugal heuristics, which are highly eective for the
kinds of problems that people actually encounter (Gigerenzer and Todd, 1999; Goldstein
and Gigerenzer, 2002; Todd and Gigerenzer, 2012). For example, if the other players in
an environment are using a wide variety of decision strategies, then a heuristic that
ignores the other players' payos entirely may perform best (Spiliopoulos and Hertwig,
2020). However, while proponents of ecological rationality explicitly reject the notion of
optimization under constraints (e.g., Gigerenzer and Todd, 1999, Ch. 1), optimization
is at the heart of resource rationality. This makes it possible to predict when people will
use one heuristic rather than another (Lieder and Griths, 2017) and even to discover
novel heuristics (Lieder et al., 2017; Krueger et al., 2022).
One important commonality between our approach and ecological rationality is the
recognition that the quality or adaptiveness of a heuristic depends on the environment
in which it is used. For example, in an environment in which most interactions are
characterized by competing interests (e.g., zero-sum games), a good heuristic is to look
for actions with high guaranteed payos. On the other hand, if most interactions have
common interests, a better heuristic might be to look for outcomes that would be good
for everyone (cf. Spiliopoulos and Hertwig, 2020). Our theory thus predicts that people
will use dierent heuristics in cooperative vs. competitive environments.
To test our theory's prediction that people adapt their heuristics to the environment,
we conduct a large, preregistered1behavioral experiment. In our experiment, partic-
ipants play a series of normal-form games in one of two environments characterized
by dierent correlations in payos. In the common-interests environment, there is
a positive correlation between the payos of the two players over the set of strategy
proles; i.e., outcomes that are good for one player tend to be good for the other as
1https://osf.io/hcnzg
3
well. In the competing-interests environment, the payo correlation is negative; i.e., one
player's loss is the other's gain, which is essentially a soft version of zero-sum games.
Interspersed among these treatment-specic games, we include four comparison games
that are the same for both treatments. If the participants are using environment-adapted
heuristics to make decisions, and dierent heuristics are good for common-interests
and competing-interests environments, the participants should behave dierently in the
comparison games since they are employing dierent heuristics. Indeed, this is what we
observe.
To provide further support for the claim that participant behavior is consistent with
an optimal tradeo between the expected payo and the cognitive cost, we dene two
parameterized families of heuristics and cognitive costs that can make quantitative
predictions about the distribution of play in each game. However, rather than identifying
the parameters that best t human behavior (as is commonly done in model comparison),
we instead identify the parameters that strike an optimal tradeo between expected
payos and cognitive costs, and ask how well they predict human behavior. Although
we t the cost function parameters that partially dene the resource-rational heuristic,
these parameters are t jointly to data in both treatments. Strikingly, we nd that
this model, which has no free parameters that vary between the treatments, achieves
nearly the same out-of-sample predictive accuracy as the model with all parameters
t separately to each treatment. Both the optimized and tted versions of this model
predicted the modal action with an accuracy of 88%, compared to 80% for a quantal
cognitive hierarchy model.
In Section 2, we provide an overview of our theory and present a stylized example to
illustrate how resource-rational heuristics depend on the structure of the environment.
In Section 3, we present our experiment and model-free analyses, which demonstrate a
strong causal link between previous experience and current behavior in one-shot games.
In Section 4, we provide a more detailed description of the theory and introduce two
dierent parameterized models, an interpretable, low-parameter model, and a black-box
neural network model. Using these models, we demonstrate in Section 5 that the
dierences in behavior can be accurately predicted out-of-sample by assuming that
participants use the optimal heuristics for the respective environments. In Section 5.4,
we compare our models to alternative models, including quantal cognitive hierarchy
and prosocial preference models, and show that our models provide better predictions
of behavior than these alternatives.
4
2 Theory Overview and Stylized Predictions
The central tenet of our theory is that individuals use heuristics that maximize the
expected payo minus the cognitive cost in a given environment. This can be summarized
in the following equation:
h= argmax
hi2HEE
i
hi(G);hi(GT)
c(hi)
: (1)
Here,hi2H is a heuristic, Gis a game, and hi(G) is the distribution of play
produced by applying heuristic hito gameG. The optimal heuristic his the one
that maximizes the expected payo i
hi(G);hi(GT)
minus the cognitive cost c(hi),
wherehi(GT) gives the distribution of play by the opponent. The expectation is taken
with respect to an environment E, which denes a distribution over possible games G
and opponent heuristics hi.
A key implication of this theory is that the heuristics we expect people to use depend
on the types of games and opponents they encounter frequently; that is, h
idepends on
E. Below, we illustrate this idea with a simple example.
Consider two possible environments: one consisting entirely of coordination games
(where the players want to coordinate on the same action), and one consisting entirely
of constant-sum games (where the players' interests are exactly opposed). In both
environments, all other players follow a heuristic where they pick the strategy with
the highest average payo (level-1 in the language of level-k reasoning). Now consider
what you would do as the row player when faced with the following games from each
environment.
8;80;0
0;09;6
Coordination game5;42;7
3;63;6
Constant-sum game
In the coordination game, the column player will select column 1because 8 is
larger than 6; row 1is thus the optimal play. In the constant-sum game, the column
player will select column 2because 7 + 6 >4 + 6; thus, row 2is the optimal play.
Clearly, simulating the other player as we have done here will always lead to the optimal
choice. However, in each case, the optimal action could also be found by a simpler, less
cognitively demanding heuristic. In the coordination game, the best action is the one
that produces the outcome with the highest minimum value for each player (we will
later call this the \jointmax heuristic"). In the constant-sum game, the best action is
5
the one that has the highest guaranteed payo (the \maximin" heuristic).
The central claim of our theory is that people will use heuristics that identify good
actions with minimal cognitive cost. Critically, a \good" action is one that achieves
high payos on average across all the games a person encounters. Thus, if we take one
person, \Lucy," and we put her in an environment where she repeatedly plays games
like the one on the left, she will learn to use the jointmax heuristic because it usually
selects the same action as simulation, but with less cognitive cost. If we put another
person, \Rodney," in an environment where he repeatedly plays games like the one on
the right, he will learn to use the maximin heuristic for the same reason. Now consider
what actions each will select in a new game:
7;70;9
9;04;4
Prisoner's dilemma game
Here, the second action strictly dominates the rst, and so it has to be the choice of
a perfectly rational decision-maker. Rodney will play this action, as it is selected by the
maximin heuristic, which has performed well in his previous experience. Importantly, he
may choose this action without ever realizing that it dominates the other. By contrast,
Lucy will be likely to play the rst, \incorrect" action, as it is selected by the jointmax.
She makes this mistake because identifying the outcome that is best for both players
is easy, and it has worked well for her before. Although she might have fared better
on this specic game if she had simulated the possible outcomes of each action, the
cognitive cost of such an approach would not be justied by the relatively small increase
in payo across the full set of games she has played.
To summarize, the principled but costly approach of simulating the other player in
order to select one's own action can sometimes be approximated by simpler heuristic
strategies. When this approximation is suciently accurate, a resource-rational agent
will use the heuristic to avoid the mental eort of simulation. But if we present the
unwitting agent with a new game that lacks the structure the heuristic was taking
advantage of, the agent will make predictable errors. This is the key intuition underlying
our behavioral experiment.
6
3 Experiment
Our overarching hypothesis is that individuals choose actions in one-shot games using
heuristics that optimally trade o between the expected payo and the cognitive cost.
Critically, as discussed above, this optimization occurs with respect to an environment
rather than a single game. This results in a central prediction: the action a player takes
in a given game will depend not only on the nature of that particular game but also on
the other games she has previously played. From this central prediction, we derived
four hypotheses, which we tested in a large, preregistered online experiment.
3.1 Methods
We recruited 600 participants on Amazon Mechanical Turk using the oTree platform
(Chen et al., 2016). Each participant was assigned to one of 20 populations of 30
participants. They then played 50 dierent one-shot normal-form games, with each
participant randomly matched to another player from their population after each game.2
Each population was assigned to one of two experimental treatments, which deter-
mined the distribution of games played. Specically, we manipulated the correlation
between the row and column players' payos in each cell (cf. Spiliopoulos and Hertwig,
2020). In the common-interests treatment, the payos were positively correlated, such
that a cell with a high payo for one player was likely to have a high payo for the
other player as well. By contrast, in the competing-interests treatment, the payos were
negatively correlated, such that a cell with a high payo for one player was likely to have
a low payo for the other player. Concretely, the payos in each cell were sampled from
a bivariate Normal distribution truncated to the range [0 ;9] and discretized such that
all payos were single-digit nonnegative integers.3Examples of each type of treatment
game are shown in Tables 1 and 2.
5;66;45;3
9;45;56;7
2;00;16;43;45;59;7
4;25;75;7
2;42;12;39;75;97;8
6;79;94;6
6;43;16;2
Table 1: Three games from the common-interests treatment.
2To facilitate running the experiment online, we used an asynchronous scheme in which participants
could play \against" an opponent who had played the game earlier. Participants were informed of this;
see Figure 8 in Appendix A.
3The normal distribution is given by N((5;5);) with  = 51
1
, where= 0:9 in the
common-interests treatment and =0:9 in the competing-interests treatment.
7
5;56;25;3
5;31;88;4
3;67;44;62;44;44;6
1;72;69;1
7;14;88;64;51;57;1
2;78;55;7
2;68;33;9
Table 2: Three games from the competing-interests treatment.
For each population, we sampled 46 treatment games, with each participant playing
every game once. The remaining four games were comparison games , i.e., treatment-
independent games that we used to compare dierences in the participants' behavior
between the two treatments. The comparison games were played in periods 31, 38,
42, and 49. We located these comparison games later in the experiment so that the
participants would have time to adjust to the treatment environment rst, while leaving
intervals between them to minimize the chance that participants would notice that
these games were dierent from the others they had played.
3.1.1 The Comparison Games
We selected four comparison games that we expected to elicit dramatically dierent
distributions of play in the two treatments. In these games, there is a tension between
choosing a row with an ecient outcome or choosing a row with a high guaranteed
payo. For two of the games, the ecient outcome was also a Nash equilibrium (NE),
and for the other two games, the ecient outcome was not a NE.
8;82;60;5
6;26;62;5
5;05;25;5
Comparison game 18;82;91;0
9;23;31;1
0;11;11;1
Comparison game 2
4;44;65;0
6;43;35;1
0;51;59;9
Comparison game 34;49;11;3
1;98;81;8
3;18;13;3
Comparison game 4
Table 3: The four comparison games.
The rst game is a weak-link game, where all the diagonal strategy proles are
Nash equilibria, but each has a dierent eciency. The most ecient NE yields the
payos (8,8), but it is also possible to get 0. The least ecient equilibrium yields the
8
payos (5,5), but 5 is also the guaranteed payo. The equilibrium (6,6) is in between
the aforementioned payos in terms of both risk and eciency. The third row has the
highest average payo and is the best response to itself, and so any standard recursive
reasoning model would predict (5,5) being the outcome.
The second comparison game is a normal prisoner's dilemma game, with an added
dominated and inecient strategy. In this game, strategy 2dominates the other
strategies. However, we still expect strategy 1to be played more often in the common-
interests treatment since, overall, it is a good heuristic to look for ecient outcomes in
that environment.
The third comparison game is a game with two NE, where one is the pure NE with
both players playing strategy 3, and the other is a mixed NE involving 1and2. This
game is constructed so that the row averages are much higher for strategies 1and2
than for 3, meaning that any level-k heuristic would result in strategy 1or2being
played, while the NE yielding (9 ;9) is much more ecient. Thus, there is a strong
tension between the ecient payo and the guaranteed payo.
In the fourth comparison game, the risky ecient outcome (8 ;8) is not a NE. A
standard level-k player of any level higher than 0 would play strategy 3.
3.2 Model-free Results
We organize our results based on four preregistered hypotheses. The rst two are
model-free and concern behavior in the comparison games; they are presented here.
The next two are model-based and concern behavior in the treatment games; these will
be presented later.
Our rst hypothesis is that the treatment environment has an eect on behavior in
the comparison games.
Hypothesis 1. The distribution of play in the four comparison games will dier between
the two treatments.
This hypothesis follows from the assumption that people learn to use heuristics that
are adaptive within their treatment and that dierent heuristics are adaptive across
the two treatments. Figure 1 visually conrms this prediction, and Table 4 conrms
that these dierences are statistically signicant ( 2-tests, as preregistered).
Inspecting Figure 1, we see that the distribution of play is not just dierent between
the two groups; it is dierent in a systematic way. In particular, players in the common-
interests treatment tend to coordinate on the ecient outcome, even in games 2 and 4,
where the ecient outcome is not a Nash equilibrium. We expected this divergence
9
8, 8 6, 2 5, 0
2, 6 6, 6 5, 2
0, 5 2, 5 5, 5Common Interests
8, 8 6, 2 5, 0
2, 6 6, 6 5, 2
0, 5 2, 5 5, 5Competing Interests
8, 8 9, 2 0, 1
2, 9 3, 3 1, 1
1, 0 1, 1 1, 1
8, 8 9, 2 0, 1
2, 9 3, 3 1, 1
1, 0 1, 1 1, 1
4, 4 6, 4 0, 5
4, 6 3, 3 1, 5
5, 0 5, 1 9, 9
4, 4 6, 4 0, 5
4, 6 3, 3 1, 5
5, 0 5, 1 9, 9
4, 4 1, 9 3, 1
9, 1 8, 8 8, 1
1, 3 1, 8 3, 3
4, 4 1, 9 3, 1
9, 1 8, 8 8, 1
1, 3 1, 8 3, 3 Human
8, 8 6, 2 5, 0
2, 6 6, 6 5, 2
0, 5 2, 5 5, 5Common Interests
8, 8 6, 2 5, 0
2, 6 6, 6 5, 2
0, 5 2, 5 5, 5Competing Interests
8, 8 9, 2 0, 1
2, 9 3, 3 1, 1
1, 0 1, 1 1, 1
8, 8 9, 2 0, 1
2, 9 3, 3 1, 1
1, 0 1, 1 1, 1
4, 4 6, 4 0, 5
4, 6 3, 3 1, 5
5, 0 5, 1 9, 9
4, 4 6, 4 0, 5
4, 6 3, 3 1, 5
5, 0 5, 1 9, 9
4, 4 1, 9 3, 1
9, 1 8, 8 8, 1
1, 3 1, 8 3, 3
4, 4 1, 9 3, 1
9, 1 8, 8 8, 1
1, 3 1, 8 3, 3 ModelFigure 1: Distribution of play in the four comparison games. Each panel shows the
joint and marginal distributions of row/column plays in a single game. The cells show
the players' payos for the given outcome. The two columns to the left show the
actual behavior in the two environments, while the two columns to the right show the
predictions of the rational (optimized) metaheuristics.
10
Frequencies
1 2 3 2p-value
Comparison Game 1 98.39p<: 001
Common interests 193 53 54
Competeting interests 75 82 143
Comparison Game 2 22.08p<: 001
Common interests 160 139 1
Competeting interests 103 195 2
Comparison Game 3 61.75p<: 001
Common interests 40 73 187
Competeting interests 106 97 97
Comparison Game 4 91.36p<: 001
Common interests 78 173 49
Competeting interests 115 62 123
Table 4:2tests for each comparison game. The results are signicant at the preregis-
tered 0.05 level.
in behavior when we constructed the comparison games, which motivates our second
hypothesis.
Hypothesis 2. The average payo in the four comparison games will be higher in the
common-interests treatment than in the competing-interests treatment.
The comparison games were designed to create tension between eciency and risk,
with one outcome having a high payo for both players, but requiring each player to take
an action that could yield a very low payo. We expected that the common-interests
players would be more likely to coordinate on the ecient outcome, and therefore
receive higher payos. The model makes this prediction because identifying mutually
benecial outcomes is typically an eective heuristic in common-interests games, while
identifying high guaranteed payos is an eective heuristic in competing-interests games.
Table 5 conrms this prediction. The common-interests players had a higher average
payo in all four comparison games, and the dierence is statistically signicant in each
case (at the preregistered level of p<: 05).
4 Theoretical Framework
We have seen that dierent environments lead to drastically dierent behavior in the
comparison games. While these results are suggestive of rational adaptation, they do
not directly imply that the participants are using heuristics in an optimal way. One
way to strengthen this claim is to show that one can accurately predict human behavior
11
Treatment average payo
Common interests Competing interests t-value p-value
Comparison game 1 5.09 3.64 6.851 p<: 001
Comparison game 2 5.52 4.04 6.28 p<: 001
Comparison game 3 5.00 4.31 2.86 p= 0:004
Comparison game 4 5.19 3.42 7.21 p<: 001
Table 5: Two-sided t-tests for the dierence in the average payo between the two
treatments in the comparison games.
by assuming optimal use of heuristics. In order to do this, we need to specify concrete
parameterizations of the space of heuristics Hand the cognitive costs. But rst, we
provide a more detailed description of the general theory.
4.1 General Model
We consider a setting where individuals in a population are repeatedly randomly
matched with another individual to play a nite normal-form game. We assume they
use some heuristic to decide what strategy to play.
LetG=hf1;2g;S1S2;ibe a two-player normal-form game with pure strategy
setsSi=f1;:::;migfori2f1;2g, wheremi2N. A mixed strategy for player iis
denotedi2(Si). The material payo for playerifrom playing pure strategy si2Si
when the other player iplays strategy si2Siis denotedi(si;si). We extend the
material payo function to the expected material payo from playing a mixed strategy
i2(Si) against the mixed strategy i2(Si) withi(i;i), in the usual way.
A heuristic is a function that maps a game to a mixed strategy hi(G)2(Si):For
simplicity, we will always consider the games from the perspective of the row player,
and consider the transposed game GT=hf2;1g;S2S1;(2;1)iwhen talking about
the column player's behavior.
Each heuristic has an associated cognitive cost4,c(h)2R+. Simple heuristics,
such as playing the uniformly random mixed strategy, have low cognitive costs, while
complicated heuristics involving many precise computations have high cognitive costs.
Since a heuristic returns a mixed strategy, the expected material payo for player i
using heuristic hiwhen playeriuses heuristic hiis
i
hi(G);hi(GT)
:
4In general, the cognitive cost could depend on both the heuristic and the game. For example, it
might be more costly to apply a heuristic to a 5 5 game than to a 2 2 game. But since all our
games are 33, we can dispense with that dependency.
12
Since each heuristic has an associated cognitive cost, the actual expected utility derived
from it is
ui(hi;hi;G) =i
hi(G);hi(GT)
c(hi):
A heuristic is neither good nor bad in isolation; its performance has to be evaluated
with regard to some environment, in particular, with regard to the games and other-
player behavior one is likely to encounter. Let Gbe the set of possible games in the
environment,Hbe the set of heuristics the other player could use, and Pbe the joint
probability distribution over G;H. In the equations below, we will assume that Gand
Hare countable. An environment is given by E= (P;G;H). Thus, an environment
describes which game and other-player heuristic combinations a player is likely to face.
Given an environment, we can calculate the expected performance of a heuristic as
follows:
V(hi;E) =EE[ui(hi;hi;G)] =X
G2GX
hi2Hui(hi;hi;G)P(G;hi): (2)
We can also calculate the expected performance of a heuristic conditional on the specic
game being played as follows:
V(hi;E;G) =EEjG[ui(hi;hi;G)] =X
hi2Hui(hi;hi;G)P(hijG):
We can now formally dene what it means for a heuristic to be rational (or optimal).
A rational heuristic his a heuristic that optimizes (2), i.e.,
h= argmax
hi2HV(hi;E); (3)
or, in slightly expanded form,
h= argmax
hi2HEE
i
hi(G);hi(GT)
c(hi)
: (4)
That is, a rational heuristic chooses actions that yield high rewards for the games and
opponents one tends to encounter, while not being costly to evaluate; more specically,
a rational heuristic achieves the best tradeo between these two (typically, but not
always, competing) desiderata. We here also see that by varying the environment, E,
we can vary which heuristics are optimal. In our experiment, we will manipulate the
distribution over games, thereby varying the predictions we get by assuming rational
13
heuristics.
One natural critique of this approach is that the problem of selecting an optimal
heuristic is actually much more complex than the problem of selecting an optimal action.
Critically, however, while the optimality of an action is dened with respect to a single
game, the optimality of a heuristic is dened with respect to an environment. Thus, it
is possible to for a player learn an optimal heuristic (but not an optimal action) even
if she has limited experience with the specic game being played. In Appendix E, we
show that a simple learning model can reproduce the performance of the optimizing
metaheuristic model.
4.2 Specic Parameterizations
We consider two parameterizations of Handc. Importantly, we don't claim that either
parameterization perfectly matches the actual spaces of heuristics or cognitive costs
faced by human beings. Instead, they are constructed to be rich enough, and close
enough to actual cognitive costs and heuristics, to be able to capture the essence of
behavior. As we will see, the optimal heuristics found with both these parameterizations
give accurate predictions.
The rst parameterization we call metaheuristics . It consists of three primitive
heuristics that together with a selection rule create the metaheuristic. The primitive
heuristics and selection rule are chosen based on existing models and descriptive evidence
on choice processes. This parameterization is intuitive and interpretable, but its design
involves many somewhat arbitrary researcher decisions.
The second parameterization, deep heuristics , makes much weaker assumptions
about the space of heuristics. This parameterization is based on the neural network
architecture for normal-form games proposed by Hartford et al. (2016). It captures
a much larger space of possible heuristics and thus removes the researcher degrees of
freedom that are a concern for the rst parameterization (e.g., the choice of primitive
heuristics), at the cost of losing interpretability and some control over the cognitive
cost.
4.2.1 Metaheuristics
To build a formal model of heuristics for one-shot games, we begin by specifying a few
general types of reasoning that such heuristics might employ: row-based reasoning, cell-
based reasoning, and simulation-based reasoning. For each of these types, we specify a
precise functional form with a small number of continuous parameters and an associated
14
cognitive cost function. The cognitive cost of a heuristic is a function of its parameters,
and the cost function is itself parameterized. Finally, we consider a higher-order heuristic,
which we call a metaheuristic , that selects among the candidate rst-order heuristics
based on their expected values for the current game. We emphasize that we do not
claim that this specic family captures all the heuristics people might employ in a game.
However, we hypothesized|and our results conrm|that this family is expressive
enough to illustrate the general theory's predictions and provide a strong quantitative
account of human behavior. Since this specic parameterization (metaheuristics) is not
the main focus of the paper, the details can be found in Appendix B.
Row Heuristics Arow heuristic calculates a value, v(si), for each pure strategy,
si2Si, based only on the player's own payos associated with si. Formally, a row
heuristic is dened by the row-value function vsuch that
v(si) =f(i(si;1);:::;i(si;mi)))
for some function f:Rmi!R. The specic parameterization of fwe consider goes
from min to max, passing by the mean, via a single parameter
such that
v
(si) =X
si2Sii(si;si)exp [
i(si;si)]P
s2Siexp [
i(si;s)]:
For example,
= 0 implies that fis the mean function, corresponding to a level-1
strategy.
!1 corresponds to the min function, and yields the maximin strategy.
More generally, the row heuristic captures a weighted mean of each row's payos that
might overweight either good or bad outcomes.
Once the function vis specied, we assume that the computation of vis subject to
noise but that this noise can be reduced through cognitive eort, which we operationalize
as a single scalar '. In particular, following Stahl and Wilson (1994), we assume that
the noise is Gumbel-distributed and thus recover a multinomial logit model with the
probability that player iplays strategy sibeing
hsi
row(G) =exp ['v(si)]P
k2Siexp ['v(k)]:
Naturally, the cost of a row heuristic is a function of cognitive eort. Specically,
15
we assume that the cost is proportional to eort,
c(hrow) ='Crow;
whereCrow>0 is a free parameter of the cost function.
Cell Heuristics An individual might not necessarily consider all aspects connected
to a strategy but nd a good \cell," meaning a payo pair ( i(si;si);i(si;si)). In
particular, previous research has proposed that people sometimes adopt a team view ,
where each player looks for outcomes that are good for both players, and chooses actions
under the (perhaps implicit) assumption that the other player will try to achieve this
mutually benecial outcome as well (Sugden, 2003; Bacharach, 2006). Alternatively,
people may engage in virtual bargaining , where each player selects the outcome that
would be agreed upon if she could negotiate with the other player (Misyak and Chater,
2014). Importantly, these approaches share the assumption that people reason directly
about outcomes (rather than actions) and that there is some amount of assumed
cooperation.
We refer to heuristics that reason directly about outcomes as cell heuristics . Based
on preliminary analyses, we identied one specic form of cell heuristic that participants
appear to use frequently: the jointmax heuristic, which identies the outcome that is
most desirable for both players. Formally, the joint desirability of a cell is given by
vjointmax(si;si) = minfi(si;si);i(si;si)g
and the probability of playing a given strategy, with cognitive eort 'is given by
hsi
jointmax (G) =X
si2Siexp
'vjointmax(si;si)
P
(ki;ki)2SiSiexp ['vjointmax (ki;ki)]:
This can be interpreted as applying a softmax to all possible outcomes and taking the
probability of each strategy to be the sum of the probabilities in the corresponding row.
Cognitive cost is again proportional to eort, and so
c(hcell) ='Ccell;
whereCcell>0 is a free parameter of the cost function.
16
Simulation Heuristics: Higher-level Reasoning Most previous behavioral mod-
els of initial play have a basic structure of belief formation and best response. Such
models assume that people rst form a belief about which strategy the other player
will choose and then select the strategy with the maximal expected value given that
belief. In general, eective heuristics do not necessarily have this form; indeed, for many
parameter values, the row and cell heuristics described earlier might not be compatible
with any beliefs. However, explicitly forming beliefs and calculating the best responses
(following a simulation heuristic ) may be a good decision-making strategy in some
situations.
If a row player uses a simulation heuristic, she rst considers the game from the
column player's perspective, applying some heuristic (a row, cell, or simulation heuristic)
that generates a distribution of likely play. She then plays a noisy best response to that
distribution.
The cognitive cost of a simulation heuristic is a combination of the cognitive cost of
the heuristic for the column player, a constant cost for updating the payo matrix using
that belief ( Cmul), and a cost that is proportional to the cognitive eort parameter in
the last step, as for a row heuristic,
c(hsim) =c(hcol) +Cmul+Crow':
Notice that once the beliefs have been formed the last cost for taking a decision is
based onCrowsince this process is the same as averaging over the rows as with a row
heuristic.
Selection Rule We don't expect a person to use the same heuristic in all games.
Instead, they may have a set of heuristics, and choose which one to use in each situation
based on an estimate of the candidate heuristics' expected values. We model this
selection process as a higher-order selection rule that selects among the rst-order
heuristics described above. This selection rule allows the decision-maker to select from
a few dierent primitive heuristics, and hence the term \metaheuristic."
Rather than explicitly modeling the process by which players select among the
candidate heuristics, for example, by using the approach in Lieder and Griths (2015),
we use a reduced-form model based on the rational inattention model of Mat ejka and
McKay (2015). We make this simplifying assumption since it allows us to focus on the
central parts of our theory. This functional form captures the three key properties a
metaheuristic should have: (1) there is a prior weight on each primitive heuristic, (2) a
17
primitive heuristic will be used more on games in which it is likely to perform well, and
(3) this adjustment from the prior based on expected value is incomplete and costly.
See Equation 6 in the Appendix for details.
4.2.2 Deep Heuristics
A drawback of using explicitly formulated heuristics, as above, is that the results depend
on somewhat arbitrary decisions made by the researchers (in particular, the set of
primitive heuristics). To minimize the risk of our conclusions being driven by such
decisions, we also consider a nonparametric family of heuristics implemented with neural
networks. While not as interpretable as the metaheuristics, this new class includes a
much larger set of possible heuristics.
......
......
......
...ARR(0)UR
ARR(1) ARR(2)
ARC(0)
UCARC(1)Output
F1
F2
FnHR
1;1
HR
1;k1
HC
1;1
HC
1;k1HR
2;1
HR
2;k2
HC
2;1
HC
2;k2SR
1
SR
k2
SC
1
SC
k2Input
layerHidden
layersSoftmax
layerAction Response
layers
Figure 2: Architecture of the deep heuristic.
We use a neural network architecture similar to the one developed by Hartford et al.
(2016), with some adjustments to allow for modeling cognitive costs. The architecture,
illustrated in Figure 13, has two key properties specically adapted to nite normal-
form games. First, the connectivity structure ensures that predictions are invariant
to relabeling of the strategies, thereby vastly reducing the size of the parameter space
(playing a similar role to convolution in computer vision). Second, the architecture
explicitly separates recursive reasoning (e.g., level-k) and direct reasoning about the
18
payo matrix. This allows us to capture belief formation and best response, as well as
simpler heuristics like our row and cell heuristics. Furthermore, we can assign a dierent
cognitive cost to each type of reasoning. A detailed description of the architecture is
given in Appendix C.
5 Model-based Analysis
Having formally specied our theoretical framework, we can now take the models to
our experimental data. Specically, we ask whether the behavioral dierences found
in our experiment are consistent with the rational use of heuristics. To do this, we
rst compare the predictive accuracy and payos achieved by models that are either
payo-optimized or t directly with the data; in particular, we conrm two preregistered
hypotheses generated by the general theory. Next, we compare our model to previously
proposed models, and demonstrate that our model yields more accurate predictions.
5.1 Model Estimation
We take an out-of-sample prediction approach to model comparison. Each data set is
divided into a training set on which model parameters are estimated and a test set on
which predictive performance is evaluated. We used the rst 30 treatment games from
each population as the training set and the remaining 16 treatment games as the test
set. We chose this split so that we could test the predictions on the later games when
people would be most likely to be using a consistent decision strategy. We consider each
game to consist of two observations: the empirical distribution of play for each player
role (row and column). The games are sampled separately for each population but are
the same within a population, and we have 10 populations for each treatment. For
each treatment, we thus have 600 observations in the training set and 320 observations
in the test set. This separation was preregistered and can thus be considered a \true"
out-of-sample prediction.
We dene separate environments for the two treatments using the actual games and
empirical distributions of play in all populations of the corresponding treatment. We
thus dene the common-interests environment, E+, by lettingG+be all the treatment
games played in the common-interests treatment, and letting the opponent's behavior,
h+(G), be the actual distribution of play in G. Lastly,Pis a uniform distribution over
all games inG+and always returns h+as the heuristic for the opponent. We dene the
competing-interests environment Ecorrespondingly. Lastly, we divide the games into
19
the training games, i.e., G+
train, and test gamesG+
test.
The measure of the t we use is the average negative log-likelihood (or, equivalently,
the cross-entropy), where a lower value means a better t. If pis the observed
distribution of play for the row or column player role in some game, and qis the
predicted distribution of play from some model, the negative log-likelihood (NLL) is
dened as
NLL(q;p) =X
spslog(qs):
We dene the total NLL of a metaheuristic mwith cognitive costs Cevaluated on
the common-interests training set E+
trainas
NLL(m;E+
train;C) =X
G2G+
trainNLL(m(G;h+;C);h+(G));
and analogously for the three remaining training and test sets. We write m(G;h+;C)
since the actual prediction of the metaheuristic min a given game depends on the
performance of the dierent primitive heuristics, which in turn depend on the opponent's
behavior,h+, and the cognitive costs, C, as given by Equation (6).
The behavior of the metaheuristic model depends on three factors: the consideration
set of possible primitive heuristics, the cognitive cost of those heuristics, and the prior
distribution for the selection rule. We assume that the consideration set includes one
of each type of primitive heuristic: a cell heuristic, a row heuristic, and a simulation
heuristic. The model thus has twelve free parameters: six that specify the behavior of
the primitive heuristics, four for the cognitive costs, and two for the selection rule's
prior.
The cognitive cost parameters are xed from the decision-maker's perspective,
re
ecting constraints imposed by the decision-maker's cognitive abilities. We thus t
the cost parameters to data. By contrast, the parameters of the heuristics and the
selection rule prior are under the decision-maker's control. We consider two methods
for estimating the parameters of the heuristics: tting them to the data, or optimizing
them such that they maximize expected utility. The latter method instantiates our
theory that people use heuristics in a resource-rational way. For a given set of cognitive
cost parameters C= (Crow;Ccell;Cmul;), the tted common-interests metaheuristic is
given by
mt(E+
train;C) = argmin
m2MNLL(m;E+
train;C);
whereMis the space of metaheuristics we restrict our analysis to. The tted parameters
20
thus capture the heuristics that empirically best explain human behavior.
The optimal common-interests metaheuristic, for cognitive cost C, is instead given
by
mopt(E+
train;C) = argmax
m2MV(m;E+
train;C) = argmax
m2MX
G2G+
trainu(m;h+;G;C )
whereu(m;h+;G;C ) is the expected utility from employing metaheuristic magainst
behaviorh+in gameGwith cognitive cost parameters C. The optimized parameters
thus identify the heuristics that objectively achieve the best cost-benet tradeo, given
the tted cost parameters. The tted and optimal metaheuristics for the competing-
interests environment are dened analogously.
Having dened the tted and optimal heuristics with cognitive costs C, we now
turn to the question of how to estimate the cognitive costs. Since the participants are
drawn from the same distribution and are randomly assigned to the two treatments, we
assume that the cognitive costs are always the same for both treatments.
To estimate the costs, we nd the costs that minimize the average NLL of the
optimized or tted heuristics on the training data. Therefore
Ct= argmin
C2R4
+NLL(mt(E+
train;C);E+
train;C) + NLL(mt(E
train;C);E
train;C);
and
Copt= argmin
C2R4
+NLL(mopt(E+
train;C);E+
train;C) + NLL(mopt(E
train;C);E
train;C):
Notice the crucial dierence between the tted and optimized metaheuristics. For the
tted metaheuristics, we t both the cognitive cost parameters and the heuristic parame-
ters to match actual behavior in the two training sets. For the optimized metaheuristics,
we t only the cognitive cost parameters; the heuristic parameters are set to maximize
the payo minus the cognitive cost. As a result, any dierence between the optimal
common-interests metaheuristic and the optimal competing-interests metaheuristic is
entirely driven by dierences in performance between the dierent heuristics in the two
environments.
5.2 Results for Metaheuristics
Next, we consider our two model-based hypotheses regarding the metaheuristic model's
ability to capture the dierence in the participants' behavior between the two treatments.
21
Hypothesis 3. Participants' behavior will dier between the two treatments in a way
that the model can capture. Specically, their behavior in the common-interests test
games should be better predicted by the common-interests metaheuristic than by the
competing-interests metaheuristics. Conversely, their behavior in the competing-interests
test games should be better predicted by the competing-interests metaheuristics than by
the common-interests metaheuristic. This should hold both for the tted metaheuristics
and for the optimized metaheuristics.
Concretely, this hypothesis states that the following four inequalities should hold:
NLL(mt(E
train);E
test)<NLL(mt(E+
train);E
test)
NLL(mopt(E
train);E
test)<NLL(mopt(E+
train);E
test)
NLL(mt(E
train);E+
test)>NLL(mt(E+
train);E+
test)
NLL(mopt(E
train);E+
test)>NLL(mopt(E+
train);E+
test);
where the notation for CtandCoptis omitted for brevity.
In order to facilitate comparisons between treatments and between games, we use
\relative prediction loss", that is, the dierence in NLL between the model's predictions
and the theoretical minimum NLL. Let ybe the observed empirical distribution of
play in some game G. Then the lowest possible NLL in that game is NLL (y;y). The
relative prediction loss for model min gameGis thus given by5
NLL (m;G;C )NLL (y;y):
We compute condence intervals of the relative prediction loss over all the games in the
test set. Since we consider each game separately for the two dierent player roles, there
are 320 observations per test set.
Figure 3 shows the relative prediction loss on the test data in each treatment achieved
by each possible method of tting the model. We clearly see that the models that were
trained on data from the same treatment as the test set outperform models trained on
the other treatment. This conrms Hypothesis 3.
An even more striking result is that the optimized metaheuristics achieve nearly the
same predictive performance as the tted metaheuristics. That is, a model that uses the
5The resulting measure of performance is related to the completeness measure of Fudenberg et al.
(2022). However, since we have only fteen participants per game and role, and there is randomness
in behavior, even the perfect model would not be able to get the exact distribution of play right.
Therefore, the theoretical minimum is truly theoretical.
22
Figure 3: Predictive performance of the metaheuristics. Each panel shows the relative
prediction loss (average negative log-likelihood minus lowest possible value) of the
test data for one treatment (competing interests or common interests). Models are
tted or optimized to either the competing-interests training games or the common-
interests training games. The error bars show 95% condence intervals. The dashed
line corresponds to uniform random play, which assigns the same probability to each
action in each game.
same set of cognitive cost parameters in both treatments (with the heuristic parameters
set to optimize the resultant expected payo-cognitive cost tradeo) explains participant
data almost as well as the fully parameterized model, in which the heuristic parameters
are separately tted to each treatment.
Not only do we conrm our hypothesis and show that the rational heuristic is a
strong predictor, but we also see that we capture most of the distance between the
uniform random play and the theoretical minimum NLL. Table 6 in the Appendix shows
the accuracy and average NLL for all models we consider in the paper. There, we see
that the average accuracy of the optimal metaheuristic is 88%, meaning that in 88% of
the games, the modal action is assigned the highest probability. It should also be noted
that in the games where the optimal metaheuristic makes an incorrect prediction, the
modal action is on average only played by 54% of the participants, while the modal
action was played by 75% of the participants in all of the test games. Therefore, in the
games where the proposed model fails to assign the highest probability to the modal
action, play is quite even and hence dicult to predict.
Our nal model-based hypothesis provides an additional test that the metaheuristics
23
Figure 4: Payo performance of the metaheuristics. Each panel shows the regret
(best possible expected payo minus true expected payo) attained by models that
are trained and tested in dierent combinations of common-interests and competing-
interests environments. The dashed line shows the performance of uniform random
play.
that participants use are adapted to their treatment environment:
Hypothesis 4. The tted heuristics estimated for a given treatment should achieve
higher expected payos on the test games for that treatment, as compared to the heuristics
estimated for the other treatment.
The logic of this hypothesis is that even if we do not assume that participants use
optimal heuristics, we should still see that the heuristics that best describe participants'
behavior in each treatment achieve higher payos in that treatment. As with prediction
loss, we use a relative performance measure that accounts for dierences in maximal
payo in the two treatments. Specically, we quantify performance in terms of regret, the
dierence between the expected payo given the predicted behavior and the maximum
expected payo in each game.
As illustrated in Figure 4, the results conrm our hypothesis. When testing on games
from either treatment, the models tted to human behavior in the same treatment
achieved lower regret than those tted to the other treatment, although the dierence
is larger for the common-interests games.
In Appendix D.2 we present results from pairwise tests of both Hypotheses 3 and 4.
We see there that all the dierences in both relative prediction loss and regret are
24
signicant at the 0.01 level.6
5.3 Results for Deep Heuristics
By applying the same estimation method to the deep heuristics as we did to the
metaheuristics, we can test whether Hypotheses 3 and 4 also hold for a completely
dierent specication of the space of heuristics and cognitive costs. In Figure 5, we see
that Hypothesis 3 holds for this specication as well: the models make more accurate
predictions for the treatment on which they were trained or optimized. We also see
that the predictive performance of the optimal heuristic is close to the tted heuristic,
given optimized cognitive costs.
Figure 5: Predictive performance of the deep heuristics.
We can also test Hypothesis 4 in the same way by looking at the expected payo
from the two dierent deep heuristics tted to the behavior of the participants in the
two dierent treatments. As before, we see that the tted models achieved lower regret
in the treatment on which they were trained, again suggesting that the heuristics people
use are well adapted to their environment.
6In the preregistration, we did not specify a formal testing procedure for these dierences and did
not originally include such a test in the paper. However, after discussions and presentations, it became
clear that such tests are sought after by readers, and we have therefore added them.
25
Figure 6: Payo performance of the deep heuristics.
5.4 Model Comparison
In the previous sections, we have shown that the rational use of heuristics can explain
and predict people's behavior in one-shot games, in particular how their behavior
depends on the previous games they have played. Next, we ask whether our proposed
theory provides a more accurate account of that behavior when compared to previously
proposed models. Specically, we consider three alternative models: quantal cognitive
hierarchy (QCH), QCH with prosocial preferences, and noisy best-response to the true
distribution of play with prosocial preferences.
Quantal Cognitive Hierarchy. In previous comparisons between behavioral
models of one-shot games, variations of cognitive hierarchy models are usually the best
performing (Camerer et al., 2004; Wright and Leyton-Brown, 2017). In such a model,
we consider agents of dierent cognitive levels. In the quantal cognitive hierarchy (QCH)
model we consider here, a level-0 agent plays the uniformly random strategy, playing
each action with an equal probability. A level-1 player (logit) best responds to a level-0.
Finally, a level-2 player best responds to a combination of level-0 and level-1 players.7
The model has 4 parameters: the share of level-0 and level-1 players (which together
determine the share of level-2 players), the sensitivity 1of level-1 players, and the
sensitivity2of level-2 players.
Prosocial Preferences
We have attributed the dierence in the participants' behavior between the two
7We found that adding higher levels of play did not improve predictive performance.
26
treatments to their learning dierent heuristics. However, this pattern of behavior
could be explained by a change not in their decision-making strategy but in their
underlying preferences. In particular, participants in the common-interests environment
may develop a sense of camaraderie that makes them care about the other players'
payos, while participants in the competing-interests environment may become jaded
or even spiteful, leading them to disregard the others' payos.
To test this alternative explanation, we augmented the QCH model with a prosocial
utility function (Fehr and Schmidt, 1999; Bruhin et al., 2019), i.e.,
ui(si;si) = (1sr)i(si;si) + (s+r)i(si;si); (5)
wheresindicates whether i(si;si)<i(si;si) andrindicates whether i(si;si)>
i(si;si). In other words, determines how much player ivalues the payo of player
iwhenigains less thani, andhow much player ivalues the payo of player i
whenigains more thani. This augmentation thus adds two parameters to the QCH
model,and. In this model, beliefs are formed using a standard QCH model, but the
payos are changed according to the prosocial preferences model (Equation 5) before
the last quantal best-response step.8This model can account for dierences in behavior
between the two treatments both by assuming dierent levels of prosociality and by
assuming dierent levels of reasoning or sensitivity in the QCH step.
Diering Beliefs
A second possible source of diering behavior across treatments is diering beliefs.
Since people behave dierently in the two treatments, participants may form dierent
beliefs about what they expect the other player to do. In particular, participants in the
common-interests treatment may expect the other player to cooperate by selecting an
action with a jointly benecial outcome, while participants in the competing-interests
treatment may expect the other player to select the safest action for themselves.
To test this account, we replace the recursively formed beliefs of QCH with the
correct (empirical) belief. This model thus plays a noisy best response to the actual
distribution of participants' play. In this model, we additionally allow for prosocial
preferences, resulting in a three-parameter model.
Results. In Figure 7, we compare the out-of-sample predictive performance of these
two alternative models and our two suggested specications for the space of heuristics.
While the alternative models are estimated by tting the parameters to match the
8We also considered another model combining QCH and prosocial preferences, in which the player
also has some beliefs about the other player's prosociality that informs the beliefs formed during the
QCH steps. This didn't make a meaningful dierence in t.
27
Figure 7: Out-of-sample relative prediction loss for alternative models of behavior.
All the models are estimated on the training games of the same environment as the
test games. The error bars show a 95% condence interval. Legend: QCH = quantal
cognitive hierarchy, Pro = prosocial preferences, EB = empirical beliefs.
participants' behavior, we also include the optimized versions of our two specications.
For the common-interests games, it is clear that both the tted and optimized
versions of our models outperform both the quantal cognitive hierarchy model and the
noisy best response with prosocial preferences (prosociality) model. The model with
both prosocial preferences and recursive reasoning (Pro+QCH) performs better, but
is still outperformed by three of our models (excluding the optimized deep heuristics).
For the competing-interests games, the prosociality model is still clearly performing
worst, but the other models all perform similarly. This suggests that the QCH model
predicts participants' behavior better in the competing-interests environment than in
the common-interests environment. Taken together, our proposed models are better
at predicting behavior than alternative models, including the current best-performing
model in the literature (QCH).
We also see clearly in Figure 7 that the predictive performance of the metaheuristics
and tted deep heuristics are very close, even though the deep heuristics encompasses a
much larger space of heuristics. This suggests that we have managed to capture the
relevant space of heuristic strategies with our parameterization of the metaheuristics.
That is, the metaheuristic model is nearly \complete" in the sense of Fudenberg et al.
(2022).
28
6 Discussion
In the theory presented, we combine two perspectives. On the one hand we assume
that people use simple cognitive strategies to choose actions that are often inconsistent
with rational behavior in any given game. On the other hand, we don't assume that the
specic heuristics used are predetermined or insensitive to incentives. On the contrary,
we assume that the heuristics people use are chosen resource-rationally, such that they
strike an optimal balance between expected payos and cognitive costs. We have seen
that by combining these two perspectives, we can predict behavior more accurately and
better understand the in
uence of the larger environment on behavior in a given game.
In particular, the proposed approach can help us predict when we should expect
behavior to coincide with rational behavior and when we might see systematic deviations
from a rational benchmark. Behavior will coincide with rational behavior if two
conditions are satised. Firstly, there has to exist a simple heuristic that leads to the
optimal action. Secondly, that heuristic has to perform well in the larger environment
so that the decision-maker can learn to use it. When there doesn't exist a simple
and high-performing heuristic, or when the heuristic that normally works well leads
to the wrong decision, we will observe consistent deviations. This latter case is nicely
illustrated in our comparison games.
The optimal heuristic will focus on the features of the games that are often of
importance, but miss opportunities that are rare. Specically, a person used to common-
interests games might miss an opportunity for personal gain at the other player's expense
while a person used to competing-interests games might fail to notice an outcome that
is actually best for everyone.
Our ndings relate to those of Peysakhovich and Rand (2016), who showed that
varying the sustainability of cooperation in an initial session of the repeated prisoner's
dilemma aects how much prosocial behavior and trust is shown in later games, including
the one-shot prisoner's dilemma. Our results provide a qualitative replication of this
idea. In particular, we found that putting people in an environment in which prosocial
heuristics (such as jointmax) perform well leads them to choose prosocial actions in the
comparison games and in some cases, even to select dominated options. By contrast,
putting people in an environment where prosocial actions often result in low payos
prevents people from achieving ecient outcomes, even when they are Nash equilibria.
Consistent with our theory, Peysakhovich and Rand interpreted their ndings as the
result of heuristic decision-making. We build on this intuitively appealing notion
by specifying formal models of heuristics in one-shot games that make quantitative
29
predictions. We also emphasize the in
uence of cognitive costs (in addition to payos)
on the heuristics people use.
Finally, we would like to emphasize an important dierence between our theory and
previously proposed models of learning in games. Previous learning models have been
posed at the level of action ; people learn which action to take in a specic (repeatedly
played) game (e.g. Jehiel, 2005; Grimm and Mengel, 2012). In contrast, in our theory,
learning happens at the level of reasoning ; people learn how to decide what to do
in a new game. We believe that this more abstract form of learning is more broadly
applicable in the real world, as it is rare that we ever encounter the exact same situation
twice (a feature that is captured by the randomly generated games in our experiment).
7 Conclusion
We have proposed a theory of human behavior in one-shot normal-form games based
on the resource-rational use of heuristics. According to this theory, people select their
actions using simple cognitive heuristics that
exibly and selectively process payo
information; the heuristics people choose to use are ones that strike a good tradeo
between the expected payos and the cognitive cost.
In a large preregistered experiment, we conrmed one of the primary qualitative
predictions of the theory: people learn which heuristics are resource-rational in a
given environment, and thus their recent experience aects the choices they make.
In particular, we found that placing participants in environments with common (vs.
competing) interests leads them to select the most ecient (or least ecient) equilibrium
in a weak-link game and to cooperate (or defect) in a prisoner's dilemma.
Furthermore, we found that our theory provides a strong quantitative account of our
participants' behavior, making more accurate out-of-sample predictions than both the
quantal cognitive hierarchy model and a model with prosocial preferences and a noisy
best response. Strikingly, we found that a resource-rational model, in which behavior in
both common-interests and competing-interests treatments is predicted using a single
set of tted cost parameters (with the heuristic parameters set to optimize the resultant
expected payo-cognitive cost tradeo), achieved nearly the same accuracy as the fully
parameterized model in which the heuristic parameters are estimated separately to
match the behavior in each treatment. Coupled with the overall high predictive accuracy
of the model, this provides strong evidence in support of the theory that people use
heuristics that optimally trade o between the expected payo and the cognitive cost.
We also found similar results using an entirely dierent neural network-based family
30
of heuristics, indicating that these ndings are robust to the parameterization of the
heuristics.
From a broader perspective, our theory speaks to a decades-long debate on the ratio-
nality of human decision-making. In contrast to classical models based on optimization
and utility maximization, which fail to capture systematic patterns in human choice
behavior, recent models instead emphasize our systematic biases, suggesting that we
rely on simple and error-prone heuristics to make decisions. In this paper, we hope
to have oered a synthesis of these two perspectives, by treating heuristics as things
that can themselves be optimized in a utility-maximization framework. We hope this
approach will prove to be a valuable step forward toward a more unied understanding
of economic decision-making.
31
A Instructions for the experiment
Figure 8: The instructions one the rst page when a participant joins the experiment.
32
Figure 9: The participants have to complete three questions like this in a row in order
to be allowed to participate in the experiment.
Figure 10: In each round, the participant chose a row by clicking on it. Once it is
clicked it is highlighted and they have to click the next button to proceed.
33
Figure 11: Once the matched participant chooses a column, either by making a decision
or by sampling from previous decisions in the game from the same population, the
result is shown.
B Complete Description of Metaheuristics
To exemplify the dierent heuristics, we will apply them to the following example game.
1 2 3
10;10;28;8
25;65;52;2
36;56;61;1
Figure 12: Example normal-form game represented as a bi-matrix. The row player
chooses a row and the column player chooses a column. The rst number in each cell is
the payo of the row player and the second number is the payo of the column player.
B.1 Row Heuristics
Arow heuristic calculates a value, v(si), for each pure strategy, si2Si, based only on
the player's own payos associated with si. That is, it evaluates a strategy based only
on the rst entry in each cell of the corresponding row of the payo matrix (see Figure
12). Formally, a row heuristic is dened by the row-value function vsuch that
v(si) =f(i(si;1);:::;i(si;mi)))
34
for some function f:Rmi!R. For example, if fis the mean function, then we have
vmean(si) =1
miX
si2Sii(si;si);
which evaluates each strategy by the average payo in the corresponding row of the
payo matrix. Deterministically selecting arg maxsivmean(si) gives exactly the behavior
of a level-1 player in the classical level-k model.
If, instead, we let fbemin, we recover the maximin heuristic, which calculates
the minimum value associated with each strategy and tries to choose the row with the
highest minimum value,
vmin(si) = min
si2Sii(si;si);
and, similarly, if we let fbe max, we recover the maximax heuristic,
vmax(si) = max
si2Sii(si;si):
While one can imagine a very large space of possible functions f, we consider a
one-dimensional family that interpolates smoothly between minandmax. We construct
such a family with the following expression:
v
(si) =X
si2Sii(si;si)exp [
i(si;si)]P
s2Siexp [
i(si;s)];
which approaches vmin(si) as
!1 ,vmax(si) as
!1 , andvmean(si) when
= 0.
Intuitively, we can understand this expression as computing an expectation of the payo
forsiunder dierent degrees of optimism about the other player's choice of si. In
the above example game (Figure 12), the heuristic will assign the highest value to 1
(the top row) when
is large and positive, to 2when
is large and negative, and to 3
when
= 0. Notice that if
6= 0, the values associated with the dierent strategies do
not necessarily correspond to a consistent belief about the other player's action. For
example, if
is positive, the highest payo in each row will be overweighted, but this
might correspond to a dierent column in each row; in the example game (Figure 12),
column 3 is overweighted when evaluating row 1 but downweighted when evaluating
rows 2 and 3. Although this internally inconsistent weighting may appear irrational, it
provides an extra degree of freedom that can increase the expected payo in a given
environment without additional cognitive eort.
We assume that the computation of vis subject to noise, but that this noise can
35
be reduced through cognitive eort, which we operationalize as a single scalar '. In
particular, following Stahl and Wilson (1994), we assume that the noise is Gumbel-
distributed and thus recover a multinomial logit model with the probability that player
iplays strategy sibeing
hsi
row(G) =exp ['v(si)]P
k2Siexp ['v(k)]:
We assume that the cost is proportional to the eort, i.e.,
c(hrow) ='Crow;
whereCrow>0 is a free parameter of the cost function.
B.2 Cell Heuristics
We refer to heuristics that reason directly about outcomes as cell heuristics . Based on
preliminary analyses, we identied one specic form of cell heuristic that participants
appear to use frequently: The jointmax heuristic, which identies the outcome that is
most desirable for both players. Formally, the joint desirability of a cell is given by
vjointmax(si;si) = minfi(si;si);i(si;si)g
and the probability of playing a given strategy with cognitive eort 'is given by
hsi
jointmax (G) =X
si2Siexp
'vjointmax(si;si)
P
(ki;ki)2SiSiexp ['vjointmax (ki;ki)]:
This can be interpreted as applying a softmax to all possible outcomes and taking the
probability of each strategy to be the sum of the probabilities in the corresponding
row. In the example game (Figure 12), the jointmax heuristic would assign the highest
probability to row 1because the cell ( 1;3) with payos (8 ;8) has the highest minimum
payo.
The cognitive cost is again proportional to eort, and so
c(hcell) ='Ccell;
whereCcell>0 is a free parameter of the cost function.
36
B.3 Simulation Heuristics: Higher-Level Reasoning
If a row player uses a simulation heuristic, she rst considers the game from the column
player's perspective, applying some heuristic that generates a distribution of likely play.
She then plays a noisy best response to that distribution. Let GTdenote the transposed
game, i.e., the game from the column player's perspective. Let hcolbe the heuristic the
row player uses to estimate the column player's behavior; then, hsim(G) is given by
hsi
row=exph
'P
si2Sii(si;si)hsi
col(GT)i
P
si2Siexph
'P
si2Sii(si;si)hsi
col(GT)i;
where'is the cognitive eort parameter. A simulation heuristic is thus dened by a
combination of a heuristic and an eort parameter ( hcol;').
The cognitive cost for a simulation heuristic is calculated by rst calculating the
cognitive cost associated with the heuristic used for the column player's behavior, then a
constant cost for updating the payo matrix using that belief ( Cmul), and one additional
cost that is proportional to the cognitive eort parameter in the last step, as if it was a
row heuristic,
c(hsim) =c(hcol) +Cmul+Crow':
Notice that once the beliefs have been formed and the beliefs have been incorporated,
the last cost for taking a decision is based on Crowsince this process is the same as
averaging over the rows as for a row-heuristic.
B.4 Selection Rule
We model the selection of primitive heuristics using the rational inattention model
of Mat ejka and McKay (2015). While we don't think about the underlying selection
process as inherently one of rational inattention, the rational inattention model captures
the key properties we expect from the selection rule: (1) there is a prior weight on each
heuristic, (2) a heuristic will be used more on games in which it is likely to perform
well, and (3) the adjustment from the prior based on expected value is incomplete and
costly.
Assume that an individual is choosing between nheuristicsH=fh1;h2;:::;hNg.
37
Then the probability of using heuristic hnwhen playing game Gis given by
P[fusehninGg] =exp [(an+V(hn;E;G))=]PN
j=1exp [(aj+V(hj;E;G))=]
=pnexp [V(hn;E;G)=]PN
j=1pjexp [V(hj;E;G)=](6)
whereiis an adjustment cost parameter and anare weights that give the prior
probability of using the dierent heuristics, pn=exp(an=i)PN
j=1exp(aj=i).
A metaheuristic is dened by a tuple m=hH;PiwhereHi=fh1;h2;:::;hNgis
a nite consideration set of heuristics, and P=fp1;p2;:::;pNga prior over those
heuristics. We can express the performance of a metaheuristic in an environment E,
analogously to (2) for heuristics, as
Vmeta(m;E) =X
G2GX
h2HV(hn;E;G)pnexp [V(hn;E;G)=]PN
j=1pjexp [(V(hj;E;G))=]P(G):(7)
The optimization problem faced by the individual, subject to the adjustment cost ,
is then to maximize (7), i.e., to choose the optimal consideration set and corresponding
priors,
m= argmax
H2Pfin(H)argmax
P2(H)Vmeta(hH;Pi;E);
wherePfin(H) is the set of all nite subsets of all possible heuristics. In practice, this
is not a solvable problem when the consideration set of possible heuristics, H, is large.
Therefore, we will assume a small set of heuristics and jointly nd optimal parameters
of those heuristics and priors P.
C Deep Heuristics
Our neural network architecture is based on that developed by Hartford et al. (2016).
The idea is to let every element of the input and hidden layers be a matrix of the same
size as the game, instead of a single value as is typical. Each cell in those matrices
is then treated in the same way. This ensures that the deep heuristic is invariant to
relabeling of strategies, as should be expected from any decision rule for normal-form
games.
Higher-level reasoning is incorporated by rst having two separated neural networks,
representing a \level-0" heuristic for the row player and the column player separately,
and then possibly taking into account the thus formed beliefs about the column player's
38
......
......
......
...ARR(0)UR
ARR(1) ARR(2)
ARC(0)
UCARC(1)Output
F1
F2
FnHR
1;1
HR
1;k1
HC
1;1
HC
1;k1HR
2;1
HR
2;k2
HC
2;1
HC
2;k2SR
1
SR
k2
SC
1
SC
k2Input
layerHidden
layersSoftmax
layerAction Response
layers
Figure 13: Architecture of the deep heuristic.
behavior in separate \action response" layers. The dierent action response layers are
then combined into a response distribution. A heuristic that did not explicitly form
beliefs about the other player's behavior would let ARR(0) be the output, a person who
applies a heuristic to estimate the opponent's behavior and then best responds to it
would only use ARR(1), etc. The neural network architecture is illustrated in Figure
13.
C.1 Feature Layers
The hidden layers are updated according to
HR
l;k=l X
jwR
l;k;jHR
l1;j+bR
l;k!
HR
l;k2RmRmC
and similarly for HC. For the rst hidden layer HR
0;i=HC
0;i=Fi, and so the two disjoint
parts have the same feature matrices, but dierent weights.
The feature matrices consist of matrices where each cell contains information
associated with the row or column of one payo matrix. The payo matrices for the
row and column players are denoted by URandUC, respectively. More specically, we
39
0
@minR;C
UR
1;1;UC
1;1
minR;C
UR
1;2;UC
1;2
minR;C
UR
1;3;UC
1;3
minR;C
UR
2;1;UC
2;1
minR;C
UR
2;2;UC
2;2
minR;C
UR
2;3;UC
2;3
minR;C
UR
3;1;UC
3;1
minR;C
UR
3;2;UC
3;2
minR;C
UR
3;3;UC
3;3	1
A
Figure 14: Examples of input feature matrices.
calculate the maximum, minimum, and mean of each row and column for both payo
matrices. Furthermore, F1andF2are the payo matrices as they are, and lastly, we
have a feature matrix where each value is the minimum payo that either one of the
players receives from the strategy prole. Below are three examples of such feature
matrices.
0
B@maxiUR
i;1maxiUR
i;2maxiUR
i;3
maxiUR
i;1maxiUR
i;2maxiUR
i;3
maxiUR
i;1maxiUR
i;2maxiUR
i;31
CA;0
B@maxjUR
1;jmaxjUR
1;jmaxjUR
1;j
maxjUR
2;jmaxjUR
2;jmaxjUR
2;j
maxjUR
3;jmaxjUR
3;jmaxjUR
3;j1
CA
C.2 Softmax and Action Response Layers
After the last feature layer, a play distribution is calculated from each feature matrix in
the last layer. This is done by rst summing over the rows (columns) and then taking
a softmax over the sums. The rst action response layer is then given by a weighted
average of those dierent distributions. For example, the distribution SR
12mRis
given by
SR
1= softmax X
i(HR
2;1)1;i;X
i(HR
2;1)2;i;:::;X
i(HR
2;1)mR;i!
while the sums for the column player taken over the columns are given by
SC
1= softmax X
j(HC
2;1)j;1;X
j(HC
2;1)j;2;:::;X
j(HC
2;1)j;mC!
:
The rst action response distribution is then ARR(0) =Pk2
lwR
lSR
lforwR2k2,
and similarly for the column player.
The ARR(0) corresponds to a level-0 heuristic, i.e., a heuristic where the column
player's behavior isn't explicitly modeled and taken into account. To do this, we
move to Action Response layer 1, and use ARC(0) as a prediction for the behavior of
the opposing player. Once the beliefs of the column player are formed, the ARR(1)
40
calculates the expected value from each action, conditioned on that expected play, and
takes a softmax over those payos:
ARR(1) = softmax
X
jUR
1;jARC(0)j;:::;X
jUR
mR;jARC(0)j!
As in the cognitive hierarchy model, the second Action Response layer, ARR(2),
forms a belief about the other player by taking a weighted average of the ARR(1) and
ARR(0) layers and computing a noisy best response to it:
ARR(2) = softmax
X
jUR
1;j

ARC(0)j+ (1
)ARC(1)j
;:::!
C.3 Output Layer
The output layer takes a weighted average of the row player's action response layers.
This is the nal predicted distribution of play for the row player.
C.4 Cognitive Costs
When the deep heuristic is optimized with respect to the received payo, the cognitive
cost comes from two features of the network. Firstly, there is an assumed xed cost
associated with simulating, which is then proportional to the weight given to ARR(1).
Secondly, it is assumed that more exact predictions are cognitively more costly. The
second cognitive cost is thus proportional to the reciprocal of the entropy of the resulting
prediction.
D Detailed Results
D.1 Accuracy and Prediction Loss
In Table 6 we see the accuracy (how often the modal action is assigned the highest
probability) and the average NLL of the dierent models.
D.2 Pairwise Tests
For Hypotheses 3 and 4 we can test signicance with pairwise tests. For each of the
games in the test set, we compare the dierence in either the prediction loss or the
payo between the relevant models. For each game, we get two observations, one for
41
Common Competing Total
Model Estimation Accu NLL Accu NLL Accu NLL
Deep heuristics Fitted 89.4% 0.593 85.3% 0.709 87.3% 0.651
Metaheuristics Fitted 88.4% 0.599 86.6% 0.715 87.5% 0.657
Metaheuristics Optimized 89.1% 0.598 86.6% 0.726 87.8% 0.662
QCH+Pro Fitted 85.3% 0.638 85.6% 0.722 85.5% 0.68
Deep heuristics Optimized 85.3% 0.636 85.0% 0.739 85.2% 0.687
QCH Fitted 82.2% 0.686 84.1% 0.737 83.1% 0.711
EB+Pro Fitted 80.9% 0.717 71.2% 0.838 76.1% 0.777
Table 6: Average accuracy and negative log-likelihood for dierent models. Here we
only report the models when estimated and evaluated on the same environments.
each role. For each of these comparisons, we perform both a t-test and a nonparametric,
Wilcoxon rank test. As can be seen in the tables below, all of these tests are signicant.
Model Test set Estimation Dierence t-test Wilcoxon
Metaheuristics Common Fitted -0.065 p<: 001p<: 001
Metaheuristics Common Optimized -0.165 p<: 001p<: 001
Metaheuristics Competing Fitted -0.058 p<: 001p<: 001
Metaheuristics Competing Optimized -0.080 p<: 001p<: 001
Deep heuristics Common Fitted -0.113 p<: 001p<: 001
Deep heuristics Common Optimized -0.120 p<: 001p<: 001
Deep heuristics Competing Fitted -0.118 p<: 001p<: 001
Deep heuristics Competing Optimized -0.231 p<: 001p<: 001
Table 7: Pairwise tests for dierences in prediction loss in the test sets between
the models estimated on training data from the same vs. the dierent environment.
The prediction loss is lower for the model estimated on training data from the same
environment for all pairs.
In Table 9 we see a pairwise test for the dierence in the predictive ability between
the optimized metaheuristic and the alternative models. Prosocial EB is a model with
prosocial preferences and correct beliefs. We see that the optimized metaheuristic model
is signicantly better than the alternative models QCH, prosociality, and prosocial
QCH.
Considering pairwise comparisons of models for each treatment in isolation, we see
that the optimized metaheuristic makes better predictions than alternative models in
the common-interests treatment. For the competing-interests treatment, the dierence
is not signicant for either the QCH model with prosocial preferences or the standard
QCH model.
42
Model Test set Estimation Dierence t-test Wilcoxon
Metaheuristics Common Fitted -0.145 p<: 001p<: 001
Metaheuristics Common Optimized -0.302 p<: 001p<: 001
Metaheuristics Competing Fitted -0.083 p<: 001p<: 001
Metaheuristics Competing Optimized -0.165 p<: 001p<: 001
Deep heuristics Common Fitted -0.238 p<: 001p<: 001
Deep heuristics Common Optimized -0.333 p<: 001p<: 001
Deep heuristics Competing Fitted -0.276 p<: 001p<: 001
Deep heuristics Competing Optimized -0.502 p<: 001p<: 001
Table 8: Pairwise tests for dierences in regret in the test sets between the models
estimated on training data from the same vs. the dierent environment. Regret is lower
for the model estimated on training data from the same environment for all pairs.
Model Estimation Dierence t-test Wilcoxon
Deep heuristics Fitted -0.011 p=:003p=:001
Metaheuristics Fitted -0.005 p=:079p=:052
QCH+Pro Fitted 0.018 p<: 001p=:001
Deep heuristics Optimized 0.025 p<: 001p<: 001
QCH Fitted 0.049 p<: 001p<: 001
EB+Pro Fitted 0.115 p<: 001p<: 001
Table 9: Pairwise tests for dierences in prediction loss between the optimized meta-
heuristic model and the alternative models across both treatments.
Model Estimation Dierence t-test Wilcoxon
Deep heuristics Fitted -0.004 p=:384p=:194
Metaheuristics Fitted 0.001 p=:801p=:373
Deep heuristics Optimized 0.038 p<: 001p<: 001
QCH+Pro Fitted 0.040 p<: 001p<: 001
QCH Fitted 0.088 p<: 001p<: 001
EB+Pro Fitted 0.119 p<: 001p<: 001
Table 10: Pairwise tests for dierences in prediction loss between the optimized meta-
heuristic model and the alternative models for the common-interests games.
E Explaining Adaptation via Learning
In the main text, we assume that the participants manage to nd rational heuristics
without going into the details about how that is done. Here, we show that a learning
model could explain this adaptation to rational metaheuristics.
We assume that all individuals arrive at the experiment with the same initial
metaheuristic m(j(0)), where are the parameters of the metaheuristic, including
the parameters of both the primitive heuristics and the priors.
43
Model Estimation Dierence t-test Wilcoxon
Deep heuristics Fitted -0.017 p<: 001p=:001
Metaheuristics Fitted -0.011 p=:011p=:059
QCH+Pro Fitted -0.004 p=:483p=:619
QCH Fitted 0.010 p=:146p=:062
Deep heuristics Optimized 0.013 p=:090p=:151
EB+Pro Fitted 0.112 p<: 001p<: 001
Table 11: Pairwise tests for dierences in prediction loss between the optimized meta-
heuristic model and the alternative models for the competing-interests games.
For each experimental population , the players play a sequence of (G;t)50
t=1, each
time with a single realized action of the other player. Given the observed behavior of
playeri, the utility in round tfor playeriis given by
u(m(j);G;t;si;c) =G;t(m(G;tj);si)c(m(j));
wherem(j) is the metaheuristic with parameters ,G;tis the game played in round
tby population ,cis the cognitive cost function, and siis the action taken by the
other player.
After observing the action sitaken by the other player, player ican calculate the
gradient with respect to the parameters to see how the metaheuristic used could have
been improved, i.e.,
ru(m(j);G;t;si;c):
A simple learning model is one where each individual changes the metaheuristic
used in the direction of the gradient after each round of the experiment, with some
step-size. We can write this as
;i(t+ 1) =;i(t) +ru(m(j;i(t));G;t;si;c):
In other words, after each game, the metaheuristic is moved in the direction that would
have yielded a higher utility in that game.
For simplicity, we consider a population-level model, rather than modeling the
behavior of each individual player separately. The behavior in round tis given by
(t+ 1) =(t) +EsiP(jG;t)[ru(m(j;i(t));G;t;si;c)];
whereP(sijG;t)is the empirical probability that s1is used in game G;t. Thus,
after each game, the population parameters for the next round move in the average
44
direction of improvement dened by the empirical behavior in that game.
In our estimation of the learning model, we use the costs estimated for the optimal
metaheuristics. To estimate this model we thus need a baseline heuristic, (0), and a
learning parameter, . To make the performance of this model comparable to that of
the other models, we estimate the common starting parameters (0) and the common
learning rate in order to minimize loss on the rst 30 games of each population in
both treatments. We then predict the remaining 16 treatment games of each population
in both treatments.
Model Estimation Common Competing Both
Deep heuristics Optimize 0.636 0.739 0.687
Deep heuristics Fit 0.593 0.709 0.651
Metaheuristics Optimize 0.598 0.726 0.662
Metaheuristics Fit 0.599 0.715 0.657
Learning 0.605 0.724 0.664
Table 12: Out-of-sample NLL prediction loss.
In Table 12 we see that the performance of the learning model is comparable
to but slightly lower than the performances of the tted models. In Table 13 the
expected payos in the test set games are shown for the learning model, the optimized
metaheuristics, the optimized deep heuristics, and relevant benchmarks. It is clear that
the expected payos from this learning model are similar to both the actual payos
and those of the optimization-based models.
Model Estimation Common Competing
Metaheuristic Optimize 6.69 5.43
Deep heuristic Optimize 6.65 5.45
Learning 6.68 5.38
Random 5.38 4.45
Human behavior 6.74 5.43
Maximum 7.17 5.95
Table 13: Out-of-sample expected payos.
In conclusion, this simple learning model appears to be a possible explanation for
how the participants come to use these near-optimal heuristics in our experiment with
simple adjustments of the heuristics used after each game.
45
References
Bacharach, M. (2006). Beyond individual choice: teams and frames in game theory .
Princeton University Press.
Bardsley, N., J. Mehta, C. Starmer, and R. Sugden (2010). Explaining focal points:
Cognitive hierarchy theory versus team reasoning. Economic Journal 120 (543),
40{79.
Bruhin, A., E. Fehr, and D. Schunk (2019). The many faces of human sociality:
Uncovering the distribution and stability of social preferences. Journal of the European
Economic Association 17 (4), 1025{1069.
Camerer, C. F. (2003). Behavioral Game Theory: Experiments in Strategic Interaction .
Princeton University Press.
Camerer, C. F., T.-H. Ho, and J.-K. Chong (2004, aug). A Cognitive Hierarchy Model
of Games. The Quarterly Journal of Economics 119 (3), 861{898.
Caplin, A. and M. Dean (2013). Behavioral implications of rational inattention with
shannon entropy. Technical report, National Bureau of Economic Research.
Chen, D. L., M. Schonger, and C. Wickens (2016, March). oTree|An open-source
platform for laboratory, online, and eld experiments. Journal of Behavioral and
Experimental Finance 9 , 88{97.
Costa-Gomes, M. A. and G. Weizs acker (2008, jul). Stated Beliefs and Play in Normal-
Form Games. Review of Economic Studies 75 (3), 729{762.
Crawford, V. P., M. A. Costa-Gomes, and N. Iriberri (2013, mar). Structural Models of
Nonequilibrium Strategic Thinking: Theory, Evidence, and Applications. Journal of
Economic Literature 51 (1), 5{62.
Devetag, G., S. Di Guida, and L. Polonio (2016, mar). An eye-tracking study of
feature-based choice in one-shot games. Experimental Economics 19 (1), 177{201.
Dhami, S. (2016). The foundations of behavioral economic analysis . Oxford University
Press.
Ert, E. and I. Erev (2013). On the descriptive value of loss aversion in decisions under
risk: Six clarications. Judgment and Decision Making 8 (3), 214{235.
Fehr, E. and K. M. Schmidt (1999). A theory of fairness, competition, and cooperation.
The Quarterly Journal of Economics 114 (3), 817{868.
Fudenberg, D., F. Drew, D. K. Levine, and D. K. Levine (1998). The theory of learning
in games , Volume 2. MIT press.
Fudenberg, D., J. Kleinberg, A. Liang, and S. Mullainathan (2022). Measuring the
completeness of economic models. Journal of Political Economy 130 (4), 956{990.
46
Fudenberg, D. and A. Liang (2019, dec). Predicting and Understanding Initial Play.
American Economic Review 109 (12), 4112{4141.
Gershman, S. J., E. J. Horvitz, and J. B. Tenenbaum (2015). Computational ra-
tionality: A converging paradigm for intelligence in brains, minds, and machines.
Science 349 (6245).
Gigerenzer, G. and P. M. Todd (1999). Simple Heuristics That Make Us Smart . Oxford
University Press, USA.
Goeree, J. K. and C. A. Holt (2004). A model of noisy introspection. Games and
Economic Behavior 46 (2), 365{382.
Goldstein, D. G. and G. Gigerenzer (2002). Models of ecological rationality: The
recognition heuristic. Psychological Review 109 (1), 75{90.
Griths, T. L., F. Lieder, and N. D. Goodman (2015). Rational use of cognitive
resources: Levels of analysis between the computational and the algorithmic. Topics
in Cognitive Science 7 (2), 217{229.
Grimm, V. and F. Mengel (2012). An experiment on learning in a multiple games
environment. Journal of Economic Theory 147 (6), 2220{2259.
Hartford, J. S., J. R. Wright, and K. Leyton-Brown (2016). Deep Learning for Predicting
Human Strategic Behavior. In Advances in Neural Information Processing Systems ,
Volume 29. Curran Associates, Inc.
Heap, S. H., D. R. Arjona, and R. Sugden (2014). How portable is level-0 behavior?
a test of level-k theory in games with non-neutral frames. Econometrica 82 (3),
1133{1151.
Hebert, B. and M. Woodford (2019). Rational inattention when decisions take time.
Journal of Chemical Information and Modeling 53 (9), 1689{1699.
Howes, A., R. L. Lewis, and A. Vera (2009). Rational Adaptation Under Task and
Processing Constraints: Implications for Testing Theories of Cognition and Action.
Psychological Review 116 (4), 717{751.
Imai, T., T. A. Rutter, and C. F. Camerer (2020, sep). Meta-Analysis of Present-Bias
Estimation Using Convex Time Budgets. The Economic Journal 186 (2), 227{236.
Izard, V. and S. Dehaene (2008). Calibrating the mental number line. Cognition 106 (3),
1221{1247.
Jehiel, P. (2005, aug). Analogy-based expectation equilibrium. Journal of Economic
Theory 123 (2), 81{104.
Krueger, P., F. Callaway, S. Gul, T. Griths, and F. Lieder (2022, January). Discovering
Rational Heuristics for Risky Choice.
47
Lewis, R. L., A. Howes, and S. Singh (2014). Computational rationality: Linking
mechanism and behavior through bounded utility maximization. Topics in Cognitive
Science 6 (2), 279{311.
Lieder, F. and T. L. Griths (2015). When to use which heuristic: A rational solution
to the strategy selection problem. Proceedings of the 37th annual conference of the
cognitive science society 1 (3), 1{6.
Lieder, F. and T. L. Griths (2017, November). Strategy selection as rational metarea-
soning. Psychological Review 124 (6), 762{794.
Lieder, F. and T. L. Griths (2020). Resource-rational analysis: Understanding human
cognition as the optimal use of limited computational resources. Behavioral and
Brain Sciences 43 , e1.
Lieder, F., P. M. Krueger, and T. Griths (2017). An automatic method for discovering
rational heuristics for risky choice. In Proceedings of the 39th Annual Meeting of the
Cognitive Science Society .
Mat ejka, F. and A. McKay (2015, jan). Rational Inattention to Discrete Choices: A New
Foundation for the Multinomial Logit Model. American Economic Review 105 (1),
272{298.
Mengel, F. and E. Sciubba (2014). Extrapolation and structural similarity in games.
Economics Letters 125 (3), 381{385.
Misyak, J. B. and N. Chater (2014). Virtual bargaining: A theory of social
decision-making. Philosophical Transactions of the Royal Society B: Biological Sci-
ences 369 (1655).
Nagel, R. (1995). Unraveling the Guessing Game. American Economic Review 85 (5),
1313{1326.
Peysakhovich, A. and D. G. Rand (2016). Habits of virtue: Creating norms of coopera-
tion and defection in the laboratory. Management Science 62 (3), 631{647.
Polonio, L., S. Di Guida, and G. Coricelli (2015, nov). Strategic sophistication and
attention in games: An eye-tracking study. Games and Economic Behavior 94 , 80{96.
Savage, L. J. (1954). The Foundations of Statistics . John Wiley & Sons.
Simon, H. A. (1976). From substantive to procedural rationality. In 25 years of economic
theory , pp. 65{86. Springer.
Sims, C. A. (1998). Stickiness. Carnegie-Rochester Conference Series on Public
Policy 49 , 317{356.
Spiliopoulos, L. and R. Hertwig (2020, March). A map of ecologically rational heuristics
for uncertain strategic worlds. Psychological Review 127 (2), 245{280.
48
Stahl, D. O. and P. W. Wilson (1994). Experimental evidence on players' models of
other players. Journal of Economic Behavior & Organization 25 (3), 309{327.
Stahl, D. O. and P. W. Wilson (1995). On players' models of other players: Theory
and experimental evidence.
Steiner, J., C. Stewart, and F. Mat ejka (2017). Rational Inattention Dynamics: Inertia
and Delay in Decision-Making. Econometrica 85 (2), 521{553.
Stewart, N., S. G achter, T. Noguchi, and T. L. Mullett (2016). Eye Movements in
Strategic Choice. 156(October 2015), 137{156.
Sugden, R. (2003). The logic of team reasoning. Philosophical explorations 6 (3),
165{181.
Todd, P. M. and G. E. Gigerenzer (2012). Ecological Rationality: Intelligence in the
World. Oxford University Press.
Tun cel, T. and J. K. Hammitt (2014). A new meta-analysis on the WTP/WTA disparity.
Journal of Environmental Economics and Management 68 (1), 175{187.
Wright, J. R. and K. Leyton-Brown (2017, nov). Predicting human behavior in
unrepeated, simultaneous-move games. Games and Economic Behavior 106 (2),
16{37.
49