Bayesian Theory of Mind: Modeling Joint Belief-Desire Attribution
Chris L. Baker (clbaker@mit.edu)
Rebecca R. Saxe (saxe@mit.edu)
Joshua B. Tenenbaum (jbt@mit.edu)
Department of Brain and Cognitive Sciences, MIT
Cambridge, MA 02139
Abstract
We present a computational framework for understanding The-
ory of Mind (ToM) : the human capacity for reasoning about
agents’ mental states such as beliefs and desires. Our Bayesian
model of ToM (or BToM) expresses the predictive model of
belief- and desire-dependent action at the heart of ToM as
apartially observable Markov decision process (POMDP) ,
and reconstructs an agent’s joint belief state and reward func-
tion using Bayesian inference, conditioned on observations of
the agent’s behavior in some environmental context. We test
BToM by showing participants sequences of agents moving in
simple spatial scenarios and asking for joint inferences about
the agents’ desires and beliefs about unobserved aspects of the
environment. BToM performs substantially better than two
simpler variants: one in which desires are inferred without ref-
erence to an agent’s beliefs, and another in which beliefs are
inferred without reference to the agent’s dynamic observations
in the environment.
Keywords: Theory of mind; Social cognition; Action un-
derstanding; Bayesian inference; Partially Observable Markov
Decision Processes
Introduction
Central to human social behavior is a theory of mind (ToM) ,
the capacity to explain and predict people’s observable ac-
tions in terms of unobservable mental states such as beliefs
and desires. Consider the case of Harold, who leaves his dorm
room one Sunday morning for the campus library. When he
reaches to open the library’s front door he will ﬁnd that it is
locked – closed on Sunday. How can we explain his behav-
ior? It seems plausible that he wants to get a book, that he
believes the book he wants is at the library, and that he also
believes (falsely, it turns out) that the library is open on Sun-
day.
Such mental state explanations for behavior go well be-
yond the observable data, leading to an inference problem
that is fundamentally ill-posed. Many different combinations
of beliefs and desires could explain the same behavior, with
inferences about the strengths of beliefs and desires trading
off against each other, and relative probabilities modulated
heavily by context. Perhaps Harold is almost positive that the
library will be closed, but he needs a certain book so badly
that he still is willing to go all the way across campus on the
off chance it will be open. This explanation seems more prob-
able if Harold shows up to ﬁnd the library locked on Saturday
at midnight, as opposed to noon on Tuesday. If he arrives
after hours already holding a book with a due date of tomor-
row, it is plausible that he knows the library is closed and is
seeking not to get a new book, but merely to return a book
checked out previously to the night drop box.Several authors have recently proposed models for how
people infer others’ goals or preferences as a kind of Bayesian
inverse planning or inverse decision theory (Baker, Saxe, &
Tenenbaum, 2009; Feldman & Tremoulet, 2008; Lucas, Grif-
ﬁths, Xu, & Fawcett, 2009; Bergen, Evans, & Tenenbaum,
2010; Yoshida, Dolan, & Friston, 2008; Ullman et al., 2010).
These models adapt tools from control theory, econometrics
and game theory to formalize the principle of rational ac-
tionat the heart of children and adults’ concept of intentional
agency (Gergely, N ´adasdy, Csibra, & Bir ´o, 1995; Dennett,
1987): all else being equal, agents are expected to choose ac-
tions that achieve their desires as effectively and efﬁciently
as possible, i.e., to maximize their expected utility. Goals
or preferences are then inferred based on which objective or
utility function the observed actions maximize most directly.
ToM transcends knowledge of intentional agents’ goals and
preferences by incorporating representational mental states
such as subjective beliefs about the world (Perner, 1991). In
particular, the ability to reason about false beliefs has been
used to distinguish ToM from non-representational theories
of intentional action (Wimmer & Perner, 1983; Onishi &
Baillargeon, 2005). Our goal in this paper is to model hu-
man ToM within a Bayesian framework. Inspired by mod-
els of inverse planning, we cast Bayesian ToM (BToM) as a
problem of inverse planning and inference, representing an
agent’s planning and inference about the world as a partially
observable Markov decision process (POMDP), and invert-
ing this forward model using Bayesian inference. Critically,
this model includes representations of both the agent’s de-
sires (as a utility function), and the agent’s own subjective
beliefs about the environment (as a probability distribution),
which may be uncertain and may differ from reality. We test
the predictions of this model quantitatively in an experiment
where people must simultaneously judge beliefs and desires
for agents moving in simple spatial environments under in-
complete or imperfect knowledge.
Important precursors to our work are several computational
models (Goodman et al., 2006; Bello & Cassimatis, 2006;
Goodman, Baker, & Tenenbaum, 2009) and informal theo-
retical proposals by developmental psychologists (Wellman,
1990; Gopnik & Meltzoff, 1997; Gergely & Csibra, 2003).
Goodman et al. (2006) model how belief and desire infer-
ences interact in the classic “false belief” task used to assess
ToM reasoning in children (Wimmer & Perner, 1983). This
model instantiates the schema shown in Fig. 1(a) as a causal
Bayesian network with several psychologically interpretable,
but task-dependent parameters. Goodman et al. (2009) model
adult inferences of an agent’s knowledge of the causal struc-
ture of a simple device (“Bob’s box”) based on observing
the agent interacting with the device. To our knowledge, our
work here is the ﬁrst attempt to explain people’s joint infer-
ences about agents’ beliefs and desires by explicitly inverting
POMDPs – and the ﬁrst model capable of reasoning about the
graded strengths and interactions between agents’ beliefs and
desires, along with the origins of agents’ beliefs via environ-
mentally constrained perceptual observations.
Computational Framework
This section describes Bayesian Theory of Mind (BToM): a
theory-based Bayesian framework (Tenenbaum, Grifﬁths, &
Kemp, 2006) that characterizes ToM in terms of Bayesian in-
ference over a formal, probabilistic version of the schema in
Fig. 1(a). BToM represents an ideal observer using a the-
ory of mind to understand the actions of an individual agent
within some environmental context. This ideal-observer anal-
ysis of ToM asks how closely human judgments approach the
ideal limit, but also what mental representations are necessary
to explain human judgments under hypothetically unbounded
computational resources. We will ﬁrst describe BToM in gen-
eral, but informal terms before progressing to the mathemati-
cal details involved in modeling our experimental domain.
Informal sketch
For concreteness, we use as a running example a simple spa-
tial context (such as a college campus or urban landscape)
deﬁned by buildings and perceptually distinct objects, with
agents’ actions corresponding to movement, although in gen-
eral BToM can be deﬁned over arbitrary state and action
spaces (for example, a card game where the state describes
players’ hands and actions include draw or fold). The ob-
server’s representation of the world is composed of the en-
vironment state and the agent state (Fig. 1(a)). In a spatial
context, the state of the environment represents its physical
conﬁguration, e.g., the location of buildings and objects, and
the state of the agent speciﬁes its objective, external proper-
ties, such as its physical location in space.
The observer’s theory of the agent’s mind includes repre-
sentations of the agent’s subjective desires and beliefs, and
the principles by which desires and beliefs are related to ac-
tions and the environment. Similar to previous models, the
content of the agent’s desire consists of objects or events in
the world. The agent’s degree of desire is represented in terms
of the subjective reward received for taking actions in certain
states, e.g., acting to attain a goal while in close proximity to
the goal object. The agent can also act to change its own state
or the environment state at a certain cost, e.g., navigating to
reach a goal may incur a small cost at each step.
The main novel component of the current model is the in-
clusion of a representation of beliefs. Like desires, beliefs are
deﬁned by both their content and the strength or degree with
which they are held. The content of a belief is a representa-
tion corresponding to a possible world. For instance, if the
(a) (b)
Agent
StateEnvironment
State
Belief Desire
ActionPrinciple of
rational actionPrinciple of
rational beliefXt-1
Yt-1R
Ot-1
Bt-1
At-1Xt
Yt
Ot
Bt
AtAgentWorldFigure 1: Causal structure of theory of mind. Grey shaded nodes
are assumed to be observed (for the observer; not necessarily for the
agent, as described in the main text). (a)Schematic model of theory
of mind. Traditional accounts of ToM (e.g. Dennett, 1987; Wellman,
1990; Gopnik & Meltzoff, 1997) have proposed informal versions of
this schema, characterizing the content and causal relations of ToM
in commonsense terms, e.g., “seeing is believing” for the princi-
ple of rational belief. (b)Observer’s grounding of the theory as a
dynamic Bayes net (DBN). The DBN encodes the observer’s joint
distribution over an agent’s beliefs B1:Tand desires Rover time,
given the agent’s physical state sequence x1:Tin environment y.
agent is unsure about the location of a particular object, its
belief contents are worlds in which the object is in different
locations. The agent’s degree of belief reﬂects the subjective
probability it assigns to each possible world.
The principles governing the relation between the world
and the agent’s beliefs, desires and actions can be naturally
expressed within partially observable Markov decision pro-
cesses (POMDPs). POMDPs capture the causal relation be-
tween beliefs and the world via the principle of rational belief,
which formalizes how the agent’s belief is affected by obser-
vations in terms of Bayesian belief updating. Given an ob-
servation, the agent updates its degree of belief in a particular
world based on the likelihood of receiving that observation in
that world. In a spatial setting, observations depend on the
agent’s line-of-sight visual access to features of the environ-
ment. POMDPs represent how beliefs and desires cause ac-
tions via the principle of rational action, or rational planning.
Intuitively, rational POMDP planning provides a predictive
model of an agent optimizing the tradeoff between exploring
the environment to discover the greatest rewards, and exploit-
ing known rewards to minimize costs incurred.
On observing an agent’s behavior within an environment,
the beliefs and desires that caused the agent to generate this
behavior are inferred using Bayesian inference. The observer
maintains a hypothesis space of joint beliefs and desires,
which represent the agent’s initial beliefs about the environ-
ment state and the agent’s static desires for different goals.
For each hypothesis, the observer evaluates the likelihood of
generating the observed behavior given the hypothesized be-
lief and desire. The observer integrates this likelihood with
the prior over mental states to infer the agent’s joint belief
and desire.
As an example of how this works, consider Fig. 2. The
“college campus” environment is characterized by the cam-
pus size, the location and size of buildings, and the location of
)UDPH
)UDPH
)UDPH
Figure 2: Example experimental stimulus. The small blue sprite represents the location of the agent, and the black trail with arrows superim-
posed records the agent’s movement history. The two yellow cells in opposite corners of the environment represent spots where trucks can
park, and each contains a different truck. The shaded grey area of each frame represents the area that is outside of the agent’s current view.
several different goal objects, here “food trucks”. The agent
is a hungry graduate student, leaving his ofﬁce and walk-
ing around campus in search of satisfying lunch food. There
are three trucks that visit campus: Korean (K), Lebanese (L)
and Mexican (M), but only two parking spots where trucks
are allowed to park, highlighted with a yellow background
in Fig. 2. The student’s ﬁeld of view is represented by the
unshaded region of the environment.
In Fig. 2, the student can initially only see where K (but not
L) is parked. Because the student can see K, they know that
the spot behind the building either holds L, M, or is empty.
By frame 10, the student has passed K, indicating that they
either want L or M (or both), and believe that their desired
truck is likely to be behind the building (or else they would
have gone straight to K under the principle of rational action).
After frame 10, the agent discovers that L is behind the build-
ing and turns back to K. Obviously, the agent prefers K to
L, but more subtly, it also seems likely that the agent wants
M more than either K or L, despite M being absent from the
scene! BToM captures this inference by resolving the desire
for L or M over K in favor of M after the agent rejects L.
In other words, BToM infers the best explanation for the ob-
served behavior – the only consistent desire that could lead
the agent to act the way it did.
Formal modeling
In the food-truck domain, the agent occupies a discrete state
spaceXof points in a 2D grid. The environment state Yis
the set of possible assignments of the K, L and M trucks to
parking spots. Possible actions include North, South, East,
West, Stay, and Eat. Valid actions yield the intended transi-
tion with probability 1−/epsilon1and do nothing otherwise; invalid
actions (e.g., moving into walls) have no effect on the state.
The agent’s visual observations are represented by the iso-
vistfrom the agent’s location: a polygonal region contain-
ing all points of the environment within a 360-degree ﬁeld
of view (Davis & Benedikt, 1979; Morariu, Prasad, & Davis,
2007). Example isovists from different locations in one en-
vironment are shown in Fig. 2. The observation distributionP(o|x,y)encodes which environments in Yare consistent
with the contents of the isovist from location x. We model
observation noise with the simple assumption that ambigu-
ous observations can occur with probability ν, as if the agent
failed to notice something that should otherwise be visible.
The observer represents the agent’s belief as a probabil-
ity distribution over Y; fory∈Y,b(y)denotes the agent’s
degree of belief that yis the true state of the environment.
Bayesian belief updating at time tis a deterministic function
of the prior belief bt−1, the observation ot, and the world state
/angbracketleftxt,y/angbracketright. The agent’s updated degree of belief in environment
ysatisﬁesbt(y)∝P(ot|xt,y)bt−1(y).
The agent’s reward function R(x,y,a )encodes the subjec-
tive utility the agent derives from taking action afrom the
state/angbracketleftxt,y/angbracketright. Each action is assumed to incur a cost of 1.
Rewards result from taking the “Eat” action while at a food
truck; the magnitude of the reward depends on the strength
of the agent’s desire to eat at that particular truck. Once the
student has eaten, all rewards and costs cease, implying that
rational agents should optimize the tradeoff between the num-
ber of actions taken and the reward obtained.
The agent’s POMDP is deﬁned by the state space, the
action space, the world dynamics, the observation model,
and the reward function. We approximate the optimal value
function of the POMDP for each hypothesized reward func-
tion using a point-based value iteration algorithm over a uni-
form discretization of the belief space. The agent’s policy is
stochastic, given by the softmax of the lookahead state-action
value function QLA(Hauskrecht, 2000): P(a|b,x,y )∝
exp(βQLA(b,x,y,a )). Theβparameter establishes the de-
gree of determinism with which the agent executes its policy,
capturing the intuition that agents tend to, but do not always
follow the optimal policy.
Our approach to joint belief and desire inference is closely
related the model of belief ﬁltering in Zettlemoyer, Milch, and
Kaelbling (2009), restricted to the case of one agent reasoning
about the beliefs of another. Fig. 1(b) shows the observer’s
dynamic Bayes net (DBN) model of an agent’s desires, states,
observations, beliefs and actions over time. The observer’s
belief and reward inferences are given by the joint posterior
marginal over the agent’s beliefs and rewards at time t, given
the state sequence up until T≥t:P(bt,r|x1:T,y). This com-
putation is analogous to the forward-backward algorithm in
hidden Markov models, and provides the basis for model pre-
dictions of people’s joint belief and desire inferences in our
experiment.
To perform inference over the multidimensional, contin-
uous space of beliefs and rewards, we uniformly discretize
the hypothesis spaces of beliefs and reward functions with
grid resolutions of 7. The range of reward values was cali-
brated to the spatial scale of our environments, taking values
−20,0,...,100for each truck. Model predictions were based
on the student’s expected reward value for each truck (K, L,
M) and the expected degree-of-belief in each possible world
for each trial.
Alternative models
To test whether the full representational capacity of our model
is necessary to understand people’s mental state attributions,
we formulate two alternative models as special cases of our
joint inference model. Each alternative model “lesions” a
central component of the full model’s representation of be-
liefs, and tests whether it is possible to explain people’s infer-
ences about agents’ desires in our experiment without appeal
to a full-ﬂedged theory of mind.
Our ﬁrst alternative model is called TrueBel. This model
assumes that the state is fully observable to the agent, i.e.,
that the agent knows the location of every truck, and plans
to go directly to the truck that will provide the maximal re-
ward while incurring the least cost. We hypothesized that this
model would correlate moderately well with people’s desire
judgments, because of the statistical association between de-
sired objects and actions.
Our second alternative model is called NoObs. In this
model, the agent has an initial belief about the state of the
environment, but there is no belief updating – the initially
sampled belief remains ﬁxed throughout the trial. We hy-
pothesized that this model might ﬁt people’s belief and desire
inferences in situations where the agent appeared to move to-
ward the same truck throughout the entire trial, but that for ac-
tions that required belief updating or exploration to explain,
for instance, when the agent began by exploring the world,
then changed direction based on its observation of the world
state, NoObs would ﬁt poorly.
Experiment
Fig. 4 illustrates our experimental design. Truck labels were
randomized in each trial of the experiment, but we will de-
scribe the experiment and results using the canonical, un-
scrambled ordering Korean (K), Lebanese (L), Mexican (M).
The experiment followed a 3 ×5×2×3×2 design.
These factors can be divided into 30 (3 ×5×2) unique paths
and6(3×2) unique environmental contexts. There were
3 different starting points in the environment: “Left”, “Mid-
dle”, or “Right”; all shown in Fig. 4. These starting pointswere crossed with 5 different trajectories: “Check-Left, go
to K”; “Check-Left, go to L/M”; “Check-Right, go to K”;
“Check-Right, go to L/M”; and “No-check, go straight to K”.
Four of these trajectories are shown in Fig. 4. Each path was
shown with 2 different judgment points, or frames at which
the animation paused and subjects gave ratings based on the
information shown so far. Judgment points were either at the
moment the student became able to see the parking spot that
was initially occluded (“Middle”; e.g., frame 10 in Fig. 2), or
at the end of the path once the student had eaten (“Ending”;
e.g., frame 15 in Fig. 2). All potential paths were crossed with
6 environmental contexts, generated by combining 3 different
building conﬁgurations: “O”, “C” and “backwards C”, (all
shown in Fig. 4) with 2 different goal conﬁgurations: “One
truck” or “Two trucks” present; both shown in Fig. 4.
After all possible trials from this design were generated,
all invalid trials (in which the student’s path intersected with
a building), and all “Ending” trials in which the path did not
ﬁnish at a truck were removed. This left 78 total trials. Of
these, 5 trials had a special status. These were trials in the
“O” environment with paths in which the student began at
the Right starting point, and then followed a Check-Left tra-
jectory. These paths had no rational interpretation under the
BToM model, because the Check-Right trajectory was always
a more efﬁcient choice, no matter what the student’s initial
belief or desire. These “irrational” trials are analyzed sepa-
rately in the Results section.
Several factors were counterbalanced or randomized.
Stimulus trials were presented in pseudo-random order. Each
trial randomly scrambled the truck labels, and randomly re-
ﬂected the display vertically and horizontally so that subjects
would remain engaged with the task and not lapse into a
repetitive strategy. Each trial randomly displayed the agent
in 1 of 10 colors, and sampled a random male or female name
without replacement. This ensured that subjects did not gen-
eralize information about one student’s beliefs or desires to
students in subsequent trials.
The experimental task involved rating the student’s degree
of belief in each possible world (Lebanese truck behind the
building (L); Mexican truck behind the building (M); or noth-
ing behind the building (N)), and rating how much the student
liked each truck. All ratings were on a 7-point scale. Belief
ratings were made retrospectively, meaning that subjects were
asked to rate what the student thought was in the occluded
parking spot before they set off along their path, basing their
inference on the information from the rest of the student’s
path. The rating task counterbalanced the side of the monitor
on which the “likes” and “believes” questions were displayed.
Subjects ﬁrst completed a familiarization stage that ex-
plained all details of our displays and the scenarios they de-
picted. To ensure that subjects understood what the students
could and couldn’t see, the familiarization explained the vi-
sualization of the student’s isovist, which was updated along
each step of the student’s path. The isovist was displayed
during the testing stage of the experiment as well (Fig. 2).
Participants were 17 members of the MIT subject pool, 6
female, and 11 male. One male subject did not understand
the instructions and was excluded from the analysis.
Results & Discussion
Debrieﬁng of subjects suggested that many were confused by
the “Middle” judgment point trials; this was also reﬂected
by greater variability in people’s judgments within these tri-
als. Because of this, our analyses only include trials from the
“Ending” judgment point condition, which accounted for 54
out of the 78 total trials.
We begin by analyzing the overall ﬁt between people’s
judgments and our three models, and then turn to a more de-
tailed look at several representative scenarios. Two parame-
tersβandνwere ﬁt for the BToM model; only the determin-
ism parameter βis relevant for the TrueBel and NoObs mod-
els. Parameter ﬁts are not meant to be precise; we report the
best values found among several drawn from a coarse grid.
BToM predicts people’s judgments about agents’ desires
relatively well, and less well but still reasonably for judg-
ments about agents’ initial beliefs (Fig. 3). In Fig. 3, data
from the “irrational” trials are plotted with magenta circles,
and account for most of the largest outliers. TrueBel and
NoObs ﬁt signiﬁcantly worse for desire judgments and pro-
vide no reasonable account of belief judgments. TrueBel’s
belief predictions are based on the actual state of the world
in each trial; the poor correlation with people’s judgments
demonstrates that people did not simply refer to the true world
state in their belief attributions. The NoObs model in prin-
ciple can infer agents’ beliefs, but without a theory of how
beliefs are updated from observations it must posit highly
implausible initial beliefs that correlate poorly with subjects’
judgments over the whole set of experimental conditions.
Fig. 4 shows several revealing comparisons of human judg-
ments and model predictions in speciﬁc cases. When the
agent follows a long path to an unseen goal (A1) it is sug-
gestive of a strong initial belief that a more desirable truck is
present behind the wall. In contrast, going straight to a nearby
observed truck says only that this truck is likely to be desired
more than the others (A2). When the agent goes out of its
way to check an unseen parking spot, sees the second truck
there, and returns to the previously seen truck, it suggests a
strong desire for the one truck not present (compare B1 to
B2). Finally, the relative strengths of inferences about desires
and initial beliefs are modulated by how far the agent must
travel to observe the unseen parking spot (compare C1 to C2,
and C3 to C4). In each of these cases people reﬂect the same
qualitative trends predicted by the model.
The ﬁnding that people’s inferences about agents’ desires
are more robust than inferences about beliefs, and more con-
sistent with the model’s predictions, is intriguingly consistent
with classic asymmetries between these two kinds of men-
tal state attributions in the ToM literature. Intentional actions
are the joint consequence of an agent’s beliefs and desires,
but inferences from actions back to beliefs will frequently
be more difﬁcult and indirect than inferences about desires.
People
Model (β=1.0; ν=0.1)0 0.2 0.4 0.6 0.8 100.20.40.60.81
Model (β=1.0; ν=0.1)PeopleDesire Inference (r=0.90) Belief Inference (r=0.76)
1 2 3 4 5 6 71234567BToM TrueBel NoObs
Desire 0.90 0.67 0.61
Belief 0.76 0.11 0.39
Figure 3: Scatter plots show overall correlations between BToM
model predictions and human judgments about desires and beliefs in
our experiment. Each dot corresponds to the mean judgment of sub-
jects in one experimental condition. Magenta circles correspond to
trials which had no rational interpretation in terms of POMDP plan-
ning. The table shows correlations with human judgments for BToM
and two simpler variants, which do not represent beliefs (TrueBel)
or do not update beliefs based on observations (NoObs).
Actions often point with salient perceptual cues directly to-
ward an agent’s goal or desired state. When a person wants
to take a drink, her hand moves clearly toward the glass on
the table. In contrast, no motion so directly indicates what
she believes to be inside the glass. Infants as young as ﬁve
months can infer agents’ goals from their actions (Gergely &
Csibra, 2003), while inferences about representational beliefs
seem to be present only in rudimentary forms by age one and
a half, and in more robust forms only by age 4 (Onishi &
Baillargeon, 2005).
Conclusion & Future Work
Our experiment showed that human ToM inferences come
surprisingly close to those of an ideal rational model, per-
forming Bayesian inference over beliefs and desires simul-
taneously. By comparing with two alternative models we
showed that it was necessary to perform joint inference about
agents’ beliefs and desires, and to explicitly model the agent’s
observational process, as part of modeling people’s theory of
mind judgments. Crucially, it was also necessary to represent
initial uncertainty over both the agent’s beliefs and desires.
We have not attempted to distinguish here between agents’
general desires and their speciﬁc goals or intentions at par-
ticular moments of action. In previous work we showed
that inferences about which object is most likely to be an
agent’s instantaneous goal were well explained using a simi-
lar Bayesian inverse planning framework (Baker et al., 2009).
However, goals are not always about objects. In the present
experiments, it feels intuitive to describe agents as attempting
to maximize their overall expected utility by adopting a com-
bination of object- and information-seeking goals (or goals
intended to update the agent’s beliefs). For instance, in Fig. 4,
B1 it looks as if the agent initially had a goal of ﬁnding out
which truck was parked on the other side of the wall, and then
after failing to ﬁnd their preferred truck (M) there, set a goal
of returning to the previously observed second-favorite truck
KL.A1
LM N00.51
K LM
KLC1
LM N00.51
K LM1357KL.C3
LM N00.51
K LM1357KLB1
LM N00.51
K LMKLA2
LM N00.51
K LM
KLC2
LM N00.51
K LM1357KL.C4
LM N00.51
K LM13571357
1357
1357
1357B2
K
LM N00.51
K LMModel
People
Desires Beliefs Desires Beliefs Desires Beliefs Desires BeliefsFigure 4: Eight representative scenarios from the experiment, showing the agent’s path, BToM model predictions for the agent’s desires (for
trucks K, L or M, on a scale of 1 to 7) and beliefs about the unseen parking spot (for trucks L, M or no truck (N), normalized to a probability
scale from 0 to 1), and mean human judgments for these same mental states. Error bars show standard error (n=16).
(K). Our model can produce and interpret such behavior, but it
does so without positing these explicit subgoals or the corre-
sponding parse of the agent’s motion into subsequences, each
aimed to achieve a speciﬁc goal. Extending our model to
incorporate a useful intermediate representation of goal se-
quences is an important direction for future work. Even with-
out these complexities, however, we ﬁnd it encouraging to see
how well we can capture people’s joint attributions of beliefs
and desires as Bayesian inferences over a simple model of
rational agents’ planning and belief updating processes.
Acknowledgments This work was supported by the JSMF
Causal Learning Collaborative, ONR grant N00014-09-0124
and ARO MURI contract W911NF-08-1-0242. We thank
L. Kaelbling, T. Dietterich, N. Goodman, N. Kanwisher, L.
Schulz, and H. Gweon for helpful discussions and comments.
References
Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009). Action under-
standing as inverse planning. Cognition ,113, 329–349.
Bello, P., & Cassimatis, N. L. (2006). Developmental accounts of
theory-of-mind acquisition: Achieving clarity via computational
cognitive modeling. In Proceedings of the Twenty-Eighth Annual
Conference of the Cognitive Science Society (pp. 1014–1019).
Bergen, L., Evans, O. R., & Tenenbaum, J. B. (2010). Learning
structured preferences. In Proceedings of the Thirty-Second An-
nual Conference of the Cognitive Science Society (pp. 853–858).
Davis, L. S., & Benedikt, M. L. (1979). Computational models of
space: Isovists and isovist ﬁelds. Computer Graphics and Image
Processing ,11, 49–72.
Dennett, D. C. (1987). The intentional stance . Cambridge, MA:
MIT Press.
Feldman, J., & Tremoulet, P. D. (2008). The attribution of mental
architecture from motion: Towards a computational theory (Tech.
Rep. No. RuCCS TR-87). Department of Psychology, Rutgers
University.
Gergely, G., & Csibra, G. (2003). Teleological reasoning in infancy:
The na ¨ıve theory of rational action. Trends in Cognitive Sciences ,
7(7), 287–292.Gergely, G., N ´adasdy, Z., Csibra, G., & Bir ´o, S. (1995). Taking the
intentional stance at 12 months of age. Cognition ,56, 165–193.
Goodman, N. D., Baker, C. L., Bonawitz, E. B., Mansinghka, V. K.,
Gopnik, A., Wellman, H., et al. (2006). Intuitive theories of mind:
A rational approach to false belief. In Proceedings of the Twenty-
Eighth Annual Conference of the Cognitive Science Society (pp.
1382–1390).
Goodman, N. D., Baker, C. L., & Tenenbaum, J. B. (2009). Cause
and intent: Social reasoning in causal learning. In Proceedings
of the Thirty-First Annual Conference of the Cognitive Science
Society (pp. 2759–2764).
Gopnik, A., & Meltzoff, A. N. (1997). Words, thoughts, and theo-
ries. Cambridge, MA: MIT Press.
Hauskrecht, M. (2000). Value-function approximations for partially
observable Markov decision processes. Journal of Artiﬁcial Intel-
ligence Research ,13, 33–94.
Lucas, C. G., Grifﬁths, T. L., Xu, F., & Fawcett, C. (2009). A
rational model of preference learning and choice prediction by
children. In Advances in Neural Information Processing Systems
21(pp. 985–992).
Morariu, V. I., Prasad, V. S. N., & Davis, L. S. (2007). Human
activity understanding using visibility context. In IEEE/RSJ IROS
Workshop: From sensors to human spatial concepts (FS2HSC).
Onishi, K. H., & Baillargeon, R. (2005). Do 15-month-old infants
understand false beliefs? Science ,308(5719), 255–258.
Perner, J. (1991). Understanding the representational mind . Cam-
bridge, MA: MIT Press.
Tenenbaum, J. B., Grifﬁths, T. L., & Kemp, C. (2006). Theory-based
Bayesian models of inductive learning and reasoning. Trends in
Cognitive Sciences ,10(7), 309–318.
Ullman, T. D., Baker, C. L., Macindoe, O., Evans, O., Goodman,
N. D., & Tenenbaum, J. B. (2010). Help or hinder: Bayesian
models of social goal inference. In Advances in Neural Informa-
tion Processing Systems 22 (pp. 1874–1882).
Wellman, H. M. (1990). The child’s theory of mind . Cambridge,
MA: MIT Press.
Wimmer, H., & Perner, J. (1983). Beliefs about beliefs: Represen-
tation and constraining function of wrong beliefs in young chil-
dren’s understanding of deception. Cognition ,13(1), 103–128.
Yoshida, W., Dolan, R. J., & Friston, K. J. (2008). Game theory of
mind. PLoS Computational Biology ,4(12), 1–14.
Zettlemoyer, L. S., Milch, B., & Kaelbling, L. P. (2009). Multi-agent
ﬁltering with inﬁnitely nested beliefs. In Advances in Neural In-
formation Processing Systems 21 (pp. 1905–1912).

Chris L. Baker
Rebecca R. Saxe
Joshua B. Tenenbaum