Identifying category representations for complex stimuli using discrete Markov chain Monte Carlo with people.

Identifying category representations for complex stimuli using discrete
Markov chain Monte Carlo with people
Anne S. Hsu1&Jay B. Martin2&Adam N. Sanborn3&Thomas L. Griffiths4
Published online: 13 February 2019
#The Author(s) 2019
Abstract
With the explosion of Bbig data, ^digital repositories of texts and images are growing rapidly. These datasets present new
opportunities for psychological research, but they require new methodologies before researchers can use these datasets to yieldinsights into human cognition. We present a new method that allows psychological researchers to take advantage of text andimage databases: a procedure for measuring human categorical representations over large datasets of items, such as arbitrary
words or pictures. We call this method discrete Markov chain Monte Carlo with people (d-MCMCP). We illustrate our method by
evaluating the following categories over datasets: emotions as represented by facial images, moral concepts as represented by
relevant words, and seasons as represented by images drawn from large online databases. Three experiments demonstrate that d-
MCMCP is powerful and flexible enough to work with complex, naturalistic stimuli drawn from large online databases.
Keywords Category representation .Markov chain Monte Carlo .Words representations .Image databases
Big data is transforming society and offers a significant op-
portunity for psychology research: Large databases of text and
images are potentially rich resources for understanding humancognition. However, the current usefulness of big data in psy-
chology is limited, in part because of its size —constructingpsychological tasks that engage with large datasets is a chal-lenge. Labels for images and other kinds of data can be col-
lected through online experiments and crowdsourcing(Carvalho, Lease, & Yilmaz, 2011;G r e e n g a r d , 2011;H o w e ,
2006 ; Kittur, Chi, & Suh, 2008 ; Kleemann, Vob, & Rieder,
2008 ), but there is still a need for methods that can extract the
psychological content of large databases more efficiently and
meaningfully. In particular, the relevance of a piece of data
(e.g., text, image, or sound) to a given psychological conceptis often more suitably represented as a subjective probabilistic
judgment than as a binary label. Understanding the distribu-
tional representations of people ’s mental categories is relevant
both for automating tasks that simulate cognitive processes,
such as for artificial intelligence applications and also for un-
derstanding principles of human psychology. For example, ifone were building an emotion detector for automated conver-
sational agents, one would need an estimate of how likely it
would be that different combinations of features of spokenstimuli (combinations of words and tone) would be indicative
of the category of anger. Another example is gaining insight
into the biases that inform our social prejudices by under-standing how likely various combination of profiling features
are to be associated with the concept of Bsocially threatening, ^
and comparing this with actual statistics of criminal profiles.
Capturing the labels corresponding to such subjective judg-
ments becomes prohibitively time-consuming for large,Electronic supplementary material The online version of this article
(https://doi.org/10.3758/s13428-019-01201-9 ) contains supplementary
material, which is available to authorized users.
*Anne S. Hsu
anne.hsu@qmul.ac.uk
Jay B. Martinjbmartin@nyu.edu
Adam N. Sanborn
A.N.Sanborn@warwick.ac.uk
Thomas L. Griffiths
tom_griffiths@berkeley.edu
1School of Electronic Engineering and Computer Science, QueenMary University of London, London, UK
2Department of Psychology, New York University, New York, NY ,
USA
3Department of Psychology, University of Warwick, Coventry, UK
4Department of Psychology, University of California, Berkeley,
Berkeley, CA, USABehavior Research Methods (2019) 51:1706– 1716
https://doi.org/10.3758/s13428-019-01201-9
discrete datasets. This article presents a new method for effi-
ciently inferring the structure of psychological categories de-
fined over arbitrarily large datasets, such as of text and images.
Taking the view that categories are often viewed as
subjective probability distributions over stimuli (Ashby& Alfonso-Reese, 1995), categorical representations can
be revealed through asking people for their judgments of
how relevant a stimulus is to a category. Similarly, insight
into categorical representations can be revealed through aset of stimuli in which each sample is labeled with a
subjective probability rating that represents how relevant
the sample is to the category. However, it is difficult toefficiently obtain such judgments over large sets of stim-
uli. One solution to this problem comes from the key
insight that if an experimenter can draw samples from thissubjective distribution, the samples will be concentrated
on the relevant stimuli, and irrelevant stimuli will be
ignored. Using this insight, Sanborn, Griffiths, andShiffrin ( 2010) adapted the well-known Markov chain
Monte Carlo (MCMC) algorithm from computer scienceto sample from subjective probability distributions, amethod they called Markov chain Monte Carlo with
people (MCMCP). MCMCP has been used to estimate
the structure of categories defined on continuous, easily
parameterized stimuli, such as stick-figure animals and
basic fruit shapes (Sanborn et al., 2010) or computer-
generated faces (Martin, Griffiths, & Sanborn, 2012 ;
McDuff, 2010 ).
Although the introduction of MCMCP has made it easier to
explore complex, high-dimens ional representations, it re-
quires parameterized stimuli, and thus is inappropriate for da-tabases of discrete items. To allow exploration of representa-tions over large sets of discrete items we introduce a new
method called discrete Markov chain Monte Carlo with
people (d-MCMCP). This method allows estimation of sub-
jective probability distributions over arbitrary datasets, such asimages or text snippets. The resulting distributions can be used
to identify the structure of people ’s psychological representa-
tions in these domains.
The outline of this article is as follows. The next section
introduces the key ideas behind MCMCP. We then presentour new method, d-MCMCP, for discrete sets of stimuli. The
remainder of the article focuses on three experiments showing
a range of applications of d-MCMP. The first experiment ex-plores categories of happy andsadfaces using real photo-
graphic images, allowing us to compare against previous re-sults using the MCMCP algorithm (Martin et al., 2012). The
second experiment applies d-MCMCP to text, exploring the
words perceived to be representative of the concept of morality
for people with liberal and conservative political affiliations(Graham, Haidt, & Nosek, 2009 ). The third experiment dem-
onstrates how d-MCMCP can be applied to large databases ofimages, by investigating how people represent seasons.Markov chain Monte Carlo with people
A standard approach to measuring human categories is to askpeople to rate how likely a stimulus is to be a member of a
given category. However, this approach has two serious lim-
itations. First, when datasets become too large, collecting in-dividual ratings is impractical. Second, questions such as
BHow good an example is this of a happy face ?^can be dif-
ficult to answer. A better-defined question is to ask for
pairwise judgments —for example, BWhich is a better example
of a happy face ?,^but collecting all possible pairwise judg-
ments requires even more trials, on the order of n
2fornitems.
To address these issues, we introduced a class of algo-
rithms from computer scie nce and statistics called
MCMC. Although algorithms exist for sampling from cer-tain distributions with well- defined mathematical forms —
for example, a normal or binomial distribution —it is often
difficult to directly generate samples from arbitrary prob-ability distributions, especial ly distributions that are not
readily described with a mathematical formula, such assubjective category representations over arbitrary stimuli.MCMC algorithms solve the problem of generating sam-
ples from arbitrarily comple x probability distributions
using a statistical method. These algorithms work bydrawing samples from a simpl er distribution (from which
direct samples are possible; e.g., a Gaussian or binomial
distribution), and keeping o r discarding these samples on
the basis of a function of the difference between the sim-pler distribution and the targ et distribution. The samples
that are kept result in a chain, and once the number of
sequences in the chain is long enough, the frequencies
represented in the chain become equivalent to those ofsamples from the true underlying distribution, p(x)
(Gilks, Richardson, & Spiegelhalter, 1996).
A popular method for constructing Markov chains is the
Metropolis –Hastings algorithm (Me tropolis, Rosenbluth,
Rosenbluth, & Teller, 2012). The sequence of the chain ’s state
is initialized at an arbitrary value, x. The next value in the
sequence is generated via a two-s t e pp r o c e s s .F i r s t ,ac a n d i d a t e
for the next value, x', is chosen by sampling from a proposal
distribution conditioned on x,q(x';x)—for example, a normal
distribution with mean x. Second, a decision is made as to
whether that proposed value will be accepted, using an accep-tance function that evaluates the relative probability of xtox'
under the target distribution p(x). For example, one might com-
pare the value of p(x) to the value of a normal distribution with
mean xevaluated at x'. If the proposal is accepted, the state of
the chain moves to x',o re l s ei tr e m a i n sa tx .T h i sp r o c e s s
continues until the chain converge s to its stationary distribution.
A variant of the Metropolis –Hastings acceptance function
is at the heart of MCMCP. In a standard categorization task,
people make a series of pairwise decisions, choosing the best
category member from two proposed stimuli. The stimuli inBehav Res (2019) 51:1706 –1716 1707
each pair actually correspond to the values xandx'in the
Metropolis –Hastings algorithm. The choices that people make
determine which proposals are accepted. If we offer people a
choice between xandx'and they choose x'with probability
Pchoice x0;xjc/C16/C17
¼px0jc/C0/C1
px0jcðÞ þ pxjcðÞ; ð2Þ
where p(x|c) indicates the degree to which stimulus xis
perceived to be a member of category c, then their judgment
is equivalent to the Barker acceptance function (Barker,1965 ). A Markov chain based on this acceptance function will
converge to p(x|c). Fortunately, Eq. 2is a well-known model
of human choice probabilities, referred to as the Luce choice
ruleorratio rule (Luce, 1963 ; Shepard, 1987 ). This rule has
provided a good fit to human data for people choosing be-
tween two stimuli on the basis of a particular property
(Bradley, 1954 ; Clarke, 1957 ; Hopkins, 1954 ), and is also
the decision rule used in many models of cognition (Ashby,
1992 ; McClelland & Elman, 1986 ;N o s o f s k y , 1986 ,1987 ).
With enough of these decisions, MCMCP converges to the
probability distribution associated with that category, and in-
dividual stimuli will be encountered with probability p(x|c).
On the basis of this correspondence, the MCMCP method
implements Metropolis –Hastings by using people ’s choices
to determine which proposals are accepted (Sanborn et al.,2010 ). Furthermore, a variety of different assumptions for
how participants are responding in proportion to Bayesian
probabilities result in the same decision rule. For example,
motivated participants can produce behavior that is more de-terministic than probability matching. This kind of behavior is
usually modeled using an exponentiated Luce choice rule, in
which each term on the right-hand side of Eq. 2is raised to an
exponent γ(Ashby & Maddox, 1993). Previous work
(Sanborn et al., 2010 ) has shown that if choices are made
according to the exponentiated Luce choice rule, thenMCMCP (and, so, also d-MCMCP) will converge to the dis-
tribution p(x)
γ.T h u s ,i f γis unknown, we can only estimate
p(x) up to a constant exponent, meaning that the relative prob-
abilities cannot be directly evaluated, but the probabilities ofdifferent objects will remain in the same order. The efficiency
of the d-MCMCP algorithm will depend on the target distri-bution, and thus on γas well.
Although MCMCP is able to tackle large datasets by ask-
ing people well-defined questions, it introduces a new issue:MCMCP requires a symmetric proposal distribution q(x';x),
meaning that the probability of choosing a proposal given thecurrent state would be the same if the proposal and the currentstate were reversed. When stimuli are described by a fixed set
of parameters, this is easy —Gaussian or uniform distributions
can be used to generate proposals for continuous parameters,
and multinomial distributions can be used for discrete fea-
tures. However, datasets consisting of images and text arenot composed of easily parameterized items. In the remainderof this section we present a scheme for making reasonable
proposals for exploring stimuli that are not easily parameter-ized. We call this method d-MCMCP.
The d-MCMCP method requires three additional steps.
The first step is obtaining a database of stimuli. The secondstep is computing a symmetric similarity matrix, S, as a rough
measure of the similarity between all possible item pairs. Thesimilarity metric is chosen as is appropriate for the domain andneed only provide a rough guide to the perceived similarity of
human participants. For example, similarity between color
histograms can be used to quantify the similarity of colorimages. The third step is constructing a graph of the stimuli
based on these similarities. The d-MCMCP proposal distribu-
tion is then defined by a random walk on this graph. How wellthese assumed similarities correspond to the similarities of the
human participants does not affect the algorithm in the long
run: As long as the resulting graph is fully connected, free ofclosed cycles, and aperiodic, we are guaranteed to converge to
the stationary distribution over time. Only the quality of the
match between the graph and people should affect the algo-rithm’s efficiency, and indeed the efficiency of d-MCMCP
does degrade as the difference between the assumed and ac-tual similarities becomes large. See the supplementary mate-
rialsfor simulation results relating to the effects of noise in the
similarity measure used.
To produce symmetric propos als using a random walk
on a graph, the edges must be symmetric (i.e., the walk cantraverse an edge in each direction), and each node in thegraph must have the same degree (i.e., each node must
have the same number of neighbors). Just choosing the b
most similar neighbors for each node does not suffice, be-
cause although node imight be one of bnearest neighbors
of node j, the reverse is not necessarily true, and the nodes
could have different degrees.
To address this issue, we construct the graph that maxi-
mizes the similarity along edges while keeping the degree ofeach node constant. Formally, we want to find
argmax
G∑ijGijSijsuch that ∑ijGij¼b;Gii¼0;Gij¼Gji;
where Gis the adjacency matrix of the graph, with Gij=1i f
there is an edge from itoj,a n dG ij= 0 otherwise. This is an
instance of the maximum-weight b-matching problem
(Papadimitriou & Steiglitz, 1998 ). Exact algorithms exist for
solving this problem, such as the blossom algorithm
(Edmonds, 1965 ), but these are impractical for large-scale
applications. Consequently, we use an approximate algorithm
based on max-product message-passing in order to find a b-
matching (Jebara & Shchogolev, 2006 ).
Given this similarity graph, proposals for the d-MCMCP
algorithm can be made in many ways. The most straightfor-
ward is to choose a proposal uniformly from all bneighbors,1708 Behav Res (2019) 51:1706 –1716
where the value of bis chosen at the experimenter ’sd i s c r e t i o n .
Alternatively, an iterative geometric proposal of step length
ngeomcan be chose. Here, the proposal is generated iteratively
using a number of steps, ngeom, that is chosen from a geometric
distribution with a fixed parameter. A random walk of lengthn
geom is then performed, choosing the next node uniformly
from the bneighbors of the most recent one, and the proposal
is the node at the end of the random walk. Geometric pro-posals allow a wider search over the stimulus space than withthe uniform proposal. Additionally, it is prudent to allow for
some small probability of choosing uniformly from all possi-
ble stimulus items, to allow the algorithm to move betweenlocal maxima. See Fig. 1.
D-MCMCP expands on the initial possibilities of the
MCMCP method to measure distributional mental representa-tions over real discrete datasets. As we discussed above, our
representation of structure in the world is probabilistic, and
thus the most complete understanding of our mental represen-tation of world structures comes from a distributional repre-
sentation. Furthermore, the distributions underlying many of
our representations are complex and may be bimodal or havenonlinear interactions in the feature space. For example, an
exceptionally quiet and an exceptionally loud voice could
both be indications of upset emotions. As another example,
larger containers may be preferred for cheaper olive oils,
whereas smaller containers are preferred for high-end varie-ties. D-MCMCP offers the capability to efficiently reveal a
richer and more accurate understanding of the distributions
that underlie our categorical representations.
Experiment 1: Happy and sad faces
As a first test of d-MCMCP, we examined the categories ofhappy andsadfaces using images of real faces. Previous work
had applied the MCMCP method to estimating these catego-ries using parameterized face stimuli, where a continuousspace was derived from eigenfaces computed from a set ofimages (Martin et al., 2012 ). We used the same image data-
base to directly compare the results of d-MCMCP andMCMCP on a matched stimulus set. Example code and theface stimuli can be found at https://osf.io/u5sz4 /.
Method
Participants A total of 60 undergraduates participated in ex-
change for course credit. Sample sizes were selected on the
basis of previous experience with this experimental paradigm.
Stimuli A database of 1,271 images of faces was assembled
from the California Facial Expression (CAFE) database, a
collection of 1,280 normalized 40 × 64 pixel grayscale por-traits containing 64 individuals (Dailey, Cottrell, & Reilly,
2001). These images express approximately eight distinct
BFACS-correct ^emotions, classified according to the taxono-
my of the Facial Action Coding System (Ekman & Friesen,
1978 ). Details of how the graph was constructed are available
in the supplemental online material .
Procedure Face images were convolved with Gabor filters at
eight scales and five orientations. Principal component analy-sis (PCA) was then applied to the whole set of convolved
images, and the Euclidean distance between the top 50 com-ponents was used as the similarity metric for defining the
matrix S. Two graphs Gwere produced using the approximate
b-matching algorithm from Jebara and Shchogolev ( 2006 ),
one with b= 6 and one with b= 16. This algorithm gives an
approximate solution to the b-matching problem, so there was
still some minor variation in the degree of individual nodes.
Our empirical evaluation of the performance of the d-
MCMCP procedure will thus also help indicate whether this
residual variation affects the results. There is no guarantee thatam a x i m a l b-matching will be connected, so we used the larg-
est connected component as the basis for the d-MCMCP pro-cedure. The largest connected components contained 1,216images with b= 6, and all 1,271 images with b=1 6 .
Fig. 1 Steps for measuring category representations using d-MCMCP. ( a)
Assemble the database of items that are candidate category members. ( b)
Quantify similarity (using a suitably chosen metric) for all item pairs inthe database. ( c) Enter the similarity matrix into a b-matching algorithm to
obtain a graph in which each item is connected to its Nneighbors. ( d)R u n
d-MCMCP using the neighbors as nearby proposalsBehav Res (2019) 51:1706 –1716 1709
We compared three different methods for defining the pro-
posal distributions, run as three separate conditions. For all
three proposal methods, we allowed a 10% chance of propos-ing a jump to a node chosen uniformly at random. The three
methods for choosing the remaining proposals were uniform
random walk on the graph with b= 6 (U6), uniform random
walk on the graph with b= 16 (U16), and the geometric
proposal with n
geom= .5 on the graph with b= 6 (G6).
Participants were randomly assigned to conditions. Trials
were presented on three different computers, one for each pro-
posal type. Following previous work, each participant complet-
ed trials corresponding to dif ferent d-MCMCP chains (Martin
et al., 2012; Ramlee, Sanborn, & Tang, 2017). There were four
chains: two for happy faces and two for sadfaces. There were
100 trials for each of the four chains. On a given trial, theparticipant decided which of a pair of faces was either more
happy or more sad. Twelve initial practice trials were not in-
cluded in the analysis. Forty catch trials were presented of facepairs for which the more happy orsadface was clearly obvious
(in this case, we used the emotion designations in the CAFEdatabase to select faces that should be clearly happy or sad).Thus, each participant responded to 100 × 4 + 12 + 40 = 452
trials, which took approximat ely 25 min. The responses were
linked in chains of ten participants each: The last trial of each ofthe four chains was passed along to the next participant as his or
her first nonpractice trial, to form a linked chain of 1,000 trials.Participants who did not correc tly answer at least 27 of the catch
trials ( p< .01 under random guessing) were not included in the
results or added into a chain. We collected two chains of tenparticipants for each proposal type, corresponding to four
happy andsadchains with 1,000 trials in each chain.Results
The previously used MCMCP method drew samples gen-
erated from a continuous eigenface space, and thus did
not result in ratings being applied directly to the discrete
images from the dataset. Previous work showed by aver-aging the generated images in the chain that the samples
g e n e r a t e db yM C M C Pm o r eq u i c k l yc o n v e r g e dt ob e i n g
representative of people ’s mental representations of each
category than did alternativ e methods (i.e., reverse
correlation; Martin et al., 2012 ). Here, to directly compare
the previous MCMCP with the present d-MCMCP results,the images selected on each d-MCMCP trial were aver-
aged together to produce the average faces shown in Fig.
2. All three proposal methods produced mean faces that
appeared reasonably consistent with the target emotions.
Also included in Fig. 2are the results reported in Martin
et al. ( 2012 ), using MCMCP in a parameterized space
based on the eigenfaces derived from the image databasewe used for d-MCMCP. Qualitatively, the results from d-
MCMCP are at least as good as, and perhaps better than,those produced using eigenfaces.
To quantify the performance of the different variants of the
algorithm, we conducted a follow-up experiment in which a
group of 40 participants recruited via Amazon Mechanical
Turk provided ratings of the emotions exhibited by the facesderived from our chains. For each proposal type (and for the
chains based on eigenfaces used in Martin et al., 2012 ), cu-
mulative average faces were computed for each of 40 loga-
rithmically spaced numbers of trials, averaging across all four
chains that corresponded to each emotion. Trial numbers
Eigenfaces
HappyU6 U16 G6
Sad
Fig. 2 Average faces for MCMCP (eigenfaces) and for each type of d-MCMCP proposal. Averages are taken across all trials and all four chains
corresponding to happy andsad1710 Behav Res (2019) 51:1706 –1716
greater than 50 images were averaged only over the 50 most
recent trials, meaning that no more than 200 faces contributed
to any single image. Participants rated the emotion exhibitedby each of these mean faces on a scale from 1 to 9, where 1
indicates very sad and 9 indicates very happy . All participants
rated all faces and received $1 in compensation for their time.
No participant data were excluded.
The results of our follow-up experiment are shown in Fig.
3. The d-MCMCP method results in statistically significantly
higher ratings for faces derived from happy chains, regardless
of the proposal type, perhaps as a consequence of being able toexplore a larger space of faces than the eigenface method.Looking at the 95% confidence intervals of differences in
the mean ratings across all trials (i.e., each participant ’sr a t i n g s
across all trials are averaged into one rating), a d-MCMCP
face is significantly more likely to be perceived as happy for
all proposal types (positive values are Bbetter ^ratings for hap-
py faces): CI=[ 1 . 1 8 ,1 . 6 8 ] , d= 1.43 for U6; CI=[ 1 . 3 6 ,1 . 8 2 ] ,
d=1 . 5 9f o rU 1 6 ; CI= [0.95, 1.40], d=1 . 1 7f o rG 6 .T h e
results for sadchains are more comparable to the eigenface
method (negative values are Bbetter ^ratings for sad faces): U6
was significantly better, CI=[–0.64,–0.11], d=–0.37; U16
was significantly worse, CI= [ 0.023, 0.52], d= 0.27 ; and G6
was not significantly different, CI=[–0.35, 0.17], d=–
0.092. For both happy andsadchains, there is some variation
in the emotion ratings of mean faces over time, consistent with
MCMCP exploring the distribution of faces associated with
the category (and possibly moving between modes of that
distribution) rather than finding the most extreme instance ofthat category.Experiment 2: Moral foundations
Our second experiment applied d-MCMCP to concepts intext. Here, we used d-MCMCP to examine and further support
a theory from social psychology that conservatives and lib-
erals have different concepts of morality. In particular, conser-vatives ’moral intuitions have been claimed to reflect values in
the five Bmoral foundations ^of harm/care, fairness/reciproci-
ty, ingroup/loyalty, authority /respect, and purity/sanctity,
whereas liberals primarily endorse only the first two principles
(Graham et al., 2009 ). This difference has previously been
observed in abstract assessments of moral judgments of state-
ments and scenarios, reactions to taboo trade-offs, and use of
foundation-related words in the moral texts of sermons in
liberal versus conservative churches. Here, we examined dif-ferences in the concept of morality between liberals and con-
servatives by employing d-MCMCP over a set of words.
Method
Participants A total of 251 US-based participants were recruit-
ed via Amazon Mechanical Turk and were paid $1 each for
their time. The sample sizes were selected on the basis of
previous experience with this experimental paradigm.
Stimuli In Graham et al. ( 2009 ), a dictionary of words was
assembled that were closely related to each of the moral foun-
dations. We chose 32 of these words from each of the fivefoundations, for a total of 160 words.
9 27 88 293 1000
TrialSadder2345678HappierEmotion RatingHappy
9 27 88 293 1000
TrialSadder2345678HappierSad
Eigenfaces
U6
U16
G6
Fig. 3 Comparison of MCMCP and d-MCMCP: MCMCP using
eigenfaces compared with d-MCMCP with a variety of proposal
methods, executed on the same set of face stimuli. Happiness ratingsare shown for average faces for the three types of d-MCMCP proposalsas well as for the original MCMCP method as a function of trial number
(error bars show 95% confidence intervals). Averages are taken across the50 most recent trials (or starting from the first trial, for trials less than 50)and across all four chains corresponding to the same emotion, happy or
sad. Also included are face ratings for the results of a previous MCMCP
experiment that used eigenfaces derived from the same image database
(Martin et al., 2012 ).Behav Res (2019) 51:1706 –1716 1711
Procedure Similarities between all possible word pairs were
measured using latent semantic analysis results made avail-
able online at http://lsa.colorado.edu/cgi-bin/LSA-matrix.
html (Landauer & Dumais, 1997 ). The similarity between all
pairs was represented as a similarity matrix that was fed intotheb-matching algorithm. A graph was found using b=1 0 ,
which was roughly between the values used in Experiment 1.
We used a proposal distribution corresponding to a uniformrandom walk on this graph. Also, as in Experiment 1,w e
allowed for a 10% chance of proposing a jump to a node
chosen uniformly at random. Each participant completed trials
corresponding to four d-MCMCP chains, all of which ex-plored the single concept of Bmorality. ^On a given trial, par-
ticipants were given the choice of two words and answered thequestion: BWhich of these two words is most relevant to mo-
rality? ^We used a proposal distribution corresponding to a
uniform random walk on this graph, but with a 10% chance ofproposing a jump to a node chosen uniformly at random.There were 100 trials for each of the four chains. Twelve trials
in the beginning were offered as practice, which were not
included in the analysis. Thus, each participant responded to100 × 4 + 12 = 412 trials, which took approximately 25 min.
The responses were not linked between participants (as had
been done in Exp. 1). At the end of the experiment, following
Graham et al. ( 2009 ), participants were also asked BWhich
best describes your political identification? ^and given the
choices (1) extremely liberal ,( 2 ) somewhat liberal ,( 3 )
middle ,( 4 ) somewhat conservative ,( 5 ) extremely
conservative , and (6) none of these .Results
Because we were interested i n contrasting responses be-
tween people who self-identified as conservatives versus
liberals, we only analyzed the responses of participants
who responded extremely liberal and somewhat liberal
(classified as liberals, N= 131) or somewhat conservative
orextremely conservative (classified as conservatives, N=
52) to the political identification question. For each ofthese participants, we calculated the frequency with which
words corresponding to the five foundations were select-
ed. Figure 4shows the normalized frequencies for conser-
vatives (top) and liberals (bottom). Normalized frequen-
cies over all 160 words, along with the 20 most morally
relevant words for conservatives and liberals, are shownin Supplementary Figs. 2and3. Under the statistically
conservative assumption that each participant counted asa single sample, the differe nce between the normalized
histograms from the two conditions was significant by a
Pearson ’sχ
2test [χ2(4) = 9.76, p< .05]. A two-way
analysis of variance (ANOV A) over moral foundations
and political identification showed a main effect of foun-dations [ F(4, 905) = 24.24, MSE =. 9 5 , p< .00001, η
p2=
.097], as well as an interaction [ F(4, 905) = 15.87, MSE =
.62, p< .0001, ηp2= .066], supporting the idea that con-
servatives and liberals find different amounts of relevance
in the five moral foundations. An ANOVA over founda-
tions for conservative participants showed a main effect of
foundations [ F(4, 255) = 17.41, MSE =. 7 4 , p< .00001,
00.10.20.30.40.5p(relevance|morality)
00.10.20.30.40.5p(relevance|morality)Conservatives N=52
harm fairness ingroup authority purityLiberals N=131
Fig. 4 Relevance of moral foundations for conservatives versus liberals:
Results of d-MCMCP over words that are associated with the five moral
foundations hypothesized by Graham et al. ( 2009 ) for participants who
self-identified as conservatives (top) versus liberals (bottom). The valuesp(relevance | morality) are normalized counts —that is, the proportions of
times words associated with the given moral foundation were chosen asb e i n gm o r er e l e v a n tt ot h ec o n c e p to fm o r a l i t y .1712 Behav Res (2019) 51:1706 –1716
ηp2= .79], with the foundation of purity (M =. 4 1 , SD=
.31) being significantly more relevant (with Bonferroni
correction) than the other four foundations: harm ( M=
.15, SD=. 1 5 ) , t(51) = 4.67, p< .0001, d=1 . 0 8 , CI=
[.15, .37]; fairness ( M= .19, SD=. 2 1 ) , t(51) = 3.45, p=
.0011, d=0 . 8 2 , CI= [.091, .34]; in-group M=. 1 0 , SD=
.14), t(51) = 5.57, p< .0001, d=1 . 2 7 , CI= [.19, .41]; and
authority (.15, SD=. 1 8 ) , t(51) = 4.60, p<. 0 0 0 1 , d=
1.015, CI= [.14, .36]. An ANOVA over foundations for
liberal participants also showed a main effect of founda-
tions [ F(4, 650) = 23.36, MSE = .88, p< .00001, ηp2=
.87]. Post-hoc analysis sho ws the foundation of purity ( M
=. 2 1 , SD= .22) being more relevant (with Bonferroni-
adjusted significance) than in-group (M =. 1 0 , SD=. 1 2 ) ,
t(130) = 4.36, p< .0001, d= .57, CI= [.055, .15], and
authority ( M=. 1 3 , SD=. 1 3 ) , t(130) = 3.15, p=.0020, d
=0 . 4 1 , CI= [.03, .12]. There was also a trend (not sig-
nificant after Bonferroni adju stment) toward purity being
less relevant than harm ( M=. 2 7 , SD= .24), t(130) = –
2.03, p=.045, d=–0.30, CI=[–.13,–.0017], and
fairness ( M=. 2 9 , SD= .23), t(130) = –2.43, p=.016, d
=–0.36, CI=[–.14,–.015]. Harm and fairness were
both significantly more relevant than the other three foun-
dations. The foundation of harm was significantly more
relevant (with Bonferroni adjustment) than in-group,
t(130) = 6.91, p< 1e-09, d=. 9 1 , CI= [.12, .22], and
authority, t(130) = 5.30, p< e-06, d=. 7 5 , CI=[ . 0 9 0 ,
.20]. Harm also had a trend (not significant withBonferroni adjustment) toward being more relevant thanpurity, t(130) = 2.0, p=.045, d=0 . 3 0 , CI= [.0017, .13].Similarly the foundation of fairness was significantly
more relevant (with Bonferroni adjustment) than in-group,
t(130) = 7.38, p<1 e - 1 0 , d=1 . 0 , CI= [.13, .23] and
authority, t(130) = 6.44, p<1 e - 0 8 , d= .84, CI=[ . 1 1 ,
.20]. Fairness also had a trend (not significant withBonferroni adjustment) toward being more relevant thanpurity, t(130) = 2.43, p=.016404, d= 0.35613, CI=
[.015, .14].
These results are consistent with those of Graham et al.
(2009 ), who found that conservatives believed that all five
moral foundations were relevant to morality, whereas liberalstended to mostly value the foundations of harm and fairness.However, the relative importance of purity as compared with
all other foundations for conservatives, and in comparison
with in-group and authority, was not seen in Graham et al.,for whom the relative importance of various foundations var-
ied between experiments, suggesting an interesting direction
for future research.
Experiment 3: Seasons
In our third experiment, we used d-MCMCP to explore cate-gories defined on images from a large online database.
Specifically, we explored the categories associated with the
seasons spring, summer, autumn, and winter, using 4,000 im-
ages obtained from online image databases. By applying the
d-MCMCP procedure to these stimuli, we could identify high-probability images and compute informative aggregate
Spring
Summer
Autumn
Winter
Fig. 5 Seasonal images: The top ten most popular images over all chains for each season, decreasing in popularity from left to right.Behav Res (2019) 51:1706 –1716 1713
statistics for each category, such as determining the distribu-
tion of colors associated with each season.
Method
Participants A total of 90 participants were recruited using
Amazon Mechanical Turk. Each participant was paid $1 for
completing the 25-min experiment. The sample size was se-lected on the basis of previous experience with this experi-
mental paradigm.
Stimuli A set of 4,000 colored season-related images was as-
sembled by searching for public-domain web images using the
phrases Bspring season, ^Bsummer season, ^Bautumn season, ^
and Bwinter season ^in Google Image Search and on Flickr.
com. The top 500 results for searches on Google and Flickr
for each season were downloaded. All images were resized so
that the maximum dimension was 250 pixels, while preserving
the original ratio of image height to width.
Procedure The similarity between all possible image pairs
(7,998,000 pairs for 4,000 images) was quantified using both
the basic color histogram (BCH) descriptor (Griffin, 2006)a n d
the scale-invariant feature transform (SIFT; Lowe, 1999 ). BCH
classifies and counts pixels as belonging to one of 11 basiccolors (black, white, gray, red, orange, yellow, green, blue,
purple, pink, and brown). SIFT applies local filters to transform
images into collections of local feature vectors that are invari-ant to scaling, rotation, and translation of the image. Similarity
results over all pairs of images for both methods were normal-
ized to have unit variance and then added together, thus yield-ing a similarity measure that combined the results of both BCH
and SIFT. The similarity between all pairs was represented as a
similarity matrix that was fed into the b-matching algorithm. A
g r a p hw a sf o u n du s i n g b= 5, which was the smallest value
such that all 4,000 images remained fully connected. We used aproposal distribution corresponding to a uniform random walk
on this graph.
Each participant made pairwise choices between images by
answering questions such as BWhich image is more represen-
tative of spring? ^We used a proposal distribution correspond-
ing to a uniform random walk on this graph. There were 100
trials for each of four chains, one for each season. There were
also 12 practice trials, as well as 40 catch trials for which one
image of the pair obviously corresponded to a particular sea-son (as judged by the experimenter). Thus, each participant
completed 452 trials. We included catch trials with the inten-
tion of excluding participants who did not get at least 27 of thecatch trials correct (corresponding to a less than 1% chance
that the answers were completely random guesses) from the
chains and later analyses. All participants passed our criteriaand were included in the analysis. We collected data by
linking three sets of ten participants, forming three chains of
1,000 trials for each of the four seasons.
Results
The top ten images that were chosen most often over all
three chains for each season are shown in Fig. 5.C l e a r l y ,
the images are very indicative of each season. Figure 6a
shows, as a function of the number of trials, the L1 distance
between 11-bin color histograms calculated for the cumu-lative images, both between chains for the same season and
between chains corresponding to different seasons. The
within-chain distance decreases over time and is typicallylower than the similarity between chains, supporting the
idea that chains are converging toward different parts of
the space of images. Figure 6b shows a simple example
of the kind of statistical analyses that can be done on the
resulting samples. The color histograms for the different
seasons are quite different from one another and corre-spond to palettes that intuitively match the seasons.
0 1000012Spring
TrialsDistancea)
0 1000012Summer
Trials0 1000012Autumn
Trials0 1000012
Winter
Trials
b)
Fig. 6 Color histograms for seasonal images. 11-bin color histograms
were calculated for all cumulative images in all three chains as a function
of the number of trials. ( a) Average L1 distance between the cumulative
histograms of a single chain and of the other two chains that correspond tothe same season (solid line) or of another three chains that correspond to adifferent season (one dotted line for each other season). ( b) Color histo-
grams of all images, averaged over all chains for each season.1714 Behav Res (2019) 51:1706 –1716
Conclusion
We have introduced a new method that can be applied to
arbitrary sets of items, making it possible to measure people ’s
representations of categories defined over those items. Ofcourse, a wide variety of methods can be applied to elicitingcategories from people. Our d-MCMCP method outperformed
MCMCP on eliciting the categories of happy faces. Previous
work has shown that MCMCP both outperforms complexregression and predicts an individual ’s category preferences
better than those of other individuals do (Huszár, Noppeney,& Lengyel, 2010). Thus, it seems likely that d-MCMCP
would show similar advantage over these other approaches.
In future work, it would be useful verify this by directly com-
paring d-MCMCP with these methods, as well as comparing itto approaches, such as active learning (Myung & Pitt, 2009 )
and cognitive tomography (Houlsby et al., 2013 ). D-MCMCP
generally makes weaker assumptions about the decision-making process than do these other methods, so it may well
prove more advantageous with real-world tasks and stimuli, a
context in which the assumptions the other methods makeabout cognitive processes are less likely to be correct.
Although our textual example used only words, one could
easily extend the method to evaluate lines of texts, paragraphs,
and documents. Furthermore, one can also use d-MCMCP to
tag video clips and other data types. As with most data-drivenmodels, the quality and generalizability of d-MCMCP ’sr e -
sults are influenced by biases present in the dataset used. Forexample, if one wanted to compare representative male versusfemale faces, and if the data had more examples of young
female faces and of old male faces, the final categorical rep-
resentations might be dominated by age-dependent rather thangender-dependent features. Thus, the limitations and biases
present in the dataset used should be considered when making
conclusions about mental representations. Our method is like-ly to be valuable to data scientists, as well as to machine-
learning and computer vision researchers interested in training
systems to automate categorizing images and other complexstimuli. Our method can be used to retrieve human judgments
for concepts in a wide variety of domains, including stereo-
typed judgments from individual characteristics, consumerpreferences as a function of variations in product offerings,
or mood/conversational intent from word usage patterns. For
psychologists, conducting experiments using d-MCMCP on alarge scale will allow us to build up a catalog of human cate-
gory representations, taking a step toward understanding how
those categories are formed.
Author note All authors contributed to developing the study concept.
T.L.G. and J.B.M. contributed to the design of Experiment 1,a l la u t h o r s
contributed to the design of Experiment 2, and A.S.H. and T.L.G. con-
tributed to the design of Experiment 3. A.S.H. programmed and ran the
experimental studies and analyzed the results. A.S.H. drafted the manu-
script, T.L.G. and J.B.M. provided critical revisions, and all authorsapproved the final version for submission. Part of this work was pub-
lished in a conference proceeding for the Cognitive Science Society (Hsu
et al., 2012). This work was supported by Grant No. IIS-0845410 fromthe National Science Foundation and by Grant No. ERC-2013-AdG339182-BAYES_KNOWLEDGE from the European Research
Council.
Open Access This article is distributed under the terms of the Creative
Commons Attribution 4.0 International License (http://
creativecommons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided you give
appropriate credit to the original author(s) and the source, provide a link
to the Creative Commons license, and indicate if changes were made.
Publisher ’sn o t e Springer Nature remains neutral with regard to jurisdic-
tional claims in published maps and institutional affiliations.
References
Ashby, F. G. (1992). Multidimensional models of perception and
cognition .H i l l s d a l e ,N J :E r l b a u m .
Ashby, F. G., & Alfonso-Reese, L. A. (1995). Categorization as proba-
bility density estimation. Journal of Mathematical Psychology ,39,
216–233.
Ashby, F. G., & Maddox, W. T. (1993). Relations between prototype,
exemplar, and decision bound models of categorization. Journal of
Mathematical Psychology ,37, 372–400. https://doi.org/10.1006/
jmps.1993.1023
Barker, A. A. (1965). Monte Carlo calculations of the radial distribution
functions for a proton-electron plasma. Australian Journal of
Physics ,18,1 1 9–133.
Bradley, R. A. (1954). Incomplete block rank analysis: On the appropri-
ateness of the model of a method of paired comparisons. Biometrics ,
10,3 7 5–390.
Carvalho, V. R., Lease, M., & Yilmaz, E. (2011). Crowdsourcing for
search evaluation. In ACM Sigir forum 44 (2), 17-22.
Clarke, F. R. (1957). Constant-ratio rule for confusion matrices in speech
communication. Acoustical Society of America ,29,7 1 5–720.
Dailey, M., Cottrell, G. W., & Reilly, J. (2001). California facial expres-
sions (CAFE) (Unpublished digital images). University of
California, San Diego
Edmonds, J. (1965). Paths, trees and flowers. Canadian Journal of
Mathematics ,17,4 4 9–467.
Ekman, P., & Friesen, W. (1978). Facial action coding system: A tech-
nique for the measurement of facial movement . Palo Alto, CA:
Consulting Psychologists Press.
F. Kleemann, G. V ob, and K. Rieder (2008). Under paid Innovators: The
Commercial Utilization of Consumer Work through Crowdsourcing,
Science Technology and Innovation Studies, 4 ,5–26.
Gilks, W. R., Richardson, S., & Spiegelhalter, D. J. (1996). Markov chain
Monte Carlo in practice . Boca Raton, FL: Chapman & Hall.
Graham, J., Haidt, J., & Nosek, B. (2009). Liberals and conservatives use
different sets of moral foundations. Journal of Personality and
Social Psychology ,96,1 0 2 9 –1046.
Greengard, S. (2011). Following the crowd. Communications of the
ACM, 54 (2), 20-22
Griffin, L. D. (2006). The basic colour categories are optimal for classi-
fication. Journal of the Royal Society Interface ,3,7 1–85.
Hopkins, J. W. (1954). Incomplete block rank analysis: Some taste test
re
sults. Biometrics ,10,3 9 1–399.
Howe, J. (2006). The rise of crowdsourcing. Wired magazine, 14 (6), 1-4.
Houlsby, N. M. T., Huszár, F., Ghassemi, M. M., Orbán, G., Wolpert, D.
M., & Lengyel, M. (2013). Cognitive tomography reveals complex,Behav Res (2019) 51:1706 –1716 1715
task-independent mental representations. Current Biology ,23,
2169–2175. https://doi.org/10.1016/j.cub.2013.09.012
Huszár, F., Noppeney, U., & Lengyel, M. (2010). Mind reading by ma-
chine learning: A doubly Bayesian method for inferring mental rep-
resentations. In S. Ohlsson & R. Catrambone (Eds.), Cognition in
flux: Proceedings of the 32nd Annual Meeting of the CognitiveScience Society (pp. 2810 –2815). Austin, TX: Cognitive Science
Society.
Jebara, T., & Shchogolev, V . (2006). B-matching for spectral clustering.
In J. Fürnkranz, T. Scheffer, & M. Spiliopoulou (Eds.), Proceedings
of the 17th European Conference on Machine Learning (pp. 679 –
686). Berlin, Germany: Springer.
Kittur, A., Chi, E. H., & Suh, B. (2008). Crowdsourcing user studies with
Mechanical Turk. In Proceedings of the SIGCHI conference on hu-
man factors in computing systems (pp. 453 –456). ACM.
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato ’s problem:
The Latent Semantic Analysis theory of the acquisition, induction,
and representation of knowledge. Psychological Review ,104,2 1 1–
240. https://doi.org/10.1037/0033-295X.104.2.211
Lowe, D. G. (1999). Object recognition from local scale-invariant fea-
tures. In Proceedings of the Seventh IEEE International Conference
on Computer Vision ( V o l .2 ,p p .1 1 5 0 –1157). New York, NY: IEEE
Press. https://doi.org/10.1109/ICCV .1999.790410
Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush,
& E. Galanter (Eds.), Handbook of mathematical psychology (pp.
103–190). New York, NY: Wiley.
Martin, J. B., Griffiths, T. L., & Sanborn, A. N. (2012). Testing the
efficiency of Markov chain Monte Carlo with people using facial
affect categories. Cognitive Science ,36,1 5 0–162.
McClelland, J. L., & Elman, J. L. (1986). The TRACE model of speech
perception. Cognitive Psychology ,18,1–86.https://doi.org/10.
1016/0010-0285(86)90015-0McDuff, D. (2010). A human –Markov chain Monte Carlo method for
investigating facial expression categorization. In D. D. Salvucci &G. Gunzelmann (Eds.), Proceedings of the 10th International
Conference on Cognitive Modeling (pp. 151 –156). Philadelphia,
PA: Drexel University.
Metropolis, A. W., Rosenbluth, A. W., Rosenbluth, M. N., & Teller, A. H.
(2012). Equations of state calculations by fast computing machines.
J o u r n a lo fC h e m i c a lP h y s i c s ,21,1 0 8 7 –1097.
Myung, J. I., & Pitt, M. A. (2009). Optimal experimental design for
model discrimination. Psychological Review ,116,4
9 9 –518.
https://doi.org/10.1037/a0016104
Nosofsky, R. M. (1986). Attention, similarity, and the identification –
categorization relationship. Journal of Experimental Psychology:
General ,115,3 9–57.https://doi.org/10.1037/0096-3445.115.1.39
Nosofsky, R. M. (1987). Attention and learning processes in the identifi-
cation and categorization of integral stimuli. Journal of
Experimental Psychology: Learning, Memory, and Cognition ,13,
87–108. https://doi.org/10.1037/0278-7393.13.1.87
Papadimitriou, C., & Steiglitz, K. (1998). Combinatorial optimization:
Algorithms and complexity. New York, NY: Dover.
Ramlee, F., Sanborn, A. N., & Tang, N. K. Y . (2017). What sways peo-
ple’s judgement of sleep quality? A quantitative choice-making
study with good and poor sleepers. Sleep ,40,1–13.
Sanborn, A. N., Griffiths, T. L., & Shiffrin, R. (2010). Uncovering mental
representations with Markov chain Monte Carlo. Cognitive
Psychology ,60,6 3–106. https://doi.org/10.1016/j.cogpsych.2009.
07.001
Shepard, R. N. (1987). Toward a universal law of generalization for
psychological science. Science ,237, 1317 –1323. https://doi.org/
10.1126/science.36292431716 Behav Res (2019) 51:1706 –1716

Anne S
Hsu
Jay B
Martin
Adam N
Sanborn
Thomas L
Griffiths


Demonstrating the Impact of Prior Knowledge in Risky Choice.

Demonstrating the Impact of Prior Knowledge in Risky Choice
Mathew D. Hardy (mdhardy@princeton.edu)
Department of Psychology, Princeton University
Thomas L. Grifﬁths (tomg@princeton.edu)
Departments of Psychology and Computer Science, Princeton University
Abstract
Bayesian models that optimally integrate prior probabilities
with observations have successfully explained many aspects
of human cognition. Research on decision-making under risk,
however, is usually done through laboratory tasks that attempt
to remove the effect of prior knowledge on choice. We ran a
large online experiment in which risky options paid out accord-
ing to the distribution of Democratic and Republican voters in
US congressional districts to test the effects of manipulating
prior probabilities on participants’ choices. We ﬁnd evidence
that people’s risk preferences are appropriately inﬂuenced by
prior probabilities, and discuss how the study of risky choice
can be integrated into the Bayesian approach to studying cog-
nition.
Keywords: risk; decision-making; Bayesian modeling
Introduction
You sample six voters from a recent US election in an un-
known Florida congressional district. Three voted Democrat
and three voted Republican. Would you prefer $5 for sure, or
$10 if Democrats won that district? What if these voters were
from a district in Massachusetts, a district in Oklahoma, or the
$10 pays if the next voter sampled is Republican? Political
analysts have to make bets like this all the time, but even ca-
sual observers of politics known enough that they could make
intelligent choices.
Whether people do so is an open question. When faced
with related everyday tasks, people’s judgments and percep-
tions are often best described as inferences and predictions
that incorporate prior probabilities with observations in a
Bayesian manner (Anderson, 1990; Grifﬁths & Tenenbaum,
2006). These results contrast with other research showing
that people often disregard base-rate information when mak-
ing laboratory judgments and predictions (Tversky & Kahne-
man, 1974; Kahneman et al., 1982). Given the prevalence of
risk in everyday choice and the breadth of biases that charac-
terize people’s preferences in laboratory studies (Friedman et
al., 2014), understanding the relationship between prior ev-
eryday knowledge and risk preferences is crucial to under-
standing how these decisions are made.
The importance of prior probabilities is especially no-
table in decisions from experience, in which people must
learn reward structures through sampling, rather than being
given this information directly (Hertwig et al., 2004). How-
ever, research on decisions from experience is usually done
through abstract laboratory tasks, in which people’s contex-
tual, domain, and prior knowledge is assumed not to inﬂuencechoice. This has enabled results from decisions from experi-
ence to be directly compared with those from decisions from
description, in which probabilities and rewards are given di-
rectly to participants, and has led to a number of notable ﬁnd-
ings on systematic differences in people’s behavior in these
domains (Hertwig & Erev, 2009; Rakow & Newell, 2009;
Weber et al., 2004; Ert & Haruvy, 2017; Ert & Lejarraga, in
press). However, this has come at the cost of testing behavior
on choice tasks that are systematically different from those
people face in everyday life.
To determine the effects of prior probabilities on people’s
risk preferences, we ran a large online experiment in which
participants chose between safe and risky options. In each
case, participants ﬁrst viewed a sample of Democratic and
Republican voters in a congressional district in a US state.
While participants knew the state, they did not know the dis-
trict. The risky option paid out depending on either the iden-
tity of the next voter sampled or the majority of voters in the
district. By keeping the voter sample constant and varying
only the state being sampled from, we were able to measure
the impact of prior probabilities on people’s risk preferences
as well as compare people’s choices with those of the optimal
Bayesian model.
Background
While the impact of prior knowledge on risk preferences is
not well understood, the effects of prior probabilities on judg-
ments and risk on choice are active areas of research. In both,
there is debate as to the best way to model these processes, as
well as the extent to which people behave optimally.
Bayesian models of human judgment
Research on the role of prior probabilities on people’s judg-
ments is traditionally done by studying how people incorpo-
rate base-rate knowledge with given information in labora-
tory tasks. Early work appeared to show that people tend
to disregard base-rate information in their judgments and
predictions (Tversky & Kahneman, 1974; Kahneman et al.,
1982), a phenomenon termed the base rate fallacy. However,
other researchers have questioned the robustness of this fal-
lacy. Gigerenzer and Hoffrage (1995) showed that changing
the presentation format of statistical information from prob-
abilities to frequencies caused people to make decisions that
better incorporated base-rate information. Furthermore, Grif-
ﬁths and Tenenbaum (2006) showed that people’s conditional
predictions of many real-world phenomena are often indistin-
guishable from those of the optimal Bayesian model. In their
setup, participants were not explicitly given prior distribution
information but appeared able to accurately incorporate it into
their predictions. Thus, in everyday tasks, it appears that peo-
ple’s judgments can and do incorporate prior information in
the optimal manner.
Decision-making under risk
Somewhat at odds with the colloquial concept, the riskiness
of an option in the behavioral sciences is generally deﬁned
as the variance of its expected outcomes. Research in this
area has shown that people’s choices often reﬂect risk aver-
sion. That is, people prefer safe options – those with low or
zero outcome variance – to more risky options, all else be-
ing equal. This is most often experimentally studied in the
simplest case, in which people make choices between a risky
option that pays off only with some probability p, and a safe
option than pays off with certainty.
This aversion to risk led Bernoulli (1954/1738) to propose
a model of diminishing marginal utility of wealth. In this
model, later expanded and formalized as Expected Utility
Theory (Neumann & Morgenstern, 1953), people maximize
their expected utility of wealth, rather than their expected
wealth, where utility is a monotonic but concave function
of wealth. Under these assumptions, it can be shown that a
utility-maximizing decision-maker will prefer an option with
low outcome variance to another with equal expected value
but higher variance.
Others have questioned both the descriptive validity of this
model and the ubiquity of risk aversion across decision con-
texts (Friedman et al., 2014). Perhaps most notably, Kahne-
man and Tversky (1979, 1992) developed Prospect Theory
as an alternative descriptive model of decision-making under
uncertainty. Rather than a purely concave function a wealth,
utility in Prospect Theory is deﬁned with respect to a refer-
ence point; outcomes above this reference point are gains, and
outcomes below it are losses. For gains, the classic utility
function holds, but for losses this curve is convex. In addi-
tion to this value function, Prospect Theory models the effect
of probability on choice through a nonlinear weighting func-
tion that overweights low probabilities and underweights high
probabilities. The combination of these utility and probability
weighting functions account for what Kahneman and Tversky
called the fourfold pattern of risk attitudes; people are risk
averse for gains and risk seeking for losses with high prob-
ability, but risk seeking for gains and risk averse for losses
with low probability.
Decisions from description and experience
Both Expected Utility Theory and Prospect Theory were de-
veloped to explain people’s preferences in decisions from de-
scription, in which people are given both the probabilities and
outcomes for every option. However, the predictions of these
models do not always generalize to tasks in which proba-bilities and outcomes have to be learned through sampling
(Barron & Erev, 2003). For example, in these decisions from
experience people tend to overweight high-probability events
and underweight low-probability events, contradicting the ex-
perimental ﬁndings in decisions from description. These dis-
crepancies have lead some to call for separate models for the
two paradigms (Rakow & Newell, 2009), while others have
argued that traditional models can be adapted to explain be-
havior in both (Fox & Hadar, 2004) or have developed new
models that do so (Lieder et al., 2018).
While learning outcome information through sampling ap-
pears to have systematic effects on people’s risk preferences,
the impact of people’s expectations on these preferences is
less clear. Ert and Trautmann (2014) found that people’s es-
timates of payout probabilities after sampling tended to lie
between their initial estimates and the observed frequencies,
as would be predicted by a Bayesian model. However, sam-
pling impacted choice in ways that this model does not fully
account for. After sampling payouts from a choice with an
unknown payout probability, participants were then given a
choice between taking another sample from this option and a
sample from a distribution with a known payout probability.
Participants tended to prefer continuing to sample from this
distribution when payoff probabilities were high, and sam-
ple from the known probability when payoff probabilities
were low. This behavior contradicts both the predictions of
a Bayesian updating model and the probability judgments of
other participants on an identical task. Sampling may there-
fore affect people’s risk preferences in ways that are not ex-
plained by the optimal Bayesian model.
A Bayesian approach to modeling risk
To test the effects of varying prior probabilities on partici-
pants’ risk preferences, we need to estimate the effects these
probabilities have on an option’s value. In this section, we
review how these values are traditionally estimated as well as
how to do so in a Bayesian framework.
Rational decision-making
Risk preferences on decisions from experience are estimated
by comparing the proportion of people that choose a risky
option ORover a safe option OS. This risky option pays
outVRwith some unknown probability qand 0 otherwise,
and the safe option pays out VSwith certainty. Before mak-
ing a decision, the decision maker is given these payoff val-
ues as well as nindependent samples from the risky option
D=fd1;d2;:::;dng,DB(n;q). In this sample are zsuc-
cesses – that is observations that would cause the risky option
to pay – and nzfailures. Observations that are successes are
encoded as 1 and failures as 0 so that ån
idi=z. Computing
the expected value of ORafter observing the data, E[ORjD]
thus requires estimating the probability p(dn+1=1) =q.
This is usually done using the maximum likelihood estimate
ofqgiven byz
n. In this case, the rational choice is to choose
ORif:
E[ORjD] =zVR
n>VS (1)
and choose the safe option otherwise. These values are often
equal, and if participants on average choose OSmore than OR
they are said to be risk averse for the choice. We will refer
to the maximum likelihood estimate of the value of the risky
option as the sample-based estimate.
This conditional expected value E[ORjD]can also be esti-
mated using a Bayesian model. This approach combines the
maximum likelihood estimate used above with prior probabil-
ities for possible values of q. While these prior probabilities
may not be useful in many laboratory studies, they are cru-
cial for solving many real-world tasks optimally (Anderson,
1990).
One of the strengths of Bayesian models is their ability to
perform both inference and prediction. Inference tasks in-
volve questions about the true parameters of a distribution,
while prediction tasks involve estimating the distribution of
future samples. While in most studies on risky choice es-
timating the value of the risky option requires prediction,
we believe that both are important in the risky choices peo-
ple make in everyday life. We ﬁrst show how traditional
prediction-based risky choices can be modeled in a Bayesian
framework, and then how inference-based choices can be
modeled as extensions of this setup.
Prediction
The prediction-based setup is identical to the one described
above, but the methods used for estimating E[ORjD]differ.
Instead of estimating a single value of q, we assume that there
is a set of Mpossible values qcan take on and estimate the
probabilities of each. In this case, the risky option should be
chosen if:
E[ORjD] =M
å
i
p(qijD)E[ORjqi]
>VS (2)
To make this choice, both p(qjjD), the posterior distribution
ofqjas well as E[ORjqj], the expected value of the risky
option conditional on a certain value qjneed to be estimated
for all Mpossible values of q. We show how to do both below.
Estimating the posterior distribution The posterior dis-
tribution of qj– its distribution conditional on sample D–
can be computed using Bayes’ rule:
p(qjjD) =p(Djqj)p(qj)
p(D)(3)
Because we assume that Dis a binomial sample and that ob-
servations are independent given the true value of q, Equation
(3) can be rewritten as follows:
p(qjjD) =qz
j(1qj)nzp(qj)
åM
i
qz
i(1qi)nzp(qi) (4)Estimating the expected value of the risky option The
probability that the next sample will pay given observations
Dcan be estimated using the posterior predictive distribution.
This gives a distribution of possible future samples condi-
tioned on the data already observed. In our setup, the risky
option pays VRif the next sampled value equals 1, and 0 oth-
erwise. This can be estimated as follows:
E[ORjqj] =p(dn+1=1jqj)VR=qjVR (5)
Equation (5) can then be combined with the posterior prob-
ability estimates from Equation (4) to estimate the expected
value of the risky option after observing data D:
E[ORjD] =M
å
i
p(qijD)qi
VR (6)
Inference
Inference-based risky choices can be constructed similarly to
the prediction setup described above, except that option OR
pays depending on the true value of q. Thus, the payoff of the
risky option is deterministic, but stochastic from the perspec-
tive of the decision-maker and modeler. In this case, the risky
option ORpays VRif the true value of qis in some set C. This
can be used to compute the expected value of the risky option
given probability qj:
E[ORjqj] = 1C(qj)VR (7)
where 1C(qj)is the indicator function that returns 1 if qj2C
and 0 otherwise. This value can be combined with the pos-
terior distribution estimates from Equations (3) and (4) to es-
timate the expected value of the risky option conditional on
observations D:
E[ORjD] =M
å
i
1C(qj)p(qijD)
VR (8)
Testing the model predictions
We test the predictions of the Bayesian model above by study-
ing risk preferences when Dis presented as a random sample
of Democratic and Republican voters. This sample is from a
known state but unknown congressional district in that state.
This allows us to test the model predictions in a setting in
which many Americans have relatively strong prior knowl-
edge, and in which the true prior distributions can be obtained
from election data. Furthermore, this setup allows prior prob-
abilities to be easily manipulated by changing the state Dis
sampled from.
In this setup, the prior probability distribution p(q)for a
state is the distribution of Democratic and Republican vot-
ers by district in that state. qjthus denotes the proportion
of Democratic voters (among only Democratic and Republi-
can voters) in district jfor the Mtotal districts in the state.
Inference-based risky tasks pay out depending on the true
distribution of Republican and Democratic voters in the dis-
trict, and prediction-based risky tasks pay out according to
Massachusetts California New York Florida Texas Tennessee OklahomaPrior Posterior
2550751002550751002550751002550751002550751002550751002550751000.00.20.4
0.00.20.4
Percent DemocraticProbabilityFigure 1: Distribution of the percentage of Democratic voters in the 2016 Presidential Election by congressional district.
The top row shows prior distributions, and the second row posterior distributions after observing three Democratic and three
Republican voters. These percentages are among only Democratic and Republican voters.
the identity of the next voter sampled. Histograms of prior
and posterior distributions for the seven US states included in
our study are given in Figure 1. All voter data used in the ex-
periment and results are from the 2017 Cook Political Report
Partisan V oter Index.
Participants
We recruited 1,384 participants from Amazon Mechanical
Turk, limiting our study to those living in the United States.
Participants earned $0.50 for completion of the experiment,
with the ability to earn an additional bonus of either $0.25
or $0.50. 24 participants were excluded for failing a basic
attention check, and an additional 134 participants for fail-
ing to identify the region of the US containing the state they
were assigned to. This left a total of 1,226 participants whose
choices we report.
Procedure
Participants were assigned randomly to either the district con-
dition or the individual condition. Estimating the value of the
risky choice in the district condition was an inference task
and paid off depending on the party afﬁliation of the majority
of the voters in the district. Estimating this value in the in-
dividual condition was a prediction task, and the risky option
paid off depending on the next sampled voter’s party afﬁlia-
tion. This risky option paid off either $0.50 or $0, and in both
conditions participants made a choice between this risky op-
tion and a safe option that always paid $0.25. In total, of the
1,226 participants whose data we report, 619 were assigned
to the district condition, and 607 to the individual condition.
In both conditions, the political party that determined whether
the risky option paid off was randomized across participants.Participants were instructed that they would see a random
sample of six voters from the 2016 US Presidential election
in a certain congressional district. They were told that all of
these voters would be sampled from a single congressional
district in a US state, and while they would know the state,
they would not know the district. They were also told that
each district had the same chance of being selected. Partici-
pants then saw a map of the congressional districts in the state
they were assigned to as well as the number of districts in that
state before moving onto the choice phase. Seven total states
were possible: Massachusetts, California, New York, Florida,
Texas, Tennessee, and Oklahoma. These states were chosen
to cover as much of the prior distribution space as possible.
After a practice round, participants completed a single test
trial. Participants were assigned to the same condition (dis-
trict or individual) for the practice and test trial, but different
states were always assigned for both. The six sampled vot-
ers on the test trial were always three Democrats and three
Republicans. Thus, the sample-based estimate of the risky
option’s value and the value of the safe option were always
equal, and changes in participants’ preferences in different
states would be determined only by their prior knowledge.
If the risky option was chosen, it then paid out with its true
posterior probability.
Results
A scatter plot of the proportion of participants who chose the
risky option for both conditions and for every state is given
in Figure 2. In both the district and the individual conditions,
a majority of participants made the optimal choice for 13 of
the 14 possible state-party combinations. These proportions
MAMA
CACA
NYNY
FL
FLTX
TXTN
TNOK
OKMAMA
CACA
NYNY FL
FLTX
TXTN
TNOK
OKDistrict Condition Individual Condition
0.00 0.25 0.50 0.75 1.00 0.4 0.5 0.60.20.40.6
Posterior Probability of Risky Option PayingProportion Choosing Risky OptionRisky Option Pays if
 District or Individual
Rep.
Dem.Figure 2: A comparison of the posterior value of the risky choice and the proportion of participants who chose it. Red points
indicate that the risky option paid if the district was majority Republican or the next voter sampled was Republican, and blue
points indicate the same for Democratic districts and voters. States are indicated with two-letter abbreviations, and vertical
lines show standard error estimates. In each case, the safe option paid $0.25 with certainty, and the risky option paid $0.50 or
$0. The maximum likelihood estimate of the probability of the risky option paying was 0.5 for all points. Note that in the 2016
US Presidential Election, all of Massachusetts’ districts voted Democratic and all of Oklahoma’s districts voted Republican.
were signiﬁcantly greater than 0.5 when the risky choice was
optimal in both conditions ( p<0:017 for both), as well as
signiﬁcantly less than than 0.5 when the safe choice was opti-
mal ( p<0:0001 for both). Thus, given the prior probabilities,
participants’ risk preferences were on average consistent with
the Bayesian model.
While participants on average chose optimally, a higher
proportion did so when the safe option was optimal than when
the risky option was optimal. This difference was signiﬁcant
in the district condition (difference = 0.117, p<0:01), but not
in the individual condition (difference = 0.077, p=0:056).
Thus, although participants adapted their choice strategies to
changes in the prior probabilities, there is some evidence that
they did this more often when the optimal choice was the safe
one.
To model the effects of the posterior value of the risky op-
tion on choice, we model utility for participant kas a latent
variable Uk. Every participant thus has utility UR;kfor the
risky option and US;kfor the safe option. These utilities are
deﬁned as follows:
UR;k=aR+bE[ORjD]+eR;k (9)
US;k=aS+bVS+eS;k (10)
That is, participant k’s utility for an option is a sum of the
value of the option and an unobserved component ek. Higher
values of bindicate that a larger proportion of participants’
utility is explained by the value of the option, and aRand
aSdenote the expected utility when the unobserved compo-
nent is zero and the risky and safe options have expected
value of zero. Under the assumption that these unobservedcomponents have independent Type I extreme value distribu-
tions, it can be shown that for participant k, the probability
p(ORjE[ORjD])of choosing the risky option conditional on
the data can be written as:
p(ORjE[ORjD]) =1
1+e[aRaS]b
E[ORjD]VS (11)
This is the standard equation for the logit model with
E[ORjD]VSas a predictor of choice (McFadden, 1973). If
participants estimate E[ORjD]using a sample-based estimate,
their utility for the risky and safe option will be equal, and
thusbwill not be a signiﬁcant predictor of choice. However,
if people use prior probabilities in their evaluations, options
with higher posterior value will have higher utility, and so b
will be positive.
To test this model, we ran a logistic regression of the pos-
terior value of the risky option minus the value of the safe
option on choice for both the district and individual condi-
tions. In both regressions, the coefﬁcient bwas signiﬁcant
and positive (district condition estimate = 3.0062, individual
condition estimate = 7.6859, p<0:0001 for both), conﬁrm-
ing our hypothesis that people’s choices are inﬂuenced by the
prior probabilities in the optimal way. That is, increasing the
posterior value of the risky option by changing the prior dis-
tribution caused people to choose it signiﬁcantly more often
due to its positive effect on utility U.
Discussion
Consistent with previous work on human judgment and per-
ception, we ﬁnd strong evidence that people’s behavior on
a risky choice task is inﬂuenced by prior probabilities in a
way that is consistent with Bayesian inference. While there
is some evidence that participants chose optimally more often
when the safe option was optimal, this bias is relatively small
when compared to the effects of changes in prior probabil-
ities. In contrast with most previous work combining real-
world data with Bayesian modeling, however, our results are
notable as they are not based on a task people commonly face.
That people were still inﬂuenced by the prior probabilities in
the optimal way suggests that people can use their knowledge
in decisions in relatively task-independent ways. We argue
that this provides additional strength to the case that the study
of risky choice can be integrated into the Bayesian approach
to studying cognition.
The assumptions of the Bayesian model are especially
notable in their similarity to and differences from those of
Prospect Theory. For example, the Bayesian model can clar-
ify aspects of Prospect Theory that were not rigorously de-
ﬁned, such as the origins and importance of references points,
which we model as prior probability distributions. Using
these distributions gives both a hypothesis of what someone’s
expectations could be – not just in terms of base-rates, but
also in terms of variance and distribution type – but also a ra-
tional basis of what these expectations should be. However,
the relationship between reference points in Prospect Theory
and in the Bayesian model is not entirely straightforward. In
our model, we make the assumption that participants have no
expectations about the possible payoff values of an option, but
do have expectations about probabilities. In Prospect Theory,
however, reference points are deﬁned in terms of payouts and
not probabilities.
Part of this confusion is due to the fact that in both mod-
els, the payoff amount of the risky option has no relationship
with its probability. These interactions may not be impor-
tant in laboratory tasks where people do not have strong prior
knowledge, but they may be crucial in the everyday risky
decisions that people make. Previous work has shown that
the distribution of rewards and probabilities in real-world do-
mains has a general structure that is absent in most studies
on the topic (Pleskac & Hertwig, 2014). Furthermore, people
use this structure to predict probabilities from outcomes, and
outcomes from probabilities. Further generalizations of the
Bayesian model in which expectations exist and interact for
both payouts and probabilities may help to disentangle these
effects and enable clearer and more robust behavioral predic-
tions.
Conclusion
While the study of decision making under risk and uncer-
tainty is a core focus of research in behavioral economics and
psychology, it has yet to be uniﬁed with Bayesian models of
cognition. These models have proved successful in explain-
ing a wide range of psychological and behavioral phenom-
ena, and we show that they can be applied to explain the ef-
fects of prior probabilities on risky choice. We believe thatthis approach will be crucial to better understanding decision-
making under uncertainty in the future.
References
Anderson, J. R. (1990). Studies in cognition. the adaptive character
of thought . Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Barron, G., & Erev, I. (2003). Small feedback-based decisions
and their limited correspondence to description-based decisions.
Journal of Behavioral Decision Making ,16, 215-233.
Bernoulli, D. (1954). Exposition of a new theory on the measure-
ment of risk. Econometrica ,22(1), 23–36.
Ert, E., & Haruvy, E. (2017). Revisiting risk aversion: Can risk
preferences change with experience? Economics Letters ,151, 91
- 95.
Ert, E., & Lejarraga, T. (in press). The effect of experience on
context-dependent decisions. Journal of Behavioral Decision
Making ,31(4), 535-546.
Ert, E., & Trautmann, S. (2014). Sampling experience reverses pref-
erences for ambiguity. Journal of Risk and Uncertainty ,49(1),
31-42.
Fox, C. R., & Hadar, L. (2004). “Decisions from experience” =
sampling error + prospect theory: Reconsidering Hertwig, Bar-
ron, Weber Erev (2004). Judgment and Decision Making ,1(2),
159-161.
Friedman, D. R., Isaac, M., James, D., & Sunder, S. (2014). Risky
curves: On the empirical failure of expected utility theory . Lon-
don: Routledge.
Gigerenzer, G., & Hoffrage, U. (1995). How to improve bayesian
reasoning without instruction: Frequency formats. Psychological
Review ,102(4), 684-704.
Grifﬁths, T. L., & Tenenbaum, J. B. (2006). Optimal predictions in
everyday cognition. Psychological Science ,17(9), 767-773.
Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions
from experience and the effect of rare events in risky choice. Psy-
chological Science ,15(8), 534-539.
Hertwig, R., & Erev, I. (2009). The description-experience gap in
risky choice. Trends in Cognitive Sciences ,13(12), 517-523.
Kahneman, D., Slovic, P., & Tversky, A. e. (1982). Judgment under
uncertainty: heuristics and biases . Cambridge, England: Cam-
bridge University Press.
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis
of decision under risk. Econometrica ,47(2), 263-91.
Lieder, F., Grifﬁths, T. L., & Hsu, M. (2018). Over-representation
of extreme events in decision making reﬂects rational use of cog-
nitive resources. Psychological Review ,125(1), 1-32.
McFadden, D. (1973). Conditional logit analysis of qualitative
choice behaviour. In P. Zarembka (Ed.), Frontiers in economet-
rics(p. 105-142). New York, NY , USA: Academic Press New
York.
Neumann, J. v., & Morgenstern, O. (1953). Theory of games and
economic behavior . Princeton, NJ: Princeton University Press.
Pleskac, T. J., & Hertwig, R. (2014). Ecologically rational choice
and the structure of the environment. Journal of Experimental
Psychology ,143(5), 2000 - 2019.
Rakow, T., & Newell, B. R. (2009). Degrees of uncertainty: An
overview and framework for future research on experience-based
choice. Journal of Behavioral Decision Making ,23(1), 1-14.
The Cook Political Report. (2017). 2017 cook political report par-
tisan voter index. (https://www.cookpolitical.com/introducing-
2017-cook-political-report-partisan-voter-index, Last accessed
on 2019-02-01)
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty:
Heuristics and biases. Science ,185(4157), 1124–1131.
Tversky, A., & Kahneman, D. (1992). Advances in prospect theory:
Cumulative representation of uncertainty. Journal of Risk and
Uncertainty ,5(4), 297–323.
Weber, E. U., Shaﬁr, S., & Blais, A.-R. (2004). Predicting risk-
sensitivity in humans and lower animals: Risk as variance or co-
efﬁcient of variation. Psychological Review ,111, 430-445.