Serial reproduction reveals the geometry of
visuospatial representations.

PSYCHOLOGICAL AND
COGNITIVE SCIENCESSerial reproduction reveals the geometry of
visuospatial representations
Thomas A. Langloisa,b,c,1
, Nori Jacobyc,d,1
, Jordan W. Suchowe
, and Thomas L. Grifﬁthsb,2
aDepartment of Psychology, University of California, Berkeley, CA 94704;bDepartment of Computer Science, Princeton University, Princeton, NJ 08542;
cComputational Auditory Perception Research Group, Max Planck Institute for Empirical Aesthetics, 60322 Frankfurt am Main, Germany;dThe Center for
Science and Society, Columbia University, New York, NY 10027; andeSchool of Business, Stevens Institute of Technology, Hoboken, NJ 07030
Edited by Richard M. Shiffrin, Indiana University, Bloomington, IN, and approved February 19, 2021 (received for review June 29, 2020)
An essential function of the human visual system is to locate
objects in space and navigate the environment. Due to limited
resources, the visual system achieves this by combining imper-
fect sensory information with a belief state about locations in a
scene, resulting in systematic distortions and biases. These biases
can be captured by a Bayesian model in which internal beliefs
are expressed in a prior probability distribution over locations
in a scene. We introduce a paradigm that enables us to mea-
sure these priors by iterating a simple memory task where the
response of one participant becomes the stimulus for the next.
This approach reveals an unprecedented richness and level of
detail in these priors, suggesting a different way to think about
biases in spatial memory. A prior distribution on locations in a
visual scene can reﬂect the selective allocation of coding resources
to different visual regions during encoding (“efﬁcient encoding”).
This selective allocation predicts that locations in the scene will
be encoded with variable precision, in contrast to previous work
that has assumed ﬁxed encoding precision regardless of location.
We demonstrate that perceptual biases covary with variations
in discrimination accuracy, a ﬁnding that is aligned with simula-
tions of our efﬁcient encoding model but not the traditional ﬁxed
encoding view. This work demonstrates the promise of using
nonparametric data-driven approaches that combine crowdsourc-
ing with the careful curation of information transmission within
social networks to reveal the hidden structure of shared visual
representations.
visual perceptionjspatial memoryjiterated learningjBayesian statistics
The formation of accurate memories poses a difﬁcult problem
for the human visual system, which must process complex
and noisy scenes while keeping pace with a relentless stream
of incoming information. Because not all information is equally
useful, the visual system must allocate its limited resources selec-
tively, which leads to simpliﬁed and distorted internal represen-
tations (1–9). An essential function of the human visual system
is to locate objects and navigate visual scenes, and understand-
ing how it accomplishes this depends on detailed and accurate
measures of visuospatial memory representations (10).
Previous work has probed visuospatial memory distortions
using a task in which participants reproduced the locations of
points within visual scenes, ﬁnding that participants’ responses
were systematically biased (11–14). These systematic distortions
have been described in terms of an attraction toward prototyp-
ical locations in the scenes (11–15), with perceptual attractors
located at the centers of mass of visual objects (12); centered
around prototype locations, such as the quadrant centers of a
circle (11, 13, 14); or located along the medial axis (“shape
skeleton”) of geometric shapes (16).
The state of the art in characterizing human visual memory
biases relies on the long-standing category adjustment model
(CAM) (11, 13), which asserts that each reconstruction Rfrom
memory linearly interpolates between the stimulus Sand a
prototype P, with
R=wS+ (1w)P+n [1]for some weight w, where nis a perceptual noise term. Using the
CAM relies on ﬁtting the prototype location and other model
parameters to the data, a process that is sensitive to estima-
tion noise, particularly when using a relatively small number of
human judgments (11, 13). In situations where multiple proto-
types need to be estimated, the risk of overﬁtting to noise is even
greater, and the number of prototypes must be predetermined
(Materials and Methods).
Here, we propose a method that overcomes these limitations.
Our approach is based on two innovations. First, we lever-
age online crowdsourcing platforms to increase the number of
human judgments obtained signiﬁcantly, and second, we apply an
adaptive sampling technique based on serial reproduction (17)
to estimate the prototype locations nonparametrically, sidestep-
ping any model-ﬁtting approach. In our paradigm, information is
repeatedly retrieved from memory by a sequence of people, with
the reconstruction of one person becoming the stimulus for the
next, forming a transmission chain analogous to the “telephone
game.” The ﬁrst participant views a point overlaid on an image
and must later reproduce the location of the point from mem-
ory following a delay. The next participant views the same image
but with the point located in the position reconstructed by the
previous participant. This process is repeated for each partici-
pant in the chain (Fig. 1 AandSI Appendix, Fig. S1). Unlike the
traditional approach, which typically attempts to ﬁt a descriptive
Signiﬁcance
A primary function of human vision is to encode and recall
spatial information about visual scenes. We developed an
experimental paradigm that reveals the structure of human
spatial memory priors in unprecedented detail. We ran a
series of 85 large-scale online experiments with 9,202 par-
ticipants that paint an intricate picture of these priors. Our
results suggest a way to understand visuospatial representa-
tions as reﬂecting the efﬁcient allocation of coding resources.
In a radical departure from traditional theory, we introduce
a model that reinterprets spatial memory priors as reﬂecting
an optimal allocation of perceptual resources. We validate the
predictions of the model experimentally by showing that per-
ceptual biases are correlated with variations in discrimination
accuracy.
Author contributions: T.A.L., N.J., J.W.S., and T.L.G. designed research; T.A.L. and N.J.
performed research; T.A.L., N.J., and J.W.S. contributed new reagents/analytic tools;
T.A.L. and N.J. analyzed data; T.L.G. advised on the development of the theory and the
statistical modeling; and T.A.L., N.J., J.W.S., and T.L.G. wrote the paper.y
The authors declare no competing interest.y
This article is a PNAS Direct Submission.y
This open access article is distributed under Creative Commons Attribution License 4.0
(CC BY).y
1T.A.L. and N.J. contributed equally to this work.y
2To whom correspondence may be addressed. Email: tomg@princeton.edu.y
This article contains supporting information online at https://www.pnas.org/lookup/suppl/
doi:10.1073/pnas.2012938118/-/DCSupplemental .y
Published March 26, 2021.
PNAS 2021 Vol. 118 No. 13 e2012938118 https://doi.org/10.1073/pnas.2012938118 j1 of 11Downloaded by guest on March 31, 2021
ITERATION 2
Stimuli
(1000 ms):
Blank
(1000 ms):
Jittered probe:
Response:
...
...The transmission chain memory paradigm
BA
Transmission chain results for a simple shape and natural image
UNIFORM 1ST ITERATION 5TH ITERATION 10TH ITERATION 15TH ITERATION 20TH ITERATION
UNIFORM 1ST ITERATION 5TH ITERATION 10TH ITERATION 15TH ITERATION 20TH ITERATION
CTransmission chain shape results: superposition of points across all iterations and KDEsITERATION 1 LAST ITERATION INITIAL SEED
DNatural image results: superposition of points across all iterations and KDEsFig. 1. Visuospatial memory distortions, serial reproduction paradigm, and results. (A) Illustration of the serial reproduction method. The ﬁrst partici-
pant views an image with a point overlaid in a random position and is then asked to reproduce its location from memory. The next participant views
the same image but with the point located at the position reconstructed by the previous participant. The process is repeated for a total of 20 iterations.
We adopted a between-subject design, where participants contributed to a given chain only once. (B) Serial reproduction results for the remembered
position of points overlaid on a simple shape (triangle) and a natural image (lighthouse). The initial uniform distributions of 500 points are shown
(column 1) as well as the distributions of the same points at iterations 1, 5, 10, 15, and 20 of the transmission chains. (C ) Scatterplots showing the
superposition of responses across all iterations of the chains for each of the shapes and the corresponding KDEs. (D) KDEs and scatterplots for complex
natural scenes.
2 of 11jPNAS
https://doi.org/10.1073/pnas.2012938118Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsDownloaded by guest on March 31, 2021
PSYCHOLOGICAL AND
COGNITIVE SCIENCESmodel to noisy and unreliable estimates following only a single
iteration of this process, we repeat it until convergence, which
allows us to discern the prototypes toward which the responses
converge. Intuitively, serial reproduction “ampliﬁes” shared per-
ceptual biases by compounding systematic errors (18). In terms
of the CAM, it is straightforward to show that repeating the
paradigm eventually converges to the CAM’s prototypes. In the
simple case described in Eq. 1, the distance to the prototype
decreases on average with each iteration, and the prototype P
is approximately a ﬁxed point of the iterated process.
Indeed, in the case of simple shapes, our paradigm reveals
a pattern of results that is consistent with previous literature
and the CAM (Fig. 1C andSI Appendix, Fig. S12) (13). How-
ever, it is also visually apparent that our technique paints
a far more nuanced picture of visuospatial memory biases,
revealing patterns missed by previous estimation approaches
and that are inconsistent with a bias toward category centers
(11, 12). Representative results are shown in Fig. 1 BandC.
We found spatial memory distortions toward the edges and
vertices of the geometric shapes, revealing a greater number
of modes at different locations than previously thought (11–
13). For natural images, the patterns are even more complex
(Fig. 1D).
How can we explain the complex patterns of visual memory
biases revealed by our method? The CAM has traditionally been
given a Bayesian interpretation (11). In this formulation, proto-
type point locations (landmarks) are replaced by a continuous
probability density function [the prior p(S), which represents
a belief state about probable point locations] where the land-
marks correspond to the modes of the distribution. Intuitively,this distribution quantiﬁes the degree of “landmarkness” of dif-
ferent visual regions. According to this view, participants infer
point locations by combining noisy sensory information with the
belief state. As a result, participants produce responses that are
systematically biased toward nearby landmarks ( SI Appendix,
Fig. S2). The Bayesian interpretation has an important impli-
cation when it comes to understanding our serial reproduction
paradigm because under experimentally veriﬁable assumptions,
one can show that with multiple iterations of the serial repro-
duction process, distributions estimated from the chain results
converge to the prior (refs. 18–20 and Materials and Methods
have a proof).
Previous literature on CAM (11) assumed that the sensory
noise is Gaussian and isotropic regardless of location (“ﬁxed pre-
cision”) with a ﬁxed SD (Fig. 2A). This assumption is common
to the classical “categorical perception” literature (21). Impor-
tantly, it has a direct mathematical implication with respect to
how discrimination accuracy changes depending on the distance
of a stimulus location to a landmark. In particular, it predicts that
discrimination is lower near the landmarks because point loca-
tions near landmarks will be biased and perceived to be closer
than they actually are, making them harder to tell apart (Fig. 2A).
This phenomenon, known as the “perceptual magnet effect,” has
been demonstrated in multiple perceptual modalities (21–24),
including spatial memory (25, 26).
An alternative to the ﬁxed precision view is the idea that
precision varies over an image. The “variable precision” view
trivially predicts variation in discrimination accuracy and can also
explain convergence in the transmission chains since it models
serial reproduction as a random walk with decreasing step sizes.
SMALL BIAS
Not a landmark
Average reproductionStimulus location
Variable precision
HIGH
DISCRIMINATION
LOW
DISCRIMINATION
TTTGGG
RRRSSS
MMO
O
O
M
MM
MM
M
O
OO
OO
OO
O
Perceptual JND units
TION
TT
N
Not a landmark
Average reproductionStimulus location
Variable precisionLandmark
Average reproductionStimulus location
Variable precisionLandmark
Average reproductionStimulus location
Fixed precisionLandmark
Average reproductionStimulus location
Variable precision
Perceptual JND units
Mode shift
LOW
DISCRIMINATION
LARGE BIAS
HIGH
DISCRIMINATION
NO BIASNot a landmark
Average reproductionStimulus locationHIGH
DISCRIMINATION
NO BIASLOW
DISCRIMINATIONLANDMARK STIMULUSFixed precision
NOT A LANDMARK
LANDMARK NOT A LANDMARKDILATION CONTRACTIONAVERAGE REPRODUCTIONINTERNAL REPRESENTATION (JND UNITS)
EUCLIDEAN SPACE
SMALL BIASLANDMARK STIMULUS NOT A LANDMARK
AVERAGE REPRODUCTION LARGE BIASSymmet ric variable precision mo del of spatial memory distortionsEcient e ncoding mo del of spat ial memory distortions
Simulated chain dynamics and d' for xed encoding ESimulated chain dynamics and d' for e cient encodingFixed precision model of spatial memory distortionsA
BC
D
Fig. 2. Models of visuospatial memory. The curves in A–C show the distributions of reproductions of a set of stimulus locations under a different model.
(A) Fixed precision view. Perceptual noise (precision) is assumed to be constant, and biases occur when participants infer the true stimulus location (red
dot). Average responses (pink dot) are pulled toward a nearby landmark (green dot). Because point locations near a mode (Left ) are perceived to be closer
to a nearby landmark, they are also harder to discriminate. Far from a mode (Right ), responses are less biased, and discrimination between nearby point
locations is higher. (B) Symmetric variable precision model. This model introduces the idea that perceptual noise (precision) varies and that it is highest near
a landmark (Left ) and lowest far from a landmark (Right ). This model trivially explains higher discrimination accuracy near a landmark but does not predict
perceptual biases. Individual responses are assumed to be independent noisy samples from a symmetric function centered on the true stimulus location,
and on average, these responses will be unbiased (pink dot). (C ) Efﬁcient encoding model. Constant precision in a perceptually warped coordinate space
(internal representation) determines how perceptual noise is skewed in Euclidean units. Visual regions near landmarks are overrepresented in the internal
representation, resulting in higher precision in external Euclidean units. The skewed perceptual noise also predicts that responses near a landmark will
be biased toward that landmark on average (Left ) but not when the stimulus is far from a landmark (Right ). In Euclidean space, the curves represent the
reproduction distributions of the responses. The same reproduction distributions are also shown in the internal representation (in JND units). (D) Simulations
of the ﬁxed encoding model and predicted discrimination accuracy map. Given the prior (column 1), the model produces perceptual biases toward the three
modes in the prior over multiple iterations of the serial reproduction process. Examples are shown for the 1st, 5th, 10th, 15th, and 20th iterations of
the process. The ﬁxed encoding model predicts that discrimination is reduced in the modes (column 8). (E ) Simulations of the efﬁcient encoding model and
predicted discrimination accuracy map. The model also produces perceptual biases over multiple iterations. Critically, it also predicts increased discrimination
accuracy in the modes of the prior (column 8).
Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsPNASj3 of 11
https://doi.org/10.1073/pnas.2012938118Downloaded by guest on March 31, 2021
With each step (iteration), responses are more likely to con-
centrate around the landmarks, which act as “absorbing states.”
Critically, the variable precision view introduces the possibility
that discrimination accuracy increases near the landmark, which
is the opposite of the prediction made by the ﬁxed precision
model. In this paper, we test these opposing theoretical pre-
dictions empirically. Testing for the perceptual magnet effect
or an increase in discrimination accuracy near a landmark is
accomplished by comparing the results of the memory exper-
iments (which reveal biases in reproduction) with the results
of two-alternative forced choice (2AFC) “same” or “different”
experiments, which quantify discrimination accuracy. We show
that discrimination accuracy is higher near the landmarks, refut-
ing the long-standing ﬁxed precision account of spatial memory
biases and supporting the variable precision view.
However, variable precision may or may not predict consis-
tent perceptual biases. In its simplest form, variable precision
can be implemented with symmetric noise (“symmetric variable
precision”) (Fig. 2B). Such a model predicts that RS(a single-
trial response) has independent noise with decreasing magnitude
closer to a landmark. However, this prediction is at odds with the
fact that people tend to produce biased responses (11, 12, 25) and
that as a result, neighboring point reconstructions tend to be ori-
ented in similar directions. For example, nearby point locations
that are close to a landmark will be consistently shifted toward
that landmark (SI Appendix, Fig. S3). We conﬁrm this effect in
our data, which we measure by quantifying the probability of
small angular differences in single-trial biases for nearby point
reconstructions, indicating that the variable precision model is ill
suited to fully explain spatial memory distortions, at least in its
simplest form.
These results demonstrate the need for a theoretical model
based on the variable precision view that can also predict the
consistent perceptual biases in the data. In this work, we inno-
vate on a recent Bayesian formulation of variable precision
developed in terms of efﬁcient encoding (7), generalizing it to
the high-dimensional case using mathematical tools from differ-
ential geometry. According to this model, convergence of the
serial reproduction chains occurs due to the combined effect
of variable precision (causing a shift in successive reproduc-
tions toward the landmark) and consistent perceptual biases
(like a “gravitational pull” of responses toward the landmarks).
This model has the advantage of being a fully Bayesian model,
just like the long-standing CAM account of spatial memory
described above, and with no additional parameters. To explain
this model, we start with a key notion from signal detection
theory (27).
A common assumption from signal detection theory is that
variable precision over an image can be measured both in
terms of changes in sensitivity using physical (Euclidean) dis-
tance units and also, in terms of constant just noticeable dis-
tance (JND) units over a transformed internal representation of
the space (Fig. 2 CandSI Appendix, Fig. S4). In other words,
increased precision in a Euclidean coordinate space is equiva-
lent to constant precision in a perceptually dilated coordinate
space. Intuitively, the geometric pattern of dilations and con-
tractions is similar to how variations in perceptual sensitivity
are reﬂected in neural representations such as the somatosen-
sory homunculus (28) or retinotopic map (29), where increased
precision corresponds to areas that are overrepresented by the
brain. Interpreting variable precision in terms of JND units is
useful because it forms the basis of a fully Bayesian formula-
tion of the variable precision view that overcomes its limitations
when it comes to predicting perceptual biases while also pre-
dicting increased discrimination accuracy near the landmarks
(Fig. 2 CandD) (7, 30).
The efﬁcient encoding model (7) is based on the idea that
encoding resources limit the ability to store all regions of a visualscene with equal accuracy, and it speciﬁes the optimal trade-off
between coding resources and precision (7, 30). The essence of
the model is that it determines the exact mathematical relation
between the magnitude of the bias and discrimination accuracy.
This is useful because it predicts the full range of empirical
results in this paper including the serial reproduction dynamics
and discrimination accuracy measures (Figs. 2 DandEand 3
AandD). Critically, it also predicts that single-trial biases for
nearby point reconstructions tend to point in the same direction
(SI Appendix, Fig. S3).
Thinking about spatial memory distortions in terms of efﬁ-
cient encoding helps to explain the structured priors revealed by
our method: As the perceptual space is condensed to Euclidean
space, it concentrates the prior probability distribution in regions
of greater precision (Fig. 2C). A uniform prior in the percep-
tual space will become a distribution in Euclidean space in
which probability is proportional to encoding precision. As a
result, the priors we estimate reveal the geometry of the per-
ceptual space. This perspective also makes additional testable
predictions. Because it explains biases in terms of an optimal
allocation of encoding resources, it predicts that limiting these
resources in the task should result in qualitative changes to
the internal representation for a given stimulus image, rather
than just introducing additive noise to the original represen-
tation. We conﬁrm this prediction empirically by reducing the
encoding time in our experiments, which reveals qualitative sim-
pliﬁcations to the transmission chain results, rather than just
additional noise. In contrast, changing the retention time or
manipulating the display during the reproduction phase had only
a minor effect on the ﬁnal results, suggesting that biases emerge
during encoding rather than the retention or reproduction
phases.
Results
Revealing Spatial Memory Priors by Serial Reproduction. We began
by running a series of serial reproduction experiments probing
memory for point locations in simple images and a selection of
complex natural scenes. For simple images, we used geometric
shapes (circle, triangle, square, and pentagon), and for natural
scenes, we used images of both natural and man-made objects
(Fig. 1). We ran approximately 500 unique chains, 1 for each ini-
tial point location, which we randomly sampled from the uniform
distribution. For each chain, the telephone game was played for
20 iterations. Fig. 1B shows the initial uniform distributions of
the points for the triangle and a natural image, as well as the
results of the 1st, 5th, 10th, 15th, and 20th iterations of the
process. As expected, initial point locations and the locations
of points in the ﬁrst iteration were not signiﬁcantly different
from a uniform distribution (P = 0.35 and P= 0.08 for initial
seeds and iteration 1, respectively). However, subsequent itera-
tions deviated considerably from the uniform distribution (P <
0.001 for iterations 2 to 20 for all shapes). The distributions esti-
mated based on data aggregated from each iteration converged
to a stationary distribution within approximately 20 iterations
(SI Appendix, SI Text and Figs. S5 and S6 have further conver-
gence analyses). Fig. 1 CandDshows scatterplots of the chain
point locations across all iterations for each of the images, as
well as kernel density estimates (KDEs; which are estimates of
the underlying distributions that produced the data, as explained
inMaterials and Methods). They reveal the intricate structure of
visuospatial memory priors.
Precision Shapes Visuospatial Memory Representations. To test the
opposing predictions of the ﬁxed and efﬁcient encoding models
(the simulated d0results of both models are in Fig. 2 DandE), we
ran a series of discrimination accuracy experiments on a separate
cohort of participants using the same images. Participants saw
the image with a point positioned over it in a random location
4 of 11jPNAS
https://doi.org/10.1073/pnas.2012938118Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsDownloaded by guest on March 31, 2021
PSYCHOLOGICAL AND
COGNITIVE SCIENCES
C Discrimination experiment results: smoothed d' grids and maps
B Discrimination experiment: experimental conditions and design D Correlations between priors and d' maps (real and predicted)0.20.40.60.811.21.41.61.8
00.20.40.60.811.21.41.6
00.20.40.60.811.200.10.20.30.40.50.60.70.80.9
00.10.20.30.40.50.60.70.80.910.10.20.30.40.50.60.70.8
00.20.40.60.811.21.41.6    INTERPOLATION    INTERPOLATION    INTERPOLATION
     INTERPOLATION      INTERPOLATION
     INTERPOLATION      INTERPOLATION
SMOOTHED D' GRID SMOOTHED D' MAP SMOOTHED D' GRID SMOOTHED D' MAP SMOOTHED D' GRID SMOOTHED D' MAP0
Smoothed d'
EFFICIENT
ENCODINGMODEL INPUT
234
Denisty relative to uniform
01234
Denisty relative to uniform1234
Denisty relative to uniform
   FIXED
ENCODING
REAL
DATA
r=0.76r=0.45
1000 ms
1000 ms
1000 ms
2AFC 2AFCSHIFTED SAME1st presentation 2nd presentation
RANDOM 6 PX RADIUSNO CHANGESAME SHIFTED2AFC
SAME
SHIFTED
0 0.01 0.02 0.03 0.04 0.05 0.06-0.8-0.6-0.4-0.200.20.40.60.8Correlation between
discrimination and prior
Efficient-encoding (simulation)
Fixed-encoding (simulation)
Data (average)A Real and simulated chain dynamics and d' maps (real and predicted)
ITERATION 0 ITERATION 1 ITERATION 5 ITERATION 10 ITERATION 15 ITERATION 20 DISCRIMINATION0
1
0Fig. 3. Visuospatial memory distortions correspond to variable encoding precision. (A) Representative example of real and simulated chain dynamics and
discrimination maps (face image). Real and simulated KDEs are shown for iterations 0, 1, 5, 10, 15, and 20. Both the efﬁcient and ﬁxed encoding models
provide good approximations to the real transmission chain data (SI Appendix, Fig. S8). Real and simulated discrimination accuracy maps are also shown,
including correlations to the prior. ( B) Discrimination experiment conditions and experimental design. Discrimination points were obtained from a regular
two-dimensional (2D) grid of points over the image. In the same condition, the red dot did not change position in the second presentation. In the shifted
condition, the red dot was shifted to a point located in a random position at a six-pixel radius distance from the original position. Two identical images were
shown for 1,000 ms sequentially with a red dot placed on top of them. The dot was either in the same location in both cases (same condition) or shifted
(shifted condition). Both the dot and the image were shifted by a random offset in the second presentation in both conditions. The starting points were
sampled from a 2D grid of possible points over the image. (C ) Discrimination results for natural images. Discrimination d0values for each grid point were
convolved with a Gaussian kernel, and ﬁnal maps were computed through cubic interpolation of the smoothed d0grid values. (D) Correlations between
priors and discrimination (natural images). For each noise magnitude , we computed the correlation predicted by the two models. The correlations were
positive (blue line) for the efﬁcient encoding model and negative (red line) for the ﬁxed encoding model. Thin lines show data for individual natural images;
error bars show SDs across images. The green line shows the mean and SD of the correlations of the empirical d0data and the priors. We exclude the edges
of the images because the ﬁxed encoding model produces predictions with noticeable edge artifacts resulting in slightly smaller correlations than the ones
we report. The ﬁxed encoding model also predicts smaller variation in d0across the images (SI Appendix, Fig. S8 ). The data support the efﬁcient encoding
model.
sampled from a regular grid of possible point locations (Fig. 3 B
andC). After a 1,000-ms delay, the same image reappeared with
the point in either the same position or in a shifted position, and
participants were asked to determine if the point was the same
or “shifted” (Fig. 3B). We obtained change sensitivity responses
from dense point grids over our images, producing detailed d0
accuracy maps (Fig. 3C andSI Appendix, Fig. S7). Smoothed d0
accuracy maps are shown for the natural images in Fig. 3C. We
found that discrimination maps were consistently highly and pos-
itively correlated with the transmission chain results. Because the
discrimination maps and prior KDEs are estimated from noisy
empirical measurements, we also computed disattenuated corre-
lations between the priors and the d0maps using estimates of the
internal reliability of the two measures ( SI Appendix). We found
that the disattenuated correlations for the d0maps, which ranged
between r= 0.76 and r= 0.93 (average r= 0.82), predict a signif-
icant portion of the variance, even though there may still be some
systematic variation originating from other sources. Note that
given the prior, both models have only a single degree of freedom
(the perceptual noise ). While both models ﬁt the transmission
chain experiment dynamics well (Fig. 3A andSI Appendix, Fig.
S8), the results of the discrimination experiment are consistent
with the predictions of the efﬁcient encoding model (we observed
positive correlations between the transmission chain results and
the simulated discrimination accuracy maps in all cases; P<
0.001; via bootstrapping) but not the ﬁxed encoding model (we
observed negative correlations in all cases where > 0). Fig. 3D
shows these opposing predictions.Consistent Perceptual Biases. A well-documented ﬁnding in the
literature describes people’s tendency to produce consistent per-
ceptual biases in the task. The bias consists in producing a
response that is oriented toward the nearest landmark. One
implication of this is that reproductions of neighboring point
locations will tend to be oriented in similar directions toward
the nearby landmark. However, the symmetric variable preci-
sion model predicts random independent bias directions (SI
Appendix, Fig. S3 AandB). To quantify this effect in our data,
we computed histograms of the angular differences between the
averaged biases of nearby point reconstructions for the trian-
gle data (SI Appendix, Fig. S3F ) and for all natural images (SI
Appendix, Fig. S3G). We found that small angular differences
(between −12and +12) tend to be 1.83 to 4.36 (mean 2.8)
times more probable than expected by chance for all images
(P<0.001; via bootstrapping) (SI Appendix, Fig. S3). The
efﬁcient encoding model predicts signiﬁcantly more probable
angular differences in this range, while the symmetric vari-
able precision model predicts a uniform distribution of angular
differences (SI Appendix has more information).
Encoding, Memory, and Reproduction. The theoretical assump-
tions of the efﬁcient encoding model predict that distortions
should change with direct experimental manipulations of encod-
ing precision (7). We conﬁrmed this through controlled experi-
ments in which we manipulated both spatial and temporal factors
of encoding precision for one of our natural images. Speciﬁ-
cally, we manipulated encoding precision temporally by reducing
Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsPNASj5 of 11
https://doi.org/10.1073/pnas.2012938118Downloaded by guest on March 31, 2021
the encoding time in our task from 1,000 to 200 ms. We also
manipulated encoding precision spatially by either adding Gaus-
sian noise to the stimulus image or reducing its contrast signiﬁ-
cantly. We found that these manipulations produced priors that
were signiﬁcantly different from the original when we compared
the resulting KDEs with the original ﬁndings (P <0.001) (SI
Appendix, SI Text and Fig. S9 have details). We also observed
simpliﬁcations to the priors for shape images following similar
experimental manipulations (SI Appendix, Fig. S11). In addi-
tion, we tested if the biases are generated during the encoding
phase or if they emerge during the testing phase of the experi-
ment when the image is reintroduced and participants produce a
response. To do this, we substituted the image during the repro-
duction phase with a completely blank gray region, as well as
the opposite: a blank gray region during the encoding phase,
followed by a naturalistic image during the reproduction phase.
If biases originate from visual processing of the images dur-
ing the reproduction phase, we would expect to see biases that
reﬂect the visual characteristics of the images shown at test time
(e.g., the natural image if the image shown during the encoding
phase was a blank gray region). However, we ﬁnd the opposite:
The pattern of biases corresponds to the visual characteristics
of the image presented during the encoding phase and not the
reproduction phase (SI Appendix , Fig. S10). Finally, we tested
the effect of increasing the delay time (from 1,000 to 2,000 ms)
and found that it did not produce any signiﬁcant differences in
the pattern of biases when compared with the original results (SI
Appendix, Fig. S9), suggesting that the visuospatial information
is preserved throughout the delay phase.
Discussion
Summary of the Results. We developed an experimental paradigm
that provides direct estimates of the geometry of visuospatial
representations. We achieved this by adopting a spatial mem-
ory task (remembering the location of a point in an image)
and incorporating it into transmission chains. Using this itera-
tive paradigm, we show that visuospatial distortions are far more
intricate and complex than previously suggested (Fig. 1). The
traditional view formulated in terms of the CAM holds that
perceptual biases are due to an attraction toward prototypical
landmarks in a scene. This view typically assumes ﬁxed preci-
sion regardless of location. As a consequence, it predicts lower
discrimination near landmarks (Fig. 1D). We tested this predic-
tion empirically and found the opposite (Fig. 3). These results are
consistent with a variable precision theory—namely, that biases
are due to reduced perceptual noise near landmarks. We formal-
ized these two interpretations in terms of Bayesian models and
found that although both predict the biases and chain dynamics
well, only the efﬁcient encoding model (which is the Bayesian
incarnation of the variable precision view) accurately predicted
the discrimination results. We also show that the efﬁcient encod-
ing model, unlike an alternative non-Bayesian variable precision
model, has the added beneﬁt of predicting the consistent percep-
tual biases that are clearly present in the data and that have been
reported in past work as well (11, 25). Furthermore, by manipu-
lating the images shown during the encoding and reproduction
phases of the experiment, we demonstrate that biases emerge
during the encoding phase rather than during the delay or repro-
duction phase. We also show an interaction between the visual
complexity of landmarks and encoding time: Shorter encoding
times result in simpliﬁed internal representations (SI Appendix,
Figs. S9 and S11). Both results are aligned with key predictions of
the efﬁcient encoding model, namely that biases emerge during
the initial process of encoding spatial locations with respect to
the image, rather than during memory retention or reproduction.
CAM. Previous work explains distortions as a consequence of
being drawn to perceptual attractors. In this tradition, broadlyreferred to as the CAM, two distinct approaches have been taken
to characterizing these attractors. The ﬁrst approach asserts that
perceptual attractors (or “prototypes”) are located at object
centers (12, 31, 32). Object centers have typically been opera-
tionalized as the centers of mass of handcrafted semantic seg-
mentations of images. We found that centers of mass were poor
predictors of the priors revealed by the transmission chains, with
an average correlation to the priors of r= 0.22 across all our
primary images (SI Appendix, Fig. S15), as well as a representa-
tive sample of images used in prior work (12, 31) (SI Appendix,
Fig. S16). In the second approach, prototypes are estimated
using a descriptive model that asserts that each reconstruction
Rfrom memory linearly interpolates between the stimulus S
and a prototype P(Eq.1andMaterials and Methods). Previ-
ous work has typically estimated prototype locations by relying
on a small number of experimentally observed reconstructions
(11–13). This approach provides good pointwise approximations
to the end result we measure in our paradigm for simple shape
images (14) (SI Appendix, Fig. S12). However, in the case of
natural scenes, where the number of modes is large and hard
to estimate, this approach is prone to overﬁtting and produces
mostly crude pointwise approximations of the distributions (SI
Appendix, Fig. S13). Finally, a bootstrapping analysis indicates
that using the CAM ﬁt to the data in the ﬁrst iteration of the
chains cannot produce estimates of the modes in the prior that
are as reliable as those obtained using serial reproduction, even
when equating the amount of data used by both methods in
the comparison (SI Appendix, Fig. S14). These results demon-
strate the practical advantages of our approach over estimation
procedures that rely on parametric model ﬁtting.
Are Visuospatial Representations Low Level? It is natural to ask
if our results can be explained away using low-level features of
the images. If internal representations are indeed more than a
simple function of low-level features, we might expect to see
biases anchored around regions that are physically absent and
only implied by contextual information. We tested this prediction
by repeating the transmission chain experiments using images
possessing illusory contours (SI Appendix, Fig. S17). Illusory con-
tours included a gray square with a smooth gradient that erased
its upper right-hand corner entirely, as well as an image of a
face in which a gradient erased its right half, with the other
half implied by symmetry and context. Transmission chain results
revealed biases concentrated around the illusory regions: a pat-
tern around the upper right-hand illusory corner of the square
that is largely identical to the pattern we observed with the orig-
inal image, as well as biases centered over the illusory eye in the
face image (SI Appendix, Fig. S17).
We also completed an additional manipulation in which we
used human segmentation data of the images and replaced the
entire textured images with uniform gray-scale regions corre-
sponding to the segmented visual objects (SI Appendix, Fig. S15).
Despite the removal of all of the ﬁne structure, we found that
the resulting KDEs are among the most predictive of the orig-
inal ﬁndings, suggesting that semantic information rather than
low-level textural information is responsible for a signiﬁcant por-
tion of the variance, with an average correlation across all images
for which the semantic segmentations were available of r= 0.57.
Finally, these experimental ﬁndings are in line with the results of
additional supporting analyses (SI Appendix, Fig. S17) indicating
that the presence of modes in the KDEs is not strictly a function
of low-level information in the images, such as corners and edges
extracted using classic image feature detectors (33).
Attention. We propose the efﬁcient encoding theory as a
Bayesian model that explains visuospatial distortions in terms
of systematic variations in encoding precision. There are sev-
eral physiological and neural processes that may support
6 of 11jPNAS
https://doi.org/10.1073/pnas.2012938118Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsDownloaded by guest on March 31, 2021
PSYCHOLOGICAL
AND
COGNITIVE SCIENCESthis process. For example, it is natural to speculate about
whether precision and spatial memory are mediated by atten-
tion in our task, including overt attention in the form of eye
movements (34).
To address this, we ran several controlled experiments in
which we combined shorter encoding times (300 ms) with for-
ward and backward noise masking. While shortening the encod-
ing time caused notable simpliﬁcations in the structure of the
priors, which is consistent with the view that biases are due to
variations in encoding precision (both spatial and temporal), the
presence of forward and backward masking had little to no effect
(SI Appendix, Fig. S10). However, another possibility is that spa-
tial memory priors reﬂect spontaneous patterns of free ﬁxations
over image regions and that these regions become spatial mem-
ory landmarks by virtue of being encoded with higher precision
following sustained foveation. However, we found that patterns
of free ﬁxations were poor predictors of our original results (SI
Appendix, Fig. S15).
Next, we tested an additional option: It is well known that
overt attention can vary substantially according to the visual
task (34) and that although free ﬁxations might not be predic-
tive of visuospatial memory priors, perhaps that ﬁxation patterns
produced by participants engaged in a different task might. In
order to address this question, we repeated our experiments
using images for which ﬁxation patterns were available not just
for free ﬁxation but for cued object search and saliency search
tasks as well (35). Although we found the ﬁxation maps to be
highly intercorrelated (SI Appendix, Fig. S18), none provided
good predictions of the spatial memory priors obtained using our
paradigm (average r<0.2 in all cases, even with optimal smooth-
ing and correction for attenuation) (SI Appendix, Fig. S19). In
addition, we found that our KDE results were also not highly cor-
related with explicit measures of image regions obtained using a
recent behavioral patch rating procedure known to be predic-
tive of overt attention (36, 37) (SI Appendix, SI Text and Fig.
S19 have details). These results suggest that overt attention only
explains part of the variability in spatial memory priors, although
we cannot completely rule out that unique eye movement pat-
terns speciﬁc to our task could be mediating precision and bias,
despite the fact that our noise masking experiments suggest oth-
erwise. Further work is required to understand how attention is
involved and whether additional mechanisms mediate how spa-
tial memory representations are encoded, such as explicit verbal
strategies (i.e., verbal descriptions of image regions to estimate
locations).
Modeling Assumptions. Our experimental method is nonparamet-
ric in that it does not rely on model ﬁtting. However, our
interpretation of serial reproduction does rely on a number of
experimentally veriﬁable assumptions. We assume that partic-
ipants possess similar perceptual priors and that they perform
the experiment by relying solely on the point location presented
to them in a given trial (a Markovian assumption). These two
assumptions are traditionally veriﬁed in experiments using trans-
mission chains by way of a strictly within-participant design, in
which each chain contains data from only a single participant (20,
38, 39) (SI Appendix, Fig. S1). We show the results of this within-
participant design for one of our shape images and a natural
image in SI Appendix, Figs. S20 and S21. The results are simi-
lar to the original ﬁndings, although the original results are less
noisy, in line with previous work studying the effects of collective
behavior on perception and decision making (40). Therefore, we
opted to present the results of the fully between-subject design
as our main ﬁndings. However, it is possible that individual dif-
ferences exist with respect to the relative strength of different
landmarks within a given image and that if this is true, the
between-subject design we adopted cannot reveal this. We illus-
trate the results of a fully within-participant serial reproductiondesign, which can be used to detect individual differences with
more data from each participant. However, further work will be
required to fully characterize the role of individual differences.
We further tested the Markovian assumption by adding uni-
form dummy trials in between experimental trials in all of the
chains, where the image was shown with a point in a ran-
dom location rather than the location produced by the previous
participant in the chain. Had participants relied on informa-
tion carried over from previous trials, this manipulation would
have produced a signiﬁcant effect. However, we found that this
manipulation had only a minor and nonsigniﬁcant effect on the
results, supporting the validity of the Markovian assumption (SI
Appendix, Fig. S9). Note that we used 20 iterations to estimate
the prior based on several metrics (SI Appendix, Figs. S5 and
S6), which reveal that convergence of the KDEs occurs by 20
iterations and that adding iterations to the chains did not alter
the estimated distributions substantially (SI Appendix, Fig. S22).
However, it is also visually apparent that there is some vari-
ation between images, so it is possible that results could be
improved with additional iterations, although that would also
come at the cost of completing longer and more data-intensive
experiments.
Finally, while our method reveals more intricate structure
than previous methods (Fig. 1), it is conceivable that even
more reﬁned details could be extracted either with more data
or with more sophisticated data aggregation methods, such as
averaging data over multiple participants before transmission
to the next participant in a chain (41). In addition, we make
a simplifying assumption in our modeling by not considering
the possibility that additional reproduction noise may be con-
tributing to the biases, but modeling reproduction noise would
not change the qualitative nature of the relation between dis-
crimination accuracy and biases (ref. 20 has a simulation that
takes production noise into account in an auditory reproduction
task).
Bayesian Inference and the Efﬁcient Encoding Model. Our empirical
ﬁndings are consistent with a variable precision interpretation of
visuospatial biases, which predicts that chaining responses in the
spatial memory task will result in a shift toward high-precision
areas that act as absorbing states. According to this view, conver-
gence in the chain is due to skewed perceptual noise toward the
landmarks. However, the simplest form of the variable precision
account might be to view the iterative process as an unbiased
random walk, where step size decreases with lower perceptual
noise, without perceptual biases (in other words, individual point
reconstructions will not necessarily consistently point toward a
nearby landmark). We favor a more complex version of the
variable precision account that innovates on a recent Bayesian
formulation of variable precision based on efﬁcient encoding (7).
The differences between these models are illustrated in Fig. 2.
We see both empirical and theoretical arguments in support of
a Bayesian interpretation and the efﬁcient encoding model in
particular.
First, unlike a simple variable precision account, the efﬁcient
encoding model predicts consistent perceptual biases. These
biases correspond to the well-documented ﬁnding (11, 25) that
people tend to produce responses that are consistently oriented
toward the nearest landmark. We conﬁrm this effect in our data
(SI Appendix, Fig. S3 CandG) and also show that the efﬁcient
encoding model captures this effect (SI Appendix, Fig. S3 C–H ),
unlike a simple variable precision model, which completely fails
to do so (SI Appendix, Fig. S3 B–H ).
Second, there is a large body of work on spatial memory
that explains systematic biases in terms of the CAM. This work
typically uses Bayesian inference to describe spatial memory
(11, 25), so it is natural that our modeling approach should
adopt the same formalisms. In addition, the Bayesian approach
Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsPNASj7 of 11
https://doi.org/10.1073/pnas.2012938118Downloaded by guest on March 31, 2021
provides a uniﬁed account for describing multiple perceptual
tasks and speciﬁes clear and testable predictions regarding the
precise mathematical relations between them, such as the rela-
tion between the magnitude of the biases and discrimination
accuracy.
Third, earlier non-Bayesian incarnations of the CAM describe
perceptual attractors in terms of discrete prototypes rather than
continuous distributions. However, our data clearly reveal modes
that vary systematically in terms of their density, elongation, ori-
entation, and shape, all characteristics that are hard to describe
using a model that can only produce discrete pointwise cat-
egorical estimates (Fig. 1D). This is especially evident in the
case of natural images, where ﬁtting a large number of dis-
crete modes provides a signiﬁcantly poorer approximation of the
biases compared with a baseline (P <0.01 for all images) (SI
Appendix, Fig. S13). By contrast, the Bayesian formulation over-
comes this problem by describing perceptual representations in
terms of continuous distributions rather than discrete pointwise
entities.
Fourth, unlike a non-Bayesian variable precision account, the
efﬁcient encoding model provides a useful theoretical motivation
for why encoding precision is higher in some visual regions and
not others. Because it explains biases in terms of an optimal allo-
cation of encoding resources, it makes a number of theoretical
commitments that are both testable and useful for understanding
perceptual biases. First, it predicts that biases emerge dur-
ing encoding, rather than delay or reproduction. Therefore, it
predicts that manipulating encoding resources directly should
interact with the structure of the biases. In fact, when we manip-
ulated encoding time, we observed a structural simpliﬁcation in
the complexity of the results. An unconstrained variable pre-
cision model does not provide any theoretical motivation for
why decreasing encoding time would generate anything beyond
increased additive noise, let alone a qualitative shift toward a
simpliﬁed representation (SI Appendix, Fig. S11B). We observed
a similar simpliﬁcation using a spatial manipulation of visual
complexity (SI Appendix, Fig. S11A).
Finally, in addition to predicting consistent perceptual biases,
the Bayesian models provide a good ﬁt to the dynamics of the
serial reproduction chains. Fig. 2A andSI Appendix, Fig. S8
provide the results of additional self-consistency tests of the efﬁ-
cient and ﬁxed encoding models in terms of how closely they
approximate the complex chain dynamics of the serial repro-
duction data for one of our images. We show that using the
data from the last iteration of the serial reproduction experi-
ments can predict the rate of convergence and the dynamics of
all previous iterations (after ﬁtting the noise-magnitude parame-
ter to the data) and in the case of efﬁcient encoding, predicts the
positive correlations between discrimination results and priors
estimated from the serial reproduction experiment (Fig. 3). This
supports the idea that in addition to predicting perceptual biases,
the efﬁcient encoding model produces good approximations
to the perceptual distortions and discrimination accuracy mea-
sures, as well as the dynamics of the transmission chain results
in our task.
However, as with any Bayesian model that invokes a “prior”
and a “likelihood,” there comes a need to make a number of
interpretative commitments that are worth discussing here. First,
it is clear that any theory of spatial memory should somehow cap-
ture and quantify the concept of a “landmark” because it is a
key concept in spatial memory. One could describe landmarks
as discrete pointwise entities (along the lines of the CAM in
its descriptive non-Bayesian form), but our empirical data show
that using a ﬁxed number of discrete pointwise estimates is not
sufﬁcient to capture the behavioral results, which reveal graded
continuous patterns (with varying elongations, orientations, and
aspect ratios). We provide some quantitative evidence for this
(SI Appendix, Fig. S13), but it is also visually apparent that theretends to be many landmarks particularly in complex scenes and
that they are not discrete. Therefore, it is natural to quantify the
concept of a landmark as some continuous function that deter-
mines the degree of landmarkness of visual regions in a scene.
The prior accomplishes this since it is a continuous function that
assigns a higher value to visual regions that are more landmark
like (because landmarks make it easier to encode nearby loca-
tions). This function is also a probability density function, which
lends it an additional interpretation in the Bayesian formulation:
It is a belief state about probable point locations in a visual scene.
However, even without adopting a Bayesian interpretation, it is
clear that some kind of continuous function p(s)is needed in
order to specify the degree of landmarkness of visual regions.
Another component of almost any theory of perception is
some way to encapsulate the notion of perceptual noise. In other
words, how accurately or noisily is a given point location per-
ceived by an observer? Assuming that the noise is ﬁxed regardless
of location results in predictions that are incompatible with our
discrimination accuracy data (this is the ﬁxed precision model,
illustrated in Fig. 2 AandD). As a result, we need to come
up with some function that captures the idea that perceptual
noise varies systematically from location to location in an image
(variable precision). In Bayesian terms, the likelihood function
is ideally suited to play this role. Again, even without adopting a
Bayesian view of this idea, the concept of a continuous function
that captures the degree to which perceptual noise inﬂuences
spatial memory remains useful. The Bayesian account only speci-
ﬁes how the prior and likelihood are combined mathematically to
form the posterior during inference. In this work, we assume that
reproductions are a sample from the posterior, although previous
work discusses alternatives to this, such as maximum a posteri-
ori estimation, which models reproduction as the mode of the
posterior rather than a sample (18). According to our model, the
reproduction distribution is the net result of the encoding (deter-
mined by the likelihood) and decoding process (determined by
the posterior). The chaining of these two processes results in
the observed reproduction distribution p(RjS)(SI Appendix,
Fig. S2).
Nevertheless, adopting the Bayesian interpretation comes at a
cost: It is signiﬁcantly more complex mathematically, although
it does not introduce any additional degrees of freedom over
a naive model where variable precision is given by an arbitrary
noise term. Speciﬁcally, both the Bayesian and non-Bayesian for-
mulations of variable precision depend solely on a scalar function
deﬁned over the entire space [ (s)in the case of a variable pre-
cision model and p(s)in our case]. All of the predictions made
by our model (e.g., the discrimination maps and chain dynamics)
are determined only from this scalar function.
Conclusion. Exploring spatial memory biases using serial repro-
duction demonstrates that the study of shared perceptual rep-
resentations can be approached by recasting experimentation as
algorithm design and through the lens of information transmis-
sion inside carefully curated social networks. More broadly, this
work demonstrates the beneﬁt of bringing innovative experimen-
tal and psychophysical methods and computational statistics to
bear on our understanding of otherwise hidden internal repre-
sentations. The advantage of this approach lies in fully charac-
terizing the structure of internal representations, revealing rich,
complex, and ecologically valid perceptual spaces. This detailed
understanding can spur theoretical insights with respect to how
perceptual systems encode and process sensory information.
Materials and Methods
Participants. Participants were recruited online using Amazon Mechanical
Turk. The experiments were approved by the Committee for Protection of
Human Subjects at the University of California, Berkeley and the Institu-
tional Review Board at Princeton University. We obtained informed consent
8 of 11jPNAS
https://doi.org/10.1073/pnas.2012938118Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsDownloaded by guest on March 31, 2021
PSYCHOLOGICAL
AND
COGNITIVE SCIENCESfrom all volunteers. SI Appendix, Fig. S23 presents the exact number of par-
ticipants in each of the 85 experiments. The overall number of participants
in all experiments was 9,202.
Stimuli. The images used in the transmission chain experiments were gray-
scale images of a few simple shapes (circle, triangle, square, and pentagon),
as well as gray-scale images of natural scenes. A detailed description of the
images is provided in SI Appendix .SI Appendix, Fig. S23 shows the list of
image ﬁle names for each of the experiments. All stimuli for the experiments
are available in our open science repository (42).
Procedure. Transmission chain memory experiments were programmed
using the Dallinger platform for laboratory automation for the behavioral
and social sciences (43). Reproducible code for the Dallinger experiments is
provided in the open science repository. Patch ratings experiments and dis-
crimination experiments were programmed using the Amazon Mechanical
Turk application programming interface (API).
Transmission chain memory experiments. Participants were shown an
image with a point overlaid on it for 1,000 ms (Fig. 1A). The initial point
locations were sampled from a uniform distribution. Participants were asked
to reproduce the position of the point as accurately as possible follow-
ing a 1,000-ms delay, when the image reappeared on the screen without
the point. To prevent participants from resorting to marking the absolute
positions of the points on the screens during the task, the displays were
shifted by a random offset on the screen during the stimulus phase and
the probe (Fig. 1A). The response was then sent to another participant who
performed the same task. A total of 20 iterations of this telephone game
procedure were completed for each chain. We terminated each experiment
after approximately 12 h. The number of total chains varied somewhat
between experiments (mean 465, range 250 to 577 chains) (SI Appendix, Fig.
S23). A typical experiment included 105 trials, and the average time needed
to complete the task was about 12 to 14 min. SI Appendix, Fig. S23 presents
the number of participants in each experiment. SI Appendix has additional
details.
Visual discrimination experiments. Participants saw an image presented for
1,000 ms with a red point overlaid on it (Fig. 2B). Following a 1,000-ms delay
with a blank screen, the image reappeared with the point either in the same
exact location relative to the image or in a shifted position (both the dura-
tions of the display and the gray-scale images were identical to those in the
transmission chain experiments). In the shifted condition, the shifted point
was offset by a six-pixel radial distance from the original point location, sam-
pled uniformly along the circumference of the circle deﬁned by the six-pixel
radius centered at the original point location. In all cases, the overall display
(the image and point) was shifted by a random offset in the second pre-
sentation to prevent participants from using absolute positions within the
display. The second display remained for 1,000 ms on the screen and was fol-
lowed by a 2AFC (“red point same” or “red point shifted”). SI Appendix has
additional details, including the 2AFC data analysis. We obtained responses
from a total of 20 participants for each grid point and for each condition
(same or shifted).
Nonparametric KDE. For each chain, we used the data for all iterations.
We computed the empirical mean and covariance matrix [ i=mean j(Rij),
i=Cov j(Rij)], where Rijis the response in chain iand iteration j. To estimate
the typical kernel width, we computed the square root of the eigenvalues
of this matrix. These values ranged between 0:015 and 0:025 for shapes and
between 0:020 and 0:040 for images (these values are reported in units of
fraction relative to an image size of 1). Since the covariance estimate is based
on a small number of points, we computed a regularized covariance matrix
0
i=  i+2Iwherewas set to 0.015 for shapes and 0.020 for natural
images and Iis the identity matrix (values were chosen based on the esti-
mates of the unregularized matrices above). For each chain, we computed
a Gaussian distribution: pi(s)=1q
(2)2j0
ijexp(1
2(si)T01
i(si)).
Next, we computed the KDE as the normalized sum over all of the pidis-
tributions. If Nis the total number of chains, the nonparametric KDE for a
given image becomes P(s)=1
NP
ipi(s). Results of this procedure are shown
for the shape image results in Fig. 1C .
Parametric KDE. KDEs were computed using the data from the last iteration
of the chains. For each point, we computed a Gaussian kernel centered at
the point with a diagonal covariance matrix. We set the kernel width to a
conservative value of 0.025 for shapes and 0.040 for natural images. These
values were chosen based on the ranges of the estimates obtained from the
unregularized nonparametric kernels. The ﬁnal KDE was calculated by sum-ming all of the Gaussian kernels and normalizing. Results of this procedure
were used for all statistical analyses.
Comparing Nonparametric Estimates with the CAM. The CAM (13) describes
the remembered position for a response vector ias a weighted average of
the actual location at which the point was presented ( Si) and the weighted
sum of the Mspatial attractor locations using the following equations:
Ri=wSi+(1w)MX
k=1vikPk
where
vik=eckSiPkk
PM
k0=1eckSiPk0k
and Riand Siare vectors in R2containing the two coordinates for the
ith initial seed point (in iteration 0) and the corresponding ith response
point in iteration 1, respectively. The Pkterms are vectors corresponding
to the prototype coordinates estimated by the model. The weight wcorre-
sponds to the relative strength of the ﬁne-grained memory representation
(as opposed to the strength of a prototype in the prior). The larger wis, the
closer the memory reconstruction approximates a perfectly unbiased spatial
location. vikcaptures the relative pull of each of the locations Pkfor each
point i. Finally, ccorresponds to a “sensitivity” parameter that models the
sharpness of the prototype boundaries.
In the case of simple shapes, we ﬁt the model with only four prototypes
using all of the data from the ﬁrst iteration of our experiment for the ﬁt-
ting process (using the same number of parameters used in ref. 13). Our
results are consistent with previous estimates (SI Appendix, Fig. S12) (13, 14).
However, when ﬁtting natural images, it is hard to estimate the number of
modes, and the results are poor predictors of the priors estimated via serial
reproduction (SI Appendix, Fig. S13). We ﬁt the CAM using 5, 10, and 20
prototype locations (P k) terms for each of the natural images. We obtained
the best estimates for the locations of the Pkterms as well as the other
parameters of the CAM using all initial point locations and the positions in
the ﬁrst iteration for each of the images. We optimized the CAM parame-
ters using Matlab’s Optimization Toolbox and the nonlinear programming
solver fmincon. The results of this comparison are presented in SI Appendix,
Fig. S13. Finally, we also completed an analysis comparing the internal reli-
ability of the transmission chain results with the predictions of the CAM for
one of our images, which shows that using the CAM ﬁt to the data in the
ﬁrst iteration of the chains cannot produce estimates of the modes in the
prior that are as reliable as those obtained using serial reproduction, even
when we equated the amount of data required (SI Appendix, Fig. S14).
Bayesian model of perceptual biases. In visuospatial memory, a point loca-
tion Sis encoded into a remembered location T. The neural implementation
of these representations in the brain can take many forms (44). However,
we are interested only in describing these representations in terms of the
distributions that are implied by them (2, 3, 7, 30, 45).
Bayesian models imply that regardless of the sensory encoding process, a
reproduction Ris based on inferring the original location Sfrom T, following
Bayesian inference:
p(R=rjT=t)=p(S=rjT=t)/p(T=tjS=r)p(S=r): [2]
According to this view, perceptual distortions correspond to systematic (and
normative) deviations between Rand S, where Rfollows the distribution of
the posterior p(SjT).
The degrees of freedom of this approach are 1) the likelihood p(TjS),
which describes the noisy observation of the stimulus location; 2) the prior
p(S), which describes beliefs that the participant possesses about the dis-
tribution of locations given an image; and 3) an assumption about how a
reproduced point location is obtained from the posterior distribution. Here,
we assume that a reproduction is a sample from the posterior, although
other assumptions are possible (18).
Bayesian model of serial reproduction and discrimination experiments. In
our serial reproduction experiment, the reconstruction becomes the basis of
another iteration, and this process is repeated. We assume that participants
use only the current point location as a basis for their perceptual decision
(the Markovian assumption) (Discussion). Formally, the transmission chain
can be described in terms of a sequence of random variables (SI Appendix,
Fig. S2A):
:::!St!Tt!Rt=St+1!:::, [3]
where St,Tt, and Rtare the veridical location, sensory encoded representa-
tion, and the inferred location at step t, respectively.
Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsPNASj9 of 11
https://doi.org/10.1073/pnas.2012938118Downloaded by guest on March 31, 2021
Formally, our modeling approach assumes the following.
Given an image, participants have a shared prior over point locations,
represented as a probability distribution over the image: p(S). We assume
that locations are encoded with respect to a sensory parsing of the image
content. As a result of this process, a prior is generated that reﬂects the
participant’s belief state about probable locations over the image.
There is a likelihood function p(TjS) that varies in its form between
the ﬁxed and efﬁcient encoding models (SI Appendix has details). The
likelihood carries information about the shape and magnitude of the
noise. Regardless of its shape, we assume it is available for sensory
inference.
Participants infer point locations by computing the posterior (Eq. 2):
p(R=rjT=t)=p(S=rjT=t)
=p(T=tjS=r)p(S=r)R
p(T=tjS=r0)p(S=r0)dr0: [4]
A participant’s response (the reproduction from memory of a point loca-
tion) is a sample from the posterior (refs. 18 and 30 have other choices,
such as choosing the mean of the posterior).
From this, we can derive
p(Sn+1=rjSn=s)=p(R n=rjSn=s)=
Z
p(R n=rjTn=t)p(T n=tjSn=s)dt: [5]
Given an initial distribution p(S0), the steps of the transmission chain
experiment are fully determined by recursively integrating the following
(this is demonstrated in Fig. 2A and SI Appendix, Fig. S2B):
p(Sn+1=r)=Z
p(Sn+1=rjSn=s)p(S n=s)ds: [6]
This formulation provides an explicit prediction with respect to dis-
crimination accuracy (7). We can write the perceptual sensitivity (d0) of a
discrimination experiment with respect to two point locations S1and S2in
the following way:
d(S1,S2)=~(S 1)~(S 2)p
(~(S1)2+ ~(S2)2)=2, [7]
where ~and~2are the mean and variance of R, which can be computed
from the formula for the posterior.
Serial reproduction converges to the prior. Here, we prove that under the
assumptions stated above, the prior p(S) is the stationary distribution of the
Markov chain in Eq. 3. We denote the prior as (s)=P(S=s). Using Eqs. 2
and5, it follows that
p(Sn+1=rjSn=s)=Zp(T n=tjSn=r)(r)R
p(T n=tjSn=r0)(r0)dr0p(T n=tjSn=s)dt:[8]
We will now show that (s) is the stationary distribution of the chain:R
p(Sn+1=rjSn=s)(s)ds=(r).This follows from a direct computation:
Z
p(Sn+1=rjSn=s)(s)ds=
ZZp(T n=tjSn=r)(r)R
p(T n=tjSn=r0)(r0)dr0p(T n=tjSn=s)dt
(s)ds=
Zp(T n=tjSn=r)(r)R
p(T n=tjSn=r0)(r0)dr0dtZ
p(T n=tjSn=s)(s)ds=
Z
p(T n=tjSn=r)(r)dt=(r): [9]
The ﬁrst equality holds true by substituting the formula above for
p(Sn+1jSn). The second equality is due to a change in the order of inte-
gration. The last equality holds true becauseR
p(T n=tjSn=r)dt=1. Note
that in past work (18, 19), Sis observed by both the participant and the
experimenter, whereas in our case, Tis observed by the participant and S
is observed by the experimenter. In the former case, the chain converges
to a stationary distribution (t) equal to the prior predictive distribution:
(t)=R
p(T=tjS=s)p(S=s)ds, whereas in our case, it converges to the
prior(s)=p(S=s).
Numerical simulations. We computed simulations of the dynamics of the
transmission chain experiments as well as the discrimination experiment
results analytically (Figs. 2D and 3A). We provide two-dimensional (2D) illus-
trations of the efﬁcient encoding and ﬁxed encoding models in Fig. 2 Dand
E, showing the opposing predictions of the two models regarding discrimi-
nation accuracy. We assumed that the prior is given as a discrete distribution
p(S=xi) on grid points xi. Note that there are Ngrid points in the one-
dimensional (1D) case and N2grid points in two dimensions (N is the number
of grid points per dimension). We also assume that the likelihood is given
as a matrix p(T=xjjS=xi). This matrix represents the probability associated
with a noisy observation xjoriginating from a veridical location xi. Note
that in the 2D case, this matrix will be of size N4. We then use Eqs. 4–6
computed numerically on the grid points. We also use Eq. 7for comput-
ing the predicted discrimination accuracy (d0). In the 2D case, we used the
following approximation: We projected the 2D distributions to the 1D line
connecting the two points. In this way, we can avoid the more complex
analysis associated with 2D signal detection theory (46). Additional details
about the simulation of the discrimination experiments are provided in SI
Appendix, Discrimination simulations. Code for the 1D and 2D simulations
and d0computation is given as part of the open science folder associated
with this paper (https://osf.io/cza25/).
Data Availability. All data and materials reported here are available
on the Open Science Framework (OSF), in the repository named Serial
Reproduction Reveals the Geometry of Visuospatial Representations, at
https://osf.io/cza25/ (46).
ACKNOWLEDGMENTS. This work was funded in part by NSF Grant SPRF-
IBSS-1408652 (to J.W.S. and T.L.G.) and the Defense Advanced Research
Projects Agency (DARPA) Cooperative Agreement D17AC00004 (to J.W.S.
and T.L.G.). The contents of this paper do not necessarily reﬂect the posi-
tion or the policy of the government, and no ofﬁcial endorsement should
be inferred. We thank Tom Morgan for his help with the Dallinger platform
and Raja Marjieh for his input.
1. D. C. Knill, W. Richards, Perception as Bayesian Inference (Cambridge University Press,
1996).
2. Y. Weiss, E. P. Simoncelli, E. H. Adelson, Motion illusions as optimal percepts. Nat.
Neurosci. 5, 598–604 (2002).
3. A. A. Stocker, E. P. Simoncelli, Noise characteristics and prior expectations in human
visual speed perception. Nat. Neurosci. 9, 578–585 (2006).
4. D. Fougnie, J. W. Suchow, G. A. Alvarez, Variability in the quality of visual working
memory. Nat. Commun. 3, 1229 (2012).
5. P. M. Bays, R. F. Catalao, M. Husain, The precision of visual working memory is set by
allocation of a shared resource. J. Vis. 9, 7 (2009).
6. W. Zhang, S. J. Luck, Discrete ﬁxed-resolution representations in visual working
memory. Nature 453, 233–235 (2008).
7. X. X. Wei, A. A. Stocker, A Bayesian observer model constrained by efﬁ-
cient coding can explain ‘anti-Bayesian’ percepts. Nat. Neurosci. 18, 1509
(2015).
8. N. S. Newcombe, J. Huttenlocher, Making Space: The Development of Spatial
Representation and Reasoning (MIT Press, 2003).
9. R. Van Den Berg, H. Shin, W. C. Chou, R. George, W. J. Ma, Variability in encoding pre-
cision accounts for visual short-term memory limitations. Proc. Natl. Acad. Sci. U.S.A.
109, 8780–8785 (2012).
10. T. Madl, K. Chen, D. Montaldi, R. Trappl, Computational cognitive models of spatial
memory in navigation space: A review. Neural Network. 65, 18–43 (2015).11. J. Huttenlocher, L. V. Hedges, S. Duncan, Categories and particulars: Pro-
totype effects in estimating spatial location. Psychol. Rev. 98, 352–376
(1991).
12. M. P. Holden, N. S. Newcombe, T. F. Shipley, Location memory in the real
world: Category adjustment effects in 3-dimensional space. Cognition 128, 45–55
(2013).
13. D. H. Wedell, S. Fitting, G. L. Allen (2007) Shape effects on memory for location.
Psychol. Bull. Rev. 14, 681–686.
14. T. Langlois, N. Jacoby, J. W. Suchow, T. L. Grifﬁths, “Uncovering visual priors in
spatial memory using serial reproduction” in Proceedings of the 39th Annual
Meeting of the Cognitive Science Society , G. Gunzelmann, A. Howes, T. Tenbrink,
E. Davelaar, Eds. (Cognitive Science Society, London, UK, 2017), pp. 712–717.
(2017).
15. B. Tversky, D. J. Schiano, Perceptual and conceptual factors in distortions in memory
for graphs and maps. J. Exp. Psychol. Gen. 118, 387 (1989).
16. C. Firestone, B. J. Scholl, “Please tap the shape, anywhere you like” shape skeletons
in human vision revealed by an exceedingly simple measure. Psychol. Sci. 25, 377–386
(2014).
17. F. C. Bartlett, Remembering: A Study in Experimental and Social Psychology
(Cambridge University Press, 1932).
18. J. Xu, T. L. Grifﬁths, A rational analysis of the effects of memory biases on serial
reproduction. Cognit. Psychol. 60, 107–126 (2010).
10 of 11jPNAS
https://doi.org/10.1073/pnas.2012938118Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsDownloaded by guest on March 31, 2021
PSYCHOLOGICAL
AND
COGNITIVE SCIENCES19. T. L. Grifﬁths, M. L. Kalish, “A Bayesian view of language evolution by iterated learn-
ing” in Proceedings of the 27th Annual Meeting of the Cognitive Science Society ,
B. G. Bara, L. Barsalou, M. Bucciarelli, Eds. (Cognitive Science Society, Stresa, Italy,
2005), pp. 827–832.
20. N. Jacoby, J. H. McDermott, Integer ratio priors on musical rhythm revealed cross-
culturally by iterated reproduction. Curr. Biol. 27, 359–370 (2017).
21. N. H. Feldman, T. L. Grifﬁths, J. L. Morgan, The inﬂuence of categories on perception:
Explaining the perceptual magnet effect as optimal statistical inference. Psychol. Rev.
116, 752–782 (2009).
22. R. L. Goldstone, Y. Lippa, R. M. Shiffrin, Altering object representations through
category learning. Cognition 78, 27–43 (2001).
23. J. Davidoff, I. Davies, D. Roberson, Colour categories in a stone-age tribe. Nature 398,
203–204 (1999).
24. B. H. Repp, “Categorical perception: Issues, methods, ﬁndings” in Speech and
Language, N. Lass, Ed. (Elsevier, 1984), vol. 10, pp. 243–335.
25. J. Huttenlocher, L. V. Hedges, J. L. Vevea, Why do categories affect stimulus
judgment? J. Exp. Psychol. Gen. 129, 220 (2000).
26. P. H. Engebretson, “Category effects on the discriminability of spatial location,”
Unpublished doctoral dissertation, University of Chicago, Chicago, IL (1995).
27. D. M. Green, J. A. Swets, Signal Detection Theory and Psychophysics (Wiley, New York,
1966), vol. 1.
28. W. Penﬁeld, E. Boldrey, Somatic motor and sensory representation in the cerebral
cortex of man as studied by electrical stimulation. Brain 60, 389–443 (1937).
29. R. B. Tootell, E. Switkes, M. S. Silverman, S. L. Hamilton, Functional anatomy of
macaque striate cortex. II. retinotopic organization. J. Neurosci. 8, 1531–1568 (1988).
30. X. X. Wei, A. A. Stocker, Lawful relation between perceptual bias and discriminability.
Proc. Natl. Acad. Sci. U.S.A. 114, 10244–10249 (2017).
31. M. Holden, T. Shipley, N. Newcombe, Memory for location is inﬂuenced by part-based
segmentation of space. J. Vis. 7, 914 (2007).
32. M. P. Holden, N. S. Newcombe, I. Resnick, T. F. Shipley, Seeing like a geologist:
Bayesian use of expert categories in location memory. Cognit. Sci. 40, 440–454 (2016).
33. G. Bradski, The OpenCV library (2000). https://www.drdobbs.com/open-source/the-
opencv-library/184404319. Accessed 10 March 2021.34. M. K. Eckstein, B. Guerra-Carrillo, A. T. M. Singley, S. A. Bunge, Beyond eye gaze:
What else can eyetracking reveal about cognition and cognitive development?. Dev.
Cognit. Neurosci. 25, 69–91 (2017).
35. K. Koehler, F. Guo, S. Zhang, M. P. Eckstein, What do saliency models predict? J. Vis.
14, 14 (2014).
36. J. M. Henderson, T. R. Hayes, G. Rehrig, F. Ferreira, Meaning guides attention during
real-world scene description. Sci. Rep. 8, 13504 (2018).
37. J. M. Henderson, T. R. Hayes, Meaning-based guidance of attention in scenes as
revealed by meaning maps. Nat. Hum. Behav. 1, 743–747 (2017).
38. N. Claidiere, K. Smith, S. Kirby, J. Fagot, Cultural evolution of systematically
structured behaviour in a non-human primate. Proc. Biol. Sci. 281, 20141541
(2014).
39. A. Ravignani, T. Delgado, S. Kirby, Musical evolution in the lab exhibits rhythmic
universals. Nat. Hum. Behav. 1, 0007 (2017).
40. J. Becker, D. Brackbill, D. Centola, Network dynamics of social inﬂuence
in the wisdom of crowds. Proc. Natl. Acad. Sci. U.S.A. 114, E5070–E5076
(2017).
41. P. Harrison et al., “Gibbs sampling with people” in Advances in Neural Information
Processing Systems 33, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan and H. Lin,
Eds. (Curran Associates, Inc., 2020), pp. 10659–10671.
42. T. Langlois, N. Jacoby, J. Suchow, T. Grifﬁths. Serial reproduction reveals the geom-
etry of visuospatial representations. Open Science Framework. https://osf.io/cza25/.
Deposited 10 March 2021.
43. J. Suchow et al., “Fully automated behavioral experiments on cultural transmission
through crowdsourcing,” presented as a poster at Collective Intelligence (Pitsburgh,
PA, 2019).
44. C. F. Doeller, J. A. King, N. Burgess, Parallel striatal and hippocampal systems for land-
marks and boundaries in spatial memory. Proc. Natl. Acad. Sci. U.S.A. 105, 5915–5920
(2008).
45. M. Jazayeri, M. N. Shadlen, Temporal context calibrates interval timing. Nat. Neurosci.
13, 1020 (2010).
46. Y. Kronrod, E. Coppess, N. H. Feldman, A uniﬁed account of categorical effects in
phonetic perception. Psychol. Bull. Rev. 23, 1681–1712 (2016).
Langlois et al.
Serial reproduction reveals the geometry of visuospatial representationsPNASj11 of 11
https://doi.org/10.1073/pnas.2012938118Downloaded by guest on March 31, 2021

Thomas A
Langloisa
Nori Jacobyc
Jordan W
Suchowe
and Thomas L
Griffiths