Using Natural Language to Guide Meta-Learning Agents towards Human-like Inductive Biases.

Using Natural Language to Guide Meta-Learning Agents towards
Human-like Inductive Biases
Sreejan Kumar1, Ishita Dasgupta2, Michael Hu1, Raja Marjieh1, Robert D. Hawkins1,
Nathaniel D. Daw1, Jonathan D. Cohen1, Karthik Narasimhan1, and Thomas L. Griffiths1
1Princeton University
2Deepmind
Abstract
Inductive biases are a key component of human
intelligence, allowing people to acquire, rep-
resent, and use abstract knowledge. Although
meta-learning has emerged as an approach to
endowing neural networks with inductive bi-
ases, agents trained via meta-learning can use
very different strategies compared to humans.
We show that co-training these agents on pre-
dicting human-generated natural language task
descriptions guides them toward human-like in-
ductive biases that more appropriately capture
the structure of the task distribution as humans
see it. We further show that the level of abstrac-
tion at which humans write these descriptions
influences the size of the effect. This work
provides a foundation for investigating how to
collect task descriptions at the appropriate level
of abstraction to leverage for approximating
human-like learning of structured representa-
tions in neural networks.
1 Introduction
Human learners are guided by strong inductive
biases towards abstract knowledge (Tenenbaum
et al., 2011; Griffiths et al., 2010); these biases
present one of the most salient differences between
humans and neural network-based learners (Lake
et al., 2017). One emerging approach to bestowing
human-like inductive biases on neural networks is
meta-learning (Griffiths et al., 2019; Hospedales
et al., 2020). In meta-learning paradigms, an agent
is trained not just on a single task but on a dis-
tribution of tasks, with the aim of acquiring the
underlying abstractions that these tasks have in
common. However, since neural networks are not
easily interpretable, it can be difficult to tell if the
resulting neural networks actually acquired this
abstract knowledge, or whether they have simply
learned statistical artifacts correlated with abstract
rules. Recently, Kumar et al. (2021) found that
neural agents are biased towards learning the latter.
Specifically, through the use of a task distributiongenerated from an abstract compositional grammar
and a corresponding control task distribution with
closely matched statistics, they found agents do bet-
ter in the control task distribution whereas humans
do better in the abstract task distribution, demon-
strating a difference in inductive biases between
humans and agents.
What explains such differences? One possi-
bility is that human biases toward abstract struc-
ture are related to our language abilities (Spelke,
2003; Lupyan and Bergen, 2016). Indeed, recent
work in machine learning has revealed how neural
network representations can be shaped and struc-
tured through natural language supervision (An-
dreas et al., 2018; Luketina et al., 2019; Wong et al.,
2021; Narasimhan et al., 2018; Mu et al., 2020).
In this work, we show that guiding meta-
reinforcement learning agents with natural lan-
guage descriptions not only increases performance
on abstract task distributions, but also results in
more human-like behavior: it decreases perfor-
mance on control task distributions where humans
perform poorly. Further, while much of language-
guided RL work focuses on synthetic descriptions,
we investigate different kinds of human-generated
descriptions. We collect human descriptions at
different levels of abstraction and find that guid-
ance with more abstract descriptions lead to more
human-like inductive biases in agents.
Our approach is to first extend and replicate the
results of Kumar et al. (2021). Specifically, instead
of developing an abstract task distribution using
handwritten rules as in Kumar et al. (2021), we di-
rectly project human priors into a task distribution
(see Fig 1B). We then test a meta-RL agent’s ability
to acquire this task distribution’s emergent abstract
priors by building a control task distribution using
the same approach as Kumar et al. (2021) (see Fig
1C). We replicated the double dissociation effect
seen in Kumar et al. (2021) (see Fig 1E) and then
further show that we can guide the agent towards
Underlying Board Start Step 1 Step 2 Step 3 Step 4 Step 10
Goal: Uncover all red with
 as little blue as possibleRL Task
ParadigmA
...
What should the
color of the white tile
be  given that the
board is generated
by a very simple rule?Gibbs Sampling with PeopleB
Train network to predict held-out tile. Sampling from Human Inductive Bias Sampling from Neural Network Inductive Bias C
Use network for Gibbs SamplingSample from
Human Distribution
and mask tileFill in masked
tile
Random Initialization
Human Samples
Control SamplesD ERL Task Performance
Control SampleHuman-Generated
(GSP)Machine-Generated
(Control)
Task DistributionBlue Tiles Revealed Z-scored
by Nearest Neighbor HeuristicFigure 1: Meta-RL task paradigm. (A) In the tile-revealing task, an agent sequentially reveals tiles to uncover
a picture on a 2D grid. We elicit (B) human priors and (C) control priors over the task distribution using Gibbs
sampling (Geman and Geman, 1984). (D) Samples from human and control distributions. (E) Performance of
(independent) humans and machine-learning agents on the tile-revealing task with human and control boards.
Performance is based on number of blue tiles revealed (lower is better; see Appendix for details). Error bars are
95% confidence intervals.
learning a human-like inductive bias through the
use of natural language co-supervision.
2 Methods
Tile-revealing task. We employ the tile-
revealing task paradigm developed in Kumar et al.
(2021) (see Fig. 1A). The observation is a 4×4
grid of tiles that are initially white except for one
red tile. Actions – clicking on white tiles – reveal
those tiles to be either red or blue. The episode
ends when the agent reveals all the red tiles. There
is a reward for each red tile revealed, and a penalty
for each blue tile revealed. The goal therefore is
to reveal all the red tiles while revealing as few
blue tiles as possible. One “board” with a fixed
configuration of red tiles defines a single task. A
distribution over tasks is defined by specifying a
distribution over different 4×4grids of red and
blue tiles (boards).
Eliciting human priors with Gibbs sampling.
In order to elicit human inductive biases, we use
a technique called Gibbs Sampling with People
(GSP; Harrison et al., 2020, see Fig. 1B). We ini-
tialize a random 4×4grid with red and blue tiles,
mask out a tile, and ask a human participant to
predict the color of the masked-out tile. We then
change that tile to match the human’s prediction
and present the updated grid to another participant,
masking out a different tile. This sequence of de-
cisions implements a Markov chain; the stationary
distribution of this chain is the implicit prior distri-
bution people hold over 4×4grid colorings (Har-
rison et al., 2020), There are several recognizable
abstract concepts that emerged within the resultinggrids, such as lines, squares, and continuous shapes
(see Fig. 1D).
Constructing a control distribution. We cre-
ated a control distribution, following Kumar et al.
2021, that matches the statistics of the GSP boards
but uses a different underlying generative process
(i.e. not produced by human decisions, see Fig. 1C).
Specifically, we train a fully connected neural net-
work to encode the conditional distributions of the
GSP boards: we mask out a random tile in each
board, and train the network to predict its value
given the other tiles (similar to masked language
models; (Devlin et al., 2018)). We then sampled
boards from the network’s learned conditionals
with Gibbs sampling. This is the same process
we used to generate the GSP boards, but using the
trained neural network to generate the conditional
distributions instead of human samples. We are
therefore sampling from the distribution the net-
work places over 4×4grids (combining its own
inductive bias and the data from the GSP boards).
We refer to this distribution as a “control” distribu-
tion, which is comprised of tasks that are generated
with different underlying generative processes but
share certain statistical properties.
Collecting natural-language descriptions. We
hypothesized that linguistic descriptions of the GSP
boards may help guide the agent’s inductive bias
towards more human-like abstractions. To test this
hypothesis, we collected natural-language descrip-
tions of 500 GSP boards from a naive group of
participants. There were three types of descrip-
tions collected: two that were human-generated
High Level Prompt: Be succinct.
• “A U shape”
• “A wide letter U”
Low Level Prompt: Be as detailed as possible
• “Square shaped cluster of white boxes to upper-middle.
An upside-down table shaped red box formation. ”
• “The red and white boxes represent 50 percent of the
picture. ’ , 'A large grid made up of 16 identical sized squares.
8 of which are coloured red. The red ones /f_ill the 3 row
down and the top 3 of column 1 and column 4' , 'squares in
the shape of a U'”
Synthetic Low Level Template
• “The reds are in /f_irst row and /f_irst column, /f_irst row and
fourth column, second row and /f_irst column, second row
and fourth column, third row and /f_irst column, third row
and second column, third row and third column, third row
and fourth column. ” Board StimulusFigure 2: Types of Text Descriptions Obtained for
GSP Boards We obtained three types of descriptions
for 500 of the GSP boards: high-level, low-level, and
synthetic low-level. The first two were collected directly
from humans using different types of prompts that em-
phasized succinctness and detail respectively. The third
was generated from a handmade template that verbalizes
the location of red tiles. When showing participants the
boards, we converted blue tiles to white tiles in order
to have them focus their description on the red tiles’
locations.
under different prompts and one that was synthet-
ically generated using a template (see Fig. 2 and
Appendix for exact wording of prompts). High-
level descriptions were collected from humans who
were given a prompt that encouraged succinctness
in descriptions. Low-level descriptions were col-
lected from humans who were given a prompt that
encouraged being verbose and detailed. Synthetic
low-level descriptions are not human generated and
were obtained by using a hand-written template that
verbalizes the location of all the red tiles.
Grounding agents with descriptions. We train
a commonly used RNN-based meta-reinforcement
learning agent (Wang et al., 2018; Duan et al.,
2016) using Proximal Policy Optimization (PPO;
Schulman et al. (2017)). See Fig. 3 and Appendix
for more details.
In order to guide the agent to learn a human-like
inductive bias, we introduce a language ground-
ing term to the loss function: loss=LPPO(θ) +
clangLlang(ˆψθ, ψ). Here LPPO(θ)is the original
Observation
x(t)enca(t-1)
V
πenc(x(t))
Underlying
Board
“A U Shape”
Board
DescriptionBERT
r(t-1)
BERT Embedding Grounding
TaskRL TaskFigure 3: Grounding architecture. A CNN encoder ob-
serves the board state and passes it onto an LSTM policy
network conditioned on the previous timestep’s action
and reward. We have the agent concurrently predict
the BERT embedding of the corresponding language
description using the encoder.
PPO loss function, clangis a hyperparameter co-
efficient that weights the language loss Llang,ψ
is the language attribute , and ˆψθis the agent’s
prediction of the language attribute. Optimizing for
an auxiliary language task jointly with the original
task has previously been found to shape the latent
representations used in the original task (Mu et al.,
2020; Lampinen et al., 2021).
In our study, ψis the BERT embedding of the
uncovered board’s corresponding language descrip-
tion, obtained using the SentenceTransformer pack-
age (https://www.sbert.net/ , based on
Reimers and Gurevych (2019)). ˆψθis generated
using a small network (two layer MLP) on top of
the board encoding shared with the RL task (see
Figure 3). Llangis the MSE between the predicted
and actual BERT embedding of the language de-
scription.
3 Results
We trained all agents on the GSP boards (see Ap-
pendix for details) and evaluated them on held-out
GSP and control boards. We then compared this
held-out test performance against human perfor-
mance on these test boards (see Fig. 1E). Perfor-
mance is based on the number of blue tiles revealed
in the episode, z-scored by the performance of a
nearest neighbor heuristic, so lower is better (see
Appendix). Results are shown in Fig. 4. First, ex-
amining the performance of human participants and
non-linguistic agents, we observe the same double
dissociation results found by Kumar et al. (2021):
humans perform better in the abstract task distribu-
tion and agents perform better in the control task
distribution. Next, we examine agents that were
co-trained with a language loss Llangon three dif-
ferent kinds of language data: low-level, high-level,
and synthetic low-level (see Fig. 2 for examples
of the different kinds of language data). As a final
baseline, we also considered an autoencoder agent
trained to predict the underlying board state (i.e.,
which tiles are red or blue) rather than the board’s
corresponding language attribute.
RL Task Performance
Human Generated (GSP)Task Distribution
Machine Generated (Control)
Blue Tiles Revealed Z-scored by
Nearest Neighbor Heuristic
Figure 4: Language-Grounded Agent Experiment
Results Performance of various agents on held-out tasks
for each task distribution using agents co-trained with
the language objective. As in Fig. 1, performance is
evaluated by z-scoring the number of blue tiles revealed
(lower is better ) relative to a nearest neighbor heuristic.
Error bars are 95% confidence intervals.
We set out to test whether grounding in human-
generated natural language descriptions will result
in our meta-RL agent producing more human-like
performance. We know humans perform better on
GSP boards than control boards (Fig. 1), while
generic agents do the opposite. An agent perform-
ing better on the GSP boards and worse on the
control boards therefore indicates more human-like
behavior.
We see that grounding on human-generated
descriptions leads to a human-like inductive bias
(low and high bars of Fig. 4). Each of them perform
better at the GSP boards than the control boards,
just like humans do. In contrast, although the au-
toencoder agent (which does not use language) is
substantially better on the GSP boards than the
original agent, the autoencoder grounding loss also
boosts its performance on the control boards, which
indicates that its boost in performance relative tothe original agent is not from acquiring a human-
like inductive bias (but could be an interesting in-
ductive bias in and of itself). We also find that
grounding in synthetic text does not seem to
lead to acquiring human-like inductive biases ei-
ther, since the agent using synthetic low-level text
closely matches the autoencoder and does better in
the control distribution than the GSP distribution.
We also find that the level of abstraction at
which humans write their description influences
the agent’s acquired inductive bias , as indicated
by the differences in performance among low and
high-level grounded agents. In all descriptions
(even in low-level ones), humans write about ab-
stract concepts (e.g. “squares,” “boxes,” “clusters,”
etc). These abstract concepts are most present in
high-level descriptions as they let humans to be
as succinct in their descriptions as possible. The
agent co-trained to predict these high-level descrip-
tion may therefore distill these abstract concepts
very strongly into the representations it learns. This
could explain why the high-level agent has a “super-
human“ inductive bias toward abstraction, where it
does best on the GSP boards (relative to all other
agents and even humans) and the worst on the con-
trol boards (worse than humans, and even worse
than the nearest neighbour heuristic).
4 Conclusion
In this work, we show how meta-reinforcement
learning agents can be guided to have human-like
inductive biases towards abstraction. To set this up,
we used the task paradigm of Kumar et al. (2021)
with a task distribution that directly embeds hu-
man priors through people using Gibbs Sampling
with People (Fig. 1B). We used the procedure in-
troduced in Kumar et al. (2021) to build a control
task distribution (see Fig. 1D) to help benchmark
for acquiring human-like inductive biases. Our
results show that having the agent predict human-
generated language descriptions while doing the
task during training can guide the agent towards
learning human-like inductive biases (Figure 4).
We also manipulated the level of abstraction at
which humans write their descriptions (Fig. 2) and
showed that this can affect how well the learner
acquires an inductive bias more consistent with hu-
man behavior. This lays the groundwork for future
research in learning human-like abstract represen-
tations to move toward closing the gap between
human and machine intelligence.
References
Jacob Andreas, Dan Klein, and Sergey Levine. 2018.
Learning with latent language. In Proceedings of
the 2018 Conference of the North American Chap-
ter of the Association for Computational Linguistics:
Human Language Technologies, Volume 1 (Long Pa-
pers) , pages 2166–2179.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and
Balázs Kégl. 2011. Algorithms for hyper-parameter
optimization. Advances in neural information pro-
cessing systems , 24.
Michael Chmielewski and Sarah C Kucker. 2020. An
mturk crisis? shifts in data quality and the impact on
study results. Social Psychological and Personality
Science , 11(4):464–473.
JH Clark. 1924. The ishihara test for color blindness.
American Journal of Physiological Optics .
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Kristina Toutanova. 2018. Bert: Pre-training of deep
bidirectional transformers for language understand-
ing. arXiv preprint arXiv:1810.04805 .
Yan Duan, John Schulman, Xi Chen, Peter L Bartlett,
Ilya Sutskever, and Pieter Abbeel. 2016. Rl Θ 2: Fast
reinforcement learning via slow reinforcement learn-
ing. arXiv preprint arXiv:1611.02779 .
Stuart Geman and Donald Geman. 1984. Stochastic re-
laxation, gibbs distributions, and the bayesian restora-
tion of images. IEEE Transactions on pattern analy-
sis and machine intelligence , (6):721–741.
Thomas L Griffiths, Frederick Callaway, Michael B
Chang, Erin Grant, Paul M Krueger, and Falk Lieder.
2019. Doing more with less: meta-reasoning and
meta-learning in humans and machines. Current
Opinion in Behavioral Sciences , 29:24–30.
Thomas L Griffiths, Nick Chater, Charles Kemp, Amy
Perfors, and Joshua B Tenenbaum. 2010. Probabilis-
tic models of cognition: Exploring representations
and inductive biases. Trends in cognitive sciences ,
14(8):357–364.
Thomas L Griffiths, Dylan Daniels, Joseph L Auster-
weil, and Joshua B Tenenbaum. 2018. Subjective
randomness as statistical inference. Cognitive psy-
chology , 103:85–109.
Peter Harrison, Raja Marjieh, Federico Adolfi, Pol van
Rijn, Manuel Anglada-Tort, Ofer Tchernichovski,
Pauline Larrouy-Maestri, and Nori Jacoby. 2020.
Gibbs sampling with people. Advances in Neural
Information Processing Systems , 33:10659–10671.
Timothy Hospedales, Antreas Antoniou, Paul Micaelli,
and Amos Storkey. 2020. Meta-learning in neural net-
works: A survey. arXiv preprint arXiv:2004.05439 .Sreejan Kumar, Ishita Dasgupta, Jonathan Cohen,
Nathaniel Daw, and Thomas Griffiths. 2021. Meta-
learning of structured task distributions in humans
and machines. In International Conference on Learn-
ing Representations .
Brenden M Lake, Tomer D Ullman, Joshua B Tenen-
baum, and Samuel J Gershman. 2017. Building ma-
chines that learn and think like people. Behavioral
and brain sciences , 40.
Andrew K Lampinen, Nicholas A Roy, Ishita Dasgupta,
Stephanie CY Chan, Allison C Tam, James L McClel-
land, Chen Yan, Adam Santoro, Neil C Rabinowitz,
Jane X Wang, et al. 2021. Tell me why!–explanations
support learning of relational and causal structure.
arXiv preprint arXiv:2112.03753 .
Jelena Luketina, Nantas Nardelli, Gregory Farquhar,
Jakob N Foerster, Jacob Andreas, Edward Grefen-
stette, Shimon Whiteson, and Tim Rocktäschel. 2019.
A survey of reinforcement learning informed by nat-
ural language. In IJCAI .
Gary Lupyan and Benjamin Bergen. 2016. How lan-
guage programs the mind. Topics in cognitive sci-
ence, 8(2):408–424.
Jesse Mu, Percy Liang, and Noah Goodman. 2020.
Shaping visual representations with language for few-
shot classification. In Proceedings of the 58th Annual
Meeting of the Association for Computational Lin-
guistics , pages 4823–4830.
Karthik Narasimhan, Regina Barzilay, and Tommi
Jaakkola. 2018. Grounding language for transfer
in deep reinforcement learning. Journal of Artificial
Intelligence Research , 63:849–874.
Antonin Raffin, Ashley Hill, Maximilian Ernestus,
Adam Gleave, Anssi Kanervisto, and Noah Dormann.
2019. Stable baselines3.
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert:
Sentence embeddings using siamese bert-networks.
InProceedings of the 2019 Conference on Empirical
Methods in Natural Language Processing and the 9th
International Joint Conference on Natural Language
Processing (EMNLP-IJCNLP) , pages 3982–3992.
John Schulman, Filip Wolski, Prafulla Dhariwal,
Alec Radford, and Oleg Klimov. 2017. Proxi-
mal policy optimization algorithms. arXiv preprint
arXiv:1707.06347 .
Elizabeth S Spelke. 2003. What makes us smart? core
knowledge. Language in mind: Advances in the
study of language and thought , page 277.
Joshua B Tenenbaum, Charles Kemp, Thomas L Grif-
fiths, and Noah D Goodman. 2011. How to grow a
mind: Statistics, structure, and abstraction. science ,
331(6022):1279–1285.
Jane X Wang, Zeb Kurth-Nelson, Dharshan Kumaran,
Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Demis
Hassabis, and Matthew Botvinick. 2018. Prefrontal
cortex as a meta-reinforcement learning system. Na-
ture neuroscience , 21(6):860–868.
Catherine Wong, Kevin M Ellis, Joshua Tenenbaum, and
Jacob Andreas. 2021. Leveraging language to learn
program abstractions and search heuristics. In In-
ternational Conference on Machine Learning , pages
11193–11204. PMLR.
ABuilding an Abstract Task Distribution
using Gibbs Sampling with People
To generate a task distribution of boards directly
from humans we used Gibbs Sampling with People
(GSP; Harrison et al. 2020, and a similar task for
binary sequences in Griffiths et al. 2018). GSP
samples internal prior distributions by putting hu-
mans “in the loop” of a Gibbs sampler. In our case,
the stimulus space consisted of the space of 4×4
boards, and each of the 16 stimulus dimensions cor-
responded to the binary color of each tile, namely,
red or blue. One of these dimensions was masked
out (i.e. “greyed”) for the prediction task. Each
GSP trial consisted of a prediction task of predict-
ing what color the single masked square is in the
grid conditional on the colors of all other squares
on the grid. Once a decision is made, the result-
ing stimulus is passed on to a new participant who
repeats the task with another masked square and
so on. A sample is generated once a full sweep
through the all sixteen squares is completed, sim-
ilar to the standard procedure of Gibbs sampling.
In each trial, participants were presented with a
board with one of its tiles covered (indicated by a
white tile) as well as the following prompt “what
should be the underlying color of the covered white
tile such that the board is described by a very sim-
ple rule?” (Fig. 4A). They then delivered their
answer by clicking on a button that corresponded
to their color of choice. Overall, we ran 100 GSP
chains in parallel for 15 sweeps each ( 24000 total
possible unique boards), and chains were initial-
ized with randomly sampled boards. The order in
which tiles were masked out within each sweep
was also randomized across chains to avoid poten-
tial biases. When sampling from this distribution,
the probability of each board is based on how fre-
quent it occurred during the GSP sampling process.
We used the 500most probable boards to collect
language descriptions.
Participants were recruited on Amazon Mechan-
ical Turk (AMT) and a total of 272 participantscompleted the study. To ensure that participants
did not suffer from any color perception deficien-
cies, we ran the Ishihara color blindness test (Clark,
1924) as a pre-screening task. This also helped in
screening out automated scripts (“bots”) that mas-
querade as participants (Chmielewski and Kucker,
2020).
B Generating the Control Task
Distribution
The same protocol in Kumar et al. 2021 was used.
We trained a fully connected neural network (3
layers, 16 units each) to learn the conditional distri-
bution of each tile given all other tiles on the GSP
boards. These conditional distributions contain all
the relevant statistical information about the boards.
The network was given a board generated with an
abstract rule that had a random tile masked out and
trained to reproduce the entire board including the
randomly masked tile. The loss was the binary
cross-entropy between each of the predicted and
actual masked tiles, summed over all tiles. The net-
work was trained on samples from the GSP boards,
and achieved an accuracy of above 99%.
We used these conditional distributions to gener-
ate samples from the distribution of boards learned
using Gibbs sampling. We started with a grid in
which each tile is randomly set to red or blue with
probability 0.5. We then masked out one tile at a
time and ran the grid through the network to ex-
tract the probability of the missing tile being red or
blue from the trained conditional model. We then
assign the color of this tile by sampling from this
binomial probability. We repeated this by masking
each tile in the 4×4grid (in a random order) to
complete a single Gibbs sweep, and repeated this
whole Gibbs sweep 20 times to generate a single
sample. We generate 25 such independent samples
from the control distribution as held-out test data
for the meta-learning agent and sample from this
distribution during training (while holding out the
test set).
C Testing Humans on Abstract and
Control Tasks
We crowdsourced human performance on our task
using Prolific ( http://www.prolific.co )
for a compensation of $2.25 (averaging $13.55
per hour). Participants were shown the 4×4grid
on their web browser and used mouse-clicks to re-
veal tiles. Each participant was randomly assigned
to either the GSP or control boards. Each partic-
ipant was evaluated on the same test set of grids
used to evaluate the models (24 grids from their as-
signed task distribution in randomized order). Note
that a key difference between the human partici-
pants and model agents was that the humans did
not receive direct training on any of the task distri-
butions. Since participants had to reveal all red tiles
to move on to the next grid, they were implicitly
incentivized to be efficient (clicking as few blue
tiles as possible) in order to finish the task quickly.
We found that this was adequate to get good per-
formance. A reward structure similar to that given
to agents was displayed as the number of points
accrued, but did not translate to monetary reward.
There were 50 participants in each condition (GSP
and control), so 100 participants in total. This was
the same protocol used in Kumar et al. 2021.
D Training Meta-Reinforcement
Learning Agents on the Grid Task
Following previous work in meta-reinforcement
learning (Wang et al., 2018), we use an LSTM
meta-learner that takes the full board as input,
passes it through a convolutional layer and feeds
that, along with the previous action and reward,
to 120 LSTM units. The agent had 16 possible
actions corresponding to choosing a tile (on the
4×4board) to reveal. The reward function was:
+1 for revealing red tiles, -1 for blue tiles, +5 for
the last red tile, and -2 for choosing an already re-
vealed tile. The agent was trained using Proximal
Policy Optimization (PPO; Schulman et al. 2017
using the Stable Baselines package Raffin et al.
2019) for one million episodes. We performed a
hyperparameter sweep separately for the agents
without the grounding loss (i.e. original agents)
and with the grounding loss, since we have to tune
the new clangweight on the grounding loss jointly.
We performed a hyperparameter sweep for: batch
size, n_steps (number of steps to run in an environ-
ment update), gamma, learning rate, learning rate
schedule (constant or linear), clip range, number of
epochs, the λfor Generalized Advantage Estimate
(GAE λ), max grad norm, activation function, value
loss coefficient, entropy coefficient, and grounding
loss coefficient for agents with the grounding loss.
The hyperparameter sweep was done by sampling
from the space of hyperparameter using the Tree-
Structured Parzen Estimator (Bergstra et al., 2011).
We evaluated 200 samples of hyperparameters fromthe space for all agents. Both grounding and non-
grounding agents used the same hyperparameter
spaces to sweep over. We initially did a separate hy-
perparameter sweep for different grounding agents,
but we found in initial experiments that they all
reached similar hyperparameter values and train-
ing reward after the search. Hyperparameters were
evaluated by training on 100,000 episodes and look-
ing at the training reward. The environments used
during test time (Fig. 4) were completely held-out
during this process.
E Performance Evaluation on the Task
Doing well on the task is indicated by the ability to
reveal all red tiles on a grid while revealing as little
blue tiles as possible. So, we measure performance
by counting the number of blue tiles revealed in the
episode. Since different boards will have different
number of red tiles/have varying levels of difficulty,
we controlled for task difficulty/length by measur-
ing the performance relative to a “nearest neighbor”
heuristic. The nearest neighbor heuristic randomly
selects covered tiles that are adjacent to currently
uncovered red tiles (or any covered tile if such a
tile does not exist). For each board, we ran this
heuristic on the board 1000 times to generate a dis-
tribution of performances (i.e., number of blue tiles
revealed) and z-scored the human/agent’s number
of blue tiles revealed according to this distribution.
A z-score of below 0 means that the human/agent
did better than the mean performance of the nearest
neighbor heuristic, and the specific value of the
z-score reflects how many standard deviations the
human/agent did better than the mean performance
of the nearest neighbor heuristic.
F Natural Language Descriptions of the
GSP Boards
We took 500 of the highest probability GSP
boards and broke them up into sets of 25. We
then randomly assigned participants on Prolific
(http://www.prolific.co ) to these sets of
25 boards. Around 10 participants did each set of
the boards given the low level prompt and around
10 participants did each of the board given the high
level prompt. As a result, each participant wrote
descriptions for 25 boards and each board has ap-
proximately 20 descriptions, with about half being
from the low level prompt and half being from the
high level prompt.
The low-level prompt was: “Your goal is to de-
scribe this pattern of red squares in words. Be as
detailed as possible. Someone should be able to
reproduce the entire board given your description.
You may be rewarded based on how detailed your
description is.” and the high-level prompt was “Be
as general as possible in your description. Your de-
scription should use as few words as possible and
focus on the pattern of red squares as a whole and
not individual squares. You may be rewarded based
on how concise your description is.” We converted
the blue tiles to white when showing the boards
to the participants so that they would focus their
descriptions on the red tiles.
Synthetic low level descriptions were non-
human generated and used the template. The tem-
plate goes through the location of every tile and
says “The reds are in: Xth column and Yth row,...,”
for every red tile location.
G Hyperparameter Values
The following table contains the hyperparameters
used.
Agent No Grounding Loss Grounding Loss
batch_size 16 256
n_steps 2048 8
gamma 0.9 0.9
learning_rate 0.000516501 0.000376021
lr_schedule linear linear
ent_coef 1.3907E-05 1.45674E-06
clip_range 0.3 0.3
n_epochs 10 5
gae_lambda 0.8 0.95
max_grad_norm 2 0.6
vf_coef 0.000914363 0.016291309
activation_fn relu tanh
grounding_coef 0 0.494866282
H Training Curves of Agents
Figure 5: Training Reward Curves for All Agents

Kumar
S
Dasgupta
I
Hu
M
Y
Marjieh
R
Hawkins
R
D
Daw
N
Cohen
J
Narasimhan
K
R
& Griffiths
T
L


Can Humans Do Less-Than-One-Shot Learning?.

Can Humans Do Less-Than-One-Shot Learning?
Maya Malaviya1,*, Ilia Sucholutsky2,*, Kerem Oktar1, Thomas L. Griffiths1,2
1Department of Psychology, Princeton University
2Department of Computer Science, Princeton University
*Equal contribution.
{mayamala, is2961, oktar, tomg }@princeton.edu
Abstract
Being able to learn from small amounts of data is a key char-
acteristic of human intelligence, but exactly how small? In this
paper, we introduce a novel experimental paradigm that allows
us to examine classification in an extremely data-scarce set-
ting, asking whether humans can learn more categories than
they have exemplars (i.e., can humans do “less-than-one shot”
learning?). An experiment conducted using this paradigm re-
veals that people are capable of learning in such settings, and
provides several insights into underlying mechanisms. First,
people can accurately infer and represent high-dimensional
feature spaces from very little data. Second, having inferred
the relevant spaces, people use a form of prototype-based cate-
gorization (as opposed to exemplar-based) to make categorical
inferences. Finally, systematic, machine-learnable patterns in
responses indicate that people may have efficient inductive bi-
ases for dealing with this class of data-scarce problems.
Keywords: Categorization, few-shot learning; soft labels;
machine-learning
Introduction
Machine learning (ML) systems now approach or exceed
human performance across a wide variety of tasks (LeCun,
Bengio, & Hinton, 2015). Much of this progress is at-
tributable to the use of highly flexible algorithms in very
data-rich settings—most benchmark tasks in classification,
for instance, provide hundreds or thousands of examples per
class (e.g. Krizhevsky, Sutskever, & Hinton, 2012). In con-
trast, human intelligence is characterized by its impressive
data efficiency. Both children and adults can effectively learn
complex concepts in few- or “one-shot” settings (i.e., after
seeing a few or one examples; see below for a detailed re-
view). This contrast in the efficiency of learning has inspired
recent ML research in few-shot learning (for a review, see
Y . Wang, Yao, Kwok, & Ni, 2020). A recent finding sug-
gests that it is technically possible to learn in less-than-one-
shot (LO-shot) settings (e.g., learning Nclasses from M<N
samples, see Sucholutsky & Schonlau, 2020). It is currently
unknown whether humans can learn in such a setting. More-
over, no existing experimental paradigms are fit to investigate
this question, as LO-shot learning requires using “soft labels”
(which provide graded category membership) as opposed to
the “hard” discrete assignments that are typically used.
In this paper, we introduce a novel experimental paradigm
that allows us to test LO-shot learning in humans, and present
the results of an experiment conducted in this paradigm.
These results are both the first demonstration of success-ful LO-shot learning in humans, and provide the follow-
ing novel insights into the psychological mechanisms en-
abling this capacity. First, the distribution of participants’
responses indicates that they accurately infer the structure
and statistics of the data-generating process, without any ex-
plicit instruction. Second, comparing participant data with
the predictions of computational models suggests that people
use a form of prototype-based categorization (as opposed to
exemplar-based). Third, participant responses show system-
atic, machine-learnable patterns, which suggests that people
may have efficient inductive biases for dealing with this class
of data-scarce problems. Combined, these results demon-
strate the power of our paradigm to reveal novel insights into
cognition and shed light on existing debates in psychology.
Few-Shot Learning in Humans
A fundamental question in cognitive science is how peo-
ple come to know so much about the world from such lit-
tle input (Russell, 1948). There is much evidence to sug-
gest that people can learn from few examples: toddlers learn
and generalize the functions of artifacts (Casler & Kelemen,
2005) and meanings of words (Carey & Bartlett, 1978; Carey,
2010; Bloom, 2000) after exposure to just one instance,
whereas adults can learn novel recursive (Lake & Piantadosi,
2020), compositional (Lake, Linzen, & Baroni, 2019), lexi-
cal (Coutanche & Thompson-Schill, 2014), and visual (Lake,
Salakhutdinov, Gross, & Tenenbaum, 2011) concepts in sim-
ilar settings. But what underlies such impressive efficiency?
Early theorists proposed that human knowledge is too com-
plex to be learned from limited input, and thus must be in-
nate (e.g., Chomsky, 1986). But mounting evidence sug-
gests that statistical inference can be sufficient (Xu & Kush-
nir, 2013; Tenenbaum, Kemp, Griffiths, & Goodman, 2011)
when supported by inductive biases resulting from hierarchi-
cal inference (Yuan, Xiang, Crandall, & Smith, 2020), ac-
tive allocation of attention (Smith, Jones, Landau, Gershkoff-
Stowe, & Samuelson, 2002), and well-calibrated priors (Rule
& Riesenhuber, 2021). LO-shot learning is a problem that
contributes to this debate by highlighting the surprising com-
plexity of statistical inferences that can be drawn from rich
and scarce data. It might seem impossible to learn a category
without seeing any examples from it—yet such inferences are
quite common.
Computational Principles of LO-shot Learning
In the machine-learning literature, few-shot learning meth-
ods can be seen as a response to the growing problem of
deep learning requiring massive models to be trained on mas-
sive datasets. This scaling trend has led to novel deep learn-
ing technologies being inaccessible to the broader research
community due to the data requirements and computational
cost, as well as potentially being harmful to the environ-
ment due to their energy cost (Strubell, Ganesh, & McCal-
lum, 2019). While few-shot learning can greatly reduce the
required number of training examples over typical deep learn-
ing techniques, this number still scales at least linearly with
the number of classes present in the dataset. For example,
one-shot learning still requires one training example for ev-
ery class in the dataset (Fei-Fei, Fergus, & Perona, 2006).
LO-shot learning emerged as an attempt to probe the lim-
its of few-shot learning and determine whether it was pos-
sible to have sub-linear scaling of the required number of
training examples. The first evidence of LO-shot learning in
neural networks came about as a by-product of research into
“dataset distillation”—a method for “distilling” large training
datasets into small synthetic datasets that will still train mod-
els to comparable test accuracies (T. Wang, Zhu, Torralba,
& Efros, 2018). Sucholutsky and Schonlau (2021) showed
that the size of the synthetic “distilled” dataset could be re-
duced to below one training example per class if each syn-
thetic training example were assigned a learnable soft label
instead of the usual hard labels (see Figure 1 for an expla-
nation of hard and soft labels). They demonstrated this by
showing a neural network can accurately recognize handwrit-
ten images of the ten digits (0-9) after being trained on a total
of just five synthetic images. Further research into LO-shot
learning focused on using analytical methods to test the theo-
retical limits of learning with small data and showed that the
number of soft-labeled training examples required to learn K
classes was actually a function of the geometry of the data
rather than of K(Sucholutsky & Schonlau, 2020). A key re-
sult from this analysis was that just two soft-label training

0.6
0.3
0.1
Set largest to 1− − − − − − − − →
and rest to 0
1
0
0

Figure 1: Soft and hard labels. The vector on the left shows a
soft label—a probability distribution over a stimulus’s mem-
bership to each class. In this example, the soft label sug-
gests 60%, 30%, and 10% probability of membership to the
first, second, and third classes, respectively. The vector on the
right shows a hard label, which indicates a stimulus belongs
to a single class. In this example, the hard label suggests strict
membership in the first class. Hard labels can be derived from
soft labels by using an argmax function (i.e. setting the high-
est probability element to 1, and the rest to 0).
Figure 2: Generated stick figures along a 1-D manifold.
examples are sufficient to characterize any finite number of
classes, if those classes are all approximately situated on the
same one-dimensional manifold. Sucholutsky, Kim, Browne,
and Schonlau (2021) proposed algorithmic methods for de-
tecting and exploiting such manifolds in datasets.
In this paper, we present a framework for testing whether
humans also have strategies for disentangling class informa-
tion in small data regimes. In particular, in this initial study
we test the simplest formulation of the result above: can hu-
mans learn to characterize three classes lying on a 1-D mani-
fold when given just two soft-label training examples?
Methods
Participants
Participants were 70 adults (25 male, 43 female, 2 non-binary,
mean age = 36) recruited from the Prolific platform in ex-
change for monetary compensation ($2.00 for a 15-minute
experiment). Participants were to be excluded if they gave
the same response for each trial throughout the experiment,
which could possibly indicate lapsed attention. However, no
participants were eliminated from the original sample as none
of them met this pre-determined exclusion criterion.
Stimuli
Testing LO-Shot learning in humans can be difficult, because
it requires eliciting people’s notions of a category that they
have never seen before. Our paradigm proposes that we can
glean this information by asking participants to classify a
stimulus into one of many unseen categories.
The stimuli for this experiment were adapted from
Sanborn and Griffiths (2008). These stick-figure models of
quadrupeds have 9 distinct, continuous features, so they are
characterized by a nine-dimensional space. The stimuli were
generated by manually instantiating feature values for two
stick figures, then taking linear combinations of these val-
ues to produce more stick figures with new feature values.
Because we only took linear combinations of the feature val-
ues, which were scaled by their respective possible ranges,
the stimuli all lie on a 1-D manifold in a 9-D feature space.
We also repeated the stimulus generation procedure, so that
we could determine if results were robust across two different
manifolds (Manifold 1 and Manifold 2). Each manifold con-
tains 20 generated stimuli. To see a subset of generated stick
figures and examples of how the features can change across
the manifold, see Figure 2.
To make the these stimuli intuitive to understand, we
framed the stick figures as models that paleontologists use to
Table 1: Soft Label Pairs (SLPs)
Dinosaur 1 Dinosaur 2
1 [0%, 0%, 100%] [25%, 50%, 25%]
2 [0%, 0%, 100%] [25%, 75%, 0%]
4 [0%, 25%, 75%] [25%, 0%, 75%]
5 [0%, 25%, 75%] [25%, 25%, 50%]
6 [0%, 25%, 75%] [25%, 50%, 25%]
7 [0%, 25%, 75%] [25%, 75%, 0%]
8 [0%, 25%, 75%] [50%, 0%, 50%]
9 [0%, 25%, 75%] [50%, 25%, 25%]
10 [0%, 25%, 75%] [50%, 50%, 0%]
11 [0%, 25%, 75%] [75%, 25%, 0%]
12 [0%, 50%, 50%] [25%, 25%, 50%]
13 [0%, 50%, 50%] [50%, 0%, 50%]
14 [0%, 50%, 50%] [50%, 25%, 25%]
16 [25%, 25%, 50%] [25%, 50%, 25%]
summarize dinosaur fossil structure. Therefore, we referred
to the stick figures as dinosaurs throughout the experiment.
The task we built took inspiration from LO-shot learning
methods in ML by adapting the concept of soft labels, which
are a way to represent an object’s simultaneous membership
to several classes to a human setting (see Figure 1 for an ex-
planation of soft labels). In order to make this concept intu-
itive to participants, we gave them soft labels in the form of
genetic information. Specifically, we indicated that dinosaurs
had a certain amount of genetic overlap with three unseen
species: Species 1, Species 2, and Species 3. If subjects build
characterizations for Species 1, 2, and 3 when given only two
soft-labelled dinosaurs, we can successfully probe whether
humans are capable of LO-shot learning in this paradigm.1
We built soft-label data in the form of soft-label pairs
(SLPs) because we planned to show two dinosaurs, each with
their own genetic information pertaining to Species 1, 2, and
3. SLPs were chosen according to the following criteria: a)
they should use probabilities that are easily interpretable in
the context of genetic background (i.e. 0, 0.25, 0.5, 0.75,
1); b) they should be informative about all three classes; c)
they should form valid probability distributions; and d) they
should capture a diverse set of possible configurations.The fi-
nal set of SLPs used in this study are listed in Table 1.
Experimental Procedure
Participants were randomly assigned into one of 14 condi-
tions. Each group was introduced to a different set of soft la-
bels which were assigned to Dinosaurs 1 and 2 for the whole
1A potential concern with this design is that participants may
also learn the 1D manifold in an unsupervised way from the test
images. Preliminary results from an ongoing pilot where partici-
pants are asked to pin the location of the three unseen classes on
the manifold after every trial suggest this to not be the case (i.e. the
participant-inferred class locations remain the same as participants
perform more trials on the same SLP but change when participants
are given a new SLP).experiment. All 14 SLPs are listed in Table 1.
To introduce the experiment, we presented participants
with a vignette, framing the stick-figure stimuli as models that
paleontologists use to compare the anatomy of dinosaur fos-
sils they have uncovered at dig sites. The soft labels were
explained as results of a long and expensive DNA analy-
sis which revealed that these labeled dinosaurs are the de-
scendants of three previously unknown species of dinosaur.
Therefore, we can offer a rationale for the classification
task: “the scientists believe that by comparing the stick-figure
model of a new fossil against those of the first two fossils
and their genetic information, it may be possible to deduce
the closest relative of the new fossil.” The participants were
then presented with two of the dinosaur fossil models (la-
beled Dinosaur 1 and Dinosaur 2) and genetic information
for each. They then saw a third dinosaur (Dinosaur 3) and
were asked to use the appearance of all of the dinosaurs and
the genetic information from Dinosaurs 1 and 2 to conclude
which species Dinosaur 3 is most closely related to.
After the participant submitted their response, the trial was
repeated, but Dinosaur 3 was a new, unseen stick figure from
the manifold. Thus, participants completed 20 trials per man-
ifold. Because we generated two manifolds, mid-way through
the experiment participants were alerted that they would be
changing to new dig sites with new dinosaurs. They pro-
ceeded to repeat the same process but with dinosaurs gen-
erated from the second manifold. The order of Manifolds 1
and 2 was randomized for each participant. Note that while
Dinosaurs 1 and 2 change across the manifolds, the genetic
information presented remains the same for each participant.
Results
Our analysis of our data had three aims. First, to deter-
mine that a) participants are actually performing some sort
of meaningful learning from the soft labels in our experiment
and that b) strategies are robust across different participants.
Second, to explore which approach to classification partici-
pants may be using when developing their strategies in this
experiment. Finally, to show that machine-learning models
can predict human behavior in this experiment.
Learning from Soft Labels in LO-Shot Settings
One of the benefits of our proposed framework is that it en-
ables evaluation of the human ability to disentangle class in-
formation in LO-shot regimes. We performed several analy-
ses which suggest that participants are discovering meaning-
ful, reproducible strategies in this setting.
First, we can qualitatively observe in the population dis-
tributions in Figure 4 that participants have non-trivial pos-
teriors over all three classes over the entire manifold and
that these posteriors vary with changes in the presented SLP.
When analyzed at a subject-level, we find that the conclusion
is the same: individual participants have non-trivial posteri-
ors over all three classes that vary with position of the tar-
get dinosaur along the manifold. Furthermore, we tested the
Figure 3: A screen capture from the experiment. In this trial,
participants classified a dinosaur from Manifold 1, with ge-
netic information from SLP 16.
variance within every SLP (i.e. a 20-by-3 contingency ta-
ble) using Pearson’s chi-squared test and find that, even after
applying the Bonferroni correction for multiple comparisons,
the results are statistically significant for all ( χ2(38)≥90,
p< .001) but two SLPs (SLP-4: χ2(38) =41.7,p=0.312;
SLP-5: χ2(38) =47.0,p=0.149). We also test the effect
of choice of SLP (i.e. a 14-by-3 table as we aggregate over
the manifold for each SLP) and find that the results are also
statistically significant ( χ2(26) =721.9,p< .001).
As mentioned above, each participant completed 40 trials
on their assigned SLP, 20 of which are with Manifold 1 and
20 with Manifold 2. Since the soft labels remain constant be-
tween these two sets of 20 trials (only the presented pair of di-
nosaurs changes), we can compute within-subject agreement
(WSA) by comparing a participant’s responses on the Man-
ifold 1 to their responses on Manifold 2. Similarly, we can
calculate between-subject agreement (BSA) by comparing re-
sponses of all pairs of participants assigned to the same SLP.
Across our dataset, we find that WSA is fairly high with an
average of 74.07% while a random selection strategy has an
expected WSA of 33.33% (binomial test, p< .001). This sug-
gests that participants develop a consistent strategy and that
the induced posterior is based on a relative comparison of the
stimuli (i.e. the manifold described by the SLP) rather than
the absolute stimuli. We find that BSA is 68.18% (chance
value is still 33.33%; binomial test, p<0.001) which sug-
gests that different participants may be discovering the samestrategies for disentangling classes.
In order to further test whether participants’ strategies
make use of the soft-label information, we compare whether
the BSA between pairs of participants is correlated with
the similarity between the two SLPs those participants were
shown. To compute similarity between pairs of SLPs, we flat-
ten each SLP into a single vector of length six (three proba-
bility values from each of the two soft labels in an SLP) and
compute cosine similarity (dot product divided by the product
of the norms) between every pair of such vectors.
In Figure 5, we present a heatmap of pairwise BSA across
all participants alongside a heatmap of cubed pairwise cosine
similarities of the SLPs presented to each participant (cubing
the similarities helps better distinguish the structure). The
Pearson correlation coefficient between the upper-triangular
entries of the two matrices is r(2483 ) =0.566, p<0.001
which suggests moderate linear correlation. Based on the
well-defined structures visible in the BSA heatmap, we con-
clude that subjects’ strategies make significant use of the pro-
vided soft labels but that a similarity metric other than cosine
similarity may be more predictive of BSA.
Prototype and Exemplar Models
A comprehensive overview of psychological models of cat-
egorization is outside the scope of this paper (instead, see
Murphy, 2002; Kruschke, 2008). However, we will intro-
duce the two main classes of models that have been used in
the literature and analyze which better matches human be-
havior in LO-shot settings. Exemplar models (e.g., Medin &
Schaffer, 1978; Nosofsky, 1987) typically store each instance
of a category that has been encountered thus far, and catego-
rize new stimuli according to the similarity of the stimulus
to every exemplar in memory. Prototype models (e.g., Reed,
1972) operate analogously, but derive summary representa-
tions of learned categories and compute similarity with these
“prototypes” (instead of computing it across all instances).
There are few restrictions on what constitutes a summary
representation—it could be the modal exemplar for a cate-
gory, the central exemplar, or an “ideal” (see Murphy, 2002).
We compare our study population results against three non-
parametrics classification models: a) a prototype model that
first fits three hard-label prototypes (one for each class) to the
data2and then uses them with a distance-weighted 3-nearest
neighbor classifier for prediction; b) a 1-nearest neighbor
(1NN) exemplar model that copies the nearest labeled di-
nosaur when predicting probabilities at a target point; and
c) a distance-weighted 2-nearest neighbor (2NN) exemplar
model. When making a prediction, the distance-weighted k-
nearest neighbor models that we used output the sum of the k
provided labels (hard or soft) weighted by their inverse square
2Prototype models can be fit a priori to the SLP and/or in post hoc
analysis to the participant responses. We found both simple rules
and gradient descent with a likelihood maximization objective can
effectively fit a priori models. In our post hoc analysis, we further
shift the fitted prototype location of each class closer to the center of
mass of participant ratings of that class.
Figure 4: Distributions of participant responses for every SLP. Each color represents one of the possible three classification
responses. The grey columns represent the locations along the manifold of Dinosaur 1 and Dinosaur 2 shown to participants.
Along the grey columns, colored markers represent the genetic information associated with the dinosaur at that location.
distance from the target point. We compare the probability
distributions predicted by all three models against the empir-
ical distribution collected from the population for all of the
SLPs and find that the prototype model best matches the em-
pirical distribution. The average of the mean squared errors
(MSE) is 1.400 (variance-weighted multi-output R2=0.609)
for the prototype model, 2.584 (variance-weighted multi-
output R2=0.284) for the 1NN exemplar model, and 2.639
(variance-weighted multi-output R2=0.299) for the 2NN ex-
Figure 5: Comparing the similarity matrix of participant re-
sponses to the similarity matrix of the SLPs they were shown.
The left shows a heatmap of pairwise between-subject agree-
ment (i.e. the similarity between participant responses). The
right shows a heatmap of cosine similarity between the actual
SLPs assigned to these participants.
Figure 6: Comparing classification strategies to population
results for four SLPs. The first column shows the distribution
in our population of human participants. The remaining three
columns show model distributions.
emplar model. In Figure 6 we visualize the empirical distri-
bution and the three model distributions for four SLPs.
Machine Learning
The human responses collected in this dataset can be used for
supervised training of machine-learning models. The moti-
vation for doing this is three-fold: a) the results can provide
evidence for or against human ability to learn in the LO-shot
setting; b) a successfully trained model could be used to simu-
late or predict human behavior which in turn can guide future
data collection; and c) it enables a new human-in-the-loop
method for training and aligning machine learning systems in
extreme low-data regimes without having to explicitly encode
strong inductive biases into the models.
We establish a proof-of-concept benchmark by fitting a
Random Forest classification model to our dataset.3To
avoid over-fitting, we perform leave-one-out cross-validation
(LOO-CV) over SLPs which ensures that no participants,
nor any SLPs, overlap between training and validation. The
population-level classification accuracy averaged over the 14
folds was 88.9%. The average MSE between the model-
estimated probability distribution and the empirical distribu-
tion was 0.086 (variance-weighted multi-output R2=0.443).
We note that R2is lower than for the prototype model due
to one fold (SLP-4) where the mean values are highly pre-
dictive for all classes leading to a large negative R2value
for the random forest model (excluding this fold, average
R2=0.627). These results suggest that machine learning
models can indeed successfully predict human behavior in
LO-shot settings which reinforces the conclusion that humans
act systematically in this low-data regime. Curiously, pro-
totype models achieve comparable performance to the much
more flexible RF model when explaining participant classi-
fications which suggests that prototype models may already
be capturing most of the modellable variance. We provide an
example of human and model predictions in Figure 7.
Discussion
In this paper, we introduced a novel paradigm for investigat-
ing LO-shot learning and soft-label classification in humans.
Experimental results from this paradigm show that people can
learn categories for which they have not seen any exemplars.
That is, people’s capacity for learning in low-data regimes
approaches the theoretical limits of sample efficiency. But
which mechanisms enable such efficient learning?
Systematic response patterns (e.g., high between-subject
agreement on trials with markedly different superficial fea-
tures) indicate that participants accurately infer and represent
3We use the RandomForestClassifier implementation from
thescikit-learn Python package (Buitinck et al., 2013). The in-
put features to the model consist of a flattened vector describing a
single trial: two coordinates corresponding to the location of the
two labeled dinosaurs along the one-dimensional manifold, the as-
sociated six soft-label values corresponding to the SLP, and the co-
ordinate corresponding to the location of the third (target) dinosaur.
The target output consists of the classification the participant made
for that third dinosaur during the trial.
Figure 7: Comparing human behavior with RandomForest
predictions for SLP-13. The top row shows human majority
votes and model predictions for class membership. The bot-
tom row shows the corresponding probability distributions.
the feature space underlying our generative model. Our mod-
eling suggests that people then form prototypes in this space,
and base their final classification judgments on these proto-
types, while constraining their inferences through machine-
learnable inductive biases.
Our paradigm can conceptually be used to investigate LO-
shot learning in other modalities (e.g., classifying auditory
stimuli) or domains (e.g., causal learning). Similarly, the cat-
egory structure underlying our experiment was chosen to be
interpretable, but our paradigm enables investigating behav-
ior at higher dimensions and categorical complexity.
How common are LO-shot learning problems in everyday
life? The novel exemplar case analyzed in this paper is likely
to occur more frequently in developmental settings: a child
might learn that a unicorn is mostly like a horse but also
a bit like a rhinoceros, for instance, learning about a novel
category without seeing any unicorns in the wild. Adults
frequently perform similar inferences as well. If you have
never heard of Queen’s music before, for instance, your friend
might tell you: “it’s a lot like Led Zeppelin and a bit like
ABBA too.” The computational principles outlined in this pa-
per could explain how you can guess what Bohemian Rhap-
sody might be like without even hearing it.
More generally, the development of finer-grained concep-
tual structure may leverage the higher information density
provided by soft labels, accessing the full potential of hierar-
chical inference, especially in domains with fuzzier category
boundaries. Future work could investigate whether the mod-
els developed in this paper can be generalized to inferences
in these domains as well.
Acknowledgments. This work was supported by grant
#61454 from the John Templeton Foundation.
References
Bloom, P. (2000). How children learn the meanings of words.
(L. Gleitman, S. Carey, E. L. Newport, & E. S. Spelke,
Eds.). MIT Press.
Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F.,
Mueller, A., Grisel, O., . . . Varoquaux, G. (2013). API de-
sign for machine learning software: experiences from the
scikit-learn project. In Ecml pkdd workshop: languages for
data mining and machine learning (pp. 108–122).
Carey, S. (2010). Beyond fast mapping. Language Learning
and Development ,6(3), 184–205.
Carey, S., & Bartlett, E. (1978). Acquiring a single new word.
Papers and Reports on Child Language Development ,15,
17-29.
Casler, K., & Kelemen, D. (2005). Young children’s rapid
learning about artifacts. Developmental Science ,8(6), 472–
480.
Chomsky, N. (1986). Knowledge of language: Its nature,
origin, and use . Greenwood Publishing Group.
Coutanche, M. N., & Thompson-Schill, S. L. (2014). Fast
mapping rapidly integrates information into existing mem-
ory networks. Journal of Experimental Psychology: Gen-
eral,143(6), 2296–2303.
Fei-Fei, L., Fergus, R., & Perona, P. (2006). One-shot learn-
ing of object categories. IEEE Transactions on Pattern
Analysis and Machine Intelligence ,28(4), 594–611.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Im-
agenet classification with deep convolutional neural net-
works. Advances in Neural Information Processing Sys-
tems,25, 1097–1105.
Kruschke, J. K. (2008). Models of categorization. In
The cambridge handbook of computational psychology (pp.
267–301). Cambridge University Press.
Lake, B. M., Linzen, T., & Baroni, M. (2019). Human few-
shot learning of compositional instructions. arXiv preprint
arXiv:1901.04587 .
Lake, B. M., & Piantadosi, S. T. (2020). People infer recur-
sive visual concepts from just a few examples. Computa-
tional Brain & Behavior ,3(1), 54–65.
Lake, B. M., Salakhutdinov, R., Gross, J., & Tenenbaum, J. B.
(2011). One shot learning of simple visual concepts. Pro-
ceedings of the Annual meeting of the Cognitive Science
Society ,33.
LeCun, Y ., Bengio, Y ., & Hinton, G. (2015). Deep learning.
Nature ,521(7553), 436–444.
Medin, D. L., & Schaffer, M. M. (1978). Context theory
of classification learning. Psychological Review ,85(3),
207–238.
Murphy, G. (2002). The big book of concepts . A Bradford
Book.
Nosofsky, R. M. (1987). Attention and learning processes
in the identification and categorization of integral stimuli.Journal of Experimental Psychology: Learning, Memory,
and Cognition ,13(1), 87-108.
Reed, S. K. (1972). Pattern recognition and categorization.
Cognitive Psychology ,3(3), 382–407.
Rule, J. S., & Riesenhuber, M. (2021). Leveraging prior
concept learning improves generalization from few exam-
ples in computational models of human object recognition.
Frontiers in Computational Neuroscience ,14.
Russell, B. (1948). Human knowledge: Its scope and limits .
Routledge.
Sanborn, A., & Griffiths, T. L. (2008). Markov chain Monte
Carlo with people. Advances in Neural Information Pro-
cessing Systems , 1265–1272.
Smith, L. B., Jones, S. S., Landau, B., Gershkoff-Stowe, L.,
& Samuelson, L. (2002). Object name learning provides
on-the-job training for attention. Psychological Science ,
13(1), 13–19.
Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy
and policy considerations for deep learning in NLP. Pro-
ceedings of the 57th Annual Meeting of the Association for
Computational Linguistics .
Sucholutsky, I., Kim, N.-H., Browne, R. P., & Schonlau, M.
(2021). One line to rule them all: Generating lo-shot soft-
label prototypes. 2021 International Joint Conference on
Neural Networks (IJCNN) , 1-8.
Sucholutsky, I., & Schonlau, M. (2020). ‘Less than one’-
shot learning: learning N classes from from M <N samples.
arXiv:2009.08449 [cs, stat] .
Sucholutsky, I., & Schonlau, M. (2021). Soft-label dataset
distillation and text dataset distillation. 2021 International
Joint Conference on Neural Networks (IJCNN) , 1-8.
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman,
N. D. (2011). How to grow a mind: statistics structure and
abstraction. Science .
Wang, T., Zhu, J.-Y ., Torralba, A., & Efros, A. A. (2018).
Dataset distillation. arXiv preprint arXiv:1811.10959 .
Wang, Y ., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). General-
izing from a few examples: a survey on few-shot learning.
ACM Computing Surveys ,53(3), 63:1–63:34.
Xu, F., & Kushnir, T. (2013). Infants are rational construc-
tivist learners. Current Directions in Psychological Sci-
ence,22(1), 28–32.
Yuan, L., Xiang, V ., Crandall, D., & Smith, L. (2020). Learn-
ing the generative principles of a symbol system from lim-
ited examples. Cognition ,200, 104243.

Malaviya
M
Sucholutsky
I
Oktar
K
& Griffiths
T
L


Predicting Human Similarity Judgments Using Large Language Models..

Predicting Human Similarity Judgments Using Large Language Models
Raja Marjieh1,*, Ilia Sucholutsky2,*, Theodore R. Sumers2,
Nori Jacoby3, Thomas L. Grifﬁths1,2
1Department of Psychology, Princeton University
2Department of Computer Science, Princeton University
3Computational Auditory Perception Group, Max Planck Institute for Empirical Aesthetics
fraja.marjieh, is2961, sumers, tomg g@princeton.edu; nori.jacoby@ae.mpg.de
*equal contribution.
Abstract
Similarity judgments provide a well-established method for ac-
cessing mental representations, with applications in psychol-
ogy, neuroscience and machine learning. However, collecting
similarity judgments can be prohibitively expensive for natu-
ralistic datasets as the number of comparisons grows quadrati-
cally in the number of stimuli. One way to tackle this problem
is to construct approximation procedures that rely on more ac-
cessible proxies for predicting similarity. Here we leverage
recent advances in language models and online recruitment,
proposing an efﬁcient domain-general procedure for predicting
human similarity judgments based on text descriptions. Intu-
itively, similar stimuli are likely to evoke similar descriptions,
allowing us to use description similarity to predict pairwise
similarity judgments. Crucially, the number of descriptions
required grows only linearly with the number of stimuli, dras-
tically reducing the amount of data required. We test this pro-
cedure on six datasets of naturalistic images and show that our
models outperform previous approaches based on visual infor-
mation.
Keywords: similarity, perception, language models, represen-
tations
Introduction
Mental representations serve as a substrate for a variety of
cognitive tasks such as decision-making, communication and
memory (Anderson, 1990). Understanding the structure of
those representation is a core problem in cognitive science
and is the subject of a large corpus of work in the psycho-
logical literature (Shepard, 1980, 1987; Ghirlanda & Enquist,
2003; Battleday, Peterson, & Grifﬁths, 2020; Peterson, Ab-
bott, & Grifﬁths, 2018; Jha, Peterson, & Grifﬁths, 2020;
Caplette & Turk-Browne, 2022; Hebart, Zheng, Pereira, &
Baker, 2020).
One important example of this research is the development
of the multi-dimensional scaling method (MDS) for uncover-
ing the structure of mental representations based on similarity
judgments (Shepard, 1980). Given a set of Nstimuli, MDS
begins by collecting pairwise similarity judgments and aggre-
gating them into a NNmatrix. Then, an iterative procedure
ﬁnds an embedding that maps the stimuli into points in a psy-
chological space such that their distance mirrors their simi-
larity. Applying MDS to different datasets revealed highly in-
terpretable organization of the stimuli (Shepard, 1980, 1987).
Aside from psychology, similarity judgments play an impor-
tant role in other disciplines such as neuroscience, e.g., in the
method of representational similarity analysis (Kriegeskorte,
Mur, & Bandettini, 2008), as well as in machine learning,e.g., as a way to regularize latent spaces so that they align
with human representations and perception (Esling, Bitton, et
al., 2018).
Despite the success of these approaches, the quadratic in-
crease of the number of pairwise comparisons as a function
of the number of stimuli poses a serious limitation on their
scalability. Indeed, even a relatively small dataset that con-
tains102stimuli would require 104judgments for con-
structing the full similarity matrix. This limitation calls for
alternative procedures that allow for efﬁcient approximation
of human similarity judgments. Previous studies have pro-
posed such a method in the visual modality by harnessing
the latent representations from convolutional neural networks
(CNNs) (Peterson et al., 2018; Jha et al., 2020). Such an
approach, however, is domain-speciﬁc and could potentially
miss important semantic dimensions that weigh on people’s
judgments.
To reduce this burden, we leverage the deep relationship
between conceptual structure and language (Murphy, 2002)
to use linguistic descriptions as a proxy for human seman-
tic representations. Intuitively, stimuli that are judged to be
highly similar are likely to evoke similar descriptions, allow-
ing us to use description similarity to predict pairwise sim-
ilarity judgments. This approach offers two key advantages
over prior work: ﬁrst, it is scalable . While pairwise similar-
ity comparisons scale quadratically with the number of stim-
uli (Shepard, 1980), text descriptions scale linearly. Second,
it is domain-general : unlike CNN representations (Peterson
et al., 2018), which are limited to visual stimuli, our proce-
dure could be applied to any domain.
Finally, we note that our approach leverages two distinct
and important advances. First, text descriptions can be easily
crowd-sourced via online recruitment platforms such as Ama-
zon Mechanical Turk (AMT; https://www.mturk.com/ )
and are part of the common practice in modern machine
learning pipelines (Parekh, Baldridge, Cer, Waters, & Yang,
2020). Second, modern language models (Speer, Chin, &
Havasi, 2017; Devlin, Chang, Lee, & Toutanova, 2018) pro-
vide rich latent representations of text. It is therefore natu-
ral to ask: how far can we go in predicting human similarity
judgments based on language alone?
We explore this question on a collection of six datasets
of naturalistic images for which the ground-truth similarity
matrices are known (Peterson et al., 2018). Our explorationarXiv:2202.04728v1  [cs.LG]  9 Feb 2022
proceeds in three stages. In Study 1, we construct similar-
ity estimates by applying a state-of-the-art word embedding
model known as ConceptNet NumberBatch (CNNB) (Speer
et al., 2017) to pre-existing semantic labels for the dataset im-
ages. In Study 2, we generalize this approach by constructing
similarity estimates based on BERT, a widely-used large lan-
guage model (Devlin et al., 2018), applied to free text descrip-
tions that we crowd-source on AMT. Finally, we combine the
concept-level representation of CNNB with the ﬁne-grained
textual representation of BERT and generate a joint predictor
of similarity judgments. In the process, we benchmark our
models’ predictive accuracy against the CNN-based approach
of Peterson et al. (2018).
General Methodology
Our general pipeline consists of collecting or using pre-
existing linguistic descriptors for the individual stimuli and
then using an embedding model to compute a proxy for pair-
wise similarity (Figure 1).
Predicting Human Similarity
Given a set of stimuli and their linguistic descriptors (se-
mantic labels or free-text descriptions) as well as a suitable
embedding scheme (e.g., a word embedding model) we used
cosine similarity between the vectors representing two stim-
uli as the metric for calculating their similarity (i.e., the dot
product of the two embedding vectors divided by the product
of their norms). Peterson et al. (2018) showed that predict-
ing human similarity using CNN representations can be sub-
stantially enhanced by linearly transforming those representa-
tions. Mathematically, this corresponds to substituting the dot
product zT
1z2with zT
1Wz 2where Wis a suitable diagonal ma-
trix and z1andz2are the embedding vectors. Moreover, Pe-
terson et al. showed that such a transformation can be found
using ridge regression with L2 normalization. We apply this
approach to our linguistic representations, using the Python
library scikit-learn’s RidgeRegression and RidgeCV imple-
mentations. To avoid overﬁtting and simulate generalization
in practice, we performed 6-fold cross-validation over images
which ensured that no images from the training set are present
in the validation set. This ensures that even when combining
BERT and CNNB representations, where the number of fea-
tures increases, overﬁtting is still avoided. To facilitate com-
parison with previous work we quantiﬁed performance by
computing Pearson R2scores (variance explained) (Peterson
et al., 2018; Jha et al., 2020).
Stimuli
The six image datasets used in this paper were taken from
Peterson et al. (2018). The datasets were organized based on
six broad categories, namely, animals, fruits, vegetables, au-
tomobiles, furniture and various objects, each comprising 120
unique images. For all categories except animals, the datasets
included semantic labels for each of the individual images. In
the case of animals, we manually labeled the images. Sample
images and labels appear in Figure 2.
Please describe the content of the image.
Wooden bookshelf composed of multiple open cubes.Dark brown wooden shelves storage with nine cubicles.A unique bookshelf that is slanted with many books sitting sideways.Bookshelf with falling shelves that are ready for all your reading needs.<bookshelf_02.png>
<bookshelf_10.png>
Language ModelSimilarity Prediction𝑠!"=𝑧!#𝑊𝑧"𝑧!,𝑧"Figure 1: Schematic of the similarity prediction procedure
based on text descriptions.
Predicting Human Similarity
Based on Semantic Labels
To initiate our investigation we ﬁrst considered using the pre-
existing semantic labels for the images in our datasets, as they
served as concise summaries of the content of the images. We
evaluated two representations for predicting human similarity
judgments based on these labels, namely, a one-hot represen-
tation and a word embedding representation.
One-hot Label Representation
The ﬁrst approach served as a baseline and consisted of us-
ing the semantic labels as class labels with a “one-hot” rep-
resentation, namely, a vector of the form (0;:::; 0;1;0;:::; 0)
where the 1 indicates which semantic label is associated with
the image. This representation implies that images with the
same semantic label are maximally similar whereas images
with different semantic labels are maximally dissimilar.
Surprisingly, this simple representation possessed non-
trivial predictive power, as indicated by its average raw R2
score of 0 :31 across the datasets shown in Table 1.
Applying a further linear transformation resulted in a small
boost in performance scores ( R2=0:40). The sparsity of
one-hot representations potentially makes linear transforma-
tion ineffective. To remedy this, we applied label smoothing
to all the one-hot vectors. If ~vis the one-hot vector, then
~vsmooth = (1e)~v+e
k1(1~v)where eis the smoothing pa-
rameter (we use a value of 0.8) and kis the number of classes
(which is equal to the length of the vector). Smoothing does
not change the relative structure of the resulting matrix but
allows linear transformation to be successfully applied to the
new vectors.
Finding positive but not strong correlations is not surpris-
ing as the one-hot representation misses ﬁne-grained similar-
Eagle
 Gorilla
 Blackberry
Ottoman
 Human Body
 Car
Beetroot
 Elevator
 End T ableFigure 2: Sample images and their semantic labels.
ity between related (though not identical) semantic labels. In-
deed, although a tiger and a leopard are distinct animals, they
nevertheless share some intuitive semantic similarity being
members of the cat family; likewise for a chair and a recliner,
or a strawberry and a blackberry. This can be seen in the ab-
sence of off-diagonal structure in the predicted similarity ma-
trix (Figure 3). Nevertheless, this preliminary study serves
as an initial evidence for the fact that people’s judgments are
indeed driven by semantic similarity.
Word-embedding Representation
To capture the structure of similarity between different se-
mantic labels we replaced the one-hot representation with
the latent representation of a state-of-art word embed-
ding model known as ConceptNet NumberBatch (CNNB).
CNNB is pre-trained on the ConceptNet knowledge graph
(https://conceptnet.io/ ) which is targeted at capturing
intuitive commonsense conceptual relations.
CNNB contains embeddings not only for single words
but also concepts consisting of several words. To make
use of these, labels consisting of multiple words needed to
have spaces replaced by underscores (e.g. ‘red onion’ be-
comes ‘red onion’). In addition, while the CNNB dictio-
nary is quite large, there are certain words or concepts that
it does not contain. In some of these cases, labels con-
sisting of multiple words whose joint form was not found
in CNNB had to be separated into individual words and
their joint embedding estimated by their normalized sum
(e.g. CNNB(animal body) CNNB(animal)+CNNB(body)p
2). In
other cases, labels had to be replaced by a synonym or the
closest matching concept available in CNNB (e.g. ‘tatsoi’
was replaced by ‘spoon mustard’).The use of CNNB representations resulted in a substantial
performance boost over one-hot representations, as reﬂected
in an R2score of 0 :71 for the transformed representations.
The predicted similarity matrix is shown in Figure 3 and it
is clear that a substantial part of the off-diagonal structure is
recovered. Similar to the CNN models used by Peterson et al.
(2018), the linear transformation ﬁne-tunes the broad repre-
sentations of the model to the speciﬁc task at hand. To ensure
that the linear transformation is not overﬁtting the similar-
ity matrices, we performed 6-fold cross-validation as men-
tioned above and computed a control cross-validated (CCV)
R2score on held-out images. These scores remained high
(R2=0:63), outperforming the CNN model of Peterson et al.
(2018) (Figure 4) on all datasets (except Animals, where it
scored lower by a small margin). This implies that CNNB
representations generalize better to new data. We also note
that the dimensionality of the latent space of CNNB ( d=300)
is much lower than that of the CNN ( d=4096) reducing the
number of possible parameters to optimize over and hence the
risk of overﬁtting.
Predicting Human Similarity
Based on Free Text Descriptions
Concise semantic labels (and corresponding embeddings) are
not always available for stimuli of interest. A more general
approach would rely on free-text descriptions, which can be
easily crowd-sourced online. Such data, however, requires
a different kind of representations capable of ﬂexibly encod-
ing entire sentences (as opposed to aggregating representa-
tions of individual words which could lose important within-
sentence structure). To that end, we used the latent represen-
tations of BERT (Devlin et al., 2018), a popular large-scale
language model based on bidirectional transformers, to em-
bed free-text descriptions for each of the individual images
which we crowd-sourced on AMT. The data collection pro-
cedure as well as example text descriptions are shown in Fig-
ure 1.
Experimental Methods
The recruitment and experimental pipeline were automated
using PsyNet (Harrison et al., 2020), a framework for ex-
perimental design which builds on top of the Dallinger plat-
form ( https://github.com/Dallinger/Dallinger ) for
recruitment automation. Overall, 328 US participants com-
pleted the study and they were paid $12 per hour. Upon com-
pleting a consent form participants had to take a standardized
LexTALE English proﬁciency test (Lemh ¨ofer & Broersma,
2012) to ensure caption quality. Participants that failed to
pass the pre-screening test were excluded from the study.
Next, participants received the following instructions: “In this
experiment we are studying how people describe images. You
will be presented with different images and your task will be
to describe their content. In doing so, please keep in mind
the following instructions, 1) describe all the important parts
of the image, 2) do not start the sentences with “There is”
AnimalsHuman
 Labels
 CNNB
 BERT
Furniture
Figure 3: Full similarity matrices for the “animals” and “furniture” datasets for human participants (left), with corresponding
predictions based on class labels, CNNB and BERT representations.
or “There are”, 3) do not describe unimportant details, 4) you
are not allowed to copy and paste descriptions, 5) descriptions
should contain at least 5 words, 6) descriptions should contain
at least 4 unique words. Note: no prior expertise is required
to complete this task, just describe what you intuitively think
is important as accurately as possible.” Participants were then
presented with nine random images from the dataset to help
give them a sense of the images they were about to describe.
In each trial of the main experiment participants saw one
of the images along with the following prompt “Please de-
scribe the content of the following image” (semantic labels
were never provided). They then provided their description
in a free text response box, subject to the constraints listed
above. Each participant provided up to 30 text descriptions
with each image receiving 15 text descriptions on average. To
ensure that participants did not provide repetitive responses
we computed the average Levenshtein edit distance between
their current response and all previous responses. Participants
for whom the average distance was close to zero ( <0:2) after
5 trials were excluded from the study. Any remaining ran-
dom or very poor quality strings were excluded in a post-
processing stage.
Computing BERT Embeddings
We used a pre-trained BERT-base-uncased model with a
standard tokenization scheme, accessed via the HuggingFace
library (Wolf et al., 2020). For each text description, we ﬁrst
passed the tokens through the BERT model, then took the av-
erage embedding across all tokens (e.g. mean-bag-of-words)
at each layer. We then averaged the embeddings at each layeracross all descriptions for a given image. Empirically, we
computed similarity scores based on layers 0 through 12 and
picked the best performing layer in each case. In order to
combine the BERT and CNNB representations, we ﬁrst nor-
malized both sets of embeddings by their respective means
and standard deviations, and then concatenated the BERT and
CNNB embeddings to get a single vector for each image.
BERT CNN CNNB CNNB+BERT0.00.10.20.30.40.50.60.7Average R2
Figure 4: Average CCV R2score for the main four models
considered (shown in bold in Table 1).
Animals
 Automobiles
 Fruits
Vegetables
 Furniture
 VariousFigure 5: Two-dimensional MDS embedding of the joint CNNB-BERT similarity predictions.
Results
We used the embeddings to produce similarity estimates as
before. We found that while the raw representations of
BERT did not constitute a strong predictor, the linearly re-
weighted BERT representations ( d=768) demonstrated gen-
eralization performance comparable to the CNN-based model
(d=4096) of Peterson et al. (2018) (Figure 4), though not as
high as CNNB. One possible explanation for this difference
is that CNNB predictors used single concise labels per im-
age whereas for BERT we averaged representations of multi-
ple descriptions which could capture different aspects of the
image (Parekh et al., 2020). A more sophisticated approach
could learn to pool embeddings from different descriptions
efﬁciently; however for the purpose of the current work we
chose to focus on simple linear transformations.
As a last step, we constructed a combined predictor that
stacked CNNB and BERT representations to capture broad
concept-level knowledge as well as ﬁne-grained descriptions.
The combined model resulted in the best aggregated perfor-
mance, improving further on the CNNB model (Figure 4).
To appreciate the semantic content of the predicted similar-
ity matrices, we computed a two-dimensional MDS represen-
tation of the images. These representations were computed
using the scikit-learn library with a maximum iteration limit
of 10;000 and a convergence tolerance of 1e-100. First met-
ric MDS was applied to get an initial embedding, then four
iterations of non-metric MDS were applied and the best solu-tion was picked. The results are shown in Figure 5, and reveal
a rich and interpretable semantic organization of the stimuli
capturing a variety of semantic dimensions such as natural
and functional classes as well as color gradients.
Discussion
We proposed a highly efﬁcient and domain-general procedure
for predicting human similarity judgments based on text de-
scriptions with linear (as opposed to quadratic) complexity.
We tested our approach on six datasets of naturalistic images,
ﬁnding evidence for its validity as well as outperforming pre-
vious models that rely on CNNs. These results suggest that
human similarity judgments are indeed grounded in semantic
understanding and language. Beyond the immediate potential
for scaling up studies of similarity, our work also provides
a new perspective on the representational similarity between
BERT and humans (Lake & Murphy, 2021): when tested on
naturalistic datasets with freely generated text descriptions,
we ﬁnd that BERT successfully captures a substantial part of
the structure of human similarity judgments.
This work represents an initial step towards a broader in-
vestigation of similarity in naturalistic domains. First, our
approach offers the possibility of predicting human similarity
in other domains such as audio and video. Second, it could
be used to explore differences between perceptual similarity
(based on raw judgments) and semantic similarity (based on
text descriptions). This discrepancy may vary by domain or
Table 1: R2scores for the different prediction models and datasets.
Model Methodology Animals Automobiles Fruits Vegetables Furniture Various hR2i
Labels Raw 0.23 0.69 0.20 0.24 0.34 0.19 0.31
CNNB Raw 0.51 0.64 0.17 0.17 0.31 0.29 0.35
BERT Raw 0.22 0.30 0.09 0.13 0.25 0.36 0.23
CNN* Raw 0.58 0.51 0.27 0.19 0.37 0.27 0.37
Labels LT-Train 0.29 0.71 0.26 0.27 0.38 0.48 0.40
CNNB LT-Train 0.85 0.86 0.53 0.60 0.67 0.72 0.71
BERT LT-Train 0.79 0.75 0.55 0.64 0.61 0.80 0.69
CNN* LT-Train 0.84 0.79 0.53 0.67 0.72 0.52 0.68
CNNB LT-CCV 0.72 0.86 0.38 0.43 0.63 0.73 0.63
BERT LT-CCV 0.52 0.53 0.23 0.40 0.47 0.62 0.46
CNNB + BERT LT-CCV 0.74 0.85 0.44 0.54 0.64 0.76 0.66
CNN * LT-CCV 0.74 0.58 0.36 0.35 0.35 0.54 0.49
Note: “Raw” corresponds to raw representations, “LT-Train” corresponds to linearly transformed representations
evaluated on training set, and “LT-CCV” corresponds to linearly transformed representations evaluated on held-out
images.hR2iis the average R2across all datasets. * indicates values reproduced from Peterson et al. (2018).
expertise. For example, in the musical domain, experts (e.g.,
trained musicians) may provide rich descriptions of stimuli
(e.g., musical chords) while novices may lack an appropriate
vocabulary, yielding a bigger gap between perception and se-
mantics for the second group. A ﬁne-grained study of this gap
as a function of expertise could be informative about the tra-
jectories of semantic development. Third, a systematic study
could, for example, use CNN and CNNB representations as
a way of isolating perceptual and semantic contributions to a
human similarity judgment. Of particular interest are cases of
maximal discrepancy whereby humans align with one of the
predictors but not the other. Figure 6 shows examples of such
pairs. These seem to suggest that people tend to focus on low-
level perceptual features when the objects of comparison are
unfamiliar, whereas they would neglect these for familiar ob-
jects. A future study could explore this hypothesis in greater
detail.
In addition to psychological applications, our paradigm
may allow for advances in machine learning. Enriching ma-
chine learning datasets with similarity judgments and behav-
ioral data more generally can endow artiﬁcial models with
a variety of useful properties, such as robustness against ad-
versarial attacks and human alignment (Peterson, Battleday,
Grifﬁths, & Russakovsky, 2019). Collecting similarity judg-
ments over all pairs is infeasible for such datasets due to the
large number of stimuli. Nevertheless, in many real-life appli-
cations similarity matrices tend to be sparse, i.e., only a small
subset of comparisons would yield non-vanishing similarity
(Parekh et al., 2020). An efﬁcient enrichment pipeline, there-
fore, must exploit this sparsity and our procedure is a promis-
ing candidate for guiding such methods by predicting which
pairs are likely to be informative a priori . Second, for more
domain-speciﬁc applications, a followup study could lever-
age recent advances in multi-modal transformer representa-
tions to construct better similarity metrics by incorporatingboth visual and semantic cues. We hope to engage with all of
these avenues in future research.
Celtuce
 SeaweedCNN CNNB Human0.00.5
Bear
 ChimpanzeeCNN CNNB Human0.00.5
Bed
 BedCNN CNNB Human01
Figure 6: Examples of image pairs that generated large dis-
crepancies between CNN and CNNB model predictions and
their relation to human similarity scores.
Acknowledgments. This work was supported by a grant
from the John Templeton Foundation.
References
Anderson, J. R. (1990). The adaptive character of thought .
Psychology Press.
Battleday, R. M., Peterson, J. C., & Grifﬁths, T. L. (2020).
Capturing human categorization of natural images by com-
bining deep networks and cognitive models. Nature com-
munications ,11(1), 1–14.
Caplette, L., & Turk-Browne, N. (2022). Computational re-
construction of mental representations using human behav-
ior.PsyArxiv . doi: https://doi.org/10.31234/osf.io/7fdvw
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K.
(2018). BERT: Pre-training of deep bidirectional trans-
formers for language understanding. arXiv preprint
arXiv:1810.04805 .
Esling, P., Bitton, A., et al. (2018). Generative timbre spaces:
regularizing variational auto-encoders with perceptual met-
rics. arXiv preprint arXiv:1805.08501 .
Ghirlanda, S., & Enquist, M. (2003). A century of
generalization. Animal Behaviour ,66(1), 15-36. doi:
https://doi.org/10.1006/anbe.2003.2174
Harrison, P., Marjieh, R., Adolﬁ, F., van Rijn, P., Anglada-
Tort, M., Tchernichovski, O., . . . Jacoby, N. (2020). Gibbs
sampling with people. In H. Larochelle, M. Ranzato,
R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in
neural information processing systems (V ol. 33, pp. 10659–
10671). Curran Associates, Inc.
Hebart, M. N., Zheng, C. Y ., Pereira, F., & Baker, C. I. (2020).
Revealing the multidimensional mental representations of
natural objects underlying human similarity judgements.
Nature human behaviour ,4(11), 1173–1185.
Jha, A., Peterson, J., & Grifﬁths, T. L. (2020). Ex-
tracting low-dimensional psychological representations
from convolutional neural networks. arXiv preprint
arXiv:2005.14363 .
Kriegeskorte, N., Mur, M., & Bandettini, P. A. (2008). Rep-
resentational similarity analysis-connecting the branches of
systems neuroscience. Frontiers in systems neuroscience ,
2, 4.
Lake, B. M., & Murphy, G. L. (2021). Word meaning in
minds and machines. Psychological review .
Lemh ¨ofer, K., & Broersma, M. (2012). Introducing Lex-
TALE: A quick and valid lexical test for advanced learners
of English. Behavior research methods ,44(2), 325–343.
Murphy, G. (2002). The big book of concepts . MIT Press.
Parekh, Z., Baldridge, J., Cer, D., Waters, A., & Yang, Y .
(2020). Crisscrossed captions: Extended intramodal and
intermodal semantic similarity judgments for MS-COCO.
arXiv preprint arXiv:2004.15020 .
Peterson, J. C., Abbott, J. T., & Grifﬁths, T. L. (2018). Eval-
uating (and improving) the correspondence between deep
neural networks and human representations. Cognitive sci-
ence,42(8), 2648–2669.
Peterson, J. C., Battleday, R. M., Grifﬁths, T. L., & Rus-
sakovsky, O. (2019). Human uncertainty makes classiﬁ-
cation more robust. In Proceedings of the ieee/cvf interna-
tional conference on computer vision (pp. 9617–9626).
Shepard, R. N. (1980). Multidimensional scaling, tree-
ﬁtting, and clustering. Science ,210(4468), 390–398. doi:
10.1126/science.210.4468.390
Shepard, R. N. (1987). Toward a universal law of gener-alization for psychological science. Science ,237(4820),
1317–1323. doi: 10.1126/science.3629243
Speer, R., Chin, J., & Havasi, C. (2017). Conceptnet 5.5: An
open multilingual graph of general knowledge. In Thirty-
ﬁrst aaai conference on artiﬁcial intelligence.
Wolf, T., Debut, L., Sanh, V ., Chaumond, J., Delangue, C.,
Moi, A., . . . Rush, A. M. (2020). Transformers: State-of-
the-art natural language processing. In Proceedings of the
2020 Conference on Empirical Methods in Natural Lan-
guage Processing: System Demonstrations (pp. 38–45).
Association for Computational Linguistics.

Marjieh
R
Sucholutsky
I
Sumers
T
R
Jacoby
N
& Griffiths
T
L