Clustering and the efficient use of cognitive resources..

JournalofMathematicalPsychology109(2022)102675
Contents lists available at ScienceDirect
JournalofMathematicalPsychology
journal homepage: www.elsevier.com/locate/jmp
TheoreticalNote
Clusteringandtheefficientuseofcognitiveresources
IshitaDasguptaa,∗,ThomasL.Griffithsb
aDepartmentofComputerScience,PrincetonUniversity,UnitedStatesofAmerica
bDepartmentsofPsychologyandComputerScience,PrincetonUniversity,UnitedStatesofAmerica
a r t i c l e i n f o
Articlehistory:
Received17November2021
Receivedinrevisedform29April2022
Accepted10May2022
Availableonline xxxx
Keywords:
Bayesianinference
Resourcerationality
Informationtheory
Probabilisticnumericsa b s t r a c t
Acentralcomponentofhumanintelligenceistheabilitytomakeabstractions,toglossoversome
detailsinfavorofdrawingouthigher-orderstructure.Clusteringstimulitogetherisaclassicexample
ofthis.However,thecrucialquestionremainsofhowone shouldmaketheseabstractions—whatdetails
toretainandwhattothrowaway?Howmanyclusterstoform?Weprovideananalysisofhowa
rationalagentwithlimitedcognitiveresourcesshouldapproachthisproblem,consideringnotonly
howwellaclusteringfitsthedatabutalsobyhow‘complex’itis,i.e.howcognitivelyexpensive
itistorepresent.Weshowthatthesolutiontothisproblemprovidesawaytoreinterpretawide
rangeofpsychologicalmodelsthatarebasedonprinciplesfromnon-parametricBayesianstatistics.
Inparticular,weshowthattheChineseRestaurantProcessprior,ubiquitousinrationalmodelsof
humanandanimalclusteringbehavior,canbeinterpretedasminimizinganintuitiveformulationof
representationalcomplexity.
©2022ElsevierInc.Allrightsreserved.
1. Introduction
Theonlyenduringaspectofourenvironmentisthatnothing
staysthesame.Weneverhave exactlythesameexperiencetwice.
As a consequence, the human mind has to form abstractions,
clustering these experiences together in a way that supports
generalization. Psychological models have applied this lens to
phenomenaasdiverseascategorization(Anderson,1991;San-
bornetal.,2010),featurelearning(Griffiths&Austerweil,2008),
theoryformation(Kempetal.,2010),classicalconditioning(Ger-
shmanetal.,2010),andwordsegmentation(Goldwateretal.,
2009).Akeyproblemthatarisesineachofthesemodelsisknow-
ingwhentogenerateanewcluster—whenanobject,stimulus,or
wordisgenuinelyofakindthathasneverbeenseenbefore.
Decidingwhentoformanewclusterinvolvesmakingatrade-
offbetweenthecomplexityoftheunderlyingrepresentationand
howwellitdescribestheenvironment.Groupingallexperiences
intoasingleclusterwheretheyarerepresentedbysomeabstract
summarystatisticsismaximallysimple,butatthecostoflosing
asignificantamountofdetail.Havingaseparateclusterforeach
experienceaccuratelycapturesthenuancesofthoseexperiences,
butismaximallycomplex.Sohow shouldweformclusters?
Inthispaper,weaddressthisquestioninthespiritofrational
analysis(Anderson,1991),askinghowitmightbesolvedbyan
idealagent.Moreprecisely,weengageinresourcerationalanaly-
sis(Gershmanetal.,2015;Griffithsetal.,2015;Lieder&Griffiths,
∗Correspondenceto:DeepMindNewYorkCity,USA.
E-mailaddress: idg@google.com(I.Dasgupta).2019),sinceouranalysisfocusesonthequestionofhowthat
agentmightmakethebestuseoflimitedcognitiveresources.For-
malizingthecomplexityofaclusteringininformation-theoretic
terms,wederiveanoptimaldistributionoverclusterings.
Thisanalysisyieldsasurprisingresult:ouroptimalsolution
hasthesamepropertiesasthedistributionoverclusteringsas-
sumedinallofthepsychologicalmodelsmentionedabove.These
modelsuseadistributionoverclusteringsoriginallyintroducedin
psychologybyAnderson(1991)inhisrationalmodelofcatego-
rization.Thisdistributionwasindependentlydiscoveredinnon-
parametricBayesianstatistics(Aldous,1985;Hjortetal.,2010),
whereitisknownastheChineserestaurantprocess(CRP).1
TheCRPhasanumberofattractivemathematicalproperties
thatcanbeusedtojustifyitsuseinpsychologicalmodels,related
toitsconvenienceortheassumptionsitmakesabouttheenvi-
ronment.OuranalysisprovidesanewreasonwhytheCRPmight
makesenseasacomponentofpsychologicalmodels:weshow
thatCRP-likedistributionscanarisefromanefforttominimize
representationalcosts,i.e.thatthisdistributionisnormativeun-
deranassumptionofresource-rationality.Inparticular,weshow
thatbestusingafixednumberofbitstostoreanobject-category
mapping(viz.controllingthecomplexityofthatmapping)can
1The name of this process comes from its creators imagining a large
restaurantthatseatsmultiplepartiesatcommunaltables,withpeoplejoining
tablesbasedontheircurrentpopularity–aphenomenonthatcouldapparently
beobservedinSanFrancisco’sChinatown.Thetablesareclustersandthepeople
theexperiencesbeingclustered.
https://doi.org/10.1016/j.jmp.2022.102675
0022-2496/ ©2022ElsevierInc. Allrightsreserved.
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
Fig. 1.SchematicillustrationofthecategorizationtasksimulatedinAnderson(1991).Inthisfigure,werepresenteachobjectwithitsbinaryfeaturesandeach
clusterasabucket.Eachtimeanobjectisadded,wecaneitherassignittoanexistingcluster,orcreateanewone.Solidarrowsindicatethehigherprobability
assignment,Anderson(1991)assumesthatthisistheassignmentchosenateachstep.Iteratingthisoverthe6stimuliintheir(fixed)orderofpresentationgives
thefinalclustering.Seemaintextforfurtherdetails.
giveCRP-likebehavior.Theseresultsprovidethefirstprocess-
levelexplanationofwhythiskindofclusteringbehaviormight
beareasonableassumptioninpsychologicalmodels.
Theplanofthepaperisasfollows.Inthenextsectionwe
provideamoredetailedintroductiontotheChineserestaurant
process.Wethenturntoouranalysisofoptimalclusteringunder
resourceconstraints.Wederiveourkeyresultsmathematically
andpresentsimulationsthatverifyouranalysis.Weconclude
withadiscussionoftheimplicationsoftheseresultsfordevel-
opingmodelsofhumancognition.
2. Background: The Chinese restaurant process
Asmentionedabove,oneofthechallengesinvolvedinclus-
teringasetofexperiencesisdecidinghowmanyclustersthere
shouldbe.ResearchersinnonparametricBayesianstatisticsde-
velopedaninnovativestrategyforsolvingthisproblem:rather
thanspecifyingaparticularnumberofclusters,weinsteadas-
sume that there could exist an infinite number of clusters of
whichonlyafinitenumberhavebeenobservedsofar.Theprob-
lemofdeterminingthenumberofclustersthenbecomesamatter
ofinferringhowmanyclustersmayhavebeenobserved,which
canbesolvedbyapplyingBayes’rule.
Pursuingthisapproachrequiresidentifyingapriordistribu-
tion over clusterings that remains well-defined regardless of
howmanyexperiencesneedtobeclustered.Acommonwayto
achievethisgoalistoassumethatthepriorprobabilityanitem
belongstoaclusterfollowsadistributionknownastheChinese
Restaurantprocess(CRP;Aldous,1985).Underthisdistribution,
theprobabilityofbelongingtoanexistingclusterisproportional
tothenumberofobjectsalreadyinthatcluster,whiletheproba-
bilityofanewclusterisproportionaltothevalueofaparameter
α.
TomakethiskindofBayesianclusteringmoreconcrete,we
considerhowitmightapplytoanempiricalcontext:thefirstex-
perimentfromMedinandSchaffer(1978).Here,participantssaw
andwereaskedtocategorizestimulithatvariedalongfourbinary
dimensions(color,form,shapeandposition)andgivenabinary
categorylabel.Thesedatawereusedtodemonstratetherational
modelofcategorizationpresentedinAnderson(1991)(illustrated
inFig.1),whichassumesaprioronclustersthatisequivalent
totheCRP.Andersonassumedthatthebinarycategorylabelis
treated as an additional feature, so there are five binary fea-
tures.Thestimuliarethenassumedtobepresentedintheorder
{11111 ,10101 ,01011 ,00000 ,01000 ,10110 }. Anderson’s model
assignsthefirststimulustoitsowncluster.Onseeingthesecond
stimulus,itdecideswhethertoassignitanewclusterortothe
sameclusterasthefirstone.Itmakesthisdecisiononthebasis
oftwofactors:thenewstimulus’overlapwiththefeaturesof
thestimulialreadyinacluster(thelikelihoodfunction),andaparameterthatdetermineshowlikelyingeneralitisthattwo
stimulibelongtothesameclustervs.differentones(theprior).
The assumption that Anderson (1991) made about the latter
parameterdefinesapriorthatisequivalenttotheCRP(Neal,
1998).2
Iteratingthisprocessforeachstimulus,Anderson’smodelpre-
dicted that the most likely clustering of these stimuli is
(11111 ,10101 ,10110),(00000 ,01000),(01011).Thisclustering
doesnotsplitthestimulibythecategorylabel(here,thelast
binaryfeature).However,themodelcangeneratepredictionsfor
thecategorymembershipofnovelstimulibasedonthisclustering
bycalculatingtheprobabilitytheygetassignedtoaclusterand
theprobabilityofthecategorylabelunderthatcluster.Ander-
sonshowedthatthesepredictionsareincloseaccordancewith
the judgments of participants in Medin and Schaffer’s (1978)
experiment,witharank-ordercorrelationof .87.
Anderson’smodelusedaparticularlysimpleinferencealgo-
rithmfortheCRP,allocatingeachstimulustotheclusterwith
highestposteriorprobabilitybasedonpreviousallocations(as
alsodepictedinFig.1).Subsequentworkhasextendedthismodel
toaccommodatedifferentinferencealgorithms(Sanbornetal.,
2010)andsharingofclustersacrosscategories(Griffithsetal.,
2007),applyingtheresultingmodelstoavarietyofresultsin
humancategorylearning(forareview,seeGriffithsetal.,2008).
TheCRPhasvariousdesirablepropertiesthatmakeitasen-
sible choice as a prior over clusters. In the infinite space of
possibleclusterings,itfavorsassigningdatatoasmallnumber
ofclusters.Theexpectednumberofclustersgrowsslowlyasthe
numberofexperiencesbeingclusteredincreases.Inparticular,
theCRPdisplays‘‘preferentialattachment’’ora‘‘rich-get-richer’’
dynamic,whereaclusterwithalargenumberofmembersis
morelikelytogrowfurther.Theresultingdistributionsofcluster
sizes(‘scale-free’distributions,wherethesizeofclustersdecays
asapowerlaw)havebeenshowntobeprevalentacrossseveral
otherdomains(Adamic&Huberman,2002;Barabási&Albert,
1999;Mandelbrot,1960;Rosen&Resnick,1980).
AnotherpracticalreasonforthesuccessoftheCRPisthatit
isagnostictotheorderofdatapresentation(i.e.exchangeable;
Aldous,1985)—changingtheorderofpresentationofexperiences
doesnotchangetheprobabilityoftheirclustermemberships.
ThismakesBayesianinferencemoretractable,asitiseasyto
computeconditionaldistributionsthatarerequiredforstandard
inferencealgorithms(see,e.g.,Gershman&Blei,2012).
2Intuitively, with a fixed probability that two objects belong to the same
cluster(c,thecouplingconstant inAnderson(1991)),itismorelikelythatthe
newobjectshouldbeinthesameclusterasanobjectthatalreadyhasseveral
objectsinit.ThisresultsintheCRPassignmentrulesoutlinedabove,withthe
couplingconstantscalinginverselywith α:α=(1−c)/c.
2
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
Fig. 2.Schematicillustrationoftheclusteringproblem.Inthisschematic,werepresenteachobjectasablueballandeachclusterasabucket.Eachobjecthasno
featuresexceptitsuniqueindex.(A)Wewanttoclusterobject4,conditionedonhowobjects1,2,and3wereclustered.Therearethreepossibilities(indicated
bytheorangearrows):assigntobucket1whichcontainstwoobjectsalready,assigntobucket2withoneobject,orstartanewclusterbyassigningtooneof
theemptybuckets.Inthiswork,theprioroverclusteringsspecifiestheseconditionalprobabilities.(B)Byiteratingtheseconditionalprobabilities,wecanderivea
probabilitydistributionovertherangeofpossiblefinalclusteringsofallobjects(inthiscase5objectsintotal).Thesefinalclusteringsarerepresentedhere,aswell
ashowtheydifferintheentropyofthemarginaldistributionoverclusters.Aprioroverclusteringsspecifiesadistributionoverthesedifferentsolutions.
In addition to these practical properties of the CRP, there
are other reasons why human minds might use this particu-
larpriordistributionforclustering.Initsfirstuseinpsychol-
ogy,Anderson(1991)derivedtheCRPfromtheassumptionthat
anytwoobjectsmusthavethesamefixedpriorprobabilityof
being in the same cluster. This is related to exchangeability,
and might be an assumption justified by the environment in
whichhumansoperate.Intheremainderofthepaperwepur-
sue another hypothesis about the appropriateness of the CRP
forcognitivemodeling:thatCRP-likeclustering(aswellasits
variousproperties,likeexchangeability)mightbeemergentprop-
ertiesofaresource-rationaltendencytobestutilizealimited
representationalbudget.
3. Resource-rational clustering
Asnotedabove,theCRPhasbeenusedtomodelclustering
problemsthatariseinavarietyofdomains.FollowingAnderson’s
(1991)originalapplicationwewillfocusonthecasewherethe
agentseekstoorganizeasetofobjectsintoclusterstosupport
theircategorization(seeFig.2).Weformalizethisproblemas
follows.Wehaveatotalof Nobjects.Sinceweareconcernedwith
examiningtheprioroverclusterings(i.e.,howeachobjectshould
beassignedtoaclusterintheabsenceofanyspecificfeatures),
we assume that these objects do not have any distinguishing
featuresexceptfortheirindex i∈ [0,N].Thegoalistoorganize
theseobjectsintoclusters.Wedonotknowapriorihowmany
clusterstheobjectswillbesortedinto,buttheywillcertainlybe
nomorebythenumberofobjects N.Wethereforeneedtolearna
mapping πfromobjectojforj∈ [0,N)toclustercifori∈ [0,N).
Foranagentwithfinitecognitiveresources,itwillbeimpor-
tanttorepresenttheseobjectsinassimpleawayaspossible
whileallowingforthepotentialdifferencesbetweenthem.We
willderiveapriorbasedonthisideaoflearning‘simpler’map-
pings π,andshowthatthissimplicitypriorcorrespondsclosely
totheCRP.Butfirst,howdowemeasuresimplicity?
3.1. Measuringsimplicityorcomplexity
Ahumanpreferencetowardsimplicity,orOccam’srazor,has
beenusedtoexplainseveralcognitivephenomenainperception,
learningandhigh-levelcognition(Chater&Vitányi,2003),with
theuseofinformationtheorytodefinethis‘simplicity’(e.g.,Bhui
&Gershman,2018;Gottwald&Braun,2019;Olshausen&Field,
1996;Ortegaetal.,2015;Todorov,2009;Zenonetal.,2019).We
followinthistraditionandusethelengthofthecoderequired
to represent a mapping πas a measure of its complexity (as
alsoseenin Chater,1996;Feldman,2016).Alonger(i.e.morecomplex)codehashigherrepresentationalcost.Wemakethis
moreprecisebelow.
Wefirstcomputeanintermediatequantity,themarginaldis-
tributionovercategoriesgivenamapping π:
Pπ(ci)=∑N
jI[π(oj)==ci]
N(1)
Eachmapping πgivesaprobabilitydistributionovercategories.
Wethendefinesimplicityorcomplexityofthisprobabilitydis-
tribution.Whatmakesonedistributionoverclustersmoreorless
complexthananother?
Theentropyofthedistributioncanactasameasureofits
representationalcostandtherebyofitscomplexity.Itisgivenby:
H(π)= −N∑
iPπ(ci)logPπ(ci) (2)
Theinformation-theoreticinterpretationoftheentropyofadis-
tributionisthatitmeasurestheaveragenumberofbits(binary
coinflips)requiredtoconveyanobjectsampledfromthatdistri-
bution,underthemostefficientcodepossible.Thenumberofbits
requiredforci,orthelengthofits‘codeword’,islog Pπ(ci)(under
themostefficientcode;Shannon,1948).Weightingthiscode-
wordlengthbytheprobabilityofeachtokengivestheentropy
ofthedistribution.Intuitively,thismeasureshowdifficultitis
(i.e.howmanybitsofinformationarerequired)toconveywhich
object is sampled, when randomly sampling objects from the
givendistribution,toanobserverthatknowsthedistributionbut
doesnotknowwhichspecificobjectwassampled.Arepresenta-
tionally‘expensive’or‘complex’distributionisonethatrequires
moresuchbits.
Inusingentropyasameasureofrepresentationalcomplexity,
wearefollowingpreviousworkinbothpsychologyandneuro-
science.Workonplanningandsequentialdecision-makinghas
used entropy as a measure of representational cost (Todorov,
2009),andotherworkhassuggestedthatatendencytominimize
thisinformation-theoreticcostiswhatcharacterizesbounded-
rationalbehaviorinagentswithlimitedresources(Olshausen&
Field,1996;Ortegaetal.,2015).Thistendencyhasbeenempiri-
callyvalidated,andusedtomodelneuralrepresentations(Laugh-
lin, 1981) as well as human behavior in high-level cognitive
tasks(Bhui&Gershman,2018).
Howdoesthismeasuremapontoourintuitionsinthisdo-
main? Thelowestentropydistributionisthedistributionthat
allocatesallofitsprobabilitytoasingleoutcome.Here,theen-
tropyiszero,sincesamplesfromthedistributionarealwaysthe
same—thereisnoinformationtobetransmittedaboutaspecific
sample.Ontheflipside,thehighestentropydistributionisone
thatisuniformoveralloutcomes.Here,sincealloutcomesare
3
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
equallylikely,eventhemostefficientcodehastoconveywhich
ofseveralpossibilitieswasactuallychosenatagivensample.
Otherdistributionsfallinbetween,asmeasuredbyEq.(2).Inthe
contextofourclusteringproblem,thismapsontotheintuition
thatitiseasytoremembertoalwaysputeveryobjectinthesame
cluster(alowentropydistribution,lowerrepresentationalcost),
buthardertorememberdifferentclustersforeachobject—with
theextremebeingtohaveaseparateclusterforeachobject(a
highentropydistribution,higherrepresentationalcost).
Wewanttousethismeasureofcomplexitytoinformaprob-
ability distribution P(π) over mappings π. We do have some
representationalcapacityandwouldliketobestutilizeit—we
donotwanttosimplyminimizetheentropyanddefaulttothe
zeroentropy π.Insteadweassumesomefixedaveragerepre-
sentationalcapacity—thismeansthatthemeanentropyaveraged
overallclusterings(weightedby P(π))isfixed.Sincethenumber
ofpossibleclustersisinfinite,theentropycangrowarbitrarily
large.Theprobabilityofthesehighentropydistributionsmust
becorrespondinglylowtoaccommodatefiniterepresentational
capacityinexpectation.Wethereforewillpreferlowcomplexity
(low entropy) mappings over higher complexity ones. Exactly
how should this preference decay as a function of complex-
ity/entropy?Thenegativeexponentialisthemaximumentropy
distributionforafixedmean(notethat‘entropy’in‘maximum
entropy’herereferstotheentropyofthepriorprobabilitydistri-
butionP(π)—notH(π)whichistheentropyofthemapping π).
Thisgives:
P(π)=exp(−kH(π))∑
π′exp(−kH(π′))
∝exp(−kH(π)) (3)
wherekisapositiveconstantandthenormalizingfactorsums
overallpossiblemappings π′.Itis(bytheprincipleofmaxi-
mumentropy;Shore&Johnson,1980)themostgeneral,least
informativedistributiongivenafixedmean,i.e.afixedaverage
representational capacity. This is the same logic used to de-
riveotherassumption-freedistributionswithfixedmeane.g.the
negativeexponentialfreeenergyfunctional(Ortegaetal.,2015).
Wehavethereforespecifiedapriorprobabilitydistribution
P(π) over different clusterings πfor when we have a fixed
meanrepresentationalcapacity,whererepresentationalcostofa
mappingisgivenby H(π).Inthefollowingsections,weshowthat
theCRPcorrespondstoexactlysuchaprobabilitydistribution.
3.2. TherelationshipbetweentheCRPandentropy
WefirstdiscussthekeypropertiesoftheCRP.Thekeyprop-
ertyoftheCRPisthewayanewobjectisaddedtoanexisting
clusteringofstates:
1.Assign it to an existing cluster with probability propor-
tionaltothenumberofitemsalreadyinthecluster.
2.Assignittoanewclusterwithfixedprobability α.
This ‘‘rich-get-richer’’ or ‘‘preferential attachment’’ also arises
when trying to reduce entropy. Adding an object to a cluster
that already has high probability reduces the entropy of the
distributionbymakingitpeakier.Ontheotherhand,addingit
toalesspopulatedonemovesthedistributionclosertouniform,
increasingitsentropy.Therefore,addinganewobjecttoacluster
thatalreadyhasmanyobjectsinitresultsinlesscostforrep-
resentingthatdistributionthanaddingittoonethathasfewer
objects.
We can formalize this intuition. By Eq. (3), the entropy of
amappingspecifiesitspriorprobability.Wecancomputethe
entropiesofallthemappingsthatarisefromdifferentpossibleassignment of a new object to an existing mapping. Inserting
theseentropiesintoEq.(3)specifiesaprobabilitydistribution
over the possible assignments of the new object. This way of
assigningnewobjectstoclustersisnotarbitrary.Rather,itisnor-
mative,undertheresource-rationalassumptionthatwewantto
bestutilizelimitedrepresentationalresources.Wetherebyprefer
mappingswithlowcomplexity(andhencelowrepresentation
cost),seeSection3.1fordetails.
Inthefollowingsection,weprovidethemathematicaldetails
oftheaboveprocedure.
4. A mathematical derivation
In this section, we show that assigning new objects based
onprobabilityunderEq.(3)recoverstheCRP’snewobjectas-
signmentruleswhenthenumberofobjectsbeingclassifiedis
reasonablylarge.
4.1. ConditionaldistributionsoftheCRPandentropy-basedpriors
Givenaclusterassignment π,anobjectisaddedtogive π′.
Thiscanbesplitintotwocases,wheretheobjectiseitheradded
toclusterjtogive πjoritisaddedtoanewcluster(i.e.onewith
nj=0)togive π0.FormalizingtheCRPintheseterms:
Pcrp(πj|π)∝nj
Pcrp(π0|π)∝α (4)
The entropy-based prior over possible cluster assignments
(as given in Eq. (3)) is proportional to the negative exponent
of the entropy of the mapping π. The equivalent conditional
distributionsforthisprioraregivenby(detailsinAppendixA):
Pentropy(πj|π)∝exp(
−k(Hj−H))
Pentropy(π0|π)∝exp(
−k(H0−H))
(5)
wheretheentropyofamapping(specifyingaclusterassignment
ofNobjects,witheachclustercontaining niobjects)isgivenby:
H= −N∑
ini
Nlogni
N(6)
Inthenextsection,wecomputethedifferences Hj−HandH0−H
intermsofnjtomoredirectlycomparetheconditionalsderived
fromtheCRPandtheentropy-basedprior.
4.2. Theeffectofaddinganobjectonentropy
Whenaddinganewobject,wecanaddittoanexistingcluster
jtogive Hj.
Hj= −K∑
i\jni
N+1logni
N+1−nj+1
N+1lognj+1
N+1
Wetakethedifferencewith HinEq.(6)andseparateoutthe
termsindependentof nj(denotedE)fromthosedependenton
nj.Thissimplifiestothefollowing(seeAppendixBfordetailed
steps):
Hj−H=E−njlog(
1+1
nj)
N+1−log(nj+1)
N+1
Wethentakethelarge Nlimitandconsideronlytheleading
orderterms.Todecidewhichofthesetermsareleading,weneed
tomakeanassumptionabouttherelationbetweenthenumberof
objectsinacluster( nj)andthetotalnumberofobjects( N).We
maketheweakassumptionthattheaveragenumberofobjects
4
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
inaclustergrowssub-linearlywiththetotalnumberofobjects—
thisholdsaslongas(a)notallobjectsareassignedtothesame
clustersor(b)notallobjectsareassignedtoanewcluster.In
otherwords,both njandNgrowwhenNislarge,but njgrows
slower.WecanthereforeTaylorexpandthefirsttermandkeep
onlyleadingorderterms.Thissimplifiestothefollowing,detailed
stepsinAppendixB:
Hj−H∼E−log(nj)
N−1
N(7)
Wehavesofarderivedthechangeinentropywhenweaddan
objecttoanexistingclusterwith njobjects.Thechangetoentropy
whenweinsteadaddanobjecttoanemptyclusterisgivenby
nj=0(beforeapplyinganyapproximationsinthelarge Nlimit):
H0−H=E
4.3. Fromentropytoprobabilitydistribution
Substituting Hj−HandH0−Hinto the expressions for
theconditionaldistributionsoftheentropy-basedprior(Eq.(5))
gives:
P(πj|π)∝exp(
−k(Hj−H))
=exp[
−k(
E−log(nj)
N−1
N)]
Wefixk=N;thisconstantk(fromEq.(3))isthereforenota
parameter.
P(πj|π)∝exp(−NE)×nj×e
∝nj
Similarly,forwhenweareaddingtoanewcluster:
P(π0|π)∝exp(
−N(H0−H))
=exp(−NE)
Wecanthennormalizetheprobabilityofthenewclusteringsas
follows:
P(πj|π)=P(πj|π)
P(π0|π)+∑
iP(πi|π)=nj
N+1/e
P(π0|π)=P(π0|π)
P(π0|π)+∑
iP(πi|π)=1/e
N+1/e
ThisisequivalenttoaCRP(asspecifiedinEq.(4))with α=1/e≈
0.36.NotethatwecangetacorrespondingCRPwithadifferent α
bytakingthelogarithminEq.(2)andtheexponentinEq.(3)with
respecttoadifferentbase.Thebaseofthelogarithmisrestricted
tobegreaterthan1,toensurethatthelogarithmisanincreasing
function, but is arbitrary beyond this constraint. We can thus
derivethefullfamilyofCRPdistributionsfor0 < α <1.
4.4. Summary
Ourgoalwastoexaminetheconsequencesoflimitedcognitive
resourcesontheclusteringprocess.Wefindthatapriorover
clustersproportionaltothenegativeexponentoftheentropyof
theclustermappinggivesCRP-likeclustering.Thispriorstrongly
prefers lower entropy mappings to higher entropy ones. This
indicatesthatCRP-likeclusteringmightcomefromatendencyto
reducerepresentationalburden.Inotherwords,CRP-likecluster-
ingcancomefromabiastowardlearning‘simple’object-category
mappings, where simplicity is defined as the entropy of the
marginalovercategories.Thisprovidesaprocess-leveltheoryfor
whyCRP-likepriorsmightbeappropriateformodelinghuman
cognition.5. Simulations
Our mathematical results establish a direct correspondence
betweenlimitedrepresentationalcapacityandtheCRP,inthe
limitsofalargenumberofobjects.Todeterminewhetherthe
clusteringproducedbythisresource-rationalclusteringscheme
producesresultssimilartothoseexpectedfromaCRPwithre-
alisticsamples,weconductedaseriesofsimulationswherewe
generated cluster assignments for both distributions and then
analyzethecorrespondence.
ThecorrespondencebetweentheCRPandtheentropy-based
distributioniscloserasthenumberofobjects( N)increases.At
very lowNtherefore, these distributions deviate slightly (see
AppendixCfordetails).Sincesubsequentclusteringbehavioris
conditionedstronglyontheobjectassignmentsthusfar,these
differences can amplify. That is, even though the conditional
distributionsgetcloserwithhigher N,themarginaldistributions
deviate further. The resultant distribution is still qualitatively
verysimilartotheCRP(asdiscussedbelow),anditisaninterest-
ingdirectionoffutureworktoexaminewhetherthisdistribution
mightbetterdescribehumanclusteringbehaviorthantraditional
CRPs.Here,however,tovalidatethesimilaritieswiththeCRP,
wecontrolforthisdeviationbyclusteringthefirst Mobjects
accordingtoanexactCRP.
WeevaluatedthecorrespondencebetweentheCRPandour
resource-rationaldistributionbasedontwocriteria.First,aprop-
ertycharacteristicofscale-freedistributionsliketheCRPisthat
thesizesofthedifferentclustersdecayasapowerlaw.There-
fore,ifwesorttheclustersbysize,thelogarithmofthecluster
size(i.e.thefractionofthetotalnumberofobjectsthatarein
thatcluster)shouldbealinearfunctionofthelogarithmofthe
clusterindex.Second,anotherkeypropertyoftheCRPisthat
thenumberofclustersincreaseslogarithmicallywiththetotal
numberofobjects.Wecanalsomeasurethatforourentropy-
baseddistributionandexaminewhetherthenumberofclusters
isalinearfunctionofthelogarithmofthenumberofobjects.
5.1. Method
Wegeneratedsamplesfromourdistributionasfollows.We
first clusterMobjects according to a CRP with α=0.368,
varyingMbetween0and80.Wethenclusteranadditional106
objectsfromthisstartingpoint,eitherwiththeCRP’sconditional
clusterassignmentrules,orbasedontheentropyasspecified
byEq.(3).Weclustersuchalargenumberofobjectstogeta
reasonablenumberoftotalclusterssothatwecanbetteranalyse
thedistributionsofobjectsacrossthem—evenwiththesemany
objects,theaveragenumberoftotalclustersisaround6.This
procedureisrepeatedtoget20uniquesetofclustersforeach
clusterassignmentruleandeach M.
5.2. Results
Wefindthattheentropy-baseddistributionproducescluster-
ingsthathavestatisticalpropertiesthatcloselyresemblethose
fromtheCRP.Toprovideaninitialillustrationofthecorrespon-
dence, we focus on the case where M=20. Fig. 3A shows
how the size of the clusters decays as a power law over the
clusterindexrankedbysize.Fig.3Bplotstheseinlog-spaceand
highlightsthelinearrelationshipcharacteristicofapowerlaw
liketheCRP.
Itisespeciallyinstructivetocomparethislinearpatternwith
whatmightbeexpectedfromothersensiblepriorsoverclusters.
Forexample,oneswheretheclustersizedecaysexponentially
withrank(intuitively,eachclusterissomefixedfactor λsmaller
thanthenextlargestone).Thispriornotablydoesnotshowthis
5
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
Fig. 3.Simulationresults:(A)Clustersizescalesasapowerlawwithclusterrank.(B)Thisrelationshipishighlightedinlogspace.Theaveragebehaviorofthe
twoclusteringassignmentsresemblesthelinearfit(dottedlines).(C)Thenumberofclustersgrowslogarithmicallywiththenumberofobjects.(D)Thecorrelation
betweentheclustersizesfromthetwoclusteringsincreaseswith M,butlevelsoff.
Fig. 4.Alternative priors: Log cluster size as a function of log rank for an
exponential prior over ranked clusters. The dotted line shows a linear fit to
thedata:thisfitdoesnotexplainthisdatawell.Incomparison,Fig.3Bshows
thattheentropy-basedpriorfollowsthesamelineartrendastheCRP.
linearstructure(asplottedinFig.4).Itisthereforeparticularly
remarkablethatourentropy-derivedprioralsoshowsthislinear
trend.
Fig.3Cshowsthechangeintheaveragenumberofclusters
(overthe20runs)asafunctionofthelogarithmofthenumber
ofobjects.Weseethatthisgrowsclosetolinearly,withthelinear
fit(dottedline)closelymatchingthedata.Thedeviationismost
apparentatsmallernumbersofobjects,asexpectedsincethe
linearrelationshipisexpectedinthelimit.Thenumberofclusters
fromtheCRPisslightlybutnotstatisticallysignificantlyhigher
thanthatfromtheentropy-basedclustering.
Toprovideamoredetailedpictureofthiscorrespondence,we
examinedthecorrelationbetweentheclustersizes(i.e.thedata
plottedinFig.3A,matchedbyclusterindex)forthefirst500objects.Clusterassignmentisstochasticandwecannotexpect
exactcorrespondence.Togetasenseoftheupperlimitofthe
correlationwecanexpectfromthismeasure,wefirstcorrelate
theclustersizesderivedfromindependentrunsofthesameclus-
teringalgorithm,repeated4times.Thisgivesacorrelationof0.72
(95%CI [0.69,0.74])fortheCRP,and0.74(95%CI [0.71,0.76])
forentropy-basedclusteringwith M=80.Thisshowsthateven
withtheclustersizesproducedbythesamealgorithm,wecannot
expectacorrelationof1.
We then look at the correlation between the CRP and the
entropy-basedclusteringafterinitializingwithvariable M.We
expectthiscorrelationtoimproveas Mincreases.Weseethat
thisisindeedthecase(Fig.3D),withthecorrelationat M=
80 being comparable to the correlation between two runs of
thesameclusteringalgorithm(CRPorentropy-based).Wealso
seethatthedifferenceincorrelationfromsmalltolarge Mis
notverydramatic(variesfrom ∼0.58to ∼0.73)andappearsto
leveloff.ThisindicatesthatthecorrespondencebetweentheCRP
andtheentropy-basedpriorisfairlyrobusttothevalueofthe
initializationM.
6. Discussion
Needingtoclusterexperiencestogetherisaubiquitousaspect
of human cognition. In this paper, we have approached this
problemfromtheperspectiveofrationalanalysis,askinghowan
idealagentshouldseektousetheirlimitedcognitiveresources.
Ourresultsshowthatthesolutiontothisproblem,whenthose
resources are expressed in information-theoretic terms, has a
directcorrespondencewithanapproachtoclusteringthathas
beenwidelyusedinprobabilisticmodelsofcognition(theChi-
nese restaurant process, or CRP). These results provide a new
cognitively-motivatedjustificationforthatassumption.
6
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
Ourfindingssuggestinterestingdirectionsforfutureempirical
work.IfCRP-likeclusteringcomesfromrepresentationalcosts,
manipulating these costs should result in different clustering
behaviors.Ourmodelpredictsthathavingmorelimitedcogni-
tiveresourcesshouldaffectclusteringbehavior,drivingtoward
a lower entropy representation and a stronger preference for
few,large,clusters.GershmanandCikara(2021)modeltheef-
fectsofcognitiveloadonstructurelearningasareductionin
theconcentrationparametergivingfewerclusters.Ourapproach
providestheoreticjustificationforwhyfewercognitiveresources
(e.g.undercognitiveload)shouldgiverisetofewerclusters.This
wouldnotbepredictedbyatraditionalCRPmodel,sinceitisa
consequenceofthecognitiveresourcesavailableandnotachange
inthebeliefsoftheagentabouttherelativepriorprobabilityof
differentclusterings.
Inourpaper,wedonotassumethatthedataaregenerated
fromaground-truthsetofclusters,ratherthatclusteringarises
solelyattherepresentationallevelfromlimitationsincapacity.
Correspondingly,wemakenoassumptionsaboutthelikelihood
functionthatinformswithin-clusterstructure–wefocusentirely
ontheapriorinumberofclusters,assumingthedatahavenodis-
tinguishingfeaturestoclusteronthebasisof.However,theseare
crucialaspectsofreal-worldclusteringbehaviorandfuturework
shouldlooktowardhowtheyinteractwithaprioriclustering
drivenbyrepresentationallimitsaspositedhere.Acommoncrit-
icismofBayesianmodelsofcognitionistheirlackofgroundingin
process-levelconsiderations,andtheriskthatthepriorsspecified
inthesemodelscanbearbitrarilychosenbypractitionerstofit
data(Bowers&Davis,2012;Jones&Love,2011).Ourworkexem-
plifiesonewaytospecify‘effectivepriors’thatareinformedand
constrained byalgorithmic considerations—andin factdirectly
arisefromresourcelimitationsatthisalgorithmiclevel.Further,
posteriorinferencewitharbitrarypriorsisoftencomputationally
intractable;thisapproachalsosuggestsatractableprocess-level
model.Thisraisesthebroaderquestionoftheepistemicroleof
thepriorinBayesianmodelsofcognition—whetheritrepresents
pre-existingknowledge,oremergentpropertiesofthealgorithm.
Ourworkhighlightsthatthisdifferencecanbenuanced.
Anopenquestioniswhetherotherubiquitouspriorsassumed
inprobabilisticmodelsofhumancognitionmightinsteadarise
fromthealgorithmicprocessesinvolvedinlearningandrepre-
sentation.Variouspriorsoverneuralnetworkmodelshavebeen
showntobeeffectivelyimplementedbyestablishedalgorithmic
approacheslikeweightdecay(Krogh&Hertz,1991),earlystop-
ping(Duvenaudetal.,2016),anddropout(Gal&Ghahramani,
2016).Thefieldofprobabilisticnumerics(Hennigetal.,2015)
hasalsoshownthatseveralclassicapproximatealgorithmsin
quadrature, linear optimization, and solving differential equa-
tionscanbereinterpretedasexactsolutionsunderspecificpri-
ors.Theseapproaches(e.g.neuralnetworks,linearoptimization)
arecommonlyusedinprobabilisticmodelsofcognition.Explor-
ingthisduality(betweenalgorithmicprocessandcomputational
prior)withintheseapproaches–andthereforetherolethese
approachesplayinmodelingcognition–isapromisingdirection
offuturework.
Acknowledgments
ThisworkwasdonewhileIDwasattheDepartmentofCom-
puterScience,PrincetonUniversity.WearegratefultoMichael
Chang,JonCohen,ZackDulberg,NoémiÉltető,SamGershman,
MarkHo,EricSchulz,SimonSegert,andShuchenWuforhelpful
discussions.ThisworkwassupportedbyDARPA,UnitedStates
ofAmericagrantFA8650-18-2-7832andAFOSR,UnitedStatesof
AmericagrantFA9550-18-1-0077.Appendix A. Conditional distributions for the entropy-based
prior
Wegivefurtherdetailonhowtoderivetheconditionaldistri-
butionsfromthenegativeexponentialprobabilityovermappings
π.Westartwith:
P(π)=exp(−kH(π))∑
π′exp(−kH(π′))
∝exp(−kH(π))
Addinganobject oitoclusterjgives πj.Theprobabilityofthis
newmappingisgivenby P(πj)∝exp(−kH(πj)).Wecanthen
considertheprobabilityof P(πj)conditionedonalreadyhaving
themapping π.
P(πj|π)=P(πj, π)
P(π)
Notethat πjspecifiesasupersetof π,i.e.πspecifiesthecluster
mappingofobjects o0. . .oi−1,while πjadditionallyspecifiesthe
mappingofoi.SoP(π, πj)=P(πj).Thisreducestheconditional
distributionto
P(πj|π)=P(πj)
P(π)∝exp(−k(H(πj)−H(π)))
IntermsofCRPnotationthatdirectlyrepresentstheprobabil-
ities of the cluster assignments zi=π(oi), we haveP(zi=
j|z1, . . . ,zi−1)=P(πj|π).
Appendix B. Simplifying the difference in entropy
Thedifferenceintheentropybetweenthenewandtheold
distributionsis:
Hj−H=K∑
i\jni(
1
N−1
N+1)
logni
+nj
Nlognj−nj+1
N+1log(nj+1)
+log(N+1)−logN
We want to separate out the terms dependent on nj, so we
separateoutthefirsttermas:
K∑
i\jni(
1
N−1
N+1)
logni
=K∑
inilogni
N(N+1)
−njlognj
N(N+1)
Thedifferencethereforereducestothefollowingterms,with E
representingthetermsindependentof nj:
Hj−H=E+nj
Nlognj−nj+1
N+1log(nj+1)−nj
N(N+1)lognj
Here,thetermindependentof nj,denotedE,isgivenby:
E=K∑
inilogni
N(N+1)+log(N+1)−logN
Wefurthersimplifythetermsdependenton nj:
nj
Nlognj−nj+1
N+1log(nj+1)−nj
N(N+1)lognj
=Nnjlognj+njlognj−Nnjlog(nj+1)−Nlog(nj+1)−njlognj
N(N+1)
7
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
Fig. C.5.Evaluating the approximation: We plot the terms in the exponents
ofEq.(C.1)asafunctionofincreasingnumberofobjectsinacluster( nj).Wesee
thatthedifferencebetweentheapproximationandexactvaluereducesquickly.
= −1
N(N+1)[
Nnjlog(
1+1
nj)
−Nlog(nj+1)]
= −njlog(
1+1
nj)
N+1−log(nj+1)
N+1
Taylorexpandingthefirsttermtoleadingordergives:
−njlog(
1+1
nj)
N+1≈ −nj(
1
nj−1
2n2
j)
N+1
InthelargeNlimit,andcorrespondinglylarge njlimit,weassume
N≈N+1andignorenon-leadingterms,toget:
−nj(
1
nj−1
2n2
j)
N+1=−1
N+1+1
2nj(N+1)≈ −1
N(B.1)
Appendix C. Evaluating the large N approximation
HerewerevisittheapproximationmadetoarriveatEq.(7)or
Eq.(B.1).Ifwehadnotmadetheapproximationrequiredtoelim-
inatetheextraterm,wewouldhaveanadditionaldependenceon
njasfollows:
P(πj)|π∝exp(
−N(Hj−H))
=exp(−NE)×nj×exp(njlog(1+1/nj))
Inoursimplification,wearemakingthefollowingapproxima-
tion:
exp(njlog(1+1/nj))∼exp(1) (C .1)
WeplottheseexponentsinFig.C.5togiveasenseforwhen
thisisagoodapproximation.Weseethatevenatsmall nj,the
valuesarerelativelyclose,withtheapproximationconverging
quickly.
Werestrictouranalysestothecorrespondenceofthecondi-
tionaldistributions P(πN|πN−1)betweentheentropy-baseddis-
tributionandtheCRP,ratherthandirectlyexaminingthejoint
distributionP(πN).Thisisbecausecomputingthenormalization
factorfortheconditionaldistributionfortheentropy-baseddis-
tribution(beforemakingtheapproximationabove)dependson
thedistributionofobjectsinthepreviousstep—unlikeafterwe
make the approximation when the normalization factor goes
toNe+1.Thismakesthepre-approximationjointdistribution
difficulttocompute.References
Adamic,L.A.,&Huberman,B.A.(2002).Zipf’slawandtheinternet. Glottometrics,
3(1),143–150.
Aldous, D. J. (1985). Exchangeability and related topics. In École d’été de
probabilitésdesaint-flourXIII—1983,Vol.1117 (pp.1–198).Springer.
Anderson,J.R.(1991).Theadaptivenatureofhumancategorization. Psychological
Review,98(3),409–429.
Barabási,A.L.,&Albert,R.(1999).Emergenceofscalinginrandomnetworks.
Science,286(5439),509–512.
Bhui, R., & Gershman, S. J. (2018). Decision by sampling implements efficient
codingofpsychoeconomicfunctions.. PsychologicalReview ,125(6),985–1001.
Bowers,J.S.,&Davis,C.J.(2012).Bayesianjust-sostoriesinpsychologyand
neuroscience..PsychologicalBulletin ,138(3),389.
Chater,N.(1996).Reconcilingsimplicityandlikelihoodprinciplesinperceptual
organization..PsychologicalReview ,103(3),566.
Chater, N., & Vitányi, P. (2003). Simplicity: a unifying principle in cognitive
science?TrendsinCognitiveSciences ,7(1),19–22.
Duvenaud,D.,Maclaurin,D.,&Adams,R.(2016).Earlystoppingasnonparametric
variationalinference.In Artificialintelligenceandstatistics (pp.1070–1077).
PMLR.
Feldman,J.(2016).Thesimplicityprincipleinperceptionandcognition. Wiley
InterdisciplinaryReviews:CognitiveScience ,7(5),330–340.
Gal,Y.,&Ghahramani,Z.(2016).Dropoutasabayesianapproximation:Repre-
senting model uncertainty in deep learning. In International Conference on
MachineLearning (pp.1050–1059).PMLR.
Gershman, S. J., & Blei, D. M. (2012). A tutorial on Bayesian nonparametric
models.JournalofMathematicalPsychology ,56(1),1–12.
Gershman,S.J.,Blei,D.M.,&Niv,Y.(2010).Context,learning,andextinction.
PsychologicalReview ,117(1),197.
Gershman,S.J.,&Cikara,M.(2021).Structurelearningprinciplesofstereotype
change.http://dx.doi.org/10.31234/Osf.Io/52f9c,PsyArXiv.
Gershman,S.J.,Horvitz,E.J.,&Tenenbaum,J.B.(2015).Computationalrational-
ity:Aconvergingparadigmforintelligenceinbrains,minds,andmachines.
Science,349(6245),273–278.
Goldwater, S., Griffiths, T. L., & Johnson, M. (2009). A Bayesian framework
forwordsegmentation:Exploringtheeffectsofcontext. Cognition,112(1),
21–54.
Gottwald, S., & Braun, D. A. (2019). Bounded rational decision-making from
elementarycomputationsthatreduceuncertainty. Entropy,21(4),375.
Griffiths, T. L., & Austerweil, J. (2008). Analyzing human feature learning as
nonparametricBayesianinference. AdvancesinNeuralInformationProcessing
Systems,21,97–104.
Griffiths,T.,Canini,K.,Sanborn,A.,&Navarro,D.(2007).Unifyingrationalmodels
ofcategorizationviathehierarchicalDirichletprocess.In Proceedingsofthe
annualconferenceofthecognitivesciencesociety .
Griffiths, T. L., Lieder, F., & Goodman, N. D. (2015). Rational use of cognitive
resources:Levelsofanalysisbetweenthecomputationalandthealgorithmic.
TopicsinCognitiveSciences ,7(2),217–229.
Griffiths,T.L.,Sanborn,A.N.,Canini,K.R.,&Navarro,D.J.(2008).Categorization
as nonparametric Bayesian density estimation. In N. Chater, & M. Oaks-
ford (Eds.),The Probabilistic Mind: Prospects for Bayesian Cognitive Science
(pp.303–328).Oxford,UK:OxfordUniversityPress.
Hennig, P., Osborne, M. A., & Girolami, M. (2015). Probabilistic numerics and
uncertaintyincomputations. ProceedingsoftheRoyalSocietyA:Mathematical,
PhysicalandEngineeringSciences ,471(2179),Article20150142.
Hjort, N. L., Holmes, C., Müller, P., & Walker, S. G. (2010). Bayesian
nonparametrics.CambridgeUniversityPress.
Jones,M.,&Love,B.C.(2011).Bayesianfundamentalismorenlightenment?On
theexplanatorystatusandtheoreticalcontributionsofBayesianmodelsof
cognition.BehavioralandBrainSciences ,34(4),169.
Kemp,C.,Tenenbaum,J.B.,Niyogi,S.,&Griffiths,T.L.(2010).Aprobabilistic
modeloftheoryformation. Cognition,114(2),165–196.
Krogh,A.,&Hertz,J.(1991).Asimpleweightdecaycanimprovegeneralization.
AdvancesinNeuralInformationProcessingSystems ,4.
Laughlin,S.(1981).Asimplecodingprocedureenhancesaneuron’sinformation
capacity.ZeitschriftfürNaturforschungC ,36(9–10),910–912.
Lieder, F., & Griffiths, T. L. (2019). Resource-rational analysis: Understanding
human cognition as the optimal use of limited computational resources.
BehavioralandBrainSciences ,43,1–85.
Mandelbrot, B. (1960). The Pareto-Levy law and the distribution of income.
InternationalEconomicReview ,1(2),79–106.
Medin,D.L.,&Schaffer,M.M.(1978).Contexttheoryofclassificationlearning.
PsychologicalReview ,85(3),207.
Neal,R.M.(1998). MarkovchainsamplingmethodsforDirichletprocessmixture
models:Technicalreport9815 ,DepartmentofStatistics,UniversityofToronto.
Olshausen,B.A.,&Field,D.J.(1996).Naturalimagestatisticsandefficientcoding.
Network.ComputationinNeuralSystems ,7(2),333–339.
Ortega, P. A., Braun, D. A., Dyer, J., Kim, K. E., & Tishby, N. (2015).
Information-theoreticboundedrationality.arXivpreprintarXiv:1512.06789.
8
I.DasguptaandT.L.Griffiths JournalofMathematicalPsychology109(2022)102675
Rosen,K.T.,&Resnick,M.(1980).Thesizedistributionofcities:anexamination
oftheParetolawandprimacy. JournalofUrbanEconomics ,8(2),165–186.
Sanborn,A.N.,Griffiths,T.L.,&Navarro,D.J.(2010).Rationalapproximations
torationalmodels:alternativealgorithmsforcategorylearning. Psychological
Review,117(4),1144–1167.
Shannon,C.E.(1948).Amathematicaltheoryofcommunication. TheBellLabs
TechnicalJournal ,27(3),379–423.Shore,J.,&Johnson,R.(1980).Axiomaticderivationoftheprincipleofmaximum
entropyandtheprincipleofminimumcross-entropy. IEEETransactionson
InformationTheory ,26(1),26–37.
Todorov,E.(2009).Efficientcomputationofoptimalactions. Proceedingsofthe
NationalAcademyofSciences ,106(28),11478–11483.
Zenon, A., Solopchuk, O., & Pezzulo, G. (2019). An information-theoretic
perspectiveonthecostsofcognition. Neuropsychologia ,123,5–18.
9