Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=sinq20
Inquiry
An Interdisciplinary Journal of Philosophy
ISSN: 0020-174X (Print) 1502-3923 (Online) Journal homepage: https://www.tandfonline.com/loi/sinq20
Existential risks: a philosophical analysis
Phil Torres
To cite this article: Phil Torres (2019): Existential risks: a philosophical analysis, Inquiry, DOI:
10.1080/0020174X.2019.1658626
To link to this article: https://doi.org/10.1080/0020174X.2019.1658626
Published online: 23 Aug 2019.
Submit your article to this journal
View related articles
View Crossmark data

Existential risks: a philosophical analysis
Phil Torres
Annandale, VA, USA
ABSTRACT
This paper examines and analyzes ﬁve de ﬁnitions of ‘existential risk. ’It
tentatively adopts a pluralistic approach according to which the de ﬁnition
that scholars employ should depend upon the particular context of use. More
speciﬁcally, the notion that existential risks are ‘risks of human extinction or
civilizational collapse’ is best when communicating with the public, whereas
equating existential risks with a ‘signiﬁcant loss of expected value ’may be the
most e ﬀective de ﬁnition for establishing existential risk studies as a legitimate
ﬁeld of scienti ﬁc and philosophical inquiry. In making these arguments, the
present paper hopes to provide a modicum of clarity to foundational issues
relating to the central concept of arguably the most important discussion ofour times.
ARTICLE HISTORY Received 2 October 2018; Accepted 12 July 2019
KEYWORDS Existential risks; analysis; global catastrophic risks; existential risk studies
Introduction
Theﬁeld of ‘existential risk studies ’was essentially founded by John
Leslie ’s 1996 book The End of the World: The Science and Ethics of Human
Extinction. However, it lingered in a ‘pre-paradigmatic ’state for several
years until the publication of Nick Bostrom ’s 2002 paper ‘Existential
Risks: Analyzing Human Extinction Scenarios and Related Hazards, ’
which drew heavily from Leslie ’s tome. Since the mid-2010s, numerous
organizations focusing on studying existential risks have sprouted up at
universities like Oxford and Cambridge, and multiple books on the topic
have been added to the scholarly literature, including Here Be Dragons:
Science, Technology, and the Future of Humanity (2016) and Morality, Fore-
sight, and Human Flourishing: An Introduction to Existential Risks (2017).
Yet there remains some disagreement, or at least a lack of agreement,
about the semantics of the ﬁeld’s central concept: existential risk. The aim
© 2019 Informa UK Limited, trading as Taylor & Francis GroupCONTACT Phil Torres philosophytorres@gmail.com 7816 Byrds Nest Pass, Annandale, VA 22003,
USAINQUIRY
https://doi.org/10.1080/0020174X.2019.1658626
of the present paper will be to examine the ﬁve most prominent
deﬁnitions employed by existential risk scholars (section 2) and then to
oﬀer some critical remarks about which de ﬁnitions ought to be used in
diﬀerent conversational settings (section 3). Ultimately, I will endorse a
context-dependent semantic pluralism whereby the best de ﬁnition to use
depends on the intended audience.1The fact is that existential risk
studies has an integral activist component: it aims not merely to generate
new knowledge about the mind-independent world, but to nontriviallyinﬂuence the developmental trajectory of human civilization. Thus, it is
essential that the ideas and insights outlined in the scholarly literatureon existential risks are e ﬀectively conveyed to non-experts, including poli-
ticians, policymakers, business leaders, and the general voting public. Thismeans that, while an agreed-upon technical vocabulary may economizediscussion between experts, it is equally important that non-esoteric
language is utilized when discussing the ideas and insights with those
not‘in the know. ’
The ultimate ambition of this paper is to o ﬀer a clear account of how the
term ‘existential risk’ has been used by scholars and some considered
thoughts about how it should be employed to best advance the normativeaims of the ﬁeld.
Five deﬁ nitions of ‘existential risk’
A risk is typically de ﬁned as the probability of an undesirable event multi-
plied by its consequences. The notion of an existential risk has nothing to
do with the ﬁrst multiplicand; all that ’s relevant are the consequences. This
section will examine ﬁve distinct de ﬁnitions of the term. These are existen-
tial risks as (i) human extinction, (ii) human extinction or civilizational col-lapse, (iii) human extinction or a permanent and drastic loss of potential,(iv) any catastrophe with pangenerational-crushing e ﬀects, and a signi ﬁ-
cant loss of expected value. Taking these in order:
Human extinction
An event X is an existential risk if and only if X could cause the extinction of
humanity. This is probably the most common de ﬁnition in the scholarly
1There is, as of this writing, a burgeoning literature on pluralism within the ﬁeld of philosophy. Of particu-
lar interest is Chalmers ( 2011 ), especially a particular version of the ‘method of elimination ’that Chal-
mers calls the ‘subscript gambit.’ (Thanks to an anonymous reviewer for calling my attention to this
paper.) I draw from this literature, although mostly implicitly.2 P. TORRES
literature and popular media. For example, Jason Matheny ( 2007 )d eﬁnes
existential risks as ‘a subset of catastrophic events –those that could extin-
guish humanity, ’and Karim Jebari ( 2015 ) asserts that existential risks
‘threaten the continued existence of mankind. ’Similarly, the journalist
Ross Anderson introduces an interview with Bostrom for The Atlantic like
this: ‘Bostrom, who directs Oxford ’s Future of Humanity Institute, has
argued over the course of several papers that human extinction risks are
poorly understood and, worse still, severely underestimated by society.
Some of these existential risks are fairly well known, especially the
natural ones ’(Anderson 2012 , italics added).
There are several reasons to accept this de ﬁnition. (a) The standard dic-
tionary de ﬁnes‘existential ’as‘of or relating to existence; involving or relat-
ing to the existence of a thing ’(OED 2018), which implies that ‘an
existential risk to humanity ’is‘a risk to the existence of the species
denoted by the word “humanity. ”’So, it aligns with common usage of
the term ‘existential. ’(b) Relatedly, it corresponds to ‘a natural division
and is easy to understand,’ as Owen Cotton-Barratt and Toby Ord (2015)
observe. Whereas phenomena like ﬂourishing are spectral, coming in gra-
dations, extinction is binary: A species is either existing or non-existing,extant or extinct, dead or alive. This makes it a non-vague concept,unlike ﬂourishing , and one that is objective in nature, in the sense that
the concept is non-axiological. (c) And ﬁnally, one could argue that the
term ‘extinction ’picks out a natural kind –speciﬁ cally, what Brian Ellis
(2001 ,2002 ) calls a ‘dynamic kind.’ It‘cuts
 the world at its joints, ’as it
were, and appears to be the sort of phenomenon that could participatein laws of nature. As it happens, almost no work within the philosophyof biology has examined the notion of species extinctions, althoughthere are no doubt many interesting issues to be dissected here.
But there are also several reasons to reject this deﬁ nition. Consider the
following two scenarios. Scenario 1: Imagine that Homo sapiens evolves
through anagenesis into a new species of ‘posthumans.’ This could
occur either naturally or through cyborgization, whereby technological
modiﬁ cations signi ﬁcantly alter ways our core capacities of cognition,
healthspan, and emotionality (see Bostrom 2008). The result would be
the‘phyletic extinction’ (or‘pseudo-extinction ’)o fHomo sapiens, and con-
sequently this transition would instantiate an existential risk on thedeﬁnition above. But now imagine that these posthumans have expanded
intellectual capacities, are free of all diseases, don ’t senesce, and are able
to experience degrees of happiness and pleasure that far surpass what anyhuman is acquainted with. While there are bioconservatives, such asINQUIRY 3
Frances Fukuyama ( 2002 ) and Leon Kass ( 2002 ), who would still oppose
the realization of this posthuman future, most people, I conjecture,
would see it as desirable. The problem is clear: If one maintains that, asBostrom puts it, ‘the objective of reducing existential risks should be a
dominant consideration whenever we act out of an impersonal concernfor humankind as a whole, ’then we should do everything necessary to
prevent Homo sapiens from becoming Posthomo supersapiens . But this
conﬂicts with the intuition that just the opposite should be the case, on
certain assumptions.
One solution is to rede ﬁne‘human ’as including what David Roden
(2014) refers to as our ‘wide descendants, ’which denotes ‘human,
machine, cyborg, etc. ’Thus, the term ‘human ’would encompass potential
posthumans that exhibit the right genealogical and axiological relation tocurrent humans.
2However, expanding the semantics of ‘human ’to count
future events in which our posthuman descendants die out would requireextra explication, as most non-experts no doubt associate the term withHomo sapiens rather than future species that share an unbroken evolution-
ary lineage with us and are related in the right sort of axiological way. Infact, a number of population ethicists have argued that the badness ofhuman extinction isn’ t just the loss of life caused by the extinction
event itself, but the loss of all the future beings who could have comeinto existence had we survived. Derek Par ﬁt and others have been explicit
that the relevant class of beings here includes posthumans; as he writes,‘life can be wonderful as well as terrible, and we shall increasingly have
the power to make life good. Since human history may be only just begin-ning, we can expect that future humans, or supra-humans, may achievesome great goods that we cannot now even imagine ’(Parﬁt2017 ). Thus,
if‘the destruction of mankind would be by far the greatest of all conceiva-
ble crimes ’(Parﬁt1984 ), the reason is partly because, if we survive, we
could undergo phyletic extinction by becoming a much superior kind ofspecies that achieves immense goods inscrutable to contemporary minds.
Scenario 2: Whereas the scenario above considers a future in which
Homo sapiens is anagenetically replaced by something better, now
imagine a future in which we develop along a ‘degenerate trajectory ’
that yields conditions of profound su ﬀering. Call such an outcome a
2The axiological constraint here aims to preclude scenarios in which current humans are supplanted by a
posthuman population whose moral and, more generally, value commitments are di ﬀerent than ours in
some problematic way. As Bostrom ( 2013 ) puts this point, ‘even if another intelligent species were to
evolve to take our place, there is no guarantee that the successor species would su ﬃciently instantiate
qualities that we have reason to value. Intelligence may be necessary for the realisation of our future
potential for desirable development, but it is not su ﬃcient.’4 P. TORRES
‘hyper-existential risk. ’Anders Sandberg ( 2014 ) appears to gesture at this
possibility when he writes that, ‘while there might be posthuman states of
great value, there are also potential existential risks threatening futures
with no or extremely negative value. Insofar as we can in ﬂuence what
future we might reach, we may have a far greater moral responsibilitythan is commonly envisioned because the stakes are higher.’ A related
idea is that of a ‘suﬀering risk, ’which David Althaus and Lukas Gloor
deﬁne as any event ‘that would bring about su ﬀering on an astronomical
scale, vastly exceeding all su ﬀering that has existed on Earth so far ’
(Althaus and Gloor 2018 ). Hyper-existential and su ﬀering risks are distinct
because, for example, the latter would include future con ﬁgurations in
which the total amount of pleasure exceeds the total amount ofsuﬀering, yet the total amount of su ﬀering is astronomical. It would also
seem to exclude situations in which, say, humanity fails to colonize
space, its population dwindles to 1 billion, and everyone alive experiences
constant torture-level misery until the oceans evaporate. Thus, thebadness of hyper-existential risks will almost certainly be less controversialthan the badness of su ﬀering risks, which are most compelling if one
champions a negative total utilitarian theory (or ‘suﬀering-focused
ethics ’) (see Gloor and Mannino 2018 ).
3The point is that there could be
future scenarios in which going extinct –in which a kind of species eutha-
nization –might be unambiguously desirable. Thus, once again, if one
maintains that existential risks should be avoided at all costs, then onewould oppose e ﬀorts to prevent our extinction even when the alternative
is a hyper-existential or su ﬀering catastrophe. This seems counter-intuitive.
In sum, the advantage of this de ﬁnition is that it is simple , but the draw-
back is that it is simplistic. I will have more to say about this in section 3.
3Incidentally, Bostrom appears to overlook these issues in one section of his 2013 paper about existential
risks. In defending his maxipok rule, Bostrom argues against the Rawlsian ‘maximin ’rule, which states
that, under ignorance, one should pick the action with the best worst-case outcome. Since it is imposs-
ible to completely rule out the possibility of extinction no matter which actions we choose, Bostrom
claims that ‘the use of maximin in the present context would entail choosing the action that has the
greatest bene ﬁt under the assumption of impending extinction, ’which would of course be Bacchanalian
revelry (Bostrom 2013 ). This is confusing, though, since the maximin rule itself is indi ﬀerent to actions
that have the same worst possible outcome; thus, decision theorists have devised the ‘leximin ’rule,
which states that if two actions have equally bad worst possible outcomes, one should choose the
action whose second worst possible outcome is the best. Thus, if one takes extinction to be theworst possible outcome —as Bostrom here does —then one should choose actions that have the best
worst possible outcomes next to extinction. Either way, the point is that Bostrom ’s assumption is
clearly wrong: There are many forms of survival, whether in a human or posthuman state, that arefar worst than extinction. It is such considerations that lead Althaus and Gloor ( 2018 ) to hortatorily
declare: ‘Rather than focusing exclusively on ensuring that there will be a future, we recommend inter-
ventions that improve the future ’s overall quality. ’INQUIRY 5
Human extinction or civilizational collapse
An event X is an existential risk if and only if X could cause either the
extinction of humanity or the collapse of civilization. This is not ascommon in the scholarly literature, although there are references to civi-
lizational collapse in the relevant contexts. For example, while Lord
Martin Rees argues that ‘the worst possible disaster [is] the foreclosure
of intelligent life ’s future through the extinction of all humankind, ’he
focuses a lot on the survival of civilization –at one point arguing that
‘the odds are no better than ﬁfty-ﬁfty that our present civilisation on
Earth will survive to the end of the present century ’(Rees 2003 ). In fact,
the Centre for the Study of Existential Risk (CSER), which Rees cofounded,initially used ‘risks of extinction’ and‘existential risks ’synonymously, but it
now describes itself as ‘dedicated to the study and mitigation of risks that
could lead to human extinction or civilisational collapse. ’
4
As with the ﬁrst deﬁnition, this one has the virtue of being easy to grasp.
Most of us have a robust intuitive understanding of what constitutes civi-lization, and thus what its collapse would entail. It also encompasses scen-arios of planetary-scale destruction that the ﬁrst de ﬁnition ignores. For
example, many scholars would describe anthropogenic climate changeas posing an existential risk, but few believe that it will directly bringabout our extinction. Rather, it is much more likely to cause civilizationalcollapse. This suggests that existential risks as human extinction orciviliza-
tional collapse is better than existential risks as human extinction, since
this de ﬁnition draws attention to the fact that some unfathomably bad dis-
asters could nonetheless be survivable .
But there are also some problems with this de ﬁnition. The most signi ﬁcant
is that, one could argue, human extinction and civilizational collapse do not
have comparable ethical implications. Since human extinction is irreversible,it would result in the loss of potentially vast amounts of future value. In con-trast, the collapse of civilization does not necessarily preclude the realizationof such value: Those who survive could rebuild civilization over the course ofdecades or centuries. Consider Bostrom ’s(2002) point that
from the perspective of humankind as a whole –even the worst of [catastrophes
so far in human history] are mere ripples on the surface of the great sea of life.
They haven ’t signi ﬁcantly aﬀ ected the total amount of human su ﬀering or hap-
piness or determined the long-term fate of our species.
4Furthermore, in a discussion of natural global pandemics, Häggström writes that not only do pandemics
pose a ‘global threat, ’but it may ‘also pose a risk on the next level, threatening to end civilization as we
know it, or even the existence of humanity ’(Häggström 2016 ).6 P. TORRES
Thus, the global village could implode without this signi ﬁcantly altering the
total amount of value that our lineage creates within our future light cone.
Even more, civilizational collapse need not entail a major loss of human life –
although it is worth noting that even if 99% of the current population wereto perish, this would still leave 76 million people alive. For such reasons,human extinction appears to be qualitatively worse than civilizational col-lapse, however unfathomably horrible the latter would be. The point isthis: Resources to mitigate existential risks are ﬁnite; thus, we should prior-
itize their use based on which threats are the most serious –holding tract-
ability and neglectedness equal. But if the ﬁrst disjunct in this de ﬁnition
refers to an outcome that is far more serious than the second disjunct,then including civilizational collapse as one form of existential catastrophecould end up taking resources away from preventing human extinction.
Another issue is that it is possible to realize a hyper-existential risk
without causing either human extinction or civilizational collapse. Thus,if one wishes for the term ‘existential risk’ to encompass hyper-existential
catastrophes as a special case –speciﬁ cally, as the worst kinds of existen-
tial catastrophes –then these de ﬁniens are inadequate. Indeed, it could be
that continued technological innovation and civilizational developmentalong certain trajectories actually enables the realization of a hyper-exis-
tential risk, perhaps in the form of an oppressive totalitarian society inwhich a majority reside in profoundly squalid conditions of inscrutablemisery. This suggests that, if existential risks are supposed to be ‘worst-
case outcomes ’for humanity, it is inadequate. Yet, as I will argue below,
it may nonetheless be useful in non-academic discussions of existentialrisk phenomena.
Human extinction or a permanent and drastic loss of potential
An event X is an existential risk if and only if X could cause either the
extinction of humanity or a permanent and drastic loss of our potentialfor desirable future development. Since this de ﬁnition emerges from the
work of Bostrom ( 2002 ,2013 ), I will call it Bostrom ’s‘lexicographic
deﬁnition ’to distinguish it from another de ﬁnition put forth in the same
manuscripts, discussed below. Although perhaps not the most commonlyused de ﬁnition in the literature, it is arguably the most canonical, since it is
theﬁrst formal de ﬁnition of the term, propounded in Bostrom ’s2002
paper that, once again, more or less o ﬃcially founded the ﬁeld.
5Notice
5Oneﬁnds myriad variations of this de ﬁnition in the literature; for example, the Future of Life Institute (FLI)
states thatINQUIRY 7
that, like the previous de ﬁnition, this one is also disjunctive, with each dis-
junct being su ﬃcient for an existential catastrophe to have occurred.
The crucial question is how one should interpret the value-laden
second disjunct. What does ‘desirable future development ’mean? For a
Benatarian anti-natalist, the most desirable future would be no future at
all–i.e. the voluntary and perhaps ‘phased ’extinction of Homo sapiens,
although this is in tension with the ﬁrst disjunct. The same goes for a
‘strong ’negative utilitarian: The most desirable future development
would be the total elimination of the very possibility of sentient life inthe universe. In contrast, for an anarcho-primitivist, it would be the cre-ation of a ‘future primitive ’state in which humans live in small-scale ega-
litarian societies that are sustainably embedded within the ecologicalsystem. And so on.
Bostrom himself appears to interpret the second disjunct by combining
total utilitarianism and transhumanism. Thus, he has previously argued forprioritizing the colonization of space
6for the purpose of ﬁlling our future
light cone with as much value as possible. On the total utilitarian view, onemight say that morality abhors a vacuum . Furthermore, just as we should
maximize the total number of happy people in the universe, we shouldalso use person-engineering technologies to optimize our individualcapacities to be happy. This would entail the creation of a new lineage,or of multiple lineages, of posthuman beings who, by deﬁ nition, have cog-
nitive abilities, healthspans, or emotional ranges that far exceed the bestattainable by any organism with a human-type genome (Bostrom 2008).Such considerations lead Bostrom, in his 2002 paper, to interpret the axio-logical term ‘potential ’as referring to ‘the transition …from a human to a
“posthuman ”society. ’He is more precise about this in his 2013 follow-up
an existential risk is any risk that has the potential to eliminate all of humanity or, at the very
least, kill large swaths of the global population, leaving the survivors without su ﬃcient
means to rebuild society to current standards of living. (FLI 2018 )
Similarly, a Global Challenges Foundation report from 2015 claims to introduce ‘a new category of global
risk,’namely, ‘risks with potentially in ﬁnite impact. ’The report de ﬁnes a risk of this sort as ‘where civi-
lisation collapses to a state of great su ﬀering and does not recover, or a situation where all human life
ends.’This comes close to de ﬁnition (ii) above, but the emphasis on civilizational collapse events that
have irreversible consequences positions it closer to Bostrom ’s lexicographic de ﬁnition. Indeed, the
authors of the report write that the idea to establish a well de ﬁned category of risks that focus on
risks with a potentially in ﬁnite impact that can be used as a practical tool by policy makers is partly
inspired by Nick Bostrom ’s philosophical work and his introduction of a risk taxonomy that includes
an academic category called ‘existential risks. ’…The in ﬁnite category is a smaller subset of ‘existential
risk’and this new category is meant to be used as a tool. Finally, one of the present authors has, on
numerous occasions, somewhat extravagantly limned the concept of existential risk as‘any future
event that either trips our species into the eternal grave of extinction or irreversibly catapults usback into the Stone Age. ’
6Just so long as doing so doesn ’t interfere with priority number one, namely, the reduction of existential
risk.8 P. TORRES
on the topic, in which he identi ﬁes the following four existential risk ‘failure
modes ’(quoting):
(1) Human extinction: Humanity goes extinct prematurely, i.e. before
reaching technological maturity.
(2) Permanent stagnation: Humanity survives but never reaches techno-
logical maturity. Subclasses: unrecovered collapse ,plateauing ,recurrent
collapse.
(3) Flawed realization: Humanity reaches technological maturity but in a
way that is dismally and irremediably ﬂawed. Subclasses: unconsum-
mated realisation ,ephemeral realisation .
(4) Subsequent ruination: Humanity reaches technological maturity in a
way that gives good future prospects, yet subsequent developments
cause the permanent ruination of those prospects.
In every case, existential risks are characterized as failures to establish a
stable state of technological maturity. By ‘technological maturity, ’Bostrom
means ‘the attainment of capabilities a ﬀording a level of economic pro-
ductivity and control over nature close to the maximum that could feasiblybe achieved ’(Bostrom 2013 ). Why does this matter? Because technological
maturity would be the situation most conducive to realizing astronomicalamounts of value, in accordance with total utilitarianism and transhuman-ism. As Bostrom puts it,
the capabilities of a technologically mature civilisation could be used to produce
outcomes that would plausibly be of great value, such as astronomical numbers
of extremely long and ful ﬁlling lives. More speci ﬁcally, mature technology would
enable a far more e ﬃcient use of basic natural resources (such as matter, energy,
space, time, and negentropy) for the creation of value than is possible with lessadvanced technology. And mature technology would allow the harvesting
(through space colonisation) of far more of these resources than is possiblewith technology whose reach is limited to Earth and its immediate neighbour-
hood (Bostrom 2013 ).
It follows from these observations that one could remove the correlative con-
junction in the lexicographic de ﬁnition and reconstruct it more succinctly as
follows: ‘An event X is an existential risk if and only if X could permanently
prevent humanity, broadly de ﬁned to include some possible posthuman descen-
dants, from attaining a stable state of technological maturity .’This is the
essence of Bostrom ’sﬁrst deﬁnition: the ultimate goal of civilizational devel-
opment should be producing mature technology in a sustainable manner, soany event that prevents this from happening must be avoided at all costs.INQUIRY 9
This de ﬁnition handles Scenario 1 from above without di ﬃculty. As
Bostrom ( 2013) writes, ‘the permanent foreclosure of any possibility of this
kind of transformative change of human biological nature may itself consti-
tute an existential catastrophe. ’Bostrom also suggests that all four scenarios –
and thus both disjuncts –are‘of comparable seriousness, entailing potentially
similarly enormous losses of expected value. ’Once again, this contrasts with
the second de ﬁnition in which the second disjunct has far less severe moral
implications than the ﬁrst disjunct, given certain assumptions.
But there are several problems with this de ﬁnition as well. First, conse-
quentialism is a minority view among philosophers, with only 23.6%accepting it as of 2013 (Bourget and Chalmers 2014 ). And, of course, not
all consequentialists are utilitarians, just as not all utilitarians accept theaggregative and impersonal perspectives of the total view. Rather, somehold that there is a diminishing marginal value to creating new people,
that the value of future lives is less than the value of current lives (temporal
discounting), or that failing to create extra people isn ’t bad even if those
people would live extremely happy lives, since acts can only be bad ifthey are bad for someone (the person-a ﬀecting intuition). According to
the latter view, Par ﬁt is wrong to argue that the di ﬀerence between
99% and 100% of humanity dying out is far greater than the di ﬀerence
between 1% and 99% dying out. Since the failure to bring into existencea potentially astronomical number of future people with worthwhile livesharms no one, human extinction is worse than the loss of 99% of humanityby only one percentage point. Note also here that the total view engen-ders the Repugnant Conclusion and violates the intuition of ‘procreational
asymmetry, ’which implies, in slogan form, that we should focus on
making people happy rather than making happy people.
Second, many would likely object to the transhumanist values that
Bostrom imports into his de ﬁnition. Bioconservatives, for example,
oppose ‘enhancive ’modiﬁcations to the human phenotype, especially
modiﬁ cations that would alter our ‘human essence ’– what Francis
Fukuyama ( 2002 ) terms ‘Factor X ’–upon which liberal democracies are
founded. But even some techno-progressives might ﬁnd the emphasis
on technological maturity o ﬀ-putting: It could be seen as too crass, tech-
nocratic, ecologically exploitative, capitalistic, colonialist, and so on. Thepresent author, in fact, knows of several scholars who balk at the lexico-graphic de ﬁnition for precisely this reason. This is worrisome because it
could lead such individuals to reason as follows: ‘Since existential risks
are scenarios that prevent the attainment of technological maturity, andsince I don ’t care about subjugating nature, maximizing economic10 P. TORRES
productivity, etc., it follows that I must not care about existential risks. ’The
same could be said about the transhumanist desideratum of exploring the
posthuman realm: why is failing to radically enhance the human species soundesirable? There are also issues relating to personal identity that arise inthis context: it could be that the transformations required to become post-human result in a genuinely new person, in the metaphysical sense (seeauthor). From this perspective, the true risk is that everyone would essen-tially commit suicide by augmenting their brains and bodies.
Yet another downside of the lexicographic de ﬁnition is that it doesn ’t
adequately address Scenario 2 from above. As Althaus and Gloor ( 2018 )
observe, this de ﬁnition
is unfortunate in that it lumps together events that lead to vast amounts of
suﬀering and events that lead to the extinction …of humanity. However,
many value systems would agree that extinction is not the worst possibleoutcome, and that avoiding large quantities of su ﬀering is of utmost moral
importance.
In other words, Bostrom doesn ’t distinguish between scenarios that could
(a) cause our extinction, and (b) result in astronomical amounts of
suﬀering. Thus, one could argue as per above, that this failure is actually
quite dangerous given the following conditions: humanity embraces themaxipok rule, has limited resources to mitigate existential risks, and couldencounter a hyper-existential or su ﬀering catastrophe. Surely we should –
as the maximin (and minipnok
7) rule implies –at least prioritize avoiding
future scenarios that are worse than extinction before aiming for a techno-
utopian future. Along these line, consider that the following two scenariosfall within the very same moral category on this de ﬁnition: (1) An asteroid
strikes Earth, resulting in an impact winter that kills every human withintwo dreadful weeks, and (2) our civilization persists at roughly its currentlevel of technological development, with the same number of people
having approximately the same quality of life for the next 1 billion years.
The latter would be an instance of ‘permanent stagnation ’in Bostrom ’s
quadripartite typology, yet it seems unambiguously better than theformer. Surely most people would concur that spending the sameamount of resources to avoid (2) as to avoid (1) would be misguided.
Aﬁnal objection concerns the ‘temporal permanence ’criterion of the
second disjunct. As Cotton-Barratt and Ord (2015) write, consider ‘at o t a l i t a r -
ian regime [that] takes control of the earth; there is only a slight chance that
7See author.INQUIRY 11
humanity will ever escape. ’Should this count as an existential catastrophe
on Bostrom ’sd eﬁnition? Perhaps it depends on when one answers: in the
midst of the regime ’s global control over civilization, one might suspect
at T1 that an existential catastrophe has occurred; but if this regime were
to dissolve at a later time T2, then, although there had been copious evi-
dence of an existential catastrophe at T1, an existential catastrophe hadnotin fact occurred. In other words, deciding that an event E is an existential
catastrophe is an epistemological issue, whereas E actually being an existen-tial catastrophe is a metaphysical one. But this does not completely solvethe problem, as Cotton-Barratt and Ord note:
Bostrom ’sd eﬁnition doesn ’t [convincingly] specify whether it should be con-
sidered as one. Either answer leads to some strange conclusions. Saying it ’s
not an existential catastrophe seems wrong as it ’s exactly the kind of thing
that we should strive to avoid for the same reasons we wish to avoid existential
catastrophes. Saying it is an existential catastrophe is very odd if humanity does
escape and recover —then the loss of potential wasn ’t permanent after all. The
problem here is that potential isn ’t binary. Entering the regime certainly seems
to curtail the potential, but not to eliminate it. (Cotton-Barratt and Ord 2015)
Put di ﬀerently, ‘potential ’isn’t binary but ‘temporal permanence ’is, and
this poses a problem for the lexicographic de ﬁnition.
In brief, this de ﬁnition is more sophisticated than the ﬁrst two, but its
connection to total utilitarianism and transhumanism could make it less
attractive to individuals who care about the future of humanity butreject one or the other of these axiological systems. It also appears suscep-tible to certain counter-examples.
A pangenerational-crushing catastrophe
An event X is an existential risk if and only if X could have undesirable conse-
quences for humanity that are ‘pangenerational ’and‘crushing. ’Let’sc a l lt h i s
Bostrom ’s‘typological de ﬁnition .’First, whereas the lexicographic de ﬁnition is
disjunctive, this one is conjunctive: it speci ﬁes two conditions –i.e. being pan-
generational and being crushing –that risks must satisfy to count as existen-
tial. Second, as the appellation implies, this de ﬁnition derives from a general
typology of risks that Bostrom delineates, and then modi ﬁes, in three papers.
The typology is based on a particular conceptual analysis of risk,a c c o r d i n gt o
which a risk is the probability of an undesirable event multiplied by its con-sequences, where the consequences of a risk are dissected into two com-ponents: Scope , or the size of the group a ﬀected, and intensity , or how bad
the eﬀects are for the group.12 P. TORRES
Bostrom ’s(2013 ) most recent typology places existential risks in the top
right box where the categories of ‘pangenerational’ and‘crushing ’intersect
(see Figure 1 ). The former refers to events ‘aﬀecting humanity over all, or
almost all, future generations, ’and the latter to events ‘causing death or a
permanent and drastic reduction of quality of life. ’Thus, Bostrom backs
away from the ‘temporal permanence ’criterion of his lexicographic
deﬁnition, although it is unclear that he recognizes this: while these two
deﬁnitions are intensionally distinct, they are presumably intended to be
extensionally equivalent. If the typological de ﬁnition includes some
events whose e ﬀects are permanent and the lexicographic de ﬁnition
does not, then they do not pick out the same class of risky happenings.
There are other problems, though. Consider the categories of the x-axis.
First, it seems that ‘imperceptible ’should be a subcategory of‘endurable, ’
and as such this seems to commit what Gilbert Ryle ( 1949 ) famously called
a‘category mistake. ’On this logic, one might as well also include as dis-
tinct categories ‘barely noticeable, ’‘moderately annoying, ’‘quite bad, ’
and so on. Furthermore, if some crushing events are by stipulation endur-
able, then distinguishing ‘crushing ’events from ‘endurable ’events is con-
fused. Similar Rylean issues problematize the y-axis. For example, noticethat ‘personal, ’‘local,’ and ‘global ’are all spatial categories whereas
Figure 1.INQUIRY 13
‘transgenerational ’and ‘pangenerational’ arespatiotemporal categories.
The result is another ontological error.
These problems yield some empirical inadequacies. For instance, con-
sider that a germline mutation, such as the hemoglobin gene mutations
that cause sickle-cell anemia, could be restricted to a local or personalscope across space yet will also have transgenerational temporaleﬀects. Yet where in Figure 1 might this risk ﬁt? Similarly, Bostrom clas-
siﬁes the ‘loss of one species of beetle ’as a transgenerational-impercep-
tible event. But this overlooks potentially important facts about the
geographical spread of the species. If the beetle is restricted to asmall region, then its disappearance may indeed be imperceptible, butif its ecological range is more global, then the loss of a single speciesof beetle could have cantharophilic e ﬀects that nontrivially harm the
health of larger ecosystems. It may thus be important to specify
whether a transgenerational event of this sort has local or global conse-
quences, yet there is no possibility in this typology of, for example, local-
pangenerational-imperceptible risks. Finally, at the non-existential risk of
caviling, it seems rather odd, rather awkward, to describe aging as a‘crushing ’risk.
Fortunately, these problems aren ’t insoluble: (i) One could replace the
three/four categories on the x-axis with the two basic categories of ‘reco-
verable ’and‘unrecoverable, ’thereby more perfectly aligning the typologi-
cal de ﬁnition with the lexicographic de ﬁnition. An ‘unrecoverable ’event
would thus be one that forever prevents humanity from attaining astable state of technological maturity, whereas a ‘recoverable ’event
would be one that doesn ’t. All four existential risk failure modes –i.e.
human extinction, recurrent collapse, and so on –would then fall within
the unrecoverable category. This would, of course, require that thedeﬁnition of ‘pangenerational ’changes from ‘aﬀecting humanity over
all, or almost all, future generations ’to‘aﬀecting humanity for all future
generations, ’but this is how Bostrom seems to think about existential
risks in most of his explorations of the topic. Incidentally, the recover-
able/unrecoverable distinction nicely maps on to a diagram put forth by
Baum, Denkenberger, and Haqq-Misra 2015 ,
seen in Figure 2 . As this
diagram makes clear, albeit tacitly, an existential catastrophe could havetwo distinct outcomes, namely, extinction and survival without recovery .
And (ii), one could split the spatiotemporal components of the cat-
egories on the y-axis, thus separating this dimension into space andtime. The resulting three-dimensional typology would be based on the fol-lowing augmented conceptual analysis of risk:14 P. TORRES
A risk is the probability of an undesirable event multiplied by its consequences;
the consequences are then analyzed into their scope and intensity, where scope
is further decomposed into its spatial and temporal properties. Thus: ‘risk =
[spatial scope, temporal scope, and intensity of consequences] × [probability
of occurrence]. ’
If we revert to an older dichotomy used by Bostrom ( 2002 )–one that dis-
tinguishes between ‘endurable ’and ‘terminal ’rather than, as per above,
‘recoverable ’and‘unrecoverable ’events –this new typology would classify
all existential risks as global-pangenerational events, where the ﬁrst term indi-
cating the risk ’s spatial scope and the second its temporal scope. Within this
conjunctive category, extinction events would constitute global-pangenera-
tional-terminal events (node A) and existential risks that correspond to the
second disjunct of Bostrom ’s lexicographic de ﬁnition would constitute the
subset of global-pangenerational-endurable events that irreversibly preclude
the stable attainment of technological maturity (node B).8In addition to these
typological accounts of existential risks, Figure 3 also easily accommodates
phenomena like aging, hereditary disease (node C), and the loss of a single
species of beetle (node D).
Figure 2. Diﬀerent Possible Long-Term Trajectories. (Adapted from Baum 2010 .)
8Note: the ‘permanence ’condition of the second disjunct is embedded within the concept of
pangenerational .INQUIRY 15
While this modi ﬁed typological de ﬁnition is more complex, it is also
more precise. It is also constant with Thomas Kuhn ’s(1962 ) observation
that as scienti ﬁcﬁelds mature, they tend to undergo ‘ar eﬁnement of con-
cepts that increasingly lessens their resemblance to their usual common-
sense prototypes. ’The three-dimensional de ﬁnition also makes explicit
that some existential risks can indeed be survived.
A signi ﬁcant loss of expected value
An event X is an existential risk if and only if X could cause the loss of a
large fraction of expected value.9This de ﬁnition comes from a short
2015 paper by Cotton-Barratt and Ord, published as a ‘technical report ’
by the Future of Humanity Institute (FHI). It is, as they write, primarily
motivated by the aforementioned fact that potential isn ’t binary but per-
manence is. Recall the totalitarian scenario mentioned when discussingBostrom ’s lexicographic de ﬁnition above; Cotton-Barratt and Ord write
that ‘if we enter into the totalitarian regime and then at a later date the
hope of escape is snu ﬀed out, that represents two existential catastrophes
under [the expected value] de ﬁnition. ’That is to say, humanity ‘lost most
of the expected value when we entered the regime, and then lost most ofthe remaining expected value when the chance for escape disappeared. ’
It may be worth pausing on this point for a moment. On the present
deﬁnition, an existential catastrophe can occur more than once in a
species ’lifetime. The same goes for an existential catastrophe that
entails ‘mere ’civilizational collapse, as speci ﬁed by the second de ﬁnition
above. But this is not the case if one accepts the ‘existential risks as
human extinction, ’‘existential risks as human extinction or a permanent
and drastic loss of potential, ’or‘existential risks as pangenerational-crush-
ing events. ’In all of these cases, humanity has one, and only one, shot at
avoiding disaster. Indeed, this is why Bostrom ( 2002 ) emphasizes that we
must replace our normal reactive approach to dealing with risks with anentirely proactive approach: if even a single existential catastrophe were
to occur, the game would be over and humanity would have lost (itschance to ‘fulﬁll its potential ’). Thus, Cotton-Barratt and Ord ’sd eﬁnition
dilutes this implication of preemptive urgency at least a little by allowingfor the possibility that humanity recovers from one or more past existential
catastrophes. This is a notable di ﬀerence with respect to the other
deﬁnitions so far explicated.
9Quoted in Cotton-Barratt and Ord (2015).16 P. TORRES
Other features of this de ﬁnition are: ﬁrst, one can interpret it as a re ﬁne-
ment of the second disjunct of Bostrom ’s lexicographic de ﬁnition that also
discards the ﬁrst disjunct (as conceptually unnecessary within the
deﬁniens). The result is a reorganization of Bostrom ’s four failure modes
of‘human extinction, ’‘permanent stagnation, ’‘ﬂawed realization, ’and
‘subsequent ruination ’under a single conceptual heading, rather than
placing the ﬁrst failure mode under the ﬁrst disjunct of the lexicographic
deﬁnition and the last three under the second disjunct. Put di ﬀerently, it
looks arbitrary and inconsistent that Bostrom ’sd eﬁnition singles out
human extinction –one of four particular failure modes –but then
hides the other three failure modes behind the phrase ‘permanently and
drastically compromising our potential for desirable future development. ’
The critical point is that all of thee failure modes are bad because they
entail huge losses of expected value within our future light cone.
Cotton-Barratt and Ord’ sd eﬁnition also foregrounds an idea –i.e.
expected value –that nebulously lurks behind Bostrom ’s discussions of
existential risk, making occasional cameos but then returning to the philo-sophical background. For example, in a discussion about the moral force of
Figure 3. A Three-Dimensional Typology of Risks.INQUIRY 17
the maxipok rule, Bostrom writes that (partly quoted above) ‘these con-
siderations suggest that the loss in expected value resulting from an exis-
tential catastrophe is so enormous that the objective of reducingexistential risks should be a dominant consideration whenever we actout of an impersonal concern for humankind as a whole ’(Bostrom 2013 ).
The point is that Cotton-Barratt and Ord ’sd eﬁnition could be seen as a
more distilled version of Bostrom ’sd eﬁnitions, which leads to the next
point: third, the expected value de ﬁnition does not explicitly mention tech-
nological maturity, and thus it does not engender the problem of potentialallies ignoring existential risk studies because its central aims are ‘too trans-
humanistic. ’For Cotton-Barratt and Ord (2015), the term ‘‘value ’refers
simply to whatever it is we care about and want in the world ’–which,
once more, connects with the following point: fourth, this ﬂexibility
opens the door for a wide range of axiological interpretations of the
deﬁnition. As mentioned above, most existential risk scholars appear to
espouse total utilitarianism and many are also sympathetic with the trans-
humanist project; for such scholars, ‘value ’refers to the well-being of senti-
ent life (which ought to be maximized) and ‘having the opportunity to
explore the transhuman and posthuman realms, ’as Bostrom ( 2005 ) puts it.
Another cluster of issues to note is: (a) The ﬁrst two de ﬁnitions above
speciﬁ cally reference scenarios that we should avoid, without saying any-
thing about which scenarios we should pursue. On these accounts, theworst-case outcome for humanity would be to fall victim to such bad scen-
arios; the same could be said of the fourth, typological de ﬁnition. (b) The
third, lexicographic de ﬁnition focuses on what we should attain,d e ﬁning
any event that prevents the attainment of technological maturity –a
techno-utopian condition of endless possibilities (see Bostrom 2008 )–
as the worst-case outcome for humanity. Or, as Cotton-Barratt and Ord(2015) put it, ‘under Bostrom ’sd eﬁnition we are comparing ourselves to
the most optimistic potential we could reach. ’
In contrast, (c) the present de ﬁnition focuses on avoiding a signi ﬁcant loss
of expected value but does not explicitly identify any particular scenario, such
as human extinction, civilizational collapse, or failing to reach technological
maturity, that humanity must dodge at all costs. Yet Cotton-Barratt and Ordargue that their conception of existential risks gives rise to the mirrorconcept of what they call ‘existential hope. ’This
 arises from the possibility
of (to borrow a term from J.R.R. Tolkien) ‘existential eucatastrophes, ’which
are the mirror phenomena of existential catastrophes, that could cause‘there to be much more expected value after the event than before.’ Thus,
t h e ya r g u et h a tj u s ta sw es h o u l d avoid existential catastrophes, we should18 P. TORRES
seek out existential eucatastrophes, exa mples being the emergence of the ﬁrst
living organisms, any successful dodging of a Great Filter, or, perhaps, the cre-
ation of a value-aligned machine superintelligence. From a pragmatic point of
view, the idea of existential hope, as Max Tegmark ( 2018) points out, usefully
encourages us to imagine ‘positive futures not only for ourselves, but also for
s o c i e t ya n df o rh u m a n i t yi t s e l f , ’an activity that may help to overcome the
occasional paralysis or defeatism that some people experience when cogitat-ing the topic of existential risks (see also Pinker 2018). In a phrase, the combi-
nation of existential risk and existential hope oﬀ ers a more complete and
comprehensive world-picture that humanity could employ when decidingwhich development trajectories we oug ht to pursue, and this makes it appeal-
ing as an organizing principle for the ﬁeld of existential risk studies.
10
But there are also some ostensive problems with this deﬁ nition. First, as
implied above, it may be tooﬂexible. As Cotton-Barratt and Ord note, the
term ‘value ’is doing much of the work in the de ﬁnition and, as we have
seen, there is a huge range of distinct and mutually exclusive value the-ories on the marketplace of ideas. Indeed, there is nothing conceptuallyincoherent about, say, an anarcho-primitivist exclaiming, ‘We must do
everything we possibly can to mitigate existential risks! Follow themaxipok rule!, ’but meaning that we ought to destroy the techno-indus-
trial system upon which modern civilization is founded. Second, if oneaccepts the ‘astronomical value thesis, ’according to which the value of
the distant future could be unimaginably great (see author), that futurelives are worth the same as present lives, and so on, the result could bewhat Steven Pinker (2011) terms ‘a pernicious utilitarian calculus ’that
encourages people to, as it were, redirect the trolly to sacriﬁ ce the lives
of current humans for the purpose of in ﬂating the probability that innu-
merably more posthumans come to exist in the future. This is a criticismthat applies no less to Bostrom ’s (utilitarian) conception of existential
risks, as Olle Häggström argues. To quote Häggström at length:
Recall …Bostrom ’s conclusion about how reducing the probability of existential
catastrophe by even a minuscule amount can be more important than saving
the lives of a million people. While it is hard to ﬁnd any ﬂaw in his reasoning
leading up to the conclusion, and while if the discussion remains su ﬃciently
abstract I am inclined to accept it as correct, I feel extremely uneasy aboutthe prospect that it might become recognized among politicians and
decision-makers as a guide to policy worth taking literally. It is simply too
10Some might argue that ‘existential despair ’better contrasts with ‘existential hope, ’although Cotton-
Barratt argues that this point isn ’t‘necessarily enough to sink the [original] term ’of‘existential hope ’
(Cotton-Barratt 2015).INQUIRY 19
reminiscent of the old saying ‘If you want to make an omelet, you must be
willing to break a few eggs, ’which has typically been used to explain that a
bit of genocide or so might be a good thing, if it can contribute to the goal of
creating a future utopia. Imagine a situation where the head of the CIA explains
to the US president that they have credible evidence that somewhere in
Germany, there is a lunatic who is working on a doomsday weapon andintends to use it to wipe out humanity, and that this lunatic has a one-in- a-
million chance of succeeding. They have no further information on the identity
or whereabouts of this lunatic. If the president has taken Bostrom’ s argument to
heart, and if he knows how to do the arithmetic, he may conclude that it is
worthwhile conducting a full-scale nuclear assault on Germany to kill every
single person within its borders.
Andﬁnally, if one interprets value in Cotton-Barratt and Ord ’sd eﬁnition as
well-being, and well-being in hedonistic terms, then, as David Pearce (2014)
observes, total utilitarians are ‘obliged to convert your matter and energy
into pure utilitronium, erasing you, your memories, and indeed human civi-lisation. ‘
11A couple of points: (a) One can view this as a reductio ad absur-
dum of hedonistic monism, along the lines of Robert Nozick ’s(1974 )
‘experience machine ’thought experiment, and (b) many people would
no doubt argue that converting the universe into utilitronium would yieldif not a worst-case outcome, then certainly not the best-case outcome.Thus, the combination of Cotton-Barratt and Ord ’s general framework
plus a widely accepted and respectable moral theory could have potentiallydevastating consequences, or so one might plausibly argue.
Evaluating de ﬁnitions
Consistent with Kuhn ’s observation in section 1, Steven Pinker ( 2014 )
observes that esoteric vocabularies are ‘unavoidable because of the abstract-
ness and complexity of [academic] subject matter[s]. ’Consequently,
every human pastime —music, cooking, sports, art —develops an argot to spare
its enthusiasts from having to use a long-winded description every time they
refer to a familiar concept in one another ’s company. It would be tedious for
a biologist to spell out the meaning of the term transcription factor every time
she used it, and so we should not expect the tête-à-tête among professionalsto be easily understood by amateurs.
12
Yet the aim of existential risk studies isn’ t just to further expand the edi ﬁce
of human knowledge, but –as alluded to in section 1 –to communicate
11Note that total utilitarianism in particular, and aggregative consequentialism in general, are also beset
by what Bostrom (2011) calls ‘inﬁnitarian paralysis. ’
12This is, of course, intended to be a sardonic reference to Pinker ’s comments in Coyne 2019 .20 P. TORRES
these ideas and insights to politicians, policymakers, business leaders, and
to all others with the power of the ballot. Informing the public about theideas and insights generated by existential risk research is integral to thebroader programme; there is simply no point to devising e ﬀective strat-
egies for navigating the obstacle course of threats before us if those strat-egies will never be implemented by governments, internationalorganizations, corporations, and other relevant entities. To paraphraseKarl Marx ’s now-hackneyed exhortation (borrowed from Voltaire), the
point is to change the world. Indeed, a number of leading scholars havemade a number of notable media appearances to discuss existentialrisks and related issues. Essential to the success of the ﬁeld are ‘ambassa-
dors of science ’–very often, thus far, active contributors to the technical
literature –capable of interfacing between the experts and non-experts.
Notice here that di ﬀerent risks require di ﬀerent outreach strategies. For
example, humanity is unlikely to prevent further anthropogenic climatechange without the public putting signi ﬁcant pressure on political
leaders. In democratic states, this means persuading the electorate tovote for candidates who vow to support legislation that reduces our col-lective carbon footprint, encourages the development of alternativeenergy sources, and so on –essentially, to resolve the tragedy of the
commons. Thus, e ﬀorts to convey information about the potentially cata-
strophic e ﬀects of climate change should target the general public. This
contrasts with risks like asteroid impacts and, arguably, value-misalignedmachine superintelligence. In the former case, public knowledge aboutnear-Earth objects (NEOs) is mostly irrelevant to the detection and de ﬂec-
tion of asteroids and comets that are on a crash course with Earth. So longas NASA and similar organizations around the world remain su ﬃciently
well-funded, the extent to which humanity is collectively safe from a cat-astrophic collision is not a function of the degree of public informedness.Similar points could be made about superintelligence: On the one hand,avoiding an AI arms race with China or Russia might depend on adequatepressure on politicians by an electorate worried about the possible conse-
quences of another ‘Cold War,’ except far less stable given the nature of AI
and the larger number of actors that could be involved (Armstrong,
Bostrom, and Shulman 2016 ; author). Thus, on the other hand, when it
comes to superintelligence, a positive outcome will depend almostentirely on experts: moral philosophers and computer scientists workingon the ‘value-alignment problem. ’Alerting the public of the dangers
posed by machine superintelligence is unlikely to directly alter progresson determine what our ‘human values ’are and how to de ﬁne these ‘inINQUIRY 21
primitives such as mathematical operators and addresses pointing to the
contents of individual memory registers ’(Bostrom 2014 ).
Thus, whereas existential risk scholars who are focused on environ-
mental degradation need to engage with the general public, thoseworking on issues like superintelligence should direct their outreacheﬀorts at theoreticians, industry scientists, CEOs, and so on. If this
picture is accurate, then it might be optimal to employ di ﬀerent de ﬁnitions
depending on the conversational context. Indeed, this is the position that Iwill here defend, which I call ‘context-dependent semantic pluralism. ’In
my view, the second de ﬁnition of existential risk as either human extinc-
tion or civilizational collapse is preferable to the ﬁrst, and the best
deﬁnition for communicating to the general public. The reasons are as
follows: ﬁrst, if the question is ‘What ought our global priorities be?, ’
then the ﬁrst de ﬁnition is much too narrow. Although the moral impli-
cations of civilizational collapse are less signi ﬁcant than those of extinc-
tion, it is useful to at least signal to the public that scholars aren ’tmerely
worried about human extinction –a planetary-scale catastrophe could cat-
apult the human species back into the Stone Age –even if the signal itself
is inexact. Indeed, a number of scenarios in the foreground of researchers ’
attention, such as climate change, seem unlikely to directly cause humanextinction but quite likely to seriously damage human civilization.
Even more, civilizational collapse may be more vivid to the average
mind than human extinction, in part because few apocalyptic Hollywoodmovies focus on extinction rather than collapse. Although scope neglectand ‘psychophysical numbing ’could apply to both scenarios, it may be
especially applicable to annihilation, thus dulling the emotional impactof warnings of the form, ‘If X happens, humanity could go extinct. ’A
recent unpublished study, in fact, found that Derek Parﬁ t was right in
his conjecture that most people don ’t see the di ﬀerence between 99%
and 100% of humanity dying out as much greater than the di ﬀerence
between peace and 99% of humanity dying out. Yet, when researchersprodded subjects to think explicitly about the far future and all the
good that it could envelop, they shifted their opinion toward Parﬁ t’s
own view.
With respect to the lexicographic, typological, and expected value
deﬁnitions of ‘existential risk, ’there is a good case that these are too eso-
teric for a general audience. Bostrom ’sd eﬁnition requires some knowl-
edge of moral philosophy and transhumanism, and few members of the
public are conversant with decision theory and the technical concept of
expected
value . In sum, the second de ﬁnition avoids the simplism of the22 P. TORRES
ﬁrst de ﬁnition as well as the esoterica of the last three; it constitutes a
middle ground between these two ends. I thus recommend its use
when discussing existential risks and related topics with lay audiences.
There remains the second question of which de ﬁnition ought to be the
canonical one used by scholars in conversations amongst themselves andwith professional scientists and philosophers outside of the ﬁeld. Here the
simplism of the ﬁrst deﬁnition and imprecision of the second quickly dis-
qualiﬁes them from serious consideration. That is to say, it would be mis-
leading to equate existential risks with either mere human extinction orcivilizational collapse; careful axiological re ﬂection implies that what
really matters are disasters with long-lasting e ﬀects that drastically com-
promise our potential to realize our various individual and collective ambi-
tions for happiness, knowledge, peace, moral growth, and so on. Both ofBostrom ’sd eﬁnitions gesture at this idea, which makes them appealing,
although as previously noted even some techno-progressives could ﬁnd
the focus on ‘technological maturity ’to be objectionable. Indeed, transhu-
manism remains a controversial normative-futurological position, andrecall from above that a relatively small minority of professional philoso-phers espouse total utilitarianism. For reasons such as these, I believethat the ﬁeld of existential risk studies would generally bene ﬁt from
eschewing both of Bostrom ’sd eﬁnitions.
This leaves the ﬁfth de ﬁnition from Cotton-Barratt and Ord, which I
believe, with modi ﬁcation, is superior to all other de ﬁnitions in the
context of tête-à-tête communication and eﬀorts to evangelize for the
ﬁeld to scientists and philosophers unfamiliar with its aims, method-
ologies, and so on. Most people value things in the world and, asSamuel Sche ﬄer (2009) observes, ‘what would it mean to value things
but, in general, to see no reason of any kind to sustain them or retainthem or preserve them or extend them into the future? ’The ambiguity
of‘value ’in the ﬁfth de ﬁnition enables a wide range of interpretations,
which could result in individuals who value seemingly disparate things –
e.g. sports, literature, philosophy, science, the wilderness, standup
comedy shows, and 3D movies –to unify behind the single ultra-goal of
mitigating existential risk. Indeed, it may be an improvement of Cotton-
Barratt and Ord ’sd eﬁnition to make explicit that ‘value ’can be understood
as
denoting whatever one happens to care about in this strange place
called ‘the universe. ’Thus, when a scientist or philosopher outside the
ﬁeld asks what existential risks are, one could respond: An event X is an
existential risk if and only if X could cause the loss of a large fraction ofexpected value, where you (the interlocutor) can deﬁ ne‘value ’howeverINQUIRY 23
you’d like. It is my considered opinion that this constitutes the best
deﬁnition for the purpose of further establishing existential risk studies
as a legitimate ﬁeld of scienti ﬁc and philosophical inquiry.
The implication of this admittedly brief discussion of context-dependent
semantic pluralism is that existential risk scholars will have to pivot from one
use to the other in response to di ﬀerent audiences.13Given that existential
risk studies is an ‘outward-facing ’(or integrally activist) ﬁeld that strives to
bring about nontrivial positive changes in the real world, terminologicalﬂexibility may be an essential skill for existential risk scholars to develop.
Conclusion
This paper falls within an emerging sub ﬁeld that I call ‘the conceptual foun-
dations of existential risk studies. ’Given the dearth of research so far con-
ducted on the relevant topics, I do not see this paper as the ﬁnal word.
Rather, my goal is much more modest: To merely stimulate further philoso-phical analyses of this concept by bringing together, for the ﬁrst time in a
single paper, the various de ﬁnitions that have so far been outlined and dis-
cussed seriously in the scholarly literature. Future research on this topicmight thus focus on devising better necessary and su ﬃcient conditions
for the de ﬁniendum, exploring how di ﬀerent theories of concepts (e.g.
the‘prototype theory, ’‘theory theory ’) might solve some of the problems
speciﬁ ed above, examining additional novel strategies for making the
idea of existential risk more appealing those outside of the ﬁeld’s perimeter,
and communicating important research results to the public –a necessary
step with respect to certain risks like climate change. Thus, my closingremark is an invitation: Much work remains to be done.
Acknowledgements
Many sincere thanks to Simon Beard for detailed comments and criticisms that greatly
improved this paper.
Disclosure statement
No potential con ﬂict of interest was reported by the author.
13There are also parallels between this account and Sally Haslanger ’s(2002 ) assertion that, ‘in deﬁning our
terms, we must keep clearly in mind our political aims both in analyzing the past and present, and in
envisioning alternative futures. ’We should, as she implores, ‘begin by asking both in the theoretical and
political sense what, if anything, we want [our concepts] to be. ’24 P. TORRES
References
Althaus, David, and Lukas Gloor. 2018 .“Reducing Risks of Astronomical Su ﬀering: A
Neglected Priority. ”Foundational Research Institute. https://foundational-research.
org/reducing-risks-of-astronomical-su ﬀer-ing-a-neglected-priority/ .
Anderson, Ross. 2012 .“We’re Underestimating the Risk of Human Extinction. ”The
Atlantic .https://www.theatlantic.com/technology/archive/2012/03/were-underesti
mating-the-risk-of-human-extinction/253821/ .
Armstrong, Stuart, Nick Bostrom, and Carl Shulman. 2016 .“Racing to the Precipice: A
Model of Arti ﬁcial Intelligence Development. ”AI & Society 31 (2): 201 –206.
Baum, Seth. 2010 .“Is Humanity Doomed? Insights from Astrobiology. ”Sustainability 2
(2): 591 –603.
Baum, Seth, David Denkenberger, and Jacob Haqq-Misra. 2015 .“Isolated Refuges for
Surviving Global Catastrophes.” Futures 72: 45– 56.
Bostrom, Nick. 2002 .“Existential Risks: Analyzing Human Extinction Scenarios and
Related Hazards. ”Journal of Evolution and Technology 9 (1).
Bostrom, Nick. 2005 .“Transhumanist Values. ”Review of Contemporary Philosophy 4.
https://nick-bostrom.com/ethics/values.html .
Bostrom, Nick. 2013 .“Existential Risk Prevention as Global Priority. ”Global Policy 4( 1 ) :1 5 –31.
Bostrom, Nick. 2014 .Superintelligence: Paths, Dangers, Strategies . Oxford University
Press.
Bostrom, Nick, and Milan Ćirković. 2008 .Global Catastrophic Risks . Oxford, UK: Oxford
University Press.
Bourget, David, and David Chalmers. 2014 .“What Do Philosophers Believe? ”
Philosophical Studies 170 (3): 465 –500.
Chalmers, David. 2011 .“Verbal Disputes. ”Philosophical Review 120 (4): 515 –566.
Coyne, Jerry. 2019 .“A Response from Steve Pinker. ”Why Evolution is True .https://
whyevolutionistrue.wordpress.com/2019/01/29/a-response-from-steve-pinker-to-sa
lons-hit-piece-on-enlightenment-now/ .
Ellis, Brian. 2001 .Scienti ﬁc Essentialism, Cambridge Studies in Philosophy. Cambridge, UK:
Cambridge University Press.
Ellis, Brian. 2002 .The Philosophy of Nature . Chesham: Acumen.
FLI.2018 .“Existential Risk.” Future of Life Institute. https://futureo ﬂife.org/background/
existential-risk/ .
Fukuyama, Francis. 2002 .Our Posthuman Future: Consequences of the Biotechnology
Revolution . New York, NY: Farrar Straus & Giroux.
Gloor, Lukas, and Adriano Mannino. 2018 .“The Case for Su ﬀering-Focused Ethics. ”
Foundational Research Institute. https://foundational-research.org/the-case-for-
suﬀering-focused-ethics/ .
Häggström, Olle. 2016 .Here Be Dragons: Science, Technology and the Future of
Humanity . Oxford, UK: Oxford University Press.
Haslanger, Sally. 2002 .“Gender and Race: (What) Are They? (What) Do We Want Them
To Be?” Noûs 34 (1): 31– 55.
Jebari, Karim. 2015 .“Existential Risks: Exploring a Robust Risk Reduction Strategy. ”
Science and Engineering Ethics 21 (3): 541 –554.INQUIRY 25
Kass, Leon. 2002 .Life, Liberty, and the Defense of Dignity: The Challenge for Bioethics .
San Francisco: Encounter Books.
Kuhn, Thomas. 1962 .The Structure of Scienti ﬁc Revolutions . Chicago, IL: University of
Chicago Press.
Matheny, Jason. 2007 .“Reducing the Risk of Human Extinction. ”Risk Analysis 27: 1335 –
1344.
Nozick, Robert. 1974 .Anarchy, State, and Utopia . New York: Basic Books.
Pamlin, Denis, and Stuart Armstrong. 2015. “12 Risks that Threaten Human Civilisation. ”
Global Challenges Foundation. https://api.globalchallenges.org/static/wp-content/
uploads/12-Risks-with-innite-impact.pdf .
Parﬁt, Derek. 1984 .Reasons and Persons . Oxford: Oxford University Press.
Parﬁt, Derek. 2017 .On What Matters: Volume Three . Oxford: Oxford University Press.
Pinker, Steven. 2014 .“Why Academics Stink at Writing ”.The Chronicle Review .http://
wiki.lib.sun.ac.za/images/9/91/2104-pinker-writing.pdf.
Pinker, Steven. 2018 .Enlightenment Now: The Case for Reason, Science, Humanism, and
Progress. New York, NY: Penguin Books.
Rees, Martin. 2003 .Our Final Hour: A Scientist ’s Warning . New York, NY: Basic Books.
Ryle, Gilbert. 1949 .The Concept of Mind . Chicago, IL: University of Chicago Press.
Sandberg, Anders. 2014. “The Five Biggest Threats to Human Existence. ”The
Conversation. https://the-conversation.com/the- ﬁve-biggest-threats-to-human-
existence-27053 .
Tegmark, Max. 2018 .Life 3.0: Being Human in the Age of Artiﬁ cial Intelligence . New York,
NY: Vintage.26 P. TORRES