How We’re Predicting AI – or Failing to

How We’re Predicting AI – or Failing to⋆
Stuart Armstrong1and Kaj Sotala2
1The Future of Humanity Institute,
Faculty of Philosophy, University of Oxford, UK
stuart.armstrong@stx.oxon.org
2The Singularity Institute, Berkeley, CA, USA
kaj@singularity.org
Abstract. This paperwill look atthevariouspredictionsthathavebeen
made about AI and propose decomposition schemas for analysing them.
It will propose a variety of theoretical tools for analysing, judging and
improving these predictions. Focusing speciﬁcally on timeline predictions
(dates given by which we should expect the creation of AI), it will show
that there are strong theoretical grounds to expect predictions to be
quite poor in this area. Using a database of 95 AI timeline predictions,
it will show that these expectations are born out in practice: expert
predictions contradict each other considerably, and are indistinguishable
from non-expert predictions and past failed predictions. Predictions that
AI lie 15 to 25 years in the future are the most common, from experts
and non-experts alike.
Keywords: artiﬁcial intelligence, predictions, experts, bias.
1 Introduction
Predictionsaboutthefuturedevelopmentofartiﬁcialintelligenceareasconﬁdent
astheyarediverse.StartingwithTuring’sinitialestimationofa30%passrateon
Turingtestbytheyear2000[1],computerscientists,philosophersandjournalists
have never been shy to oﬀer their own deﬁnite prognostics, claiming AI to be
impossible [2] or just around the corner [3] or anything in between.
Whatarewetomakeofthesepredictions?Whataretheyfor,andwhatcanwe
gain from them? Are they to be treated as light entertainment, the equivalent of
fact-free editorials about the moral decline of modern living? Or are there some
useful truths to be extracted? Can we feel conﬁdent that certain categories of
experts can be identiﬁed, and that their predictions stand out from the rest in
terms of reliability?
In this paper, we start oﬀ by proposing classiﬁcation schemes for AI predic-
tions: what types of predictions are being made, and what kinds of arguments
⋆The authors wish to acknowledge the help and support of the Singularity Insti-
tute, the Future of Humanity Institute and the James Martin School, as well as
the individual advice of Nick Bostrom, Luke Muelhauser, Vincent Mueller, Anders
Sandberg, Lisa Makros, Sean O’Heigeartaigh, Daniel Dewey, Eric Drexler and the
online community of Less Wrong.
c/circlecopyrtSpringer International Publishing Switzerland 2015 11
J. Romportl et al. (eds.), Beyond Artiﬁcial Intelligence,
Topics in Intelligent Engineering and Informatics 9, DOI: 10.1007/978-3-319-09668-1 \_2
12 S. Armstrong and K. Sotala
or models are being used to justify them. Diﬀerent models and predictions can
result in very diﬀerent performances, and it will be the ultimate aim of thisproject to classify and analyse their varying reliability.
Armed with this scheme, we then analyse some of these approaches from
the theoretical perspective, seeing wh ether there are good reasons to believe or
disbelieve their results. The aim is not simply to critique individual methods or
individuals, but to construct a toolbox of assessment tools that will both enable
us to estimate the reliability of a prediction, and allow predictors to come up
with better results themselves.
This paper, the ﬁrst in the project, looks speciﬁcally at AI timeline predic-
tions: those predictions that give a date by which we should expect to see an
actual AI being developed (we use AI in the old fashioned sense of a machine
capable of human-comparable cognitive performance; a less ambiguous modernterm would be ‘AGI’, Artiﬁcial General Intelligence). With the aid of the biases
literature, we demonstrate that there are strong reasons to expert that experts
wouldnotbe showing particular skill the ﬁeld of AI timeline predictions. The
task is simply not suited for good expert performance.
Those theoretical results are supplemented with the real meat of the paper: a
database of 257 AI predictions, made in a period spanning from the 1950s to the
present day. This database was assembled by researchers from the Singularity
Institute (Jonathan Wang and Brian Potter) systematically searching thoughthe literature, and is a treasure-troveof interesting results. A total of 95 of these
can be considered AI timeline predictions. We assign to each of them a single
‘medianAI’date,whichthenallowsustodemonstratethatAIexpertpredictionsare greatly inconsistent with each other – and indistinguishable from non-expert
performance, and past failed predictions.
With thedata,wefurthertest twofolk theorems:ﬁrstlythat predictorsalways
predict the arrival of AI just before their own deaths, and secondly that AI is
always 15 to 25 years into the future. We ﬁnd evidence for the second thesis butnot for the ﬁrst.
This enabled us to show that there seems to be no such thing as an “AI
expert” for timeline predictions: no category of predictors stands out from thecrowd.
2 Taxonomy of Predictions
2.1 Prediction Types
“There will never be a bigger plane built.”
Boeing engineer on the 247, a twin engine plane that held ten people.
The standard image of a prediction is some fortune teller staring deeply into the
mists of a crystal ball, and d ecreeing, with a hideous cer tainty, the course of the
times to come. Or in a more modern version, a scientist predicting the outcome
of an experiment or an economist pronouncing on next year’s GDP ﬁgures. Butthese “at date X, Y will happen” are just on e type ofvalid prediction. In general,
How We’re Predicting AI 13
a prediction is something that constrains our expectation of the future. Before
hearing the prediction, we thought the future would have certain properties; butafter hearing and believing it, we now ex pect the future to be diﬀerent from our
initial thoughts.
Under this deﬁnition, conditional predictions – “if A, then B will happen”
– are also perfectly valid. As are negative predictions: we might have believed
initially that perpetual motion machines were possible, and imagined what they
could be used for. But once we accept tha t one cannot violate conservation of
energy, we have a diﬀerent picture of the future: one without these wonderful
machines and all their fabulous consequences.
For the present analysis, we will divide predictions about AI into four types:
1. Timelines and outcome predictions. These are the traditional types of pre-
dictions, telling us when we will achieve speciﬁc AI milestones. Examples:
An AI will pass the Turing test by 2000 [1]; Within a decade, AIs will bereplacing scientists and other thinking professions [4].
2. Scenarios. These are a type of conditional predictions, claiming that if the
conditions ofthe scenarioaremet, the n certain types ofoutcomeswill follow.
Example: If we build a human-level AI that is easy to copy and cheap to
run, this will cause mass unemployment among ordinary humans [5].
3. Plans. These are a speciﬁc type of conditional prediction, claiming that if
someone decides to implement a speciﬁc plan, then they will be successful
in achieving a particular goal. Example: We can build an AI by scanning ahuman brain and simulating the scan on a computer [6].
4. Issues and metastatements. This ca tegory covers relevant problems with
(some or all) approaches to AI (including sheer impossibility results), andmetastatementsaboutthe wholeﬁeld.Examples:anAIcannotbebuiltwith-
out a fundamental new understanding of epistemology [7]; Generic AIs will
have certain (potentially dangerous) behaviours [8].
There will inevitably be some overlapbetween the categories,but this division
is natural enough for our purposes. In this paper we will be looking at timeline
predictions. Thanks to the eﬀorts of Jonathan Wang and Brian Potter at theSingularity Institute, the authors were able to make use of extensive databases
of this type of predictions, reaching back from the present day back to the 1950s.
Other types of predictions will be analysed in subsequent papers.
2.2 Prediction Methods
Just as there are many types of predictions, there are many ways of arriving
at them – consulting crystal balls, listening to the pronouncements of experts,
constructing elaborate models. Our review of published predictions has shown
that the prediction methods are far more varied than the types of conclusionsarrived at. For the purposes of this analysis, we’ll divide the prediction methods
into the following loose scheme:
14 S. Armstrong and K. Sotala
1. Causal models
2. Non-causal models3. The outside view4. Philosophical arguments5. Expert authority6. Non-expert authority
Causal model are the staple of physics: given certain facts about the situation
under consideration (momentum, energy, charge, etc.) a conclusion is reached
aboutwhattheultimatestatewillbe.Ifthefactswerediﬀerent,theendsituationwould be diﬀerent.
But causal models are often a luxury outside of the hard sciences, whenever
we lack precise understanding of the underlying causes. Some success can be
achieved with non-causal models: without understanding what inﬂuences what,
one can extrapolate trends into the future. Moore’s law is a highly successfulnon-causal model [9].
The outside view is a method of predicting that works by gathering together
speciﬁc examples and claiming that they all follow the same underlying trend.For instance, one could notice the plethora of Moore’s laws across the spectrum
of computing (in numbers of transistors, size of hard drives, network capacity,
pixels per dollar, ...), note that AI is in the same category, and hence argue that
AI development must follow a similarly exponential curve [10].
Philosophical arguments are common in the ﬁeld of AI; some are simple im-
possibility statements: AI is decreed to be impossible for more or less plausible
reasons. But the more thoughtful philosophical arguments point out problems
that need to be resolved to achieve AI, highlight interesting approaches to doingso, and point potential issues if this were to be achieved.
Many predictions rely strongly on the status of the predictor: their innate
expertise giving them potential insights that cannot be fully captured in their
arguments, so we have to trust their judgment. But there are problems in relying
on expert opinion, as we shall see.
Finally, some predictions rely on the judgment or opinion of non-experts.
Journalists and authors are examples of this, but often actual experts will make
claims outside their domainofexpertise. CEO’s,historians,physicists and math-ematicians will generally be no more accurate than anyone else when talking
about AI, no matter how stellar they are in their own ﬁeld [11].
Predictions can use a mixture of these approaches,and often do. For instance,
Ray Kurzweil’s‘Law of Time and Chaos’ uses the outside view to group together
evolutionary development, technological development, and computing into the
same category, and constructs a causal model predicting time to the ‘Singular-
ity’ [10]. Moore’s law (non-causal model) is a key input to this Law, and Ray
Kurzweil’s expertise is the main evidence for the Law’s accuracy.
This is the schema we will be using in this paper, and in the prediction
databases we have assembled. But the purpose of any such schema is to bring
clarity to the analysis, not to force every prediction into a particular box. Wehope that the methods and approaches used in this paper will be of general use
How We’re Predicting AI 15
to everyone wishing to analyse the reliability and usefulness of predictions, in AI
and beyond. Hence this schema can be freely adapted or discarded if a particularprediction does not seem to ﬁt it, or if an alternative schema seems to be more
useful for the analysis of the question under consideration.
3 A Toolbox of Assessment Methods
The purpose of this paper is not only to assess the accuracy and reliability of
some of the AI predictions that have already been made. The purpose is to start
building a ‘toolbox’ of assessment methods that can be used more generally,
applying them to current and future predictions.
3.1 Extracting Veriﬁable Predictions
The focus of this paper is squarely on the behaviour of AI. This is not a philo-
sophical point; we are not making the logical positivist argument that only em-
piricallyveriﬁablepredictionshavemeaning[12].Butitmustbenotedthatmany
of the vital questions about AI – can it be built, when, will it be dangerous, willit replace humans, and so on – all touch upon behaviour. This narrow focus
has the added advantage that empirically veriﬁable predictions are (in theory)
susceptible to falsiﬁcation, which mea ns ultimately agreementbetween people of
opposite opinions. Predictions like these have a very diﬀerent dynamic to those
that cannot be shown to be wrong, even in principle.
To that end, we will seek to reduce the prediction to an empirically veriﬁable
format. For some predictions, this is automatic: they are already in the correct
format. When Kurzweil wrote “One of my key (and consistent) predictions isthat a computer will pass the Turing test by 2029,” then there is no need to
change anything. Conversely, some philosophical arguments concerning AI, such
assomeofthevariantsoftheChineseRoomargument[13],arearguedto contain
noveriﬁablepredictionsatall:anAIthatdemonstratedperfecthumanbehaviour
would not aﬀect the validity of the argument.
And in between there are those predictio ns that are partially veriﬁable. Then
the veriﬁable piece must be clearly extra cted and articulat ed. Sometimes it is
ambiguitythatmustbeovercome:whenanauthorpredictsanAI“Omegapoint”in 2040 [14], it is necessary to read the paper with care to ﬁgure out what counts
as an Omega point and (even more importantly) what doesn’t.
Even purely philosophicalpredictions ca n have(or can be interpretedto have)
veriﬁable predictions. One of the most famous papers on the existence of con-
scious states is Thomas Nagel’s “What is it like to be a bat?” [15]. In this paper,
Nagel argues that bats must have mental states, but that we humans can never
understand what it is like to have these mental states. This feels purely philo-
sophical, but does lead to empirical predictions: that if the bat’s intelligencewere increased and we could develop a common language, then at some point in
the conversation with it, our understanding would reach an impasse. We would
try to describe what our internal mental states felt like, but would always failto communicate the essence of our experience to the other species.
16 S. Armstrong and K. Sotala
Many other philosophical papers can likewise be read as having empirical
predictions; as making certain states of the world more likely or less – even
if they seem to be devoid of this. The Ch inese Room argument, for instance,
argues that formal algorithms will lack the consciousness that humans possess
[13]. This may seem to be an entirely se lf-contained argument – but consider
that a lot of human behaviour revolves around consciousness, be it discussing
it, commenting on it, deﬁning it or intuitively noticing it in others. Hence if we
believed the Chinese Roomargument, and wereconfrontedwith twoAI projects,
one based on advanced algorithms and one based on modiﬁed human brains, we
would be likely to believe that the second project is more likely to result in
an intelligence that seemedconscious than the ﬁrst. This is simply because we
wouldn’t believe that the ﬁrst AI could ever be conscious, and that it is easier
to seem conscious when one actually is. And that gives an empirical prediction.
Note that the authors of the predictions may disagree with our ‘extracted’
conclusions. This is not necessarily a gam e breaker. For instance, even if there
is no formal link between the Chinese Room model and the prediction above,
it’s still the case that the intuitive reasons for believing the model are also good
reasons for believing the prediction. Our aim should always be to try and create
useful veriﬁable predictions in any way we can. In this way, we can make use of
much more of the AI literature. For instance, Lucas argues that AI is impossible
because it could not recognise the truth of its own G¨ odel sentence1[ 1 6 ] .T h i si sa
very strong conclusion, and we have to a ccept a lot of Lucas’s judgments before
we agree with it. Replacing the conclusio n with the weaker (and veriﬁable) “self
reference will be an issue with advanced AI, and will have to be dealt with
somehow by the programmers” gives us a useful prediction which is more likely
to be true.
Care must be taken when applying this method: the point is to extract a
useful veriﬁable prediction, not to weaken or strengthen a reviled or favoured
argument. The very ﬁrst stratagems in Shopenhauer’s “The Art of Always being
Right”[17]aretoextendandover-genera lisetheconsequencesofyouropponent’s
argument; conversely, one should reduce and narrow down one’s own arguments.
There is no lack of rhetorical tricks to uphold one’s own position, but if one
is truly after the truth, one must simply attempt to ﬁnd the most reasonable
empirical version of the argument; the truth-testing will come later.
This method often increases uncertainty, in that it often narrows the conse-
quences of the prediction, and allows more possible futures to exist, consistently
with that prediction. For instance, Bruce Edmonds [18], building on the “No
Free Lunch” results [19], demonstrates that there is no such thing as a universal
intelligence: no intelligence that performs better than average in every circum-
stance. Initially this seems to rule out AI entirely; but when one analyses what
this means empirically, one realises there is far less to it. It does not forbid an
1AG ¨odel sentence is a sentence G that can be built in any formal system containing
arithmetic. G is implicitly self-referential, as it is equivalent with “there cannot exist
a proof of G”. By construction, there cannot be a consistent proof of G from within
the system.
How We’re Predicting AI 17
algorithm from performing better than any human being in any situation any
human being would ever encounter, for instance. So our initial intuition, which
was to rule out all futures with AIs in them, is now replaced by the realisation
that we have barely put any constraints on the future at all.
3.2 Clarifying and Revealing Assumptions
The previous section was co ncerned with the predictions’ conclusions. Here we
will instead be looking at its assumptions, and the logical structure of the argu-
ment or model behind it. The objective is to make the prediction as rigorous as
possible
Philosophers love doing this: taking apart argument, adding caveats and
straightening out the hand-wavy logica l leaps. In a certain sense, it can be ar-
gued that analytic philosophy is entirely about making arguments rigorous.One
of the oldest methods in philosophy – the dialectic [20] – also plays this role,
with concepts getting clariﬁed during the conversationbetween philosophersand
various Athenians. Though this is perhaps philosophy’s greatest contribution to
knowledge, it is not exclusively the hunting ground of philosophers. All rational
ﬁelds of endeavour do – and should! – beneﬁt from this kind of analysis.
Of critical importance is revealing hidden assumptions that went into the
predictions. These hidden assumptions – sometimes called Enthymematic gaps
in the literature [21] – are very important because they clarify where the true
disagreements lie, and where we need to focus our investigation in order to ﬁnd
out the truth of prediction. Too often, competing experts will make broad-based
arguments that ﬂy past each other. This makes choosing the right argument a
matter of taste, prior opinions and our admiration of the experts involved. But
if the argument can be correctly deconstr ucted, then the source of the disagree-
ment can be isolated, and the issue can be decided on much narrower grounds –
and its much clearer whether the various experts have relevant expertise or not
(see Section 3.4). The hidden assumptions are often implicit, so it is perfectly
permissible to construct assumptions that the predictors were not consciously
aware of using.
Forexample,let’s lookagainattheG¨ odel argumentsmentioned in the Section
3.1. The argument shows that formal sy stems of a certain complexity must be
either incomplete (unable to see that their G¨ odel sentence is true)orinconsistent
(provingfalse statements). This iscontrastedwith humans, who– allegedly–use
meta-reasoningto know that their own G¨ odel statements are t rue. It should ﬁrst
be noted herethat noonehaswritten downan actual“humanG¨ odelstatement,”
so we cannot be sure humans would actually ﬁgure out that it is true.2Also,
humans are both inconsistent and able to deal with inconsistencies without a
complete collapse of logic. In this, the y tend to diﬀer from AI systems, though
some logic systems such as relevance logic do mimic the same behaviour [22].
In contrast, both humans and AIs are not logically omniscient – they are not
capable of proving everything provable within their logic system (the fact that
2One could argue that, by deﬁnition, a human G¨ odel statement must be one that
humans cannot recognise as being a human G¨ odel statement!
18 S. Armstrong and K. Sotala
there are an inﬁnite number of things to prove being the problem here). So
this analysis demonstrates the hidden assumption in Lucas’s argument: that thebehaviourofanactualcomputer programrunningon arealmachineismoreakin
to that of a logically omniscient formal agent, than it would be to a real human
being. That assumption may be ﬂawed or correct, but is one of the real sourcesof disagreement over whether G¨ odelian arguments rule out artiﬁcial intelligence.
Again, it needs to be emphasised that the purpose is to clarify and analyse
arguments, not to score points for one side or the other. It is easy to phrase
assumptions in ways that sound good or bad for either “side”. It is also easy to
take the exercise too far: ﬁnding more and more minor clariﬁcations or speciﬁchidden assumptions until the whole prediction becomes a hundred page mess
of over-detailed special cases. The pur pose is to clarify the argument until it
reaches the point where all (or most) parties could agree that these assumptionsare the real sources of disagreement. An d then we can consider what empirical
evidence, if available, or expert opinio n has to say about these disagreements.
There is surprisingly little published on the proper way of clarifying assump-
tions, making this approach more an art than a science. If the prediction comes
from a model, we have some standard tools available for clarifying, though [23].
Most of these methods work by varying parameters in the model and checking
that this doesn’t cause a breakdown in the prediction.
Model Testing and Counterfactual Resiliency
Though the above works from inside the model, there are very few methods that
can test the strength of a model from the outside. This is especially the case for
non-causal models: what are the assumptions behind Moore’s famous law [9],or Robin Hanson’s model that we are due for another technological revolution,
based on the timeline of previous revolutions [24]? If we can’t extract assump-
tions,we’rereducedtosaying“thatfeelsright/wrongtome”,andthereforewe’re
getting nowhere.
The authors have come up with a putative way of testing the assumptions
of such models (in the case of Moore’s law, the empirical evidence in favour is
strong, but there is still the question of what is powering the law and whether
it will cross over to new chip technologies again and again). It involves givingthe model a counterfactual resiliency check: imagining that world history had
happened slightly diﬀerently, and checking whether the model would have stood
up in those circumstances . Counterfactual changes are permitted to anything
that the model ignores.
The purpose of this exercise is not to rule out certain models depending on
one’s own preferred understanding of history (e.g. “Protestantism was essential
to the industrial revolution, and was a ﬂuke due to Martin Luther; so it’s very
likely that the industrial revolution would not have happened in the way ortimeframe that it did, hence Hanson’s model – which posits the industrial rev-
olutions’s dates as inevitable – is wrong”). Instead it is to illustrate the tension
between the given model and other models of history (e.g. “The assumptionsthat Protestantism was both a ﬂuke and essential to the industrial revolution
How We’re Predicting AI 19
are in contradiction with Hanson’s model. Hence Hanson’s model implies that
either Protestantism was inevitable or that it was non-essential to the industrial
revolution, an extra hidden assumption”). The counterfactual resiliency exercise
has been carried out at length in an online post.3The general verdict seemed to
be that Hanson’s model contradicted a lot of seemingly plausible assumptions
about technological and social development. Moore’s law, on the other hand,
seemed mainly dependent on the continu ing existence of a market economy and
the absence of major catastrophes.
This method is new, and will certainly be reﬁned in future. Again, the pur-
pose of the method is not to rule out certain models, but to ﬁnd the nodes of
disagreement.
More Uncertainty
Clarifying assumptions often ends up in creasing uncertainty , as does revealing
hidden assumptions. The previous section focused on extracting veriﬁable pre-
dictions, which often increases the range of possible worlds compatible with a
prediction. Here, by clarifying and cav eatting assumptions, and revealing hid-
den assumption, we reduce the number of worlds in which the prediction is valid.
This means that the prediction puts fewer constraints on our expectations. In
counterpart, of course, the caveatte d prediction is more likely to be true.
3.3 Empirical Evidence
The gold standard in separating true predictions from false ones must always
be empirical evidence. The scientiﬁc method has proved to be the best way
of disproving false hypotheses, and should be used whenever possible. Other
methods, such as expert opinion or unjustiﬁed models, come nowhere close.
The problem with empirical evidence is that ... it is generally non-existent
in the AI prediction ﬁeld. Since AI predictions are all about the existence and
properties of a machine that hasn’t yet been built, that no-one knows how to
build or whether it actually can be built, there is little opportunity for the whole
hypothesis-prediction-testing cycle. This should indicate the great diﬃculties in
the ﬁeld. Social sciences, for instance, ar e often seen as the weaker cousins of the
hard sciences, with predictions much mo re contentious and less reliable. And yet
the social sciences make use of the scientiﬁc method, and have access to some
types of repeatable experiments. Thus a ny prediction in the ﬁeld of AI should
be treated as less likely than any social science prediction.
That generalisationis somewhatover-harsh.Some AI prediction methods hew
closer to the scientiﬁc method, such as the whole brain emulations model [6] – it
makes testable predictions along the way. Moore’s law is a wildly successful pre-
diction, and connected to some extent wi th AI. Many predictors (e.g. Kurzweil)
make partial predictions on the road towards AI; these can and should be as-
sessed – track records allow us to give some evidence to the proposition “this
3Seehttp://lesswrong.com/lw/ea8/
counterfactual
 resiliency
 test
for
noncausal
20 S. Armstrong and K. Sotala
expert knows what they’re talking about.” And some models also allow for a
degree of testing. So the ﬁeld is not void of empirical evidence; it’s just that
there is so little of it, and to a large extent we must put our trust in expert
opinion.
3.4 Expert Opinion
Reliance on experts is nearly unavoidable in AI prediction. Timeline predictions
are often explicitly based on experts’ fee lings; even those that consider factors
about the world (such as computer sp eed) need an expert judgment about why
that factor is considered and not othe rs. Plans need experts to come up with
them and judge their credibility. And unless every philosopher agrees on the
correctness of a particula r philosophical argument, we are dependent to some
degree on the philosophical judgment of the author. It is the purpose of all the
methods described above that we can reﬁne and caveat a prediction, back it
up with empirical evidence whenever possible, and thus clearly highlight the
points where we need to rely on expert opinion. And so can focus on the last
remaining points of disagreement: the premises themselves (that is of course the
idealsituation:somepredictionsaregivendirectlywith nootherbasisbut expert
authority, meaning there is nothing to reﬁne).
Should we expect experts to be good at this task? There have been several
projects over the last few decades to establish the domains and tasks where we
would expect experts to have good performance [25, 26]. Table 1 summarises the
results:
Table 1. Table of task properties conducive to good and poor expert performance
Good performance:
 Poor performance:
Static stimuli
 Dynamic (changeable) stimuli
Decisions about things
 Decisions about behaviour
Experts agree on stimuli
 Experts disagree on stimuli
More predictable problems
 Less predictable problems
Some errors expected
 Few errors expected
Repetitive tasks
 Unique tasks
Feedback available
 Feedback unavailable
Objective analysis available
 Subjective analysis only
Problem decomposable
 Problem not decomposable
Decision aids common
 Decision aids rare
Not all of these are directly applicable to the current paper (are predictions
about human level AIs predictions about things, or about behaviour?). One of
themostimportantfactorsiswhetherexpe rtsgetfeedback,preferablyimmediate
feedback. We should expect the best expert performance when their guesses are
immediatelyconﬁrmedordisconﬁrmed.Whenfeedbackisunavailableordelayed,
How We’re Predicting AI 21
or the environment isn’t one that gives good feedback, then expert performance
drops precipitously [26, 11].
Table 1 applies to both domain and task. Any domain of expertise strongly
in the right column will be one where we expect poor expert performance. But
if the individual expert tries to move their own predictions into the left column(maybe by decomposing the problem as far as it will go, training themselves
on related tasks where feedback is available...) they will be expected to perform
better. In general, we should encourage this type of approach.
When expertsfail,thereareoftensimplealgorithmicmodelsthat demonstrate
better performance [27]. In these cases, the experts often just spell out theircriteria, design the model in consequence, and let the model give its predictions:
this results in better predictions than simply asking the expert in the ﬁrst place.
Hence we should also be on the lookout for experts who present their ﬁndingsin the form of a model.
As everyone knows, experts sometimes disagree. This fact strikes at the very
heart of their supposed expertise. We listen to them because they have the skillsand experience to develop correct insigh ts. If other experts have gone through
the same process and come to an opposite conclusion, then we have to conclude
that their insights do not derive from their skills and experience, and hence
should be discounted. Now if one expert opinion is a fringe position held by only
a few experts, we may be justiﬁed in dismissing it simply as an error. But ifthere are diﬀerent positions held by lar ge numbers of disagreeing experts, how
are we to decide between them? We need some sort of objective criteria: we are
not experts in choosing between experts, so we have no special skills in decidingthe truths on these sorts of controversial positions.
What kind of objective criteria cou ld there be? A good track record can be
an indicator, as is a willingness to make veriﬁable, non-ambiguous predictions.
A better connection with empirical knowledge and less theoretical rigidity are
also positive indications [28], and any expert that approached their task withmethods that were more on the left of the table than on the right should be
expected to be more correct. But thes e are second order phenomena – we’re
looking at our subjective interpretatio n of expert’s subjective opinion – so in
most cases, when there are strong dis agreement between experts, we simply
can’t tell which position is true.
Grind versus Insight
Some AI prediction claim that AI will result from grind: i.e. lots of hard work
and money. Other claim that AI will need special insights: new unexpected ideas
that will blow the ﬁeld wide open [7].
In general, we are quite good at predicting grind. Project managers and var-
ious leaders are often quite good at estimating the length of projects (as longas they’re not directly involved in the pro ject [29]). Even for relatively creative
work, people have suﬃcient feedback to ha zard reasonable guesses. Publication
dates for video games, for instance, though often over-optimistic, are generallynot ridiculously erroneous – even though video games involve a lot of creative
22 S. Armstrong and K. Sotala
design, play-testing, art, programming the game “AI”, etc. Moore’s law could
be taken as an ultimate example of grid: we expect the global eﬀorts of manyengineers across many ﬁelds to average ou t to a rather predictable exponential
growth.
Predicting insight, on the other hand, seems a much moredaunting task. Take
the Riemann hypothesis, a well-established mathematical hypothesis from 1885,
[30]. How would one go about estimating how long it would take to solve? How
about the P=NPhypothesis in computing? Mathematicians seldom try and
predict when major problems will be solved, because they recognise that insight
is very hard to predict. And even if predictions could be attempted (the age ofthe Riemann’s hypothesis hints that it probably isn’t right on the cusp of being
solved), they would need much larger e rror bars than grind predictions. If AI
requires insights, we are also handicapped by the fact of not knowing what theseinsights are (unlike the Riemann hypothe sis, where the hypothesis is clearly
stated, and only the proof is missing). This could be mitigated somewhat if we
assumed there wereseveraldiﬀerent insights, each of which could separatelyleadto AI. But we would need good grounds to assume that.
Does this mean that in general predictions that are modelling grind should
be accepted more than predictions that are modelling insight? Not at all. Pre-
dictions that are modelling grind should only be accepted if they can make a
good case that producing an AI is a matter grind only. The predictions aroundwhole brain emulations [6], are one of the few that make this case convincingly;
this will be analysed in a subsequent paper.
Non-experts Opinion
It should be born in mind that all the caveats and problems with expert opinion
apply just as well to non-experts. With o ne crucial diﬀerence: we have no reason
to trust the non-expert’s opinion in the ﬁrst place. That is not to say that non-experts cannot come up with good models, convincing timelines, or interesting
plans and scenarios. It just means that our assessment of the quality of the
prediction depends only on what we are given; we cannot extend a non-expertany leeway to cover up a weak premise or a faulty logical step. To ensure this,
we should try and assess non-expert predictions blind, without knowing who the
author is. If we can’t blind them, we can try and get a similar eﬀect by askingourselves hypothetical questions such as: “Would I ﬁnd this prediction more or
less convincing if the author was the Archbishop of Canterbury? What if it was
Warren Buﬀet? Or the Unabomber?” We should aim to reach the point where
hypotheticalchangesinauthorshipdonotaﬀectourestimationofthe prediction.
4 Timeline Predictions
The practical focus of this paper is on AI timeline predictions: predictions givingdates for AIs with human-comparable cognitive abilities. Researchers from the
Singularity Institute have assembled a database of 25AI predictions since 1950,of which 95 include AI timelines.
How We’re Predicting AI 23
4.1 Subjective Assessment
A brief glance at Table 1 allows us to expect that AI timeline predictions will
generally be of very poor quality. The only factor that is unambiguously positive
for AI predictions is that prediction errors are expected and allowed: apart from
that, the task seems singularly diﬃcult, especially on the key issue of feedback.
An artiﬁcial intelligence is a hypothetical machine, which has never existed on
this planet before and about whose properties we have but the haziest impres-
sion. Most AI experts will receive no fee dback whatsoever about their predic-
tions, meaning they have to construct them entirely based on their untested
impressions.
Thereisnothingstoppingexpertsfromdecomposingtheproblem,orconstruct-
ing models which they then calibrate with available data, or putting up interim
predictionstotesttheirassessment.Andsomedousethesebetterapproaches(see
for instance [10, 5, 31]). But a surprisingly large number don’t! Some predictions
are unabashedly based simply on the feelings of the predictor [32, 33].
Yet another category are of the “Moore’s law hence AI” type. They postulate
that AI will happen when computers rea ch some key level, often comparing
with some key property of the brain (number of operations per second [34], or
neurones/synapses4).Inthedivisionestablishedinsection3.4,thisispure‘grind’
argument: AI will happen after a certain amount of work is performed. But, as
we saw, these kinds of arguments are only valid if the predictor has shown that
reaching AI does not require new insights! And that step is often absent from
the argument.
4.2 Timeline Prediction Data
The above were subjective impressions, formed while looking over the whole
database.To enablemore rigorousanalysis,the varioustimeline predictions were
reduced to a single number for purposes of comparison: this would be the date
upon which the predictor expected ‘human level AI’ to be developed.
Unfortunately not all the predictions were in the same format. Some gave
ranges, some gave median estimates, some talked about superintelligent AI, oth-
ers about slightly below-human AI. In order to make the numbers comparable,
one of the authors (Stuart Armstrong) went through the list and reduced the
various estimates to a single number. He followed the following procedure to
extract a “Median human-level AI estimate”:
When a rangewas given,he took the mid-point of that range (rounded down).
Ifayearwasgivenwith a50%likelihoodestimate,he tookthatyear.Ifit wasthe
collection of a variety of expert opinions, he took the prediction of the median
expert. If the predictor foresaw some sort of AI by a given date (partial AI
or superintelligent AI), and gave no other estimate, he took that date as their
estimate rather than trying to correct it in one direction or the other (there were
4See for instance Dani Eder’s 1994 Newsgroup posting
http://www.aleph.se/Trans/Global/Singularity/singul.txt
24 S. Armstrong and K. Sotala
roughly the same number of subhuman AIs as suphuman AIs in the list, and not
that many of either). He read extracts of the papers to make judgement calls
when interpreting problematic statements like “within thirty years” or “during
this century” (is that a range or an end-date?). Every date selected was either
an actual date given by the predictor, or the midpoint of a range.5
It was also useful to distinguish between popular estimates, performed by
journalists, writers or amateurs, from those predictions done by those with ex-
pertiseinrelevantﬁelds(AI research,co mputersoftwaredevelopment,etc.)Thus
each prediction was noted as ‘expert’ or ‘n on-expert’; the expectation being that
experts would demonstrate improve d performance over non-experts.
Figure 1 graphs the results of this exer cise (the range has been reduced; there
weresevenpredictionssetting dates beyondthe year2100,three ofthem expert.)
Fig. 1.Median estimate for human-level AI, graphed against date of prediction
As can be seen, expert predictions span the whole range of possibilities and
seem to have little correlation with each other. The range is so wide – ﬁfty
year gaps between predictions are commo n – that it provides strong evidence
that experts are not providing good pred ictions. There does not seem to be any
visible diﬀerence between expert and non-e xpert performance ei ther, suggesting
that the same types of reasoning may be used in both situations, thus negating
the point of expertise.
5The data can be found at
http://www.neweuropeancentury.org/SIAI-FHI\_AI\_predictions.xls ;
readers are encouraged to come up with their own median estimates.
How We’re Predicting AI 25
Twoexplanationshavebeengenerallyad vancedto explainpoorexpertperfor-
mance in these matters. The ﬁrst, the so-called Maes-Garreau law6posits that
AI experts predict AI happening towards the end of their own lifetime. This
would make AI into a technology that would save them from their own deaths,
akin to a ‘Rapture of the Nerds’.
The second explanation is that AI is per petually ﬁfteen to twenty-ﬁve years
into the future. In this way (so the explanation goes), the predictor can gain
credit for working on something that will be of relevance, but without any possi-
bility that their prediction could be shown to be false within their currentcareer.
We’ll now look at the evidence for these two explanations.
Nerds Don’t Get Raptured
Fifty-ﬁve predictions were retained, in which it was possible to estimate the pre-
dictor’s expected lifespan. Then the diﬀ erence between their median prediction
and this lifespan wascomputed (a positive diﬀerence meaning they would expect
to die before AI, a negative diﬀerence meaning they didn’t). A zero diﬀerence
would be a perfect example ofthe Maes-Ga rreaulaw: the predictorexpects AI to
be developed at the exact end of their life. This number was then plotted again
thepredictor’sageinFigure2(theplotwasrestrictedtothosepredictionswithin
thirty years of the predic tor’s expected lifetime).
From this, it can be seen that the Maes-Garreau law is not born out by the
evidence: only twelve predictions (22% of the total) were within ﬁve years in
either direction of the zero point.
Twenty Years to AI
The ‘time to AI’ was computed for each expert prediction. This was graphed in
Figure 3. This demonstrates a deﬁnite increase in the 16–25 year predictions: 21
of the 62 expert predictions were in that range (34%). This can be considered
weak evidence that experts do indeed pr efer to predict AI happening in that
range from their own time.
But the picture gets more damning when we do the same plot for the non-
experts,asinFigure4.Here,13 ofthe33predictions areinthe 16–25yearrange.
But more disturbingly, the time to AI graph is almost identical for experts and
non-experts! Though this does not preclude the possibility of experts being more
accurate,itdoes hintstronglythatexperts andnon-expertsmaybe using similar
psychological procedures when creating their estimates.
The next step is to look at failed predictions. There are 15 of those, most
dating to before the ‘AI winter’ in the eighties and nineties. These have been
graphed in Figure 5 – and there is an uncanny similarity with the other two
graphs! So expert predictions are not only indistinguishable from non-expert
predictions, they are also indistinguishable from past failed predictions. Hence
it is not unlikely that recent predictions are suﬀering from the same biases and
errors as their predecessors
6Kevin Kelly, editor of Wired magazine, cr eated the law in 2007 af ter being inﬂuenced
by Pattie Maes at MIT and Joel Garreau (author of Radical Evolution).
26 S. Armstrong and K. Sotala
Fig. 2.Diﬀerence between the predicted time to AI and the predictor’s life expectancy,
graphed against the predictor’s age
Fig. 3.Time between thearrival of AI andthe date the prediction was made, for expert
predictors
How We’re Predicting AI 27
Fig. 4.Time between the arrival of AI and the date the prediction was made, for
non-expert predictors
Fig. 5.Time between the arrival of AI and the date the prediction was made, for failed
predictions
28 S. Armstrong and K. Sotala
5C o n c l u s i o n
Thispaper,theﬁrstinaseriesanalysingAIpredictions,focusedonthe reliability
of AI timeline predictions (predicting the dates upon which ‘human-level’ AIwould be developed). These predictions are almost wholly grounded on expert
judgment. The biases literature classiﬁed the types of tasks on which experts
wouldhavegoodperformance,and AItimeline predictionshaveallthehallmarks
of tasks on which they would perform badly.
This was born out by the analysis of 95 timeline predictions in the database
assembled by the Singularity Institute. There were strong indications therein
that experts performed badly. Not only w ere expert predictions spread across a
wide range and in strong disagreement w ith each other, but there was evidence
that experts were systematically preferring a ‘15 to 25 years into the future’
prediction. In this, they were indistinguishable from non-experts, and from past
predictionsthatareknowntohavefailed.Thereisthusnoindicationthatexperts
brought any added value when it comes to estimating AI timelines. On the other
hand, another theory – that experts were systematically predicting AI arrivaljust before the end of their own lifetime – was seen to be false in the data we
have.
There is thus strong grounds for dramatically increasing the uncertainty in
any AI timeline prediction.
References
1. Turing, A.: Computing machinery and intelligence. Mind 59, 433–460 (1950)
2. Jacquette, D.: Metamathematical criteria for minds and machines. Erkenntnis
27(1) (1987)
3. Darrach, B.: Meet Shakey, the ﬁrst electronic person. Reﬂections of the Future
(1970)
4. Hall, J.S.: Further reﬂections on the timescale of AI. In: Dowe, D.L. (ed.)
Solomonoﬀ Festschrift. LNCS, vol. 7070, pp. 174–183. Springer, Heidelberg (2013)
5. Hanson, R.: What if uploads come ﬁrst: The crack of a future dawn. Extropy 6(2)
(1994)
6. Sandberg, A.: Whole brain emulations: A roadmap. Future of Humanity Institute
Technical Report 2008-3 (2008)
7. Deutsch, D.: The very laws of physics imply that artiﬁcial intelligence must be
possible. What’s holding us up? Aeon (2012)
8. Omohundro, S.: Basic ai drives. In: Proceedings of the First AGI Conference,
vol. 171 (2008)
9. Moore, G.: Cramming more components onto integrated circuits. Electronics 38(8)
(1965)
10. Kurzweil, R.: The Age of Spiritual Machines: When Computers Exceed Human
Intelligence. Viking Adult (1999)
11. Kahneman, D.: Thinking, Fast and Slow. Farra, Straus and Giroux (2011)12. Carnap, R.: The Logical Structure of the World (1928)13. Searle, J.: Minds, brains and programs. Behavioral and Brain Sciences 3(3), 417–
457 (1980)
How We’re Predicting AI 29
14. Schmidhuber, J.: Artiﬁcial General Intelligence, pp. 177–200 (2006)
15. Nagel, T.: What is it like to be a bat? The Philosophical Review 83(4), 435–450
(1974)
16. Lucas, J.: Minds, machines and G¨ odel. Philosophy XXXVI, 112–127 (1961)
17. Schopenhauer, A.: The Art of Being Right: 38 Ways to Win an Argument (1831)
18. Edmonds, B.: The social embedding of intelligence. In: Parsing the Turing Test,
pp. 211–235. Springer, Netherlands (2009)
19. Wolpert, D., Macready, W.: No free lunch theorems for search (1995)
20. Plato: The Republic (380 BC)21. Fallis, D.: Intentional gaps in mathematical proofs. Synthese 134(1-2) (2003)
22. Routley, R., Meyer, R.: Dialectical logic, classical logic, and the consistency of the
world. Studies in East European Thought 16(1-2) (1976)
23. Morgan, M., Henrion, M.: Uncertainty: A guide to dealing with uncertainty in
quantitative risk and policy analysis. Cambridge University Press (1990)
24. Hanson, R.: Economics of brain emulations. In: Unnatrual Selection - The Chal-
lenges of Engineering Tomorrow’s People, pp. 150–158 (2008)
25. Shanteau, J.: Competence in experts: The role of task characteristics. Organiza-
tional Behavior and Human Decision Processes 53, 252–266 (1992)
26. Kahneman, D., Klein, G.: Conditions for intuitive expertise: A failure to disagree.
American Psychologist 64(6), 515–526 (2009)
27. Grove, W., Zald, D., Lebow, B., Snitz, B., Nelson, C.: Clinical versus mechanical
prediction: A meta-analysis. Psychological Assessment 12, 19–30 (2000)
28. Tetlock, P.: Expert political judgement: How good is it? How can we know? (2005)
29. Buehler, R., Griﬃn, D., Ross, M.: Exploring the planning fallacy: Why people
underestimate their task completion times. Journal of Personality and Social Psy-
chology 67, 366–381 (1994)
30. Riemann, B.: ¨Uber die Anzahl der Primzahlen unter einer gegebenen Gr¨ osse.
Monatsberichte der Berliner Akademie (1859)
31. Waltz, D.: The prospects for building truly intelligent machines. Daedalus 117(1)
(1988)
32. Good, J.: The scientist speculates: An anthology of partly-baked ideas. Heinemann
(1962)
33. Armstrong, S.: Chaining god: A qualitative approach to AI, trust and moral sys-
tems (2007) (online article)
34. Bostrom, N.: How long before superintelligence? International Journal of Futures
Studies 2 (1998)