Testing the Automation Revolution Hypothesis

Introduction Since at least 2013, many have claimed that we are entering an automation revolution, and so should soon expect large trenddeviating increases in job automation, in related job losses, and in the determinants of automation. As context for considering such claims, we study what predicts which jobs have been how automated in the recent past, how the best automation predictors have changed over time, and recent correlations between changes in automation, pay, and employment.
 Material We combine data from three sources  (Scholl and Hanson, 2020) . First, O*NET is a database of U.S. job feature scores made by surveying occupational experts and employees, who compare ✩ For financial support we thank Open Philanthropy Project for funding ''Research into Future Artificial Intelligence Scenarios.'' They did not influence our study design or interpretation. each job to related jobs. For each of 2144 jobs, O*NET includes 261 job features, many scored on a 0-5 numerical scale. In its current form O*NET started in 2002, though that year's entries include data from 1999. We project these scores onto 881 ''sixdigit'' level jobs, via scaling frequency by importance. We thus have scores of 261 job features for 881 jobs from 1999 to 2019. One key O*NET job feature is years of education, and another is ''degree of automation'', which seems to represent expert judgments on which workers would have to do each task now done by machines, absent those machines. Second, from Occupational Employment Statistics (OES), we obtain, for all these jobs and for years 1999 to 2018, U.S. annual averages for employment (i.e., number of employees) and an inflation-adjusted mean hourly pay (in U.S. dollars). Our third source is two widely-discussed expert-judgmentderived metrics regarding the vulnerability of jobs to near future automation. These metrics do not vary by year. Computerisable comes from  Frey and Osborne (2017) , first published in 2013, and ranges 0 to 1. Building on judgments made by a ''group of machine learning researchers'', they estimated 47% of U.S. jobs to be at ''high risk'' of being ''computerisable'', ''perhaps over the next decade or two''. Machine Learning Suitability, coming directly from an author of  Brynjolfsson et al. (2018) , was created from machine learning expert judgments using a 23-item rubric, and ranges 2 to 5. We transform all variables (besides time and intercept) into rough ''z-score'' variants. We apply a logarithmic transform to 0.01 plus pay, employment, and each O*NET variable. We then rescale them to have zero mean and unit standard deviation. Time is rescaled to take value zero at 1999, and one at 2019. Our transformed variables look closer to normally distributed. As we use transformed versions in our analysis, most regression coefficients say how many standard deviations of increase in a dependent variable is predicted by a one standard deviation increase in an independent variable. Most O*NET job features were not scored for each job every year, but were on average scored 3.3 times during our 1999-2019 period. From available scores, we interpolate scores for all years using two methods. In piecewise-linear interpolation, we fit straight lines in time between scorings that are adjacent in time, and fit zero-slope lines for years outside the time range of available scorings. In regression interpolation, we fit a linear regression in time to all available job-feature scorings. This method requires at least two job-feature scorings. While our dataset officially includes 881 jobs and 260 O*NET job features (besides education), data limitations force a tradeoff between how many jobs and features we can include. We choose to maximize the product of these at 832 jobs and 251 features. Table 3 further selects for jobs with enough data to allow regression interpolation for all features used.
 Methods Table  1  gives basic statistics for all variables in the other tables. Table  2  describes seven ordinary least squares regressions, all predicting automation. Each data point corresponds to a notinterpolated O*NET scoring of an automation value for a job in a year; independent variables are piecewise-linear interpolated. These 25 O*NET variables were selected via the LASSO method out of the 251 available O*NET variables via a structure like Model 3. Models 2,5 add extra columns for each variable multiplied by time, and Models 6,7 apply the same structure of Model 4 separately to times <0.5 and >0.5. A simple model not shown, using only an intercept and time, gives a time coefficient of 0.102, not significant at the 10% level. Table  3  describes regression models predicting changes in pay and employment. All change variables are constructed via regression interpolation. Changes are not renormalized into z-scores; they are differences in normalized z-scores. These models help us test these null hypotheses: (1) metrics constructed to forecast future automation do not predict past automation, (2) best predictors of automation have not changed in two decades, (3) automation changes do not predict changes in pay or employment.
 Results The first two models of Table  2  suggest that five variables, plus time and an intercept, have substantial predictive power, explaining roughly 15% of automation variation. Automation requires fixed costs, but saves on worker marginal costs. Simple theory thus predicts that, all else equal, employers are more eager to automate jobs with higher pay and employment. These two factors should thus predict job automation, and we do in fact see such effects, though more consistently for pay. Two automation vulnerability metrics built from expert judgments on which jobs seem easier to automate in the future predict which jobs were more automated in the last two decades. Education does not, after our other controls. Model 4 suggests that, aside perhaps from education, these predictors have little to add to the predictive power of the top 25 O*NET variables, which can explain over half of automation variance, even when interpolated. Our two strongest O*NET predictors are Pace Determined By Speed Of Equipment and Importance of Repeating Same Tasks, with coefficients of 0.58 and 0.23. Following these, we find a set of four predictors at roughly 0.14, and a set of five at roughly 0.08. Most of these predictors seem understandable in terms of traditional styles of job automation. For example, Pace Determined By Speed Of Equipment picks out jobs that coordinate closely with machinery, while Importance of Repeating Same Tasks picks out jobs with many similar and independent small tasks. The coefficients in Table  2  that probe time effects do not offer much support for the claim that automation predictors have changed noticeably over time. Model 5 finds only three of 31 time-interaction coefficients are 10% significant, the number to be expected at random. Models 6,7, show no clear differences between time periods. Though we expect automation to have increased over our period, we find no significant time coefficient using only an intercept and time, and adding many controls makes automation seem to have decreased with time. As scores come from expert judgments made at different times, this decrease may be due to drifting standards on what it takes for a job to seen as more automated  (Levari et al., 2018) . If so, the difference between these two time coefficients, roughly 0.37 standard deviations of automation, can be interpreted as estimating this much of an increase in the average suitability of jobs for automation over this period. In Table  3 , changes in pay or employment are only significantly predicted by changes in job automation in one model out of six, where the sign is the opposite of the usual fear. If real, that one coefficient confirms the basic theory that employers gain more from automating jobs with more workers. Changes in pay and employment consistently predict each other, and with large coefficients. This suggests that, in supply and demand terms, labor market changes are on average better seen as changes in demand, and less as changes in supply. Increases in education consistently predict declines in pay and employment, though with small coefficients, suggesting that falling labor demand has been positively correlated with increasing education. Perhaps labor shortages (surpluses) induce firms to adopt weaker (stricter) education requirements. Terms in Table  3  that interact changes in employment and pay with changes in automation suggest that the positive correlation between pay and employment changes is weaker for jobs that saw increased automation.
 Conclusions Recently, many have said we are entering an automation revolution which will soon produce large trend-deviating increases in automation levels and resulting job losses, and also big changes in which kinds of jobs are more vulnerable to automation. We do not yet see evidence of such a revolution. Using 1505 expert reports on the degree of automation of particular jobs 1999-2019, we find that reported automation levels have not changed noticeably, though changing reporting standards may mask such increases. Jobs with larger automation increases did not on average see noticeable changes in pay or employment. Both pay and employment predict automation in the direction suggested by simple theory, as do expert judgments on which jobs are vulnerable to future automation. We can explain over half the variance in which jobs have been how automated using the top 25 O*NET variables, which are relatively mundane and understandable in terms of traditional kinds of automation. All these best predictors have not noticeably changed in two decades. Table 1 1 Variable statistics. Variable Untransformed Transformed Mean Std Dev Min Max Min Max Time 2007.50 6.16 1999 2019 0 1 Education 14.31 2.60 10.28 22.88 -1.839 2.824 Employees 167538 387560 200 4612510 -3.428 2.857 Pay 24.67 13.94 7.18 129.62 -2.331 3.735 Machine Learning Suitability 3.466 0.115 2.780 3.902 -6.609 3.586 Computerisable 0.536 0.368 0.003 0.990 -2.284 0.845 O*NET: Activity 3.374 0.397 1.750 4.620 -5.297 2.629 Advancement 2.723 0.423 1.250 4.000 -4.701 2.444 Cramped 1.932 0.764 1.000 4.900 -1.562 2.680 Dynamic Strength 0.079 0.086 0.000 0.736 -1.389 2.303 Fine Arts 0.040 0.110 0.000 0.974 -0.859 3.784 Gross Body Equilibrium 0.060 0.072 0.000 0.574 -1.326 2.450 Hearing Sensitivity 0.125 0.090 0.000 0.904 -3.253 2.944 Importance of Repeating Same Tasks 2.859 0.846 1.100 5.000 -2.801 1.883 Indoors Environmentally Controlled 3.998 0.945 1.000 5.000 -4.356 0.853 Innovation 3.552 0.489 1.880 4.880 -4.345 2.276 Letters and Memos 3.264 0.791 1.090 5.000 -3.894 1.693 Mathematics 0.264 0.164 0.000 0.989 -4.648 2.218 Number Facility 0.199 0.117 0.000 0.800 -4.712 2.491 On Knees 1.982 0.689 1.000 4.660 -2.205 2.794 Pace Determined by Speed of Equipment 1.839 0.826 1.000 4.760 -1.303 2.588 Physical Proximity 3.451 0.688 1.290 5.000 -4.729 1.923 Spend Time Keeping or Regaining Balance 1.570 0.525 1.000 4.310 -1.321 3.486 Spend Time Sitting 3.126 0.975 1.010 5.000 -3.112 1.523 Supervision Human Relations 3.200 0.467 1.250 4.620 -5.539 2.274 Supervision Technical 2.781 0.585 1.120 4.620 -3.730 2.256 Support 3.778 0.979 1.250 7.000 -5.753 2.618 Thinking Creatively 0.312 0.206 0.000 0.928 -3.456 1.507 Variety 2.791 0.637 1.120 4.120 -3.583 1.699 Visualization 0.221 0.113 0.000 0.657 -5.038 2.028 Wear Common Safety Equipment 2.736 1.346 1.000 5.000 -1.578 1.355
 Table 2 2 Predicting Automation. Model (1) (2) (3) (4) (5) (6) (7) Intercept 0.2146*** 0.3397*** 0.1474*** 0.1755*** 0.1939* 0.2809*** -0.0097 (0.0550) (0.1043) (0.0435) (0.0456) (0.1011) (0.0828) (0.1545) Time -0.2982*** -1.0041** 0.7372** -0.2451*** -0.2682*** -0.5104 0.2796 -0.6632*** -0.0351 (0.0944) (0.3948) (0.3538) (0.0849) (0.0857) (0.4035) (0.3918) (0.2386) (0.1984) Education 0.0099 -0.1101 0.2452 0.0906** 0.0370 0.0630 0.0938 0.0681 (0.0444) (0.1114) (0.1925) (0.0453) (0.1093) (0.1929) (0.0657) (0.0637) Employees 0.0893*** 0.0422 0.0883 0.0362* -0.0024 0.0611 0.0339 0.0296 (0.0248) (0.0593) (0.1056) (0.0201) (0.0496) (0.0865) (0.0295) (0.0288) Pay 0.2286*** 0.2734*** -0.0937 0.0508 0.0527 -0.0267 0.0586 0.0206 (0.0369) (0.0905) (0.1555) (0.0342) (0.0856) (0.1445) (0.0518) (0.0464) Computerisable 0.3356*** 0.3771*** -0.0820 0.0048 0.0067 0.0069 0.0012 0.0232 (0.0312) (0.0754) (0.1326) (0.0276) (0.0679) (0.1180) (0.0411) (0.0368) M.L. Suitability 0.2161*** 0.2786*** -0.1212 0.0006 0.0273 -0.0463 0.0188 -0.0181 (0.0246) (0.0581) (0.1005) (0.0202) (0.0482) (0.0825) (0.0289) (0.0281) Activity 0.0336 0.0142 -0.0448 0.1116 -0.0082 0.0340 (0.0210) (0.0220) (0.0527) (0.0938) (0.0320) (0.0301) Advancement 0.0656*** 0.0551** 0.1036* -0.0991 0.0605 0.0346 (0.0250) (0.0254) (0.0618) (0.1086) (0.0370) (0.0346) Cramped -0.0527 -0.0594 -0.1391 0.1811 -0.1138* 0.0161 (0.0393) (0.0399) (0.0975) (0.1672) (0.0592) (0.0542) Dynamic Strength -0.0663 -0.0552 -0.0060 -0.1442 -0.0420 -0.1199* (0.0505) (0.0508) (0.1228) (0.2159) (0.0751) (0.0706) Fine Arts -0.0463* -0.0358 -0.0932 0.1117 -0.0706* -0.0157 (0.0267) (0.0270) (0.0637) (0.1108) (0.0401) (0.0368) Gross Body Equilibrium 0.0036 -0.0024 0.0260 -0.0247 0.0269 0.0112 (0.0489) (0.0489) (0.1182) (0.2042) (0.0732) (0.0664) Hearing Sensitivity -0.0713** -0.0757*** -0.0311 -0.0857 -0.0560 -0.0966** (0.0291) (0.0291) (0.0615) (0.1218) (0.0383) (0.0477) Importance of Repeating Same Tasks 0.2240*** 0.2254*** 0.1513** 0.1302 0.1975*** 0.2587*** (0.0317) (0.0326) (0.0770) (0.1458) (0.0449) (0.0546) Indoors Environmentally Controlled 0.1375*** 0.1357*** 0.1491*** -0.0382 0.1358*** 0.1316*** (0.0242) (0.0245) (0.0553) (0.1003) (0.0330) (0.0376) Innovation -0.0887*** -0.0924*** -0.0723 -0.0407 -0.0919*** -0.0938** (0.0248) (0.0251) (0.0545) (0.0999) (0.0346) (0.0385) Letters and Memos 0.1624*** 0.1457*** 0.0748 0.1467 0.1185*** 0.1882*** (0.0273) (0.0281) (0.0653) (0.1151) (0.0399) (0.0408) Mathematics 0.0979** 0.0838** 0.0770 0.0699 0.0734 0.1610** (0.0407) (0.0412) (0.0968) (0.1893) (0.0569) (0.0661) Number Facility 0.0415 0.0400 -0.0151 0.0795 0.0177 0.0141 (0.0359) (0.0359) (0.0794) (0.1679) (0.0467) (0.0624) On Knees -0.0462 -0.0303 -0.0072 -0.0695 0.0150 -0.0991* (0.0418) (0.0427) (0.0997) (0.1721) (0.0632) (0.0581) Pace Determined by Speed of Equipment 0.5638*** 0.5805*** 0.6974*** -0.2315* 0.6360*** 0.5221*** (0.0288) (0.0295) (0.0726) (0.1223) (0.0444) (0.0397) Physical Proximity -0.0694*** -0.0735*** -0.0440 -0.0496 -0.0755** -0.0641* (0.0217) (0.0221) (0.0508) (0.0912) (0.0301) (0.0330) Spend Time Keeping or Regaining Balance -0.0480 -0.0492 -0.0531 0.0049 -0.0599 -0.0368 (0.0395) (0.0400) (0.0953) (0.1632) (0.0587) (0.0546) Spend Time Sitting 0.0545* 0.0428 0.1454* -0.2272* 0.0472 0.0122 (0.0304) (0.0310) (0.0742) (0.1270) (0.0460) (0.0430) Supervision Human Relations -0.0117 -0.0025 -0.0562 0.0714 -0.0059 -0.0156 (0.0318) (0.0322) (0.0895) (0.1505) (0.0542) (0.0424) Supervision Technical 0.0607* 0.1068*** -0.0361 0.2352 0.0554 0.1366** (0.0350) (0.0400) (0.1013) (0.1734) (0.0617) (0.0539) Support 0.0539* 0.0363 0.2297** -0.2896* 0.1004 0.0055 (0.0314) (0.0320) (0.1067) (0.1557) (0.0725) (0.0361) Thinking Creatively -0.1375*** -0.1543*** -0.1222 0.0477 -0.1019* -0.0987 (0.0436) (0.0447) (0.0957) (0.1922) (0.0596) (0.0884) Variety -0.0647** -0.0697** -0.0950 0.0547 -0.0656 -0.0596 (0.0309) (0.0311) (0.0737) (0.1281) (0.0442) (0.0439) Visualization -0.0190 -0.0160 0.0597 -0.2230 -0.0246 -0.0508 (0.0319) (0.0322) (0.0686) (0.1395) (0.0423) (0.0534) Wear Common Safety Equipment -0.1384*** -0.1470*** -0.1847** 0.0813 -0.1579*** -0.1352*** (0.0329) (0.0333) (0.0831) (0.1426) (0.0508) (0.0444) N 1505 1505 1505 1505 1505 821 684 R2 Adjusted 0.1481 0.1509 0.5392 0.5407 0.5441 0.5288 0.5595 R2 0.1515 0.1576 0.5471 0.5501 0.5628 0.5466 0.5795 Dependent variable is Automation (A); *p < .1, **p < .05, ***p < .01, Standard errors in parentheses.
 Table 3 3 Predicting △ Pay, △ Employees. Model (1) (2) (3) (4) (5) (6) Dependent Variable: △ Pay △ Pay △ Pay △ Empl. △ Empl. △ Empl. Intercept 0.376*** 0.387*** 0.386*** -0.021** -0.002 -0.088*** (0.008) (0.011) (0.010) (0.008) (0.012) (0.022) △ A -0.006 -0.007 -0.009 0.001 0.000 0.041*** (0.005) (0.006) (0.006) (0.006) (0.006) (0.015) △ A * A(0) 0.013 0.011 0.009 0.008 (0.008) (0.008) (0.009) (0.009) △ A * △ A 0.006 0.007* 0.003 0.003 (0.004) (0.004) (0.004) (0.004) △ Education -0.015** -0.011* -0.021*** -0.016** (0.006) (0.006) (0.007) (0.007) △ Employees 0.172*** (0.040) △ A * △ Employees -0.078*** (0.028) △ Pay 0.216*** (0.048) △ A * △ Pay -0.104*** (0.035) N 495 495 495 495 495 495 R2 Adjusted 0.0004 0.0092 0.0586 -0.002 0.0115 0.0626 R2 0.0024 0.0172 0.0701 0.000 0.0195 0.0739 A = Automation. Standard errors in parentheses. * p<.1, ** p<.05, ***p<.01


Quantifying the probability of existential catastrophe: A reply to Beard et al.

Introduction A recent article by  Beard, Rowe, and Fox (Beard, Rowe, & Fox, 2020; henceforth BRF)  surveys methods used to quantify the probabilities of global and existential catastrophes. BRF is a valuable contribution to the study of global catastrophic risk and existential risk. It documents the range of methods in use and evaluates their strengths and limitations, providing both a good resource for researchers wishing to get up to speed on the topic and constructive guidance for future work. In this article, I provide some further commentary on the quantification of the probability of global/existential catastrophe, making some points not made by BRF or other prior literature. At the outset, I wish to express my wholehearted agreement with BRF in the importance of sound methodology for quantifying the probabilities (and severities) of global and existential catastrophes. I regret that I share the view that "within the nascent field of Existential Risk research people have been insufficiently discriminating" in the selection and use of quantification methods (BRF, Section 7). The following remarks join BRF in seeking to improve this situation. 2. Some considerations related to severity 2.1. The definition of existential catastrophe BRF defines existential catastrophe as "the collapse of civilization or the extinction of humanity". This is a reasonable definition, but there are others. First,  Bostrom (2002)  defines existential catastrophe as an event that "would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential". This definition emphasizes long-term effects. The collapse of civilization would presumably have some long-term effects, though whether the effects would last into the distant future is a complex matter  (Baum et al., 2019) . Second,  Tonn and Stiefel (2013)  define existential catastrophe as human extinction, i.e. loss of the existence of the human species. This paper will use the BRF definition while noting the existence of other definitions for existential catastrophe, for related concepts involving large losses of value such as global catastrophe  (Morrison, 1992) , ultimate harm  (Persson & Savulescu, 2012) , and oblivion  (Tonn, 1999) , and for other concepts involving large-scale harm such as astronomical suffering  (Sotala & Gloor, 2017) .
 Probability vs. severity BRF focuses on methodology for quantifying the probability of existential catastrophe, with limited attention other factors including severity. (This focus was an explicit decision; see BRF, Section 1) That is not unreasonable for a first pass at the topic-there is plenty to say about the probability methodology. However, it is important to recognize the interplay between probability and severity. All quantifications of the probability of existential catastrophe require at least some attention to severity for the simple reason that existential catastrophe is a type of event defined by its severity. More precisely, existential catastrophe is generally defined in terms of some minimum severity threshold; in BRF, the minimum threshold is the collapse of civilization. To assess the probability of existential catastrophe is to assess the probability of a harm greater or equal to the collapse of civilization. In the terminology of catastrophe risk analysis, the probability of existential catastrophe is an "exceedance probability", i.e. the probability of harms exceeding some minimum threshold  (Grossi, Kunreuther, & Windeler, 2005) . Quantifying the probability of specific existential catastrophe events (such as a nuclear war or Earth-asteroid collision) requires additional attention to severity. The probability can be decomposed into two constituent parts as follows: = P P P * EC 1 2 (1) In Eq. (  1 ), P EC is the probability of existential catastrophe from some event; P 1 is the probability of the initial catastrophe event; and P 2 is the probability that the event will result in a harm greater or equal to the collapse of civilization. For example, P 1 could represent the probability of nuclear war and P 2 could represent the probability that nuclear war would result in the collapse of civilization or worse. The occurrence of the initial catastrophe event does not necessarily entail the collapse of civilization-that depends on how effectively the survivors can cope with the aftermath of the event. Calculating P EC via Eq. (  1 ) requires two distinct analyses: one for each of P 1 and P 2 . Analysis of P 1 is the analysis of the probability of initial events, and can follow many conventions of probabilistic risk analysis. In contrast, quantifying P 2 requires analysis of the severity, with attention to the success of catastrophe survivors. This is a rather different sort of analysis than is needed to quantify the probability of initial catastrophe events represented by P 1 . However, P 2 is not equivalent to severity. P 2 is a probability variable representing the probability that the severity will exceed a certain threshold. P 2 can be obtained by creating a probability distribution for the severity of an initial event and then calculating the portion of that distribution that exceeds the threshold for existential catastrophe: ∫ = ∂ ∞ P P S S ( ) 2 ST (2) In Eq. (  2 ), P 2 is as in Eq. (  1 ); S is severity of some initial event; and S T is the minimum severity threshold of existential catastrophe Fig.  1 . A depiction of the relationship between P 2 and the severity of an initial event. The graphic is for illustrative purposes only; no evaluation of the actual relationship is intended. (the collapse of civilization in BRF). Eq. (  2 ) is illustrated in Fig.  1 . The studies surveyed in BRF vary in terms of which probabilities they quantify. (BRF acknowledges this; see BRF, Section 1) Out of the 66 studies in BRF, only three-  Chapman (2004) ;  Rees (2003), and Wells (2009) -explicitly quantify the probability of civilization collapse (as obtained from a keyword search of BRF, Appendix A). Of the other 63 studies, some quantify P 1 , while others quantify something similar to P EC but with a different severity threshold.  Chapman (2004)  is the only one of these studies that quantifies P EC via something resembling Eq. (1).  Chapman (2004)  is a study of asteroid impacts. As with many asteroid risk analyses,  Chapman (2004)  quantifies impact probability as a function of asteroid diameter (see Fig.  1  of  Chapman, 2004) . This provides P 1 .  Chapman (2004)  then proposes that asteroids of 2−3 km diameter may qualify as "civilization destroyers", while noting that the "predictions of consequences are fraught with uncertainty" (p.11; see also  Baum, 2018) . This roughly amounts to postulating that P 2 ≈ 0 for asteroids smaller than 2−3 km diameter and P 2 ≈ 1 for asteroids 2−3 km diameter or larger. However, there is no reason to expect such an extreme discontinuity in P 2 at 2−3 km. A more careful analysis would likely display a distribution of severities (as in Fig.  1 ) across a range of asteroid sizes, finding a continuum of P 2 values that gradually increase as a function of asteroid size. Until such analysis is conducted, one cannot rigorously quantify P EC for asteroid impact. Given that impact probability varies widely as a function of diameter (smaller asteroids have a much higher impact probability), P EC for asteroid impact will be highly sensitive to the resilience of civilization to impact events of different sizes. Similar reasoning applies to other existential risks. The study of  Wells (2009) , p.84-90) is of note because it quantifies P EC directly, without considering P 1 and P 2 . It can do this because is quantifies the overall P EC , not P EC for specific events. The analysis uses a technique that quantifies the probability of something persisting based on how long it has already existed for. Given the current age of civilization, P EC can be calculated.  Wells (2009)  also uses this technique to calculate the probability of human extinction, as do several other studies including  Gott (1993) . Because this technique does not consider any particulars of the existential risks that civilization currently faces, it provides at most a very limited amount of guidance to how the risks can be addressed. The study of  Rees (2003, p.8 ) is more ambiguous. Rees provides a subjective estimate of P EC , stating "I think the odds are no better than fifty-fifty that our present civilization on Earth will survive to the end of the present century". The text does not explain how the estimate was made. The broader study presented by  Rees (2003)  includes reasoning similar to  Wells (2009)  and analysis of catastrophe scenarios similar to  Chapman (2004) . Perhaps the subjective estimate was made in consideration both approaches, but this cannot be discerned from the text. The studies that quantify P 1 are not studies of the probability of existential catastrophe: they are studies of the probability of events that might or might not result in existential catastrophe. For example,  Lipsitch and Inglesby (2014)  quantify the probability that a highly pathogenic influenza strain would escape a research laboratory and cause a pandemic.  Lipsitch and Inglesby (2014)  also quantify the severity of the pandemic, estimating 2 million to 1.4 billion fatalities. To calculate P 2 and in turn P EC , this range of severities would have to be evaluated in terms of the probability of exceeding S T .  Hellman (2008)  quantifies the probability of a "fullscale nuclear war" started from a crisis similar to the Cuban missile crisis.  Hellman (2008)  does not quantify the severity and would need to do so to obtain P 2 and P EC . For the studies that quantify something analogous to P EC , analysis of additional scenarios is needed to obtain P EC . For example,  Gott (1993)  quantifies the probability of human extinction; additional analysis would quantify the probability of civilization collapsing without humanity going extinct.  Pamlin and Armstrong (2015)  quantify the probability of the subset of civilization collapse scenarios in which there is no subsequent recovery of civilization; additional analysis would quantify the probability of scenarios in which civilization collapses and recovers. Here it is worth recalling the multitude of definitions of existential catastrophe. The studies of  Gott (1993)  and  Pamlin and Armstrong (2015)  could classify as quantifying P EC under non-BRF definitions.
 Fast vs. slow catastrophes Existential risks can be classified in terms of how quickly the catastrophe unfolds. Specifically, it is the initial catastrophe event that unfolds at different speeds. The initial catastrophe event is the phenomenon that causes the harm, such as a nuclear war or Earthasteroid collision; it is represented in P 1 in Eq. (  1 ). The overall catastrophe scenario also includes the aftermath of the initial event. All existential catastrophes could have long-lasting effects, especially if civilization is not quickly rebuilt after it collapses. Nonetheless, the speed of initial events can vary significantly between different existential risks. The risks included in BRF group into either fast or slow risks. Fast risks have initial events that occur within years or less. Fast risks included in BRF are nuclear war, pandemics, runaway artificial intelligence, asteroid impacts, nanotechnology, particle physics experiments, space weather, super-volcanic eruptions, and synthetic biology. Slow risks have initial events that persist for decades or longer. Slow risks included in BRF are climate change and ecological catastrophe (see BRF, Appendix A; note that the fast vs. slow distinction is my own and does not appear in BRF). This is overall a reasonable list of risks and broadly consistent with other studies of existential risk. The list is not without exceptions-for example, flood basalt volcanic eruptions consist of a series of large eruptions over a period of hundreds of thousands of years; if the eruptions are sufficiently large and frequent, they could cause more prolonged harm  (Schmidt et al., 2016)  and therefore rate as a slow catastrophe. As another example, studies of artificial intelligence sometimes consider that the event could unfold in a "slow takeoff" lasting on the order of "decades or centuries"  (Bostrom, 2014, p.63) . Regardless, for probability analysis, the distinction between fast and slow is important in several respects. (The distinction can also be important for other reasons beyond the scope of this paper, such as policy decision-making.) First, event probability P 1 can have a different meaning for slow risks. The probability of climate change is approximately 1 (i.e., it is virtually certain that climate change is occurring). The only meaningful uncertainty about climate change risk is about how severe the impacts will end up being, including whether they would amount to existential catastrophe. The same probably applies to the forms of ecological degradation that could result in ecological catastrophe, though the reference cited in BRF on ecological catastrophe  (Pamlin & Armstrong, 2015)  is unclear on this point. Second, slow catastrophes may be especially important when they interact with fast catastrophes. Climate change and ecological degradation may be unlikely to destroy civilization on their own. Instead, they could weaken civilization, making it less resilient to fast catastrophes. All else equal, climate change and ecological degradation may make it more likely that civilization survives a fast catastrophe (such as a nuclear war) if the fast catastrophe occurs in 2020 instead of 2050. (All else is not equal-other factors include, but are not limited to, economic and technological change.) In other words, climate change and ecological degradation increases P 2 for nuclear war (and other fast risks). They could also affect P 1 , such as by destabilizing international politics, or alternatively by providing a rallying point that brings rival nations closer together. Additionally, fast catastrophes could also affect slow catastrophes. For example, the 2011 Fukushima Daiichi nuclear disaster, caused by the Tōhoku earthquake and tsunami, has prompted a shift from nuclear power to fossil fuels, thereby worsening climate change  (Srinivasan & Rethinaraj, 2013) . While the Tōhoku-Fukushima event was only a local catastrophe, the episode is nonetheless indicative of potential impacts of fast catastrophes on slow catastrophes. The above implies that the probability of slow catastrophe can be linked to the probability of fast catastrophe. This applies to most of the risks in BRF. One potential exception is particle physics experiments, for which P 1 may not be meaningfully affected by the slow catastrophes and for which P 2 may 1 under any circumstances (it may be impossible to survive a particle physics catastrophe). Artificial intelligence could be another exception following the same reasoning, though it is conceivable that AI development could be altered by the desire to develop AI to address the slow catastrophes  (Baum, 2014) , or by effects of slow catastrophes on the context in which AI is developed, such as related to international cooperation and competition. But for most cases, quantifying the probability of slow catastrophes should account for their effect on fast catastrophes. It has been argued that the field of existential risk is insufficiently attentive to slow catastrophes, perhaps because slow catastrophes are too "boring" or "unsexy"  (Kuhlemann, 2019; Liu, Lauta, & Maas, 2018) . My own view is that yes, slow catastrophes have been at least somewhat neglected in studies of existential risk. But it is also the case that slow catastrophes pose distinct analytical challenges, as outlined above. Furthermore, the fields dedicated to studying slow catastrophes tend to not study them in terms resembling existential risk. For example, high-profile research on planetary boundaries identifies approximately ten major global environmental threats that are nominally supposed to be unacceptable to humanity, but are defined in the research strictly in biogeophysical terms and not in terms of impacts to humanity  (Baum & Handoh, 2014) . Likewise, almost all of the studies of slow catastrophes studied in BRF (Section 1) focus on biogeophysical metrics, in particular the magnitude of average global temperature increase from climate change. The one exception is  Pamlin and Armstrong (2015) , which is also probably the only of these studies that classifies within the field of existential risk. (BRF defines the existential risk research community as "those who are consciously seeking to align their research with the goal of understanding and managing such risks".) There is a major need for research that blends the traditions of global environmental change and existential risk to analyze, in human terms, the risks of slow catastrophes.
 Multi-risk catastrophes The preceding discussion of slow vs. fast catastrophes raises a more general point. The literature surveyed in BRF consists exclusively of quantifications of specific risks or quantification of the overall probability of existential catastrophe or similar catastrophe. However, some catastrophe scenarios involve multiple risks. These scenarios can require a different set of methods than those surveyed in BRF. One type of scenario involves multiple initial catastrophe events occurring together by coincidence. For example, there could be a volcano eruption that happens to occur as a pandemic is breaking out. The initial events may be independent-for example, there is no reason to believe that infectious diseases affect plate tectonics and vice versa-in which case their probabilities (P 1 in Eq. (  1 )) could be modeled as independent random variables, with the aggregate initial event probability calculated accordingly. However, the probability of existential catastrophe given the co-occurrence of these events (P 2 in Eq. (  1 )) is unlikely to be independent. For example, a volcano eruption can prompt mass evacuations that facilitate disease transmission. Analysis of the severity of co-occurring initial catastrophe events must be done on a case-by-case basis; ditto for the accompanying probability of of existential catastrophe. Another type of scenario involves causal relations between multiple fast catastrophes. For example, there could be a war or pandemic that causes a geoengineering termination shock  (Baum, Maher, & Haqq-Misra, 2013; Parker & Irvine, 2018) , an asteroid collision that causes a nuclear war  (Baum, 2018; Tagliaferri, Spalding, Jacobs, Worden, & Erlich, 1994) , or a war that induces risky technology development (as previously occurred in World War II with the development of nuclear weapons). These scenarios cannot be modeled as independent random variables. They can be modeled using fault trees or Bayesian networks, albeit with more complex models that account for the nuances of each catastrophe within the causal chain. In principle, they could be studied via aggregated opinion surveys, weighted aggregation, or enhanced solicitation, all of which derive from expert judgment. In practice, however, there may be a dearth of qualified experts, because experts tend to focus on one risk at a time and not on the causal relations between them. (The methods mentioned in this paragraph are described in Section 3) Finally, there are scenarios in which a fast catastrophe occurs during a slow catastrophe, as discussed in Section 2.3. Analysis of these scenarios could proceed by evaluating "baseline" values of P 1 and P 2 for the fast catastrophe, and then evaluating the effects of the slow catastrophe on the baseline P 1 and P 2 . Alternatively, it could proceed by calculating scenario probabilities directly without first considering baselines. Regardless, the analysis requires a nuanced understanding of the impacts of the slow catastrophe as they relate to the onset and impacts of the fast catastrophe. (Other slow factors, such as economic and technological change, can be factored into the analysis alongside slow catastrophes.) 3. On the methods surveyed by BRF BRF survey ten methods used for quantifying existential catastrophe: 1 Analytical approachesestimating probability from theory without incorporating specific evidence on the risks, such as in the "doomsday argument". 2 Extrapolation and toy modelingsimple models that derive probabilities from data on other events on the assumption of an inherent relationship between the probability of the other events and existential catastrophe (noting that no existential catastrophe has previously occurred). 3 Fault treeslogic trees that decompose types of catastrophe events (e.g., nuclear war) into a collection of sequences of precursor events and conditions that can lead to the catastrophe event. 4 Bayesian networkssimilar to fault trees, except also permitting interactions between precursor events (hence a "network" instead of a "tree"). 5 Adapting large-scale modelsthe models depict details of specific risks (e.g., models of the global climate system to study climate change risk); these models are adapted to study extreme cases of these risks. 6 Individual subjective opinionsubjective judgments of a single person. 7 Aggregated opinion surveyssubjective judgments of multiple people, averaged into a single probability estimate. 8 Weighted aggregationsubjective judgments of multiple people, combined into a single probability estimate using a weighted average, with weights set according to some measure of the quality of the judgment expected from each person. 9 Enhanced solicitationsubjective judgments of multiple people, in which effort is made to improve judgment quality via some combination of training people to provide better opinions and being more careful in how their judgments are obtained. 10 Prediction marketsplatforms for people to place financial bets on their best-guess parameter estimates.
 The tradeoff between quality and accessibility BRF evaluate probability quantification methods according to four criteria: (1) rigor, defined as how effectively the method makes use of available information; (2) uncertainty, defined as how effectively the method handles the considerable uncertainty about the probability of existential catastrophe; (3) accessibility, defined as how readily a research group can apply the method, especially the small interdisciplinary groups that often study existential risk; and (4) utility, defined as the value of the method's results for decisionmaking purposes. BRF presents ratings of ten methods according to each of the four criteria. (The ratings were performed subjectively by the authors. In my own view, their ratings are reasonable.) Ideally, a method would rate well across all four criteria. Unfortunately, none of the methods surveyed by BRF achieve this. The BRF ratings data show a trend in which rigor, uncertainty, and utility are similar to each other and dissimilar to accessibility. Fig.  2 . Rigor vs. accessibility of each method as rated by BRF (data from BRF, Table  1 ). This trend holds across all methods (see BRF, Table  1 ). To illustrate this trend, Fig.  2  plots rigor vs. accessibility. No method is rated high in both, and only weighted aggregation is rated low in both. (More precisely, weighted aggregation is rated medium-low in rigor.) The trend in the data is entirely understandable. Rigor, uncertainty, and utility are all aspects of the quality of the probability quantification produced by the application of a particular method. Obtaining higher-quality results generally requires incorporating more detail about the risk and more nuance in how the detail is processed, both of which tend to make the method less accessible. There can be methods that are low-quality and inaccessible, though one would hope that these have gotten weeded out of the collective risk analysis toolkit. Conversely, however, it may not be possible for a method to be both high-quality and accessible. Quality analysis may inevitably require a substantial investment. The same point applies to applications of a given method. Any method can be applied in ways that are of higher or lower quality. There can be applications of a given method that are low-quality and inaccessible: a lot of resources spent implementing a method poorly. Higher quality implementations of a given method will tend to be less accessible, requiring more effort on the part of the research team, and in some cases more funding to invest in expenses such as travel to interview experts in person (instead of via phone or other mode of remote communication) or rewards to incentivize participants in prediction markets.
 On the role of existential risk quantification
 Why quantify probability (and severity)? Underlying the tradeoff between quality and accessibility is the basic point that the probabilities of potential existential catastrophe scenarios are difficult to rigorously quantify. The scenarios involve complex and unprecedented global processes. Historical data are unreliable or nonexistent. Expert judgment is often unreliable as well-or, perhaps there are no experts. Obtaining quality quantifications requires applying difficult methods, which can be a resource-intensive process in a field in which resources are scarce. Perhaps it would be better to skip the matter entirely and focus on other activities. BRF identify two major reasons to quantify the probabilities and severities of existential risks. The first is the prioritization of efforts to reduce various risks. The second is the evaluation of actions that could decrease one set of risks but increase another (i.e., risk-risk tradeoffs). Both are situations in which actions could affect multiple risks, and in which identifying the best action requires comparing the size of the effect on one set of risks to the size of the effect on another. These sizes are typically measured in terms of the product of probability and severity. Therefore, to identify the best actions in these situations, it is necessary to know the effects of the actions on the probabilities and severities of the risks in question. Quantifying the probabilities and severities is an important first step. (Additional analysis is required to assess the effects of specific actions aimed at reducing the risks, as well as the costs associated with pursuing these actions.) Quantification is not always necessary. Some actions would only affect one risk, or would affect multiple risks in the same direction (increase or decrease). These actions pose no tradeoff and require no quantification to evaluate. Where possible, it can be more efficient to focus on these actions. Indeed, one approach to handling difficult quantification tradeoffs is to search for actions that avoid the tradeoffs.  Graham and Wiener (1995)  refer to these as "risk-superior" options, analogous to the concept of Pareto superiority in welfare economics. An "existential-risk-superior" option would be one that reduces all existential risks by at least as much as all other options, and reduces at least one existential risk by more than each other option, such that there is reason to favor the existential-risk-superior option and no reason to favor any other option, setting aside reasons unrelated to existential risk. If existential-risk-superior options are identified, there is no further need to quantify the options, because one option is clearly best. However, even then, some quantification may be needed to identify existential-risk-superior options. And in many other cases, there are no existential-risk-superior options. Tradeoffs are inevitable. Quantification is an important research task, however difficult it may be.
 Analysis vs. decision-making Some further perspective can be obtained from considering the relationship between analysis and decision-making. Arguably, analysis of the probability and severity of existential catastrophe should aim to improve the quality of decisions that affect existential risk. Otherwise, the analysis is of mere intellectual interest. Reducing existential risk is (again, arguably) a moral imperative, not an intellectual curiosity. Analysis can improve decisions by enabling the decisions to result in larger reductions in the risk. Phrased differently, the expected value of the action taken is larger with the analysis than without. This is the essence of the concept of the value of information (e.g.,  Barrett, 2017) . Analysis can be valuable even if the action taken is unchanged. As a toy example, perhaps heads would have been chosen in a coin flip anyway, but it would have been a lucky guess, and so there is value in analysis indicating that the coin was indeed going to land on heads. The same reasoning holds for analysis about more complex decisions, including those about existential risk. But decisions do not occur in a theoretical vacuum, and they do not necessarily conform to the ideals of expected value maximization. Decisions are psychological and (often, especially for the most important decisions) social processes. In order to be most useful for reducing the risks, methodology for quantifying probabilities and severities should be designed with the nuances of actual decision processes in mind. This point is captured somewhat by the BRF concept of method utility, but some further elaboration is warranted. A lot of risk analysis is commissioned by clients who seek to use the analysis in their own decision-making. The field of risk analysis has roots in industry, especially nuclear power  (Thompson, Deisler, & Schwing, 2005; Wilson, 2012) . Engineering risk analysis commonly seeks to guide in-house safety decisions of industrial risk managers or regulatory decisions by policy makers at agencies like the US Environmental Protection Agency and Nuclear Regulatory Commission. For this, quantifying the overall probability of some catastrophe can be less important than identifying the specific system components that are most implicated in the risk and likewise in greatest need of attention. This is one reason that engineering risk analysis has gravitated toward methods like fault trees that decompose risks into constituent parts and therefore produce results that help managers focus on specific problems. Existential risk research is less often commissioned by clients. Instead, it is often sponsored philanthropically, whether by private funders or government agencies. The funders do not seek results that they can use for their own decisions. Instead, they wish to support the cause of existential risk reduction and hope that the analysis will in some way be useful to that end. Without decisionmaker clients, existential risk analysis projects must take additional steps to ensure that their analysis improves decision-making, including identifying decision-makers to support. Because of this, there is a greater risk of the analysis working in an academic fashion, disconnected from practical decision-making and not having constructive impact on the risks themselves. (Existential risk is not the only domain in which research lacks decision-maker clients. Another example is research by industry and environmental public interest groups that seek to inform environmental policy; von  Winterfeldt, Kavet, Peck, Mohan, & Hazen, 2012.)  Nonetheless, even without built-in decision-maker clients, it is still important for existential risk analysts to bear in mind how their analysis could be used for decision-making. That means recognizing that there is more to risk management than just numbers for probability and severity. The details of the risks themselves must be understood, and how potential actions can affect the risks, and the psychological and social and institutional contexts in which decisions will be made. This is a lot to account for-as if the probabilities and severities were not a lot on their own-but it is essential for pursuing actual reductions in existential risk and not just academic studies. It can help to recognize that risk analysis can be valuable even if it does not produce any quantification of probability and severity. Often, simply having an analysis that pulls together information about the risk in an intuitive and well-organized fashion is sufficient for improving risk management. In other words, risk analysis can be of value by improving people's mental models of risks, in addition to providing information on the rating or ranking of decision options. Additionally, risk analysis can be of value by creating an opportunity for stakeholders and experts to share risk-related information with each other in a way that crosses typical organizational barriers. The opportunity for people in different groups to talk through key issues affecting risk can be of significant valuable for the practice of risk management, even regardless of formal risk analysis outputs. It is important to not let the pursuit of numbers (of probability, severity, the timing of future events such as AI milestones, etc.) serve as a distraction from the goals of evaluating decision options and improving our understanding and management of the risks. In sum, quantifying probability and severity should be at most only one part of an overall existential risk analysis and management portfolio. But it is still a part, and an important one at that. It is likewise important to use quality methodology in performing the quantifications.
 Conclusion This article has sought to build on the excellent contribution of BRF to provide further perspective on the quantification of the probability of existential catastrophe. This article finds that the probability of existential catastrophe is inherently linked to the severity of events that could result in existential catastrophe, that achieving a higher quality of analysis of the probability will in general require a larger investment in analysis, that analysis of the probability is sometimes but not always necessary for decisionmaking, and that analysis should be designed to support risk management and not just designed as an academic exercise. If these findings are taken into account, together with the analyses of specific methodologies by BRF, then it is believed that risk analyses of existential catastrophe will tend to be more successful at understanding and reducing the risks.