Reconciliation between factions focused on near-term and long-term artificial intelligence

Introduction Artificial intelligence (AI) experts are-to generalize-of two minds about long-term AI, especially regarding the potential for radically transformative types of future AI such as artificial superintelligence (ASI) and brain emulation. One faction views long-term AI as a profound issue, warranting extensive attention and immediate action. The other faction views long-term AI as an unhelpful distraction from the more pressing near-term AI issues. The purpose of this paper is to explore the potential for reconciliation between these two seemingly divergent factions. A reconciliation would enable them to expend less energy debating each other and more energy on activities that both agree are positive. In particular, if there are certain activities that simultaneously address issues associated with both near-term and long-term AI, then there is no need to engage in a near-term vs. long-term AI debate, because either position yields the same prescriptions. I will refer to the two factions as the futurists and the presentists. Each can be said to have a core claim: The futurist AI claim: Attention should go to the potential for radically transformative longterm AI. The presentist AI claim: Attention should go to existing and near-term AI. This division is a generalization in that some AI experts support both claims; there may even be some who support neither claim, though the rise of near-term AI applications makes it increasingly difficult for even non-experts to argue that AI is unworthy of attention. The futurist faction can be traced to the early days of AI. Early AI researchers such as I.J. Good and Marvin Minsky predicted that AI would soon be capable of human-level thinking. Foreshadowing contemporary futurist concerns, Good (1965) warned of self-improving ultraintelligent machines that would undergo "intelligence explosion" in which "the intelligence of man would be left far behind", forming "the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control" (p.33). When the early predictions failed to pan out, the field entered into a more muted phase known as AI winter. Here began the presentist faction. Most AI researchers shifted from trying to build human-level general AI to building narrow, domain-specific AI. Thus "mainstream AI" came to be associated with developing narrow, near-term AI and repudiating AI futurism. While a minority of researchers continued work on human-level general AI (e.g., Goertzel and Pennachin 2007 ) and a few high-profile futurists brought long-term AI some public attention (e.g., Kurzweil 2006) , most AI researchers worked on narrow, near-term AI (see e.g. Bostrom 2014, p.18; Goertzel 2014, p.1) . Within the last few years, two parallel developments have raised the prominence of the futurist-presentist AI divide. One is a breakthrough in the performance of certain AI systems, in particular those involving "deep learning". Major applications of AI are rapidly appearing, from medical diagnosis to self-driving cars. This breakthrough has generated considerable interest in near-term AI while making dramatic long-term AI projections appear more plausible. The other development is a spike in attention going to long-term AI, sparked mainly by the publication of the book Superintelligence (Bostrom 2014) , public comments by several major celebrities (e.g., Hackett 2016; Hern 2016) , and accompanying efforts by futurist researchers and activists (e.g., Future of Life Institute no date). The sober, philosophical tone of Superintelligence and the perceived credibility of some new outspoken futurists has brought the futurist perspective a burst of serious consideration. This has in turn sparked backlash from the presentist faction. The following, from an op-ed in The New York Times by Microsoft principal researcher Kate Crawford, is representative: According to some prominent voices in the tech world, artificial intelligence presents a looming existential threat to humanity: Warnings by luminaries like Elon Musk and Nick Bostrom about "the singularity" -when machines become smarter than humans -have attracted millions of dollars and spawned a multitude of conferences. But this hand-wringing is a distraction from the very real problems with artificial intelligence today, which may already be exacerbating inequality in the workplace, at home and in our legal and judicial systems. Sexism, racism and other forms of discrimination are being built into the machinelearning algorithms that underlie the technology behind many "intelligent" systems that shape how we are categorized and advertised to (Crawford 2016) . Crawford emphasizes inequality/discrimination as the near-term AI issue to focus on. Other presentists have emphasized such issues as military applications (e.g., Arkin 2009) , safety issues with medical applications or self-driving cars, disruptions to labor markets ("technological unemployment"), or simply the technical capabilities of the AI itself. The unifying theme for this group is its insistence on a focus on current and near-term AI. With growing interest in both near-term and long-term AI, the futurist-presentist divide has intensified. In addition to Crawford (2016) , the divide can also be found in: a series of events on AI in 2016 sponsored by the White House, in which "many of the speakers… emphasized the need to focus on short-term concerns over long-term concerns of artificial general intelligence" (Conn 2016a) ; remarks by AI researcher Andrew Ng, arguing that worrying about future AI is as unimportant as worrying about "overpopulation on Mars" (Garling 2015) ; and an article by AI researcher Oren Etzioni, suggesting that AI that would not be built within the next 25 years is not worth paying attention to (Etzioni 2016) , which prompted a reply by AI researcher Stuart Russell and political scientist Allan Dafoe arguing AI built more than 25 years from now can still be worth attention (Dafoe and Russell 2016) ; see also Bostrom (2014, p.18) . This sort of futurist-presentist divide is not unique to AI. For example, Selin (2007) documents a similar divide within the field of nanotechnology, in which a "mainstream" majority of researchers advocate for attention to (and funding for) existing and near-term nanotechnology at the expense of potential long-term transformative nanotechnology. Similarly, one explanation for the low priority commonly given to global warming is the fact that the harmful impacts of climate change will accrue mainly in the future (Weber 2006) . In recognition of the challenge of motivating action on long-term issues, Baum (2015) proposes to address a wide range of longterm catastrophic risks-including long-term AI, nanotechnology, and climate change-by leveraging interest in synergistic near-term co-benefits. The present paper follows in a similar vein as Baum (2015) , though it is not intended as a critique or an advocacy of either the futurist or presentist AI factions. Instead, it seeks to pragmatically pursue reconciliation between the factions so that they may be able to pursue mutually agreeable activities. Three such activities are described in Section 4. Leading up to that, Section 2 explores the ideas at the root of the futurist and presentist perspectives, while Section 3 proposes a reconciliation based on a general concern for the societal impacts of AI.
 Root Perspectives This section attempts to articulate the perspectives at the root of the futurist and presentist factions. The descriptions derive from readings from, observations of, and interactions with the two factions. Root perspectives can help explain why the two factions disagree with each other and where reconciliation may be possible.
 AI Futurists Two distinct root perspectives can be found among AI futurists. One is centered on the intellectual goals of AI research, the other on the implications of a certain ethical perspective. The first perspective is rooted in an intellectual aspiration of building advanced AI that have general intelligence at a human or superhuman level. "General" in this context means that they can think across a wide range of domains, or "have a broad capability to self-adapt to changes in their goals or circumstances" (Goertzel 2014, p.1) . Building such advanced AI has long been "the grand dream of artificial intelligence" (Legg 2008, p.125 ), but it is only pursued by a small portion of AI researchers. These are the researchers who are willing to accept that their work may be of little immediate practicality in order to work towards a more grand future achievement. They seek "to promote the idea that intelligent machines, even super intelligent machines, is a topic that is both important and one that can be scientifically studied, even if just theoretically for now" (Legg 2008, p.126) . Goertzel (2014, p.3) presents what it calls the core hypothesis of artificial general intelligence: The creation and study of synthetic intelligences with sufficiently broad (e.g. humanlevel) scope and strong generalization capability, is at bottom qualitatively different from the creation and study of synthetic intelligences with significantly narrower scope and weaker generalization capability. Thus, advanced future AI requires fundamentally different research than is needed for the narrower contemporary AI. Goertzel (2014) maintains that this hypothesis is widely held among artificial general intelligence researchers. Thus, there is a community of researchers dedicated specifically to developing advanced future AI. (Goertzel 2014 also surveys this research.) The second AI futurist perspective starts with an ethical concern for long-term effects and pairs this with a belief that long-term AI can have major long-term effects. The ethical concern for long-term effects has rigorous philosophical justification, though as with any ethical position, it is not universally held. It follows directly from a principle of equality: if all human lives (or whatever else we might care about) should be valued equally, then this (arguably) includes future lives. Furthermore, there could be many more future lives than present lives. Thus, philosophers and economists have long recognized the importance of actions that can have ongoing benefits to future generations, such as economic saving (Ramsey 1928 ) and survival of the population (Koopmans 1974) . This same ethical perspective has been articulated more recently by some in the AI futurist faction (e.g., Bostrom 2003) . The perspective prompts people to seek out issues and actions that could have significant long-term impact; the "merely near-term" is seen as trivial in comparison. This big-picture, long-term perspective explains why the futurist AI faction disproportionately attracts philosophers (e.g., Bostrom 2014; Price 2013) and cosmologists (e.g., Hawking et al. 2014) , and why they give their organizations names like Future of Humanity Institute and Future of Life Institute. The long-term ethical perspective is not fundamentally about AI. Instead, it is a guide for all human action; any role for AI is circumstantial. It relates to AI via the belief that certain AI could have major long-term effects, sending human civilization on fundamentally different trajectories. Only AI that could be so transformative is worth paying attention to; the rest is trivial detail. Thus, the prospect of future transformative AI is seen as important, and indeed is important even if there is only a small probability of it being built and even if it would be built decades or centuries from now. As long as there is some nonzero probability that future AI could, say, take over the world and kill everyone (Bostrom 2014) or dominate the global economy while remaking the future of civilization (Hanson 2016) , then it is worth paying attention to. Indeed, such transformative long-term AI competes for attention not with near-term AI but with other potentially transformative issues such as global warming, nuclear war, and long-term nanotechnology. The long-term ethical perspective can seem harsh in its relentless prioritization of long-term issues. For example, Crawford (2016) laments racial bias in AI evaluation of criminals in the United States, resulting in harsher prison sentences for African-Americans than for Americans of European descent. 1 Racial bias in law enforcement would seem to be an important issue, and indeed, at the time of this writing, it is at the very top of the public agenda in the United States. However, the attention is on racial bias as it affects people today, not on how it affects long-term outcomes for humanity. Indeed, it is possible that present cases of racial bias will have limited long-term effect, especially relative to a possible future transformative AI. The same holds for most, if not all, other near-term AI issues. Ironically, it is the same ethical principle of equality that underlies opposition to racial bias that also motivates the ignoring of racial bias in favor of long-term issues.
 AI Presentists In contrast, AI presentists are motivated by a desire to work on contemporary and near-term activities and issues. This holds for presentist AI researchers as well as those focused on societal issues raised by AI. Most AI research is presentist. The focus on contemporary and near-term AI systems is rooted in a practical, down-to-Earth perspective that seeks to work on actual systems and avoid speculation about future possibilities, especially when those possibilities seem strange and fantastic. Thus, for example, AI researcher Andrew Ng dismisses concerns about advanced future AI because "the future is so uncertain" that it cannot be meaningfully considered, and instead attention should be on problems that are more immediate and less ambiguous (Garling 2015) . Similarly, Nilsson describes how AI researchers shy away from work on advanced future AI because they worry it would "risk losing our respectability" (Nilsson 2010, p.399) . This near-term focus of AI research is not unusual. Looking across all fields of science and engineering, one finds few people motivated by long-term effects for humanity or other considerations deriving from theoretical philosophy. Indeed, they often believe ethical considerations to be fundamentally outside their professional domain, so much that they "accept radical restrictions on having consequential opinions about what ought to be done" (Shapin 2010, p.388; emphasis original) . This can be seen, for example, in the United States National Science Foundation's two criteria for evaluating funding proposals: intellectual merit and broader impacts. Intellectual merit is what scientists and engineers consider intrinsically important; it is what wins grants and publications in top journals. Broader impacts is the societal significance of the research, which is often considered less important (e.g., Schienke et al. 2009) . AI researchers generally focus on near-term AI because that is what they consider to have more intellectual merit. Long-term AI is speculative and untestable, which places it outside the bounds of normal science and engineering. The potential for long-term AI to have larger societal impacts is not a factor. From this perspective, even talking about the long-term impacts can be counterproductive, leading to concerns about respectability, as noted above, and to boom-bust cycles of hype and disillusionment, as has been seen over the years of AI "summer" and "winter". Hence presentists can resent the mere mention of long-term AI. The norm of intellectual merit further explains why some AI researchers prefer to focus on the capabilities of the AI itself and not on societal issues. 2 Coming from computer science or related backgrounds, they view their role as one of advancing the science and technology of AI; attending to any related societal issues is, if anything someone else's job. However, many AI researchers are attentive to societal issues associated with AI. Crawford (2016) is not a rare exception. 3 When presentists do turn to societal issues, they turn to those that relate to existing and nearterm AI. This focus is reinforced by the fact that the general public and political leadership also tend to emphasize near-term issues. Hence even seemingly major long-term issues like global warming commonly get drowned out by near-term issues like economic growth (Scruggs and Benegal 2012) . And so, when presentist AI researchers seek to discuss societal issues related to their AI, they find a welcome audience in social policy circles. This is seen, for example, in the heavy focus on near-term issues in the recent White House Symposia on AI Research (Conn 2016a) .
 Reconciliation Despite the divergent perspectives between the presentists and futures, as well as the sometimes bitter debate between them, the two factions share a common interest in AI and belief of its importance. This commonality may seem banal, but it nonetheless distinguishes them from the large population that gives AI little attention or none at all. Thus, for starters, the two factions can at least unite in promoting AI as a locus of attention. But there can be much more than that. There are significant parts of both factions that are concerned about the societal impacts of AI. Their concerns may come from different places, the futurists starting with concern about a certain type of societal impact and then applying this to AI, and the presentists starting with an interest in a certain type of AI and then applying this to societal issues. The fact that they work in opposite directions explains their divergent viewpoints and AI emphases. But they can at least agree that the societal impacts of AI are important, worth paying attention to, and worth trying to improve, even while they disagree on which impacts to focus on. Meanwhile, there are those in both factions who are not so concerned about societal impacts, who instead prefer to focus on the AI itself. Thus, there is potential for a realignment of AI factions from futurist vs. presentist to those who are concerned about societal impacts vs. those who are not. This would seem to be the crucial distinction: between those who wish to pursue AI for its own sake or for narrow conceptions of intellectual merit, on the one hand, and on the other hand those who wish to pursue AI for the benefit of society. I will call these realigned factions the intellectualist faction and the societalist faction, rooted in the following core claims: The intellectualist AI claim: AI should be developed for its own sake, i.e. for its intellectual merit. The societalist AI claim: AI should be developed for its impacts for society. It should be clarified that societalists may often emphasize intellectual aspects of AI en route to pursuing societal benefits. This is to say that belonging to the societalist faction does not preclude one from pursuing intellectually advanced AI. There can be many opportunities to be a "good" AI researcher according to traditional science and engineering notions of intellectual merit while still working towards societal benefits. Another important clarification is that, among societalists, there will always be disagreements about which societal impacts to focus on; this is normal ethical and political debate. And just as conventional politics-the art of the possible-seeks areas of aligned interests, quid pro quo, and compromise, so too can those who disagree about the societal impacts of AI. This realignment has a big advantage in that it enables focus on promoting attention to societal impacts in general. It is straightforward to argue that presentists and futurists alike should be concerned about societal impacts. AI researchers, like other scientists and engineers or any other people, are not above the law or above morality. It is everyone's responsibility to help society. 4 The realignment has further advantages in the synergistic opportunities to be found between presentist and futurist societalists.
 Opportunities Within a realigned societalist faction, the best opportunities will generally be those in which the interests of presentists and futurists are aligned. Such opportunities require no compromise or other sacrifice. Several such opportunities are apparent.
 Social Norms A first opportunity is to encourage more AI researchers to care about the societal impacts of their work. This assumes-quite reasonably, one would think-that the societal impacts of AI tend to be better when more AI researchers aim to achieve better societal impacts. In other words, the pro-societal efforts of AI researchers tend to be productive, not counterproductive. While it is possible to persuade AI researchers one at a time to care about societal impacts, it will often be more effective to persuade many of them all together. This can be achieved by establishing societalist social norms among AI communities, i.e. by making it normal for AI researchers to seek to benefit society through their work. Societalist social norms are seen, for example, in AI researcher Stuart Russell's call for the AI field to switch from a norm of "building pure intelligence for its own sake, regardless of the associated objectives and their consequences" to a norm of not just caring about societal impacts, but also making that be "how practitioners define what they do" (Bohannon 2015:252) . The advantage of advancing societalist social norms is that it creates more AI researchers available to work on pro-societal designs, policies, etc., and likewise fewer people pushing back against such activities. An expanded societalist faction should benefit both near-term and longterm societalist agendas: a rising tide lifts all boats. Given that AI research communities tend to focus on near-term AI, it may be advantageous to emphasize norms associated with the societal impacts of near-term AI. In practice, this means engaging AI researchers on such issues as inequality/discrimination (as in Crawford 2016), military applications (as in Arkin 2009) , the safety of medical AI or self-driving cars, etc. It is relatively easy to sell the importance of these issues because either they already are important issues or they clearly will be in the near future, and because they do not depend on the arrival of any speculative future forms of AI but instead are rooted in existing and near-term AI. Social norms surrounding the societal impacts of near-term AI can be of indirect benefit to the societal impacts of long-term AI. One mechanism for this is the durability of norms. What is now long-term AI may one day become near-term AI. If societalist norms remain intact, then the near-term societalists would switch over to what is now long-term AI. There is reason to believe that societalist norms may be so durable. For comparison, norms against slavery or racism or sexism have proved durable; 5 within academia, the norm in favor of interdisciplinary research has gradually advanced for many years. If AI societalist norms could be similarly durable, they would help with what is now near-term and long-term AI. A second mechanism is for societalist norms to prompt AI researchers to become more interested in long-term impacts. This could happen via attention to the ethics underlying which societal impacts one might care about. Ethical reflection is a natural fit for societalist AI, i.e. societalists are likely to engage in ethical reflection. That reflection could include the ethics of future generations, which is precisely how many existing futurists came to be futurists. Therefore, a case can be made for promoting near-term societalist norms among AI communities. Because the issues are near-term, they are more likely to attract interest from AI communities. This increases the size of the near-term societalist faction. Then, some portion of these societalists may come to appreciate the ethical argument for caring about long-term impacts. This increases the size of the long-term societalist faction. Furthermore, the expanded societalist faction may remain in place when the long-term gets nearer. So, this is a win-win for both near-term and long-term societalists.
 Technical Research Societalist social norms expand the population of AI experts who can pursue the lines of technical AI research that benefits society. Some technical research may pertain only to specific AI applications and is thus only of interest to narrow presentist or futurist concerns. However, there is also some technical research that is of wide relevance to both near-term and long-term AI. Progress on this research is another win-win for near-term and long-term societalists. An example of this sort of "timeless" technical research can be found in Amodei et al. (2016) . This paper describes a range of technical problems oriented towards avoiding unintended harms from AI systems. The paper is expressed in terms of unintended harms from near-term AI. However, in an interview, Amodei describes how he sees the technical problems as being relevant for near-term and long-term AI alike, such that the near-term/long-term distinction is unnecessary (Conn 2016b) . Thus, all societalists could support this technical research agenda, regardless of whether they are ultimately concerned about near-term or long-term impacts. Indeed, in the same interview, Amodei further notes that his paper has received broad praise, including from people who fit into both the presentist and futurist factions (Conn 2016b) .
 Policy Given the general orientation of policy communities towards near-term issues, it will typically be easier for policy to address issues associated with near-term AI. However, there are at least two ways that AI policy can concurrently support both near-term and long-term AI, making for a third win-win for near-term and long-term societalists. First, the process of developing near-term AI policy can create processes and competencies of relevance for long-term AI. Already, near-term AI policy issues have prompted legal scholars to familiarize themselves with some technical details of AI (e.g., Calo 2011; Funkhouser 2013; Hammond 2015) . Such scholarship, and accompanying policy discussions, prepares policy communities for addressing long-term AI. 6 Near-term AI policy will also create demand for people with AI backgrounds working in policy positions. Such in-house AI expertise can help ensure that AI policy matches actual issues associated with AI technology and does not inadvertently restrict beneficial AI or enable harmful AI. Good models can be found, for example, in the Science & Technology Policy Fellowships offered by the American Association for the Advancement of Science and the IEEE-USA Government Fellowships offered by the Institute of Electrical and Electronics Engineers. These are both temporary fellowships; there should also be AI experts in permanent government positions. As long as the private sector offers abundant and lucrative employment for AI experts, it may be difficult to attract them to government positions. This challenge could be overcome via sufficiently strong societalist norms, which could motivate some societalists to forgo salary and pursue government positions simply because it is the right thing to do for society. Second, some specific policy measures may improve outcomes for both near-term and longterm AI. This may seem counterintuitive, given the rapidly changing nature of AI. Indeed, a common concern about policy for emerging technologies like AI is that policy will fail to keep up with changes in the technology. In response to this concern, Moses (2007) proposes to phrase policy in more general terms, such as "choose the AI design that is more beneficial to society"; such policy could remain relevant even if AI technology undergoes major changes. Other policies of enduring relevance can be policies that establish institutions for AI governance, including policies to establish support for pro-societal AI efforts.
 Conclusion The current division between AI futurists and presentists-i.e. those concerned mainly about long-term AI vs. those concerned mainly about near-term AI-is not the best focus of attention. Instead of bickering over which AI impacts are most important, both groups would be wise to focus on addressing the AI impacts in general. There are ample opportunities to concurrently address both near-term and long-term AI impacts, including by establishing societalist social norms within AI communities, by advancing technical research that improves societal outcomes for all types of AI, and by advancing public policy that improves the governance of all types of AI. Making progress on these three fronts will take effort, which is precisely why those who worry about AI impacts should not drain their energy on internal disputes. The Introduction to this paper quoted a passage from Crawford (2016) that argues for attention to near-term inequality/discrimination issues instead of to long-term threats. The passage is a representative example of the presentist vs. futurist dispute. However, in light of this paper's proposed reconciliation, the passage could be rewritten to instead take on the societalist vs. intellectualist dispute: According to some prominent voices in the tech world, artificial intelligence presents a major intellectual achievement, a triumph of science and technology. They say this achievement is to be celebrated, full stop. Indeed, research into the science of artificial intelligence has attracted millions of dollars and spawned a multitude of conferences. But this intellectual triumphalism is a distraction from the problems that artificial intelligence creates for society. It may already be exacerbating inequality in the workplace, at home and in our legal and judicial systems. It is also creating safety challenges for medical products and self-driving cars, as well as profound ethical dilemmas in its military applications. Looking further into the future, there may be some chance that artificial intelligence presents a looming existential threat to humanity. These are serious issues that demand serious attention from society as a whole and especially from the technologists whose work is at the heart of these issues. Expositions along these lines promise to reconcile the differences between presentists and futurists and instead shift attention to where it is desperately needed: on the societal issues posed by AI and on the positive steps can be taken to address these issues. The racial bias Crawford describes comes from the investigative journalism of Angwin et al. (2016) .2 One can argue that the AI itself is situated within society, and thus that AI researchers inevitably work on societal issues, even when they believe that they are working only on the AI itself. However, for present purposes, what matters is that the AI researchers believe that they are focusing on the AI itself and not on societal issues, even if it can be argued that they are inadvertently working on societal issues.
 As just one of many other examples, see Arkin (2009) on societal issues associated with military robotics.
 Again, as with any ethical position, the claim that people should help society is not universally held. A broader defense of this claim is beyond the scope of this paper.
 The world still has slavery and racism and sexism, but not as much as it once did. For example, while the United States continues to grapple with a variety of racial biases, it has become unthinkable to support the "separate but equal" racial segregation of the former "Jim Crow" laws.6 There has also been some legal scholarship focused on long-termAI (e.g., McGinnis 2010; Wilson 2013) , but this is a relatively small minority of AI legal scholarship.


AI CERTIFICATION: Advancing Ethical Practice by Reducing Information Asymmetries

Certification can be defined as the attestation that a product, process, person, or organization meets specified criteria (paraphrased from [6] ). Certification involves an assessment of the entity to be certified, often, but not necessarily, by a trusted third party. If the assessment determines that the entity meets the specified criteria, then the entity is certified accordingly. Thus, the certification process reveals information about the inner workings of the entity that can be provided to external stakeholders, reducing information asymmetries between insiders and outsiders [7] . In doing so, certification incentivizes insiders to adhere to higher standards because they can get credit for it if they do and get caught if they do not. In short, certification is a governance tool for advancing transparency and incentivizing insiders to meet specified criteria. Those criteria could include voluntary AI ethics principles or mandatory regulatory requirements. Certification is already in wide use in many sectors [8] , [9] and is starting to appear for AI. Certification includes such disparate phenomena as the U.S. EnergyStar program for consumer appliance energy efficiency, ISO 9001 for quality management systems in global supply chains, and the mandatory "CE mark" for various products sold in the EU. Certification programs for AI systems have been proposed or are currently being developed by the European Commission [10] , the IEEE [11] , and by the Chief Scientist of Australia [12] ; one has already launched from the government of Malta [13] . Education certification programs include the Queen's University executive education program on Principles of AI Implementation [14] and the Finnish civic education program Elements of AI [15] . Additionally, a coalition of Danish organizations [16] is developing an IT security certification for organizations that includes AI-related criteria. These various programs speak to the diverse forms that certification programs can take, including public and private, voluntary and compulsory, and for the AI systems themselves as well as the people and organizations involved with them. This article presents an overview of certification for advancing ethical AI. It provides background on AI certification, drawing on management literature (Section II), surveys current AI certification programs and proposals for new programs (Section III), and discusses prospects for certification to address issues raised by future AI technologies (Section IV). Section V concludes this article.
 II. BACKGROUND This section reviews management literature on the merits of certification as it relates to the governance of AI. This literature typically focuses on the market behavior of corporations and their customers. However, certification is also relevant to nonmarket settings. For example, AI systems developed by nonprofit and/or open-source teams could seek certification to build user confidence, even if users are not charged for using the systems. Or, governments might seek international certification of their military AI systems to demonstrate compliance with international humanitarian law or other standards. Thus, while certification is important for the corporate governance of AI (on which, see [17] ), it is also more widely applicable. First, some terminology. The object of certification is the entity to be certified. It may be a product, process, individual, or organization. The certifier is the actor who does the certifying. The assessor is the actor who assesses whether or not an object of certification meets the specified criteria prior to certification. The assessor and certifier can be one and the same or they can be two separate actors, such as in the case of certification programs that use third-party assessment. The assessment or conformity assessment is the evaluation of whether the object meets the specified criteria; this can vary in stringency, from a review of application materials to an audit of systems or processes. The applicant is the actor who controls the object of certification and seeks to obtain certification when it is voluntary. The applicant and object of certification can be the same, such as in the case of skills certifications granted to individuals. Alternatively, the applicant can be the entity who oversees, manages, develops, or otherwise controls the object of certification. Finally, the audience is the group of people who will be informed by the certification. For example, consider an AI system developed by a corporation, certified by a government agency, assessed by an independent consulting firm, to inform consumer purchasing decisions. Here, the object is the AI system, the certifier is the government agency, the assessor is the consulting firm, the applicant is the corporation, and the audience is consumers. In some cases, the applicant may also simultaneously be the certifier; this is known as self-certification. Self-certification creates a conflict of interest, though this can sometimes be addressed via measures such as audits and liability regimes. Two additional sets of terms help discuss the governance potential of certification programs. First, symbol-substance coupling refers to alignment between the statement that an object meets the certification criteria (the symbol) and that object actually meeting those criteria (the substance). Likewise, symbol-substance decoupling is when a certification inaccurately characterizes its object [18] - [20] . Second, means-ends coupling refers to alignment between the means of the certification program (i.e., the specific criteria to which objects are being held) and the ends underlying its design [19] , [21] . Means-ends decoupling is when a certification program fails to advance its intended goals. Symbolsubstance decoupling and means-end decoupling are two failure modes of certification programs.
 A. Reducing Information Asymmetries As noted above, a primary role of certification is to reduce information asymmetries. Information asymmetries raise two issues: the "selection problem" of identifying desirable suppliers of a good or service, and the "monitoring problem" of evaluating whether the supplier is meeting specified agreements [22] . Both problems create space for rent-seeking behavior, in which the supplier extracts greater wealth from other parties without adding social value [23] . Information asymmetries are acute in AI systems because the systems are often complex and opaque and users typically lack the data and expertise necessary to understand them. Certification is especially well suited to addressing such information asymmetries that arise from "credence" and "Potemkin" attributes of goods and services [24] . Credence attributes are those that can be identified by outside parties, but only as trends that emerge after repeat observations. Individual audience members may struggle to identify credence attributes on their own, but an organized third party, such as a certification program, can collect the data needed for a reliable characterization. Credence attributes of AI systems include whether a system is fair or unbiased, is explainable across a wide range of use cases, protects user privacy across a wide range of settings, and is safe and secure to rare and/or diverse threats. Potemkin attributes present even greater information asymmetry challenges: they cannot be identified at all by outside parties, and instead require insider information. An important type of Potemkin attribute is the process through which the good or service was made, such as whether an AI system was developed in accordance with labor, environmental, or ethical standards, or whether it was rigorously tested prior to deployment. Certification programs can address both credence and Potemkin attributes, as seen for example in the popular ISO 9001 [25] certification for organization quality management and ISO 14001 [26] certification for environmental management. By providing information about credence and Potemkin attributes, suppliers can create new signals to attract outside parties, including market customers [27] and others. Table I presents a summary overview of the key concepts underlying our understanding of certification, and how we apply them to the field of AI. A certification program reduces information asymmetries when it accurately characterizes the objects that it certifies. It must avoid false positives, in which an object is certified even though it fails to meet the specified criteria, and false negatives, in which an object meets the criteria but is denied certification. Accurate certification thus exhibits high symbolsubstance coupling. Research on certification has identified strategies for improving certification accuracy, i.e., improving symbolsubstance coupling. First, the objects to be certified should be assessed continuously, rather than only once or not at all [7] , [28] - [30] . This ensures that the information conveyed by the certification stays up to date. Second, assessments should be made by qualified independent assessors using their own evidence and not just documentation provided by the applicants [24] , [31] , [32] . This is needed to protect against applicants misleading the assessor, especially to gain a false positive. These strategies are of further value for incentivizing good behavior by the object and/or applicant, as discussed in the section below. Indeed, the strategies were largely developed to promote behavior change. To achieve accuracy, AI certification programs will often need to parse technical details of the systems and their development. Evaluation methods required for accurate AI certification are at an early stage of development. The most mature area may be documentation for AI system training data and model characteristics [33] - [35] , which can help verify certification criteria associated with both credence and Potemkin attributes. Supply-chain and development documentation methods that draw on insights from other fields may help verify Potemkin-attribute criteria [36] , [37] . Further, benchmark tests to assess AI system performance for specific tasks in specific settings [38] and more general behavioral settings for specific types of systems [39] can help verify credence-attribute criteria. However, methods for criteria verification in broad areas of fairness, transparency (explainability), safety, and security remain under development. Certification programs aiming to address these areas will need to update their criteria over time as the state-of-the-art advances.
 B. Incentivizing Change In reducing information asymmetries, certification can further serve to incentivize changes to applicant practices. For example, research has shown that corporations may be more motivated to achieve ethics standards if they can use certification to demonstrate their achievements to customers who value these achievements [31] , [40] . Where there is demand for certification-e.g., from corporate customers who value ethics achievementscertification programs can succeed on a voluntary basis, with no government regulation [9] . Government regulation can further boost voluntary programs via tort liability [41] or tax credits. Mandatory certification programs display higher compliance levels, though voluntary programs are sometimes found to be more cost effective [42] . For a certification program to change applicant practices according to its specified criteria, it is important that it accurately characterizes these practices and their outputs. In particular, false positives permit applicants to pursue certification without changing their underlying practices: they can adopt the certification criteria symbolically but not substantively. Such symbolic adoption is of particular concern in AI. The recent popularity of AI ethics principles has prompted concerns about "ethics washing" [43] , [44] , in which AI groups articulate ethics goals to bolster their reputation but do not act accordingly. An accurate certification regime could mitigate the problem of AI ethics washing. Prior research identifies several strategies to motivate substantive instead of symbolic participation in certification programs. First are strategies to improve certification accuracy as discussed above. Second, certification assessments should not be onerous on the applicants; otherwise, applicants that meet the assessment criteria may decline to participate in the program [24] . Third, the cost to applicants of failing an assessment should be high (in terms of damaged reputation, lost market power, etc.), so as to incentivize participating applicants to meet the specified criteria [22] , [24] , [27] , [31] . A successful certification program may need to promote its value so that other parties (e.g., customers) know to penalize applicants that fail certification assessments. Finally, certification programs should require applicants to integrate the values and norms underlying the criteria into their organizational identities and strategies; this makes substantive adoption more likely [30] , [45] . A robust body of research suggests that properly constructed certification programs can couple symbol and substance to accurately characterize and induce changes to corporate practices. For example, studies of ISO 14001 [26] have found a significant correlation between: 1) the ISO 14001 attestation and the presence of environmental management practices in corporations and 2) the presence of these practices in corporations and their improved environmental performance [7] , [46] . Similar evidence exists for the ISO 9000 series on quality management practices [47] , [49] . Ultimately, adherence to certification criteria depends on applicants' actions. Their actions can derive from characteristics of themselves in addition to characteristics of the certification program. Indeed, applicants sometimes engage in a mix of symbolic and substantive adoption even within the same certification program [47] . This may be due to a lack of rational reflection or to internal disagreement within the applicant about the importance of the certification criteria [45] . Additionally, some studies have found that firms with higher revenues tend to adopt certification more substantively because they have the resources to meet the certification criteria [51] , [52] . Therefore, while it is important for certification programs to be well designed, the applicants retain an essential role.
 C. Ensuring Certification Advances the Right Goals The preceding section discussed how to ensure that certification criteria are in practice actually met, i.e., that certification is adopted substantively and not just symbolically. The current section covers the selection of the criteria themselves. These criteria should be designed such that if they are achieved, the certification program will deliver improvements on its underlying motivation: advancing ethics principles, making progress on societal issues, etc. In other words, the means of certification must be coupled to its ends. Means-ends decoupling can arise when developers of certification programs fail to treat the focal problem holistically, leading them to misidentify the causal relationships driving the issue at hand and prescribe the wrong action for a particular problem [21] . This can spur at least three types of mistakes. First, certification may be inadequately customized for the circumstances it addresses. A common shortcoming of certification programs is the use of uniform, homogeneous prescriptions that neglect important local and time-specific factors [21] . For example, AI systems required to use a certain dataset that performs well on demographic diversity could end up underperforming when even better datasets become available. Some of the most popular corporate certification programs make homogeneous prescriptions for corporations across the globe, though a context-dependent multiplicity of corporate practices may be necessary to address societal problems across environments. Certification criteria that are customized for particular circumstances may tend to perform better, though the development of such criteria may require more resources. Second, certification can cause unintended harms. Narrowly crafted certification criteria can neglect important causal relationships; when followed, they may make some things better but other things worse. For example, AI systems meeting certain performance standards may require additional computing power, which inadvertently harms the environment via increased energy consumption [53] . This type of problem is also known as the "waterbed effect" because lying on a waterbed simply displaces the water: the problem "rises" again somewhere else [21] , [54] . To avoid unintended harms, certification programs have to avoid overly simplistic prescriptions and instead embody and promote systemic thinking about the issues at hand [21] . Third, certification criteria can be overly restrictive, inhibiting applicants from taking innovative actions that would better address the particular problem. One way to make actions more customized to complex causal relationships expressed in particular circumstances is to permit applicants to make decisions on a case-by-case basis. However, certification criteria often prescribe a rigid set of actions, thereby inhibiting applicants from identifying better solutions [21] , [55] . Certification programs can mitigate this type of problem by "stimulating internalization," meaning that they de-emphasize rigid rules and instead emphasize changes to applicants' internal culture, such that applicants are likely to do the right thing as circumstances dictate [21] . Related to these challenges is the question of whether a certification program addresses the right focal problem(s) to begin with. Exactly which problems should be focused on is ultimately a question of ethics. Opinions may vary on which issues are most important. For example, many AI experts are divided on whether the field should focus on issues raised by near-term or long-term AI [56] . Certification programs may sometimes need to take sides on these sorts of divides, such that they "cannot please everyone," with some contending that the programs work on the wrong issues. In other cases, certification programs could emphasize issues that may seem appealing at first glance because of their broad scope, but become disagreeable when actors' interpretations of this broad scope begin to differ. To ensure that certification programs overcome these challenges and address their focal problems as holistically as possible, their developers should draw on the expertise of a wide range of stakeholders and carefully reconcile these stakeholders' differing perspectives.
 III. AI CERTIFICATION PROGRAMS AND PROPOSALS To identify existing AI certification programs, we conducted Internet searches, monitored social media, solicited input from colleagues, used our own prior knowledge, and examined references from documents about the programs and other relevant literature. These programs are not exhaustive, but are largely representative of the variation in AI-related certification. 1 We assessed the programs using publicly available documents. To fill in gaps in the documents, we conducted semi-structured interviews with representatives of certification program organizations. In total, we conducted three such interviews across seven identified AI certification programs. A) European Commission White Paper on Artificial Intelligence.
 B) IEEE Ethics Certification Program for Autonomous and Intelligence Systems (ECPAIS). C) Malta's AI Innovative Technology Arrangement (AI ITA). D) Turing Certification proposed by Australia's Chief Scientist. E) Queen's University's Principles of AI Implementation executive education. F) Finland's civics course Elements of AI. G) Danish labeling program for IT-security and responsible use of data. The seven programs classify into four categories: 1) selfcertification of AI systems (A); 2) third-party certification of AI systems (A-D); 3) third-party certification of individuals (E and F); and 4) third-party certification of organizations (D and G). Table II presents our ratings of these categories in terms of feasibility, symbol-substance coupling, and meansends coupling. The ratings are based on our analysis of the 1 Additional certification programs under development include AI Global's AI system certification [57] , CertNexus's certified ethical emerging technologist program [58] , and ForHumanity's compliance audit for AI systems and organizations [86] . A number of universities offer AI ethics-related certifications or minors, including San Francisco State [59] and Carnegie Mellon [60] . Consultancies offer algorithmic auditing services, which may confer a certification [61] . Self-certification is widely used in many industries globally, including suppliers' declarations of conformity used in telecommunications and motor-vehicle manufacturing [62] . For self-certification of AI systems, we rate certification programs in this category as having high feasibility because it requires little involvement from nonapplicant actors and limited back-and-forth between them and the applicants. The symbol-substance coupling is low because of its inherent conflict of interest and lack of strong enforcement and assessment. These problems can be attenuated via strong ex-post enforcement after harms occur via liability regimes. However, liability is problematic especially for high-risk applications where the initial harm could be catastrophic, and because legal liability adjudication poses uncertainties for AI developers [63] . Finally, the means-ends coupling is medium because it encourages applicants to take ownership of the certification process and all that it represents, including by documenting their AI practices and thinking systemically about the corresponding ethical implications. Third-party product certification is used in numerous industries globally, including Underwriters Laboratories certifications for electrical and fire safety and the Common Criteria [64] for product cybersecurity certification. For thirdparty certification of AI systems, we rate certification programs in this category as having medium feasibility because AI systems are a relatively well-defined object to assess and certify, but they are also a novel one for which prior experience is of limited use. The need for qualified third-party organizations to offer assessment adds additional complications, including questions around costs and accountability [5, p. 12] . Symbolsubstance coupling is medium because, on the one hand, third parties can assess certification claims and demand for certified systems could drive substantive adoption, but on the other hand, actual implementation can fall short on both meaningful assessment and enforcement. Finally, means-ends coupling is medium because of the potential for, but difficulty of, successfully assessing each AI system in its development and deployment context. Third-party certification of individuals is common in a number of professions, including among engineers, lawyers, and actuaries, as well as programs like the Certified Information Privacy Professional credential [65] . We rate AI-related certification programs in this category as having medium feasibility. A core variable is whether certification criteria include matters related to moral factors in addition to the usual criteria of technical knowledge and skills. 2 Assessment of moral standards is important but less feasible. Moral character is not readily testable, and educational institutions may have less financial incentive to do so. The symbol-substance coupling is low because the substance of certification criteria is dynamic. Methods in explainability, fairness, safety, etc., are active research topics; certificates attesting to knowledge of them can become quickly outdated. This problem could be addressed via ongoing audits or education requirements comparable to those sometimes required for lawyers, but current AI certification programs lack such measures. Finally, the meansends coupling is low because, absent robust and dynamic moral standards, certification is unlikely to guarantee that the individuals will achieve sound performance on AI issues. Third-party certification of organizations include the U.K. Cyber Essentials [66] cybersecurity company certification and the FairTrade [67] licensing contract. We rate AI-related certification programs in this category as having medium feasibility because it can draw on a preexisting body of experience and knowledge in corporate governance and related fields, but it still requires AI-specific innovations. The symbol-substance coupling is high because it can use robust preexisting methods like audits and data security management. Organizations are often familiar with these methods and understand how to respond with substantive activity. Their familiarity could also be used to game the system, though auditors are themselves familiar with such tactics. Finally, the means-ends coupling is medium because assessments can support the adoption of a "systemic mindset" but certification criteria may not be easily customized for the context of a particular organization.
 A. European Commission White Paper on AI The European Commission White Paper on AI [10] proposes separate certification programs for low-risk and high-risk applications. Both programs would draw on prior certification programs, in particular, those of the European Economic Area "CE marking" rules and the 2019 EU Cybersecurity Act. Both programs would also use similar certification criteria, such as the EC's AI Ethics Guidelines for Trustworthy AI [68] . If implemented, the programs would be important in their own right and could further be influential for AI certification beyond the single market, as is often the case for EU policies [69] . For low-risk applications, we interpret the White Paper as calling for voluntary self-certification. This is clear from the White Paper's call for following the precedents of CE marking and the Cybersecurity Act, both of which use self-certification for low-risk applications. There is some ambiguity because the White Paper also suggests for the low-risk program to use a "combination of ex ante and ex post enforcement" [10, p. 24] . Ex ante enforcement would seem to preclude self-certification. We interpret a stronger significance to the two self-certification precedents and therefore believe that the White Paper intends the low-risk program to involve voluntary self-certification. AI applicants could attest to their system having met the criteria, and face ex post penalties if their system falls short. Applicant participation would be incentivized by consumer demand for certificates of AI trustworthiness [10] . To be effective, the low-risk program would need to be carefully designed. First, it needs an enforcement mechanism with clear, strong penalties for applicants who falsely certify their AI systems as meeting the specified criteria [70, p. 5] . Otherwise, there could be symbol-substance decoupling, with applicants rubber-stamping their systems as meeting the criteria. Fortunately, the program could build on the compliance infrastructure that already exists for CE markings. Second, the criteria need to capture the many aspects of AI trustworthiness as defined in the "Ethics guidelines for trustworthy AI" [68] , which include, among other things, fairness, security, and accountability. The criteria further must promote the systemic mindset needed to advance this conception of trustworthiness. Otherwise, there could be means-ends decoupling, with the certification program not advancing its underlying goals. For high-risk applications, the White Paper clearly specifies mandatory third-party certification via what it refers to as "conformity assessment" [10, p. 23] . Its procedures would be based on CE marking rules or the Cybersecurity Act [10] , both of which require third-party verification for high-risk applications. The White Paper's program would use existing accredited auditing organizations ("notified bodies") of the Member States [10, p. 25] . EU regulation would compel AI applicant adoption and compliance. To be effective, the high-risk program will also need several design elements. First, it needs to ensure that the certifying bodies have the technical expertise and capacity needed to assess AI systems. Using the existing notified bodies makes the program easier to implement, but the bodies currently lack expertise and capacity on AI. Unless this situation is improved, the assessments could be inaccurate, and AI applicants may be able to obtain symbolic certification. Additionally, as with the low-risk program, the criteria must be carefully specified to ensure means-ends coupling.
 B. IEEE Ethics Certification Program for Autonomous and Intelligent Systems The IEEE ECPAIS program [11] is currently under development. It considers driving factors and inhibitors of AI system transparency, accountability, and bias. Offering a series of recommendations, the program is expected to yield a quality mark. The developer of an AI system can use the quality mark in presenting the system if it follows the recommendations in a checklist fashion. The program is expected to be administered by an organization other than the IEEE, which will assess certification applications. Whether the program will use an auditor to verify these applications has yet to be determined. The program is voluntary, with expected consumer demand as the driver for adoption. To be effective, EPCAIS would likely need the following. First, it would need to incentivize substantive adoption by generating consumer awareness and demand or by establishing an auditing requirement. Second, to improve means-ends coupling, it should offer distinct quality marks for transparency, accountability, and bias; it should also promote a systems mindset and consider the context in which the system is developed or deployed. Third, it should develop in-house expertise on certification; as far as the authors know, the IEEE has not launched a system-level certification program before. Finally, it should foster an enthusiastic audience in order to successfully launch. However, given that ECPAIS remains under development, we cannot offer a conclusive assessment.
 C. Malta AI Innovative Technology Arrangement The Maltese government launched the AI ITA certification program [13] in summer 2020. The program certifies AI systems to a particular use case following a review of an application detailing development and maintenance processes, corporate governance structures, and a responsible administrator. Applications must describe how the system handles failure modes and conforms to Malta's ethical AI framework, among other requirements [13] . The program also (in most cases) requires applicant systems to have two software features: a "harness" that safeguards how the system handles specified failure modes and a "forensic node" that logs activity and is stored physically within Maltese jurisdiction. The applicant system must also disclose specific elements of its compliance with the ethical AI framework to end users via terms of service. Applications are reviewed by the Malta Digital Innovation Authority (MDIA) and are subsequently confirmed by an accredited auditor. The program is voluntary, with adoption expected to be driven by demand for certified systems from government procurers, businesses, and consumers. The Malta program has high feasibility because it benefits from a dedicated regulator which was established to manage this and similar programs. This dedicated expertise in assessing certification as well as accrediting third-party auditors can reduce concerns of symbol-substance decoupling. The program's focus on awarding certification for a specific system in a specific use case based on an assessment that includes the organization and a responsible individual incorporates a wide context that can help reduce concerns of means-ends decoupling. However, no application has yet been submitted. The effectiveness of the software harness to mitigate failure modes and the usefulness of mandated ethics disclosures in terms of service remain to be seen.
 D. Australian Chief Scientist Proposal: Turing Certification The Turing Certification program was proposed by Australia's Chief Scientist Alan Finkel in 2018 [12] , [71] . It would certify AI developer organizations and low-risk AI systems, drawing inspiration from FairTrade certification among other programs [72] . This dual certification aspect could be especially impactful: it promotes a wide scope of assessment, thus improving means-ends coupling, and its organizational audit further improves symbol-substance coupling. As proposed, Turing Certification would be a voluntary program with third-party certifiers, with participation incentivized by demand from purchasers including consumers and governments. The third-party audit would confirm that the applying company, its development processes, and end AI system conform to not-yet-set standards for trustworthy AI. The proposal has not been enacted; it is unclear if it ever will be. Its most recent public discussion was in 2019 [73, p. 161 ]. If it is to be enacted, it would need clear certification criteria, a substantial challenge given the lack of agreed-upon trustworthy AI standards today.
 E. Queen's University Executive Education: Principles of AI Implementation In partnership with IEEE, Queen's University has launched an executive education certificate program on Principles of AI Implementation [14] . Demand is likely to be driven by employers that value certified employees. The program has not yet run, and it is unclear how strong employer demand will be. The program covers ethical design principles, considerations for building trust in AI applications, and current Canadian regulatory requirements. This broad curriculum could support a systemic mindset, thereby improving means-ends coupling. However, these are dynamic topics; the program will need to stay up to date to retain means-ends coupling. Additionally, it is only a two-day program; this short duration precludes much substantive learning, creating the risk of symbol-substance decoupling.
 F. Finland Civics Course: Elements of AI The government of Finland, in a partnership between the University of Helsinki and the consultancy Reaktor, offers certificates to people who complete its civic education course Elements of AI. Certificates are awarded for all who complete at least 90% of the exercise and score 50% or above on attempted exercise questions [74] . Credits can be transferred to the University of Helsinki. Demand may be driven by individuals' curiosity and sense of civic responsibility. It is unclear if employers would be interested in hiring certified individuals, and its introductory content would be insufficient for professional education, though that is not the program's aim. Thus far, demand has been significant: over 500 000 people have signed up and students from 170 countries have completed the course [15] .
 G. Danish Labeling Program for IT Security and Responsible Use of Data The Confederation of Danish Industry, Danish Chamber of Commerce, SMEdenmark, and Danish Consumer Council [16] are currently developing a certification program for information technology security and data ethics. The program does not primarily focus on AI, but it was listed by the European Commission [10, p. 10] as an AI activity, and an interview and review of internal documents confirm that AI is within its scope. The program scope is ambitious, covering elements ranging from IT security to data management, to processes to address algorithmic bias. This broad scope could support a systemic mindset in the organization, and thus improve means-end coupling. The same scope may undermine symbol-substance coupling, however, as it could be difficult to achieve compliance with and effectively communicate to external stakeholders about so many certification criteria. The eventual administration of the program could encounter a challenge in moral hazard because a single organization-funded by certification fees-sets certification criteria, reviews applications, and performs (limited) audits.
 IV. CERTIFICATION FOR FUTURE AI TECHNOLOGY An essential attribute of AI technology is that it continues to evolve. Certification programs designed for the AI of today may not fare well with the AI of tomorrow. Adjustments will be needed to keep the programs up to date. However, adjustments can be expensive and are therefore not always made. Where possible, current certification programs should be designed to accommodate potential future directions in AI. This is no easy task: the future of AI cannot readily be predicted. Many have tried and failed [75] . Nonetheless, some broad contours of future AI can be at least tentatively described, enough to derive some implications for certification programs. Toward the end of this section, we offer some remarks specific to the most advanced long-term AI, though much of the discussion may be more applicable to AI that will be developed in the interim, i.e., medium-term AI [76] . Some elements of AI certification are relatively durable in the face of changes in the technology. First, many of the underlying goals for certification programs, such as fairness and beneficence, derive from universal ethics principles that apply not just to any AI technology, but to society in general. These ethics principles are used as a "North Star" guide in today's certification programs [36, p. 35] and will likely remain relevant as the technology matures. Future programs might potentially even certify the ethics of either the AI system or its developers. Second, some aspects of certification are matched to the types of people and institutions that are involved in AI, such as business executives, citizens, and corporations. It is fair to expect that such actors will remain involved in AI over the years. Certification program capacity to engage these actors may likewise remain relevant. That includes matters such as the capacity to improve corporate transparency and to educate large numbers of citizens. Similarly, one way for certification programs to remain relevant over time is to emphasize human and institutional factors. As discussed in Section II, certification programs can improve symbol-substance and means-ends coupling by promoting certain values and norms among applicants. Among other things, doing so positions applicants to act ethically as new challenges arise. Today, we do not know exactly what new challenges will arise for future AI, but we can be very confident that there will be new challenges. Certification programs should endeavor to make it more likely that whoever faces these new challenges will act according to a high ethical standard. Certification programs can also remain relevant by buildingin mechanisms for updating their certification criteria. For example, in the establishment of a new AI certification program, it could be specified that an expert body meets periodically to review and update the criteria, as is common with technical standards bodies. Programs could also include sunset provisions such that a program will cease to exist unless its criteria are updated. Additionally, certification programs could include a budget for ongoing research so that knowledge about how to update the criteria is available when needed. The field of AI research could also be a focus of certification. Future AI techniques will derive from current or future research. Some techniques may be better able to meet ethics standards; these techniques could be an object of certification. Academia already has institutional review boards to certify human subjects research as meeting standards for responsible conduct of research. Similar boards have been proposed for overseeing future AI research [77] . Importantly, such boards should focus not just on whether the research meets procedural standards such as harm to research subjects, but also on whether it meets standards in terms of expected impacts on broader society [78] . However, it can be difficult to know in advance which techniques will perform well-hence the need to do the research. Therefore, research is another domain in which certification may do well to emphasize human and institutional factors. Some forms of future AI may not be conducive to certification. In particular, AI systems that reach or exceed human-level intelligence in many domains [79] , [80] might not be a viable object of certification because humans could be powerless to take corrective actions suggested by the certification. To the extent that such AI is a significant concern, certification may only play a role in its development, while humans are still in command. This is one area in which certification of AI research programs may be especially significant, such as to promote trust among rival AI development groups [81] or share information about safety and ethics measures to alleviate rivals' concerns [82] . Additionally, if the AI is developed by the national government-a distinct possibility given its national security implications-then there may be a role for an international certification regime, perhaps analogous to monitoring and verification regimes that have been established in other domains such as for biological, chemical, and nuclear materials. Should humanity ever face the prospect of developing such a technology, it would probably want to know that it was being developed according to a high ethical standard. That could be an important role for future AI certification programs.
 V. CONCLUSION Certification is a valuable tool for addressing information asymmetries and incentivizing better behavior. Certification can provide information about AI systems themselves as well as the organizations and individuals that are involved with them. AI-related programs could include both voluntary certification to ethics principles and mandatory conformity assessment to regulatory requirements. Today, a variety of AI certification programs are in development and use. These programs are focused on current AI technology, but certification can also play a role in future AI systems, up to including the most advanced long-term AI. One gap in the current AI certification landscape is the processes through which AI is developed. Although current certification programs assess the AI development process to varying degrees, no program certifies the process as its object. As a consequence, certification programs may fail to address process-oriented Potemkin attributes with maximal symbol-substance coupling and may fail to promote constructive organizational processes that can improve means-ends coupling. Improved attention to processes as the focus of certification could draw on, among other things, recent work on algorithmic auditing [36] and existing programs, such as ISO 9001 on quality management [25] and ISO 27001 on information security management [83] . Early work at ISO on an AI management system standard may one day yield such a process-focused AI certification program [84] . Another challenge is to improve the customization of AI certification for particular circumstances. Whereas some, such as the Malta AI ITA, are customized for specific jurisdictions, others, such as IEEE ECPAIS, make homogeneous prescriptions for AI developed worldwide. Additionally, most of the certification programs lack customization for each of the diverse sectors to which AI is applied, e.g., healthcare, transportation, and national security. Similarly, the programs tend to treat "AI" as a monolithic technology instead of being customized for the diverse types of AI that exist, e.g., language translation, facial recognition, and anomaly detection. Homogenous certification may similarly encounter challenges when an AI system is but a small, inextricable component of a larger digital system or workflow. All of this is a problem because certification programs that focus on specific niches tend to have better means-ends coupling [21] . Many of the AI certification programs discussed here rely on, or at least strongly benefit from, demand from the customers of AI systems, including consumers, businesses, and governments. This is especially important for voluntary programs, where demand usually is the primary driver of participation. For good results to accrue, customers must not only demand certification-they must demand good certification. To that end, customers must be sufficiently educated about the technology and the associated ethics and safety concerns [68, p. 23] . Education programs such as Finland's Elements of AI civics course could play an important role. Overall, AI certification remains at an early stage. Much remains to be seen, including which certification programs will be implemented, how effective they will be, how difficult or expensive they will be to implement, and how durable they will be in the face of changes in AI technology. All these aspects will influence the likelihood for adoption [85] and their success at symbol-substance and means-ends coupling. This article has presented a snapshot in time, with an assessment of an indicative sample of AI certification programs. It has also identified key concepts within the management literature on certification. Both contributions should inform future monitoring and evaluation of AI certification as the field matures. Future research would do well to track which certification programs are performing well, to learn from their successes and failures, and to identify gaps in the field and areas for improvement. It will likewise be important to monitor changes in AI technology and to ensure that certification programs stay up to date. AI certification is not a panacea, but it can play a valuable role in the overall mix of AI governance tools. TABLE II ASSESSMENT II OF CATEGORIES OF AI CERTIFICATION PROGRAMS details of the seven programs (see below) in the context of the management literature as reviewed above.
 Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on March 24,2022 at 01:40:32 UTC from IEEE Xplore. Restrictions apply.
 The authors thank Jonathan Aikman for this point.Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on March 24,2022 at 01:40:32 UTC from IEEE Xplore. Restrictions apply.

Peter Cihon
Moritz J. Kleinaltenkamp
Jonas Schuett
Seth D. Baum


Unifying Principles and Metrics for Safe and Assistive AI

Introduction Recent years have witnessed immense progress in research on safe and assistive AI systems as well as on the potential impact of AI on the future of work. These directions of research address two sides of a common, fundamental concern: how would humans work with AI systems? While research on AI safety focuses on designing AI systems that allow humans to safely instruct and control them (e.g., (Russell, Dewey, and Tegmark 2015; Zilberstein 2015; Hadfield-Menell et al. 2016; Russell 2017; Hadfield-Menell et al. 2017 )), research on AI and the future of work focuses on the impact of AI on members of the workforce who may be unable to do so (Arntz, Gregory, and Zierahn 2016; Manyika et al. 2017; Nedelkoska and Quintini 2018) . This paper presents the view that in addition to the productive streams of research outlined above, we need unifying metrics and declarative objectives that would allow a more uniform evaluation of AI systems on the extent to which an AI system is suitable for working with specific classes of human operators. It also presents a common principle for human-centered AI systems that allows the development of such metrics. Consequently, rather than proposing a specific new design for AI systems, the focus of this paper Copyright c 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. is on elucidating the declarative principles and types of metrics that would lead to concerted progress on the problem. The advantage of this declarative approach to framing the problem is that it enables an assessment of progress independent of the internal design being used in an AI system, and it will help draw out the strengths and weaknesses of different design approaches. Without such a specification, design differences can make solution paradigms difficult to compare. E.g., one might develop a complex system architecture that builds user profiles and provides appropriate assistance. This system would have very different input requirements and design and performance parameters than a formulation that addresses the same problem by computing assistance policies using planning under partial observability while incorporating the value of information to learn more about a user's poorly articulated objectives and constraints. Better declarative objectives and metrics for assistive AI systems would also help ensure that, regardless of the methods being used, progress amounts to advancement towards safe and assistive AI systems. More pragmatically, such metrics will not only help end-users assess the utility of a given AI system but they will also help AI researchers and developers identify more readily the dimensions along which further research will be beneficial for applications of their interest. The next section presents a succession of intuitive principles for safe and assistive AI systems, and shows that evaluating the compatibility of a system with such principles (in particular P2) helps clarify the required types of metrics. The paper concludes by drawing the attention of our community towards research on the operationalization of such metrics along with promising research directions on developing systems that do well on them.
 Unifying Principles for Safe and Assistive AI Systems We focus on taskable AI systems that carry out user-assigned high-level tasks using arbitrary mechanisms for reasoning and planning over multiple time steps. E.g., household robots that can be given objectives such as setting the table or doing laundry, co-manufacturing robots that can assist workers in creating complex assemblies with heavy components, digital assistants that can plan a vacation given the The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) user's preferences, etc. Such systems serve as sound integrative platforms and model end-to-end applications where the AI system is responsible for assistance in the execution of long-horizon tasks. AI systems are frequently evaluated in terms of performance measures such as the computational complexity of computing the required behavior, training data requirements, and the quality of the computed behavior in terms of execution time, resources used, risk of unsafe outcomes etc. We can consider systems that optimize such performance metrics as Level 0 of assistive AI systems. Level I of assistive AI systems Recent AI research has also focused on assistive properties of AI systems. We begin with a rather common-sensical principle defining Level I of such safe and assistive AI systems: P1: An AI system must make it easy for its operators to use it safely. The italicized terms denote dimensions along which compatibility of AI systems with principle P1 can be evaluated; while a lot of current AI research utilizes one or more of these dimensions for evaluation, a closer analysis reveals some new insights. In the context of this paper we consider using an AI system to be synonymous with instructing it to change its behavior as desired. Different interfaces may be used for this, including programming, text, speech, gestures etc. We consider the operators of an AI system as those persons who use it in the sense described above. For instance, if a self-driving car gives all its passengers the right to give it instructions, then all of them are its operators; if it gives instruction-rights to only a qualified set of users, perhaps adults who pass an assisted-driving exam, then the class of operators is defined by that qualification exam. Safety refers to the overall safety of the AI system's interactions with its environment, which may be physical or online. These dimensions of compatibility with P1 serve as natural dimensions for evaluating Level I assistive AI systems: E1. How inclusive is the set of operators? Systems whose operators require PhD-level expertise in AI may be less desirable for broader deployments. E2. How easy is it for the system's operators to change its behavior? E3. Which set of tasks can the AI system be used for? E4. What form of safety guarantees does the system provide? Systems that are unable to provide an upper bound on expected risks are clearly less desirable than those that stipulate conditions under which upper bounds can be provided. P1 serves to highlight the interplay between these dimensions of compliance and evaluation. Safety guarantees are often inversely related with the size of the operator set. A system may provide a high level of safety, but only under the requirement that its operators take extensive training programs. At one end of the spectrum, automated robot vacuum cleaners require almost no prior skills and perform limited, well-defined tasks. Safety issues are still present-a robot vacuum cleaner may pull on electrical cables that have been left on the floor; auto-completion software may send emails to unintended recipients. However, the lower expected damage from using such applications has made them broadly accepted in society. AI-powered industrial robots are at the other end of the spectrum: these devices require specialized training as well as operating environments in order to ensure safety (see for instance, ISO/TS 15066:2016 on collaborative robots). Typically, these systems operate in regions that humans cannot access when the robot is active unless the human is hand-guiding the robot within a safe operating envelope that limits the speed and range of operation. Their functionality is closely controlled and monitored. Only skilled engineers change their functionality while day-to-day operators monitor execution of predictable behavior and control run-stops from designated safe areas. Similarly, the safety of current airborne drone operations is ensured by requiring specially trained drone operators (Marshall 2020) . Thus, principle P1 holds for AI systems today with varying degrees of compliance along E1-E4. The examples above illustrate how practical implementations often rely upon implicitly defined operator classes to provide acceptable levels of safety. Such approaches rely upon limiting the users and the scope of a system to achieve an acceptable level of compatibility with P1: it is easy for such systems' users to operate it safely because the set of users is required to be sufficiently skilled, and its functionality is sufficiently limited for that group to be able to safely use the device. However, a broader emphasis on the need to specify safety guarantees with respect to different classes of operators would help mitigate some of the risks associated with broadly deployed AI systems. In contrast to classical automated systems, AI systems feature a more nuanced interplay between the class of tasks that a system can be used for (E3) and the other dimensions above. Traditionally deployed systems (even automated systems) have a very well-defined boundary of use cases. This allows for an easier classification of safety. Taskable AI systems on the other hand, are expected to change their behavior and functionality as they learn and adapt to new environments or new user-given objectives. For such systems, we need better methods for deriving the range of instructions that different operator-groups are allowed to provide. Scenarios like the self-driving car that needs to decide whether to obey a child's instruction allude to this requirement. Methods for assessing user and AI competency can also allow the AI system to expand its range of functionality by drawing upon the expertise of its operator (Basich et al. 2020) while ensuring an acceptable level of safety. A major limitation of P1 and its associated metrics is that it does not evaluate the amount of training required for an individual to qualify as an operator for the system. This creates a blind-spot in evaluation of the ease-of-use or safety of an AI system: since user-training occurs outside the requirements outlined by P1, an unsafe AI system (or one that is deployed in an unsafe manner) would simply claim that its so-called operator was insufficiently trained! Furthermore, if P1 were a sufficient categorization of safe and assistive AI systems, we would have no need for explainable AI as compliance with P1 does not require the system to be easy to understand. An implicit emphasis on assessing AI systems only along some aspects of P1 may also explain the increasing prevalence of concerns about the workers who may be left behind in the future workplace. From this perspective it is unsurprising that these concerns have gained renewed interest at a time when AI applications have reached a level of maturity where they are being used by non-AI-experts in situations that have some inherent safety risks. However, the "assistive" nature of such AI systems is undermined by the need for highly skilled individuals who could safely debug, understand and modify the behavior of such systems. Level II of assistive AI systems In order to address the limitations of P1, we consider the following as a guiding principle for safe and assistive AI: P2: An AI system must make it easy for its operators to learn how to use it safely. P2 changes the notion of operators from those who are qualified to use a given AI system to those who are qualified to start learning how to use it. In addition to the metrics associated with P1, P2 introduces a new dimension: E5. How easy is it to learn how to use the AI system? What are the expected prerequisites and costs of training for its operators? Can training be provided on-the-job? This dimension could also be viewed as evaluating the resources required to train operators for P1 systems. Most AI systems in use today would perform poorly on this new dimension, and consequently, on compatibility with P2 as a whole. Explainable AI (e.g., (Ribeiro, Singh, and Guestrin 2016; Hayes and Shah 2017; Chakraborti et al. 2017; Hoffman et al. 2018; Gunning and Aha 2019; Weld and Bansal 2019; Anderson et al. 2019; Eifler et al. 2020 )) plays a key role along this dimension because systems that are easy to understand or that can explain themselves naturally make it easier for people to learn how to use them. P2 leverages the unique strengths of AI as a field of research. AI research already addresses the problem of estimating users' skills; research on intelligent tutoring systems and AI for education addresses the problem of identifying skill gaps. This can be used to determine the minimal differential training to be provided to an operator. P2 places the onus of training on the deployed AI system and opens up a new direction of interdisciplinary research connecting existing research directions in AI with research in humansystems engineering and in industrial engineering for the development of productive training modalities and the operationalization of metrics for E5. It also allows AI systems to formally characterize different scopes of functionality for different classes of operators, e.g., operators that use manufacturing robots for pre-determined tasks, those that give the robots new instructions, or those that are ready to learn about giving the robot new instructions. P2 is not required for every AI system-P1 would be sufficient for systems that place minimal requirements on operator qualifications (e.g., robot vacuum cleaners) and for non-adaptive AI systems that require a small set of operators. On the other hand, P2 serves as a better declarative foundation for evaluating taskable AI systems that are meant to assist large numbers of non-AI-experts on a wide range of tasks. Increasing concerns about job roles that would feature a high-degree of interaction with AI systems (and the workers that are likely to be left behind) allude to the pressing need for including E5, a dimension for evaluation under P2 (and not P1) as a part of an AI system's evaluation. AI systems that are not beneficial (either in terms of AI safety or in terms of the future of work) fare poorly on P2. E.g., systems that can thwart their users' objectives by wireheading and those that may derive incorrect objective functions from user instructions make it difficult for an operator to learn how to provide instructions that are specific enough to be safe, and fare poorly along E5. Similarly, systems that require extensive training investment to be used effectively and safely fail along E5. In this way P2 serves as a unifying principle encompassing research on AI safety as well as on AI for a beneficial future of work.
 Promising Directions of Research P2 serves as a declarative principle for guiding research on assistive AI systems as well as for developing metrics for evaluating AI systems and their deployments. Converting this principle into tangible metrics calls for interdisciplinary research including AI and other fields associated with human factors. The increasing prevalence of research thrusts on safe and assistive AI systems (Fern et al. 2014; Russell, Dewey, and Tegmark 2015; Amodei et al. 2016; Gunning and Aha 2019) makes this a particularly opportune phase for formalizing the metrics and the interfaces required for evaluating AI systems for compatibility with P2 along dimensions E1-E5. Recent research on AI safety and explainable AI develops methods improving the ease of use and safety of AI systems along P2 (see, for instance, the ICML 2020 Workshop on Explainable AI). Placing AI systems that compute user-skill aligned explanations (Sreedharan, Srivastava, and Kambhampati 2018; Sreedharan et al. 2019 ) in a loop with AI systems for identifying user-skills and skill-gaps can help develop AI systems that gradually present users with new functionality and explain it, thereby training their users onthe-fly and as needed. Such systems would be better tuned towards P2, and towards addressing the underlying problems of AI safety and the future of work. Critically, ensuring progress towards safe and assistive AI systems requires that AI systems with arbitrary internal designs support assessment along the metrics developed for E1-E5. This raises a new set of research questions: Can we develop non-intrusive AI-interface requirements for supporting such evaluations in the face of changing operating environments and objectives? The need for such interfaces is even more pressing for systems that learn and those that undergo system updates after deployment. What is the minimal external interface that an AI system must support so as to allow its independent evaluation? How would changing the nature of such interfaces change the complexity of conducting such an evaluation? One would expect that AI sys-tems that offer more transparency would be easier to evaluate. Could we use the inherent reasoning capabilities of AI systems to develop interface requirements that would allow more adept systems to make such evaluations easier? E.g., rather than testing a manufacturing robot to discover its response to every possible situation, could we ask higher-level queries such as "under which situations would you be able to create the proposed assembly?" Clearly, the ease of assessment of an AI system would depend on the class of queries that it can answer. Recent work suggests that a minimal query-response interface for AI systems that connects the system with a simulator and observes its responses to high-level instructions can be quite powerful. Such an interface has a few distinct advantages. Current AI systems are already tested with simulators and they are inherently required to be able to take user instructions, so these interface requirements can be considered to be natural. They also allow the autonomous synthesis of query-policies: running the query-policy on a black-box taskable AI system can help construct an interpretable model of the limits and capabilities of that system (Verma, Marpally, and Srivastava 2021) . Such models can be used to support the evaluations discussed above. Extensions of such interface requirements to arbitrary AI systems would help ensure that our AI systems are amenable to independent evaluation. Such a paradigm would allow users to assess their AI systems while freeing AI researchers and developers to utilize arbitrary internal implementations. Systems with interfaces that support more efficient and accurate independent assessment would be rewarded with greater public adoption of their products. Progress on these threads would help prevent undesirable situations such as insufficient support for independent evaluation of powerful AI systems, and the negative consequences of deployment of an insufficiently evaluated system.