The Immigration Preferences of Top AI Researchers: New Survey Evidence | GovAI

The Immigration
Preferences of Top AI
Researchers:
New Survey EvidenceJANUARY 2021
Remco Zwetsloot, Baobao Zhang, Markus
Anderljung, Michael C. Horowitz, Allan Dafoe
Perry World House is a center
for scholarly inquiry, teaching, research, international exchange, policy engagement, and public outreach on pressing global issues.
Perry World House’s mission is to bring the academic knowledge of the University of Pennsylvania to bear on some of the world’s most pressing global policy challenges, and to foster international policy engagement within and beyond the Penn community.
Located in the heart of campus at 38th
Street and Locust Walk, it draws on the expertise of Penn’s 12 schools and numerous globally-oriented research centers to educate the Penn community and prepare students to be well-informed, contributing global citizens. At the same time, Perry World House connects Penn with leading policy experts from around the world to develop and advance innovative policy proposals.
Through its rich programming, Perry
World House facilitates critical conversations about global policy challenges and fosters interdisciplinary research on these topics.  It presents workshops and colloquia, welcomes distinguished visitors, and produces content for global audiences and policy leaders, so that the knowledge developed at Penn can make an immediate impact around the world.
Perry World House—its programs and
the building itself—builds on Penn’s strengths in teaching, interdisciplinary scholarship, and policy-relevant research.  By doing so, it cultivates the broad worldview, critical thinking, and leadership required to address the most urgent issues of global affairs.The Future of Humanity Institute is a unique world-leading research centre that works on big picture questions for human civilisation and explores what can be done now to ensure a flourishing long-term future.
Its multidisciplinary research team includes several of the world’s most brilliant and famous minds working in this area. Its work spans the disciplines of mathematics, philosophy, computer science, engineering, ethics, economics, and political science.
FHI has originated or played a
pioneering role in developing many of the key concepts that shape current thinking about humanity’s future. These include: simulation argument, existential risk, nanotechnology, information hazards, strategy and analysis related to machine superintelligence, astronomical waste, the ethics of digital minds, crucial considerations, observation selection effects in cosmology and other contexts of self-locating belief, prediction markets, infinitarian paralysis, brain emulation scenarios, human enhancement, the unilateralist’s curse, the parliamentary model of decision making under normative uncertainty, the vulnerable world hypothesis, and many others.
@perryworldhouse
facebook.com/perryworldhouse
@perryworldhouse
@FHIOxford
facebook.com/FHIOxford
Contents
About the Authors            4
Acknowledgements            5
Executive Summary            6
Data               8
Findings               8
 AI Researchers’ Likelihood of Migrating
 AI Researchers’ Immigration Decisions
 AI Researchers’ Perceptions of Immigration Policy
Conclusion                                                                     15
Appendix A: Additional Results       16
 Additional Results on AI Researchers’ Likelihood of Migrating
 Additional Results on AI Researchers’ Immigration Decisions
Appendix B: Sample and Survey Methodology     20
 Respondent Representativeness
 Survey Questions
Endnotes            26
About the Authors
Remco Zwetsloot
@r_zwetsloot
Remco Zwetsloot is a Research Fellow at Georgetown’s
Center for Security and Emerging Technology
Baobao Zhang
@baobaofzhang
Baobao Zhang is Klarman Postdoctoral Fellow in the
Cornell Society of Fellows, Cornell University
Markus Anderljung
@manderljung
Markus Anderljung, Project Manager: Operations & Policy
Engagement, Centre for the Governance of AI at the Future of Humanity Institute, Oxford University
Michael C. Horowitz
@mchorowitz
Michael C. Horowitz is Richard Perry Professor and
Director of Perry World House at the University of
Pennsylvania
Allan Dafoe
@allandafoe
Allan Dafoe is Associate Professor in the International
Politics of Artificial Intelligence, and Director of the Centre for the Governance of AI at the Future of Humanity Institute, Oxford University

Acknowledgements
This paper is co-published by the Centre for the Governance
of AI at the Future of Humanity Institute at the University
of Oxford and Perry World House at the University of
Pennsylvania, with institutional support from the Center
for Security and Emerging Technology at Georgetown
University. This project was supported by the Ethics and
Governance of Artificial Intelligence Fund, and Michael
Horowitz’s work on this project was supported in part by
U.S. government grant FA9550-18-1-0194. The authors
would like to thank Catherine Aiken, Zachary Arnold, Tessa
Baker, James Dunham, Melissa Flagg, Charlie Giattino,
Roxanne Heston, Igor Mikelic-Torreira, Dewey Murdick, and
Helen Toner for feedback on the AI expert survey and the
analysis. We are also grateful for research assistance and
editorial support from Noemi Dreksler, Emmie Hine, Lauren
Kahn, Will Marks, Kwan Ye Ng, and Sacha Zimmerman.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence6
Zwetsloot, Zhang, Anderljung, Horowitz, DafoeExecutive Summary
Artificial intelligence (AI) talent
is global. AI researchers and engineers come from, and are in high demand, all over the world. Countries and companies trying to recruit and retain AI talent thus face immense competition. In order to understand current and
prospecti ve flows of t alent, we
investigate the drivers of AI
researchers’ i mmi gratio n
decisions and preferen ces.
Immigr
ation questions are
particularly salient for the United States today, as half of its current AI workforce and two-thirds of graduate students in AI-related graduate programs were born elsewhere.
1 Some
experts believe that the current U.S. immigration system will prevent or dissuade many of these international graduates from staying in the country, potentially undermining the vitality of the U.S. technology sector.
2 Many other countries
have also seen recent immigration policy debates centered on attracting AI talent.
3
To better understand the immigration decisions and preferences of this global AI workforce, we conducted a survey of more than 500 active researchers who publish in the leading machine learning conferences.
4 Key findings
include:
•Some countries are
considerably more attractiveto AI researchers than others.
oNearly 60 percent of
respondents not  cur rently
based in the United Statesthink there is a greater thanone-in-four chance they willmove there within the nextthree years. The samepercentages are 35 percentfor the United Kingdom, 28percent for Canada, and 10percent for China.
oSome countries, such as theUnited States and the UnitedKingdom, are attractive toresearchers from a diversityof backgrounds, while othercountries, such as China andFrance, mostly appeal toresearchers who grew up inthat country or its vicinity.
•Decisions about when andwhere to move depend on amix of professional, personal,and political factors.
oWhen asked what factorsaffected respondents’moving decisions, more than90 percent reported that acountry’s professionalenvironment and
opportunities wereimportant to them. “Lifestyle and culture” (79 percent), the “political climate” (66 percent), and “personal relations” (60 percent) came next. “Immigration incentives in the destination country”
The Immigration Preferences of Top AI Researchers:
New Survey Evidence7
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoeand “immigration difficulties
in the country of residence” were less important (47 percent and 24 percent, respectively).
 oAll respondents considered “professional opportunities,” as well as “lifestyle and culture,” the most important factors, regardless of where they live. However, respondents who currently live in Asia were less likely than those in Europe or North America to rate “personal relations” or the “political climate” as a key factor.
• AI researchers consider legal immigration barriers a bigger problem in some countries than in others.
 oNearly 70 percent of AI researchers based in the United States considered “visa and immigration issues” a serious problem for AI research in the country. This was significantly higher than in the United Kingdom (44 percent), Canada (29 percent), China (16 percent), and other countries.
 oAs a whole, these findings suggest that the optimal strategies for countries to attract and retain AI talent will vary, depending on national strengths and weaknesses. For countries like the United States, and to a lesser extent the United Kingdom and Canada, the biggest obstacle to international talent recruitment is not attractiveness but legal immigration barriers. For countries like China, however, the opposite is true: the main barriers to attracting AI talent are not legal but professional, cultural, and political in nature.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence8
Zwetsloot, Zhang, Anderljung, Horowitz, DafoeData
This paper draws on data from
a survey of AI experts who
answered several immigration-
related questions. The experts
were selected based on having
papers accepted at top AI research conferences. One
group of respondents consisted
of researchers who participated in the 2018 Conference on
Neural Information Processing
Systems (NeurIPS) and the 2019
International Conference on
Machine Learning (ICML). Another group had papers
accepted at NeurIPS and ICML
in 2015 and participated in a
2016 expert survey on AI,
fielded by several authors of this paper.
5
Out of the 3,030 researchers contacted to complete the survey, 524 researchers (17
percent) completed the survey.
6
The survey respondents work and study in more than two
dozen countries, although a
majority are based in North
America and Western Europe. Respondents were asked about
a range of issues, including but
not limited to immigration.
7
Appendix B discusses the
sample in more detail, compares
survey respondents with
nonrespondents to assess
representativeness, and lists the complete survey questions
and answer options.Findings
This paper presents results on
three questions:
1. How likely are AI researchers to migrate and
where?
2. What factors affect
researchers’ moving
decisions? and
3. Do researchers consider
immigration policy a
challenge to the AI sector in
their country of residence?
AI Researchers’
Likelihood of Migrating
To gauge AI researchers’
likelihood of migrating, and
different potential destination
countries’ level of appeal,
respondents were asked: “Of
the following countries, which would you have a greater than
25% likelihood of moving to for
work or study in the next three years?” Respondents saw a list
of 10 countries that they could
indicate an interest in, as well
as an open textbox where they
could list additional countries.
8
Their current country of
residence was omitted from the
answer list.
Figure 1 shows the percentage
of respondents who said they
might move to each of the listed
countries. The United States
The Immigration Preferences of Top AI Researchers:
New Survey Evidence9
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoescored highest, with 58 percent
of respondents who are not
currently residents there saying
they had a greater than 25
percent chance of moving there
within the next three years. The United Kingdom, Canada, and
Switzerland came next
(between 25 percent and 35 percent of respondents). China, despite its large investment and good performance in AI
research, does not score high in
attractiveness, with only 10
percent of nonresident
respondents indicating an interest in immigrating there.
These results provide a crucial
but incomplete picture of
different countries’
Figure 1 | Which countries are AI researchers most likely to move to? | This
graph shows the percentage of AI researchers who reported a greater than
25 percent chance of moving to a country for work or study within the
next three years. Respondents were presented with a list of countries to
choose from (see Appendix B for a full list), which omitted their country
of residence. There was also a “none of these” and an “other” option in
which respondents could write a different country they had a greater than
25 percent chance of moving to. Error bars represent 95 percent confidence
intervals.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence10
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoeattractiveness to AI researchers.
For example, out of the 524
survey respondents, 209
already lived in the United
States when they took the
survey, presumably because
they considered it an attractive
place to study or work. Their
feelings about the United States
are not represented in Figure 1,
which looks only at researchers
not already living in a given
country.
To provide a better sense of a
country’s overall attractiveness,
Table 1 in Appendix A shows the
number of respondents who
currently live in the country as
well as the number of
respondents who said they may
move there in the next three
years. For example, whereas
the share of surveyed AI
researchers who don’t live in
the United States but who may
move there within three years is
58 percent (as seen in Figure 1),
the share of researchers who
may move there or who already
live there is 78 percent. In other
words, only 22 percent of the
sample found the United States
unattractive as a place to work
or live (or is unlikely to move
there for some other reason).9
Another way to look at
attractiveness is by looking at
the percentage of researchers
in a country who were unlikely
to move at all. Figure 4 in Appendix A shows that 37
percent of U.S.-based
researchers reported they were
unlikely to move anywhere else,
compared with between 23
percent and 14 percent for most
other common countries of
residence.
We also investigated whether
researchers’ likelihood of
moving somewhere varied by
where they came from. Figure 5
in Appendix A breaks down
respondents’ answers by the
country where they did their
undergraduate degree, which is
a common and generally reliable
proxy for a person’s citizenship.10
Some countries appeared
broadly attractive. The United
States was the top destination
for almost all respondent
categories—consistently
scoring 50 percent or higher no
matter where researchers did
their undergraduate degree—
and the United Kingdom was
also popular (scoring between
23 percent and 71 percent).
Other countries had more
narrow appeal. For example,
respondents who did their
undergraduate degree in China
were much more likely to report
considering a move to China
(roughly 50 percent) than
respondents from any other
country (around 5 percent on
average). Similarly, France was
popular mainly among European
respondents. (Because there
The Immigration Preferences of Top AI Researchers:
New Survey Evidence11
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoewere only a limited number of
respondents from some of
these countries, not all of these
differences are statistically
significant; see Appendix A for
a discussion.)
AI Researchers’
Immigration Decisions
Considering the differences in
how likely AI researchers were
to move to particular countries
raises the question of what
factors determine whether
countries are considered
attractive. To answer this
question, respondents were
asked: “When considering
moving to work or study in a
country that you don’t currently
work or study in full-time, what
factors are important in your
consideration?” Respondents
saw a list of six factors
commonly cited as key to
immigration decisions in the
high-skill immigration
literature.11 Respondents could
select as many factors as they
wanted, and they could also
add other factors in an open
textbox.12
Figure 2 shows the percentage
of respondents who designated
that a given factor is important
to their moving decisions.
Having a “good professional
opportunities and environment”
clearly topped the other factors,
with 91 percent of respondents selecting it as important.
“Lifestyle and culture,” “political
climate,” and “personal
relations” were important to 79
percent, 66 percent, and 60
percent of respondents,
respectively. “Ease of
immigration and/or immigration
incentives provided to move to
destination country” were cited
as important by a little under
half, and least important were
“current immigration difficulties
in country of residence”
(selected by 24 percent of all
respondents).13
Figure 2 reports the results for
all respondents, but looking at
subcategories provides answers
to additional questions. First,
the decision factor “current
immigration difficulties in
country of residence” was
mainly relevant to respondents
who live in a country where
they are not citizens. We do not
have a direct measure of
respondents’ citizenship status,
but we can impute citizenship
by whether a respondent lives
in the same country where they
did their undergraduate degree.
Figure 6 in Appendix A shows
that about 40 percent of
(imputed) noncitizen
respondents in the United
States reported that “current
immigration difficulties” were a
factor influencing their moving
decisions, a higher percentage
than in other countries.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence12
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoe
Figure 2 | What factors affect AI researchers’ immigration decisions? |
This graph shows what AI researchers report are important considerations
in their immigration decisions. Respondents were asked what factors were
important in their consideration of moving to work or study in a country
that they don’t currently work or study in full-time. Respondents were
presented with the six options shown in the figure, alongside an “other”
and “none of the above” option. Error bars represent 95 percent confidence
intervals.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence13
Zwetsloot, Zhang, Anderljung, Horowitz, DafoeSecond, Figure 7 in Appendix A
looks at whether the importance
of these factors differed
depending on where
respondents live. Notably,
respondents living in Asia
viewed “personal relations” as
less important than those living
in the United States or Europe
(36 percent compared with
around 64 percent), and the
same is true for “political
climate of the destination
country” (41 percent compared
with around 66 percent). Yet
there are also striking similarities
across respondents who live in
different regions. For example,
“good professional
opportunities” and “lifestyle”
ranked first and second in
importance among all regional
respondent groups.
AI Researchers’
Perceptions of
Immigration Policy
National success in attracting
and retaining international
talent requires countries not
only to be attractive, but also
to have a legal immigration
system that allows researchers
and engineers to live and work
there long-term. Given the
efforts at reform and political
tensions surrounding
immigration policy in the United
States and many other
countries, it is critical to understand how immigration
policy issues factor into the
decision-making process for
talented AI researchers. This
survey presented respondents
with a list of potential issues
that could affect AI research
and asked respondents to
indicate which, if any, applied
to their country of residence.
“Visa and immigration
problems” were among the
potential issues facing foreign
researchers and students (see
Appendix B for the full list of
options).
Figure 3 shows what percentage
of respondents said that “visa
and immigration problems”
were affecting AI research in
their country of residence.
Nearly 70 percent of those in
the United States said that “visa
and immigration problems”
were a serious problem,
compared with 44 percent of
those in the United Kingdom,
29 percent of those in Canada,
and less than 20 percent of
those in other countries. Not all
of these differences are
statistically distinguishable
because there were few
respondents in some of these
countries, but it is clear that AI
researchers in the United States
considered immigration a
bigger issue than researchers
elsewhere.
This finding is consistent with
The Immigration Preferences of Top AI Researchers:
New Survey Evidence14
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoecomplaints by American
researchers and companies that
restrictive U.S. immigration
policies hurt the U.S. AI sector
and that other countries have
better-designed immigration
systems.14 However, other
interpretations are also possible. For example, the fact
that researchers in other
countries rated immigration
and visa issues as less of a
problem for their AI sector
could also be explained by
lower levels of demand for AI
talent in other countries (which
Figure 3 | Where are visa and immigration issues a serious problem for
conducting AI research? | Respondents were asked what they considered
to be serious problems for conducting high-quality AI research in their
country of residence, with an option list that included seven issues (see
Appendix B). Figure 3 presents the by-country breakdowns for one of
these issues: the percentage of AI researchers living in each country who
feel that visa and immigration issues are a serious problem for conducting
high-quality AI research in their country of residence. Error bars represent
95 percent confidence intervals.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence15
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoewould mean immigrants are not
as needed). These data do not
definitively distinguish between
these explanations.
Conclusion
The findings presented in this
paper suggest that different
countries face different
immigration-related challenges
that impact their ability to
attract top AI researchers. For
the United States, attractiveness
is not a big issue. Many top AI
researchers are already based
in the United States, and nearly
60 percent of those who are
not said they “may move to the
United States in the near future.”
However, immigration obstacles
loom large for the United States:
Nearly 70 percent of surveyed
AI researchers based in the
country said “immigration and
visa issues” posed a serious
problem for U.S. AI research, a
considerably higher share than
in any other country. This
finding bolsters long-standing
complaints by leading U.S.
machine learning scientists,
who have long argued that “visa
restrictions have been one of
the largest bottlenecks to our
collective research productivity
over the last few years” and
that immigration obstacles are
hurting the country’s AI sector.15
For countries like the United
Kingdom and Canada, the story is more mixed. The number of
researchers there who said
“immigration issues are a
problem for AI research” is
lower than in the United States
but still considerable. Similarly,
they were the second- and
third-most attractive countries
to AI researchers, respectively,
with 35 percent and 28 percent
of respondents saying they
could move there soon. For the
United Kingdom and Canada,
then, a successful AI talent
recruiting strategy would likely
have to combine reductions in
immigration barriers with
targeted steps aimed at
increasing the countries’
attractiveness.
For China and several other
countries, the main challenge
to attracting international AI
talent seems to be low overall
appeal, rather than their
immigration system. Less than
20 percent of respondents in
China said that “visa and
immigration issues” are an
important problem for the
country’s AI ecosystem.
However, only 10 percent of
respondents not currently
residing in China would consider
moving there, and most of them
appear to be Chinese citizens.
Successful AI talent recruiting
policies for countries like China
will thus have to involve
bolstering attractiveness.
Judging by the factors
The Immigration Preferences of Top AI Researchers:
New Survey Evidence16
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoerespondents said are relevant
to their immigration decisions,
this would likely require
addressing concerns about a
mix of professional, cultural,
and political issues.16
Future research on the nexus
between immigration and AI
talent should go deeper into
these and other questions. First,
similar questions should be
asked not just of researchers
publishing at top conferences,
but also other AI professionals,
such as engineers and
entrepreneurs, who are likely to
be underrepresented in our
data. Second, to validate our
findings, follow-up work could
ask the same questions in
different ways to see if
responses are robust (e.g.,
eliciting immigration
preferences through conjoint
survey experiments as opposed
to direct questions) or collect
data on researchers’ moving
behavior to assess how
predictive immigration
intentions are of actual choices.
Third, many more immigration-
related questions could be
asked, such as which specific
visa and immigration problems
are most worrisome to AI
researchers.Appendix A:
Additional Results
Additional Results on AI
Researchers’ Likelihood
of Migrating
Figure 1 showed how many
respondents said that there was
a decent chance (greater than
25 percent) that they would
move to different countries
within the next three years.
Below are additional materials
to contextualize the data
presented in Figure 1.
First, Table 1 helps provide a
better sense of a country’s
overall attractiveness. The first
two columns show the raw
frequencies and percentages
already displayed in Figure 1,
focusing on how many
nonresident respondents said
they may move to certain
countries within the next three
years. The third column shows
the number of respondents who
already live in a country, and
therefore presumably find it an
attractive place to work or live.
The fourth column sums the
first and third columns together,
showing the total number
respondents who either already
live in a country or would
consider moving there within
the next three years. The fifth
column converts this number
into a share of all 439
The Immigration Preferences of Top AI Researchers:
New Survey Evidence17
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoerespondents who were asked
this question.
Figure 4 provides another
perspective on different
countries’ attractiveness to AI
researchers. It shows what
percentage of respondents
already based in a country said they were unlikely (less than 25
percent chance) to move to any
other country within the next
three years. For example, 37
percent of researchers in the
United States answered “none
of these” to the question about
future migration destinations,
compared with 23 percent or
Table 1 | Number of current residents and potential immigrants across different
destination countries | Table 1 displays countries selected as answer options for the
survey question (see Appendix B); at least one respondent lived in an additional 16
countries not shown here (none of those countries hosted more than six respondents).
Data on country of residence was missing for 72 respondents. The denominator used
to calculate the percentage in column 2 fluctuates by country due to a varying number
of nonresident respondents. The denominator used to calculate the percentages in
column 5 is 434, representing the number of people who answered both the work
location question and the question concerning potential moving destinations.1 2 3 4 5
Number of
nonresident
respondents
with greater
than 25%
likelihood of
moving within
three yearsPercent of
nonresident
respondents
with greater
than 25%
likelihood of
moving within
three yearsNumber of
respondents
based in
countryNumber of
respondents
who are either
based in a
country or
indicate
interest in
moving therePercent of
respondents
who are either
based in a
country or
indicate
interest in
moving there
U.S. 134 58% 209 343 78%
U.K. 133 35% 54 187 43%
Canada 114 28% 22 136 31%
Switzerland 105 25% 22 127 29%
France 66 16% 25 91 21%
China 41 10% 21 62 14%
Singapore 34 8% 3 37 9%
Japan 26 6% 7 33 7%
Australia 25 6% 5 30 7%
South Korea 12 3% 11 23 5%
Israel 9 2% 4 13 3%
The Immigration Preferences of Top AI Researchers:
New Survey Evidence18
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoefewer of respondents based in
other countries.
Figure 5 considers what
destination countries are most
attractive depending on where
respondents did their
undergraduate degree. Country
of undergraduate degree is often used as a proxy for
nationality when data on actual
nationality is not available,
because the vast majority of
students do their bachelor’s in
their home country.17 Figure 5
breaks out respondents who
received their undergraduate
degrees in the United States,
Figure 4 | Which countries are AI researchers unlikely to leave? |
Respondents were asked where they had a greater than 25 percent chance
of moving to for work or study within the next three years. They were
presented with a list of countries to choose from as well as “none of these”
and “other” options (see Appendix B for the full list). This figure shows
the percentage of AI researchers who chose the option “none of these,”
sorted by current country of residence. Error bars represent 95 percent
confidence intervals.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence19
Zwetsloot, Zhang, Anderljung, Horowitz, DafoeChina, India, the United
Kingdom, and Germany—the
five most common
undergraduate countries (see
Table 2 in Appendix B)—plus a
residual “other” category. Some
countries, such as the United
States and the United Kingdom,
had broad appeal across all
respondent groups. Other
countries, such as China and to
some extent France, were attractive only to researchers
who appear to hail from those
countries or the same region.
Additional Results on AI
Researchers’ Immigration
Decisions
Figure 2 in the paper displayed
the factors that respondents
considered important to their
immigration decisions. The two
Figure 5 | Which countries are appealing to AI researchers from a broad
range of backgrounds? | Respondents were asked where they had a greater
than 25 percent chance of moving to for work or study within the next
three years. They were presented with a list of countries to choose from
as well as “none of these” and “other” options (see Appendix B for the full
list). This figure shows the percentage of AI researchers who selected the
six most popular of the possible destination countries, broken down by
where respondents did their undergraduate degree.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence20
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoefigures below analyze specific
respondent subgroups that
provide additional insights into
different AI researchers’
decisions.
First, Figure 2 showed that 24
percent of all respondents
reported “current immigration
difficulties in country of
residence” as a factor
influencing their decision to
move. However, in practice, this
factor was relevant only to
respondents who are not
citizens in their current country
of residence. It is therefore
instructive to look at this
particular factor only among
noncitizens. We do not have a
direct measure of respondents’
citizenship status, but we can
impute citizenship by whether
a respondent lives in the same
country where they did their
undergraduate degree. There
are four countries where there
are more than 10 (imputed)
noncitizens in the respondent
pool: the United States, Canada,
the United Kingdom, and
Switzerland.18
For these four countries, Figure
6 shows the percentage of
noncitizens—that is,
respondents who did their
undergraduate degree in
different countries—who
reported that current
immigration difficulties are an
important factor for their immigration decisions.
Noncitizens in the United States
were most likely to say this is
the case (40 percent), while
noncitizens in Switzerland were
least likely to (14 percent).
Second, Figure 2 considered all
respondents regardless of their
location. Figure 7 below shows
the factors that respondents
considered important to their
immigration decisions broken
out by region of current
residence: North America,
Europe, or Asia. The most
notable differences across
regions are in how important
“personal relations” and the
“political climate” in the
destination country are, two
factors considered much less
important by respondents living
in Asia. Importance scores for
the other factors are mostly
similar.
Appendix B:
Sample and Survey
Methodology
This paper draws on data from
an AI expert survey fielded by
the Centre for the Governance
of AI at the Future of Humanity
Institute at the University of
Oxford. The survey was
circulated via email from mid-
September to mid-October
2019 to 3,030 machine learning
researchers who had papers
The Immigration Preferences of Top AI Researchers:
New Survey Evidence21
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoeaccepted at either NeurIPS
(2018, 2015) or ICML (2019,
2015), two top conferences in
the field. Overall, 524
researchers responded to the
survey (response rate: 17
percent).We collected data on a variety
of respondents’ demographic
characteristics. For example,
most live in Western countries
(see Table 1), although China
and India were the second- and
third-most common countries
Figure 6 | In which countries are noncitizens most likely to say that current
immigration difficulties are an important factor influencing their
immigration decisions? | Respondents were asked what factors were
important in their consideration of moving to work or study in a country
that they don’t currently work or study in full-time. Respondents were
presented with six options, one of which was “current immigration
difficulties in country of residence.” This figure shows the percentage of
imputed noncitizen respondents who said that this factor was important
to their consideration. Figure 6 shows this percentage across the four
countries that had more than 10 imputed noncitizens. Error bars represent
95 percent confidence intervals.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence22
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoefor respondents to have
received their undergraduate
degree, respectively (Table 2).
Most respondents are relatively
young, with the average
respondent having received their undergraduate degree in
2009 (Table 3). Finally, the
majority of respondents work
in academia, though a significant
minority work in industry (Table
4).
Figure 7 | Regional differences in what factors affect AI researchers’
immigration decisions | Respondents were asked what factors were
important in their consideration of moving to work or study in a country
that they don’t currently work or study in full-time. Respondents were
presented with the six options shown in Figure 7. The figure shows the
percentage of AI researchers who reported that these six factors were
important, broken down by their current region of residence (Asia, Europe,
and North America). Error bars represent 95 percent confidence intervals.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence23
Zwetsloot, Zhang, Anderljung, Horowitz, DafoeCountry of
undergraduate
degreeNumber of
respondents
United States 115
China 84
India 43
United
Kingdom23
Germany 21
France 20
Iran 18
Italy 17
South Korea 16
Canada 14
Table 2 | Country where
respondents received their
undergraduate degree (top 10) |
There were an additional 37
countries in which seven or fewer
respondents received their
undergraduate degree. Data were
missing for 70 respondents.
Statistic Year
Mean 2009
Median 2011
Minimum 1974
Max 2019
Table 3 | Year respondents
completed their undergraduate
degree | Data were missing for 120
of the respondents.Employment
sectorNumber of
respondents
Only academic 360
Only industry 109
Table 4 | Respondents’ sector(s)
of current employment | “Other”
primarily involves government and
nonprofit organizations.
Respondents could report multiple
affiliations.
Respondent
Representativeness
Out of the 3,030 researchers
contacted, 524 researchers (17
percent) completed the survey.
This raises questions about
representativeness: Are those
who responded systematically
different from those who did
not? To assess this question, we
collected background data on a
random sample of 446
nonrespondents using
information publicly available
online (e.g., from LinkedIn).
We compared respondents with
nonrespondents on a range of
dimensions, including gender;
the region where they did their
undergraduate degree, their
Ph.D. degree, and where they
are currently employed; the
proportion employed in
academia and industry or still
enrolled in a degree program;
and their logged citation counts
and H-index. Respondents
closely resemble
The Immigration Preferences of Top AI Researchers:
New Survey Evidence24
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoenonrespondents on almost all
of these dimensions. The only statistically significant differences between the groups are in employment sector (respondents were more slightly likely to work in academia than nonrespondents) and H-index (respondents with a higher H-index were slightly less likely to respond). For a full comparison table, see Appendix B of our companion paper “Ethics and Governance of Artificial Intelligence: Evidence from a Survey of Machine Learning Researchers.”
19
Survey Questions
We reported results for the following questions:
• “Of the following countries, which would you have a greater than 25% likelihood of moving to for work or study in the next three years?” The answer options presented to respondents (in randomized order) were:
 oUnited States
 oUnited Kingdom
 oChina
 oAustralia
 oSingapore
 oJapan
 oCanada
 oSouth Korea
 oIsrael
 oFrance
 oSwitzerland oOther [textbox]
 oNone of these
These countries were selected based on studies of AI talent that show these countries as the top destinations for AI researchers.
20
• “When considering moving
to work or study in a country that you don’t currently work or study in full-time, what factors are important in your consideration?” The answer
options presented to respondents (in randomized order) were:
 oCurrent immigration
difficulties in country of residence
 oEase of immigration and/or incentives provided to move to destination country
 oPersonal relations in the destination country (e.g., friends and family)
 oGood professional opportunities and environment (e.g., you are offered a job at an attractive organization)
 oLifestyle and culture
 oPolitical climate of the destination country
 oOther: [textbox]
 oNone of the above
As noted above, these options
were chosen based on a literature review of prior immigration-related surveys among researchers and other
The Immigration Preferences of Top AI Researchers:
New Survey Evidence25
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoehigh-skill immigrants.21
• “Which of the following, if
any, are serious problems for
conducting high-quality AI
research in <INSERT NAME
OF YOUR COUNTRY OF
WORK> today?” The answer
options presented to
respondents (in randomized
order) were:
 oLack of government funding
for AI research
 oLack of corporate funding for
AI research
 oLack of funding for training
students
 oVisa and immigration
problems facing foreign
researchers or students who
want to work or study in
<INSERT NAME OF YOUR
COUNTRY OF WORK>
 oLack of a successful
ecosystem for AI startups
 oNot enough top researchers
and labs
 oThe political climate in
<INSERT NAME OF YOUR
COUNTRY OF WORK> is not
conducive to AI research
 oOther: [textbox]
 oThere are no serious problems
Other sections of the survey
asked about AI governance
challenges, AI progress
forecasts and milestones,
research priorities and norms,
and employment preferences,
among other topics. The median
respondent took 17 minutes to
complete the survey.The Centre for the Governance
of AI will publish further reports
on the full survey results.22
The Immigration Preferences of Top AI Researchers:
New Survey Evidence26
Zwetsloot, Zhang, Anderljung, Horowitz, DafoeEndnotes
1. Remco Zwetsloot, Roxanne Heston, and Zachary Arnold, Strengthening the
U.S. AI Workforce: A Policy and Research Agenda (Washington, D.C.: Center for
Security and Emerging Technology, September 2019), https:/ /cset.georgetown.
edu/research/strengthening-the-u-s-ai-workforce/ .
2. See, e.g., Association for the Advancement of Artificial Intelligence, “AAAI
Response to NITRD RFI: National Artificial Intelligence Research and Development
Strategic Plan,” October 26, 2018, https:/ /www.nitrd.gov/rfi/ai/2018/AI-RFI-
Response-2018-Yolanda-Gil-AAAI.pdf ; Oren Etzioni, “What Trump’s Executive
Order on AI Is Missing,” Wired, February 13, 2019, h ttps:/ /www.wired.com/story/
what-trumps-executive-order-on-ai-is-missing/ .
3. Tina Huang and Zachary Arnold, Immigration Policy and the Global Competition
for AI Talent (Washington, D.C.: Center for Security and Emerging Technology,
June 2020), https:/ /cset.georgetown.edu/research/immigration-policy-and-
the-global-competition-for-ai-talent/ .
4. Our study complements a recent survey by the Center for Security and
Emerging Technology that looked specifically at the immigration preferences of
AI Ph.D. graduates from U.S. universities; see Catherine Aiken, James Dunham,
and Remco Zwetsloot, Immigration Pathways and Plans of AI Talent: Findings
from a CSET Survey of Artificial Intelligence PhDs from U.S. Universities
(Washington, D.C.: Center for Security and Emerging Technology, September
2020), https:/ /cset.georgetown.edu/research/immigration-pathways-and-
plans-of-ai-talent/ .
5. For the results of the 2015–2016 expert survey, see Katja Grace, John Salvatier,
Allan Dafoe, et al., “When Will AI Exceed Human Performance? Evidence from AI
Experts,” Journal of Artificial Intelligence Research, 62: 729-754, July 2018,
https:/ /doi.org/10.1613/jair.1.11222 . This previous expert survey did not ask
immigration-related questions.
6. We incentivized responses through a lottery: One out of every 10 researchers
who completed the survey received a $250 gift card.
7. The Centre for the Governance of AI will publish the full survey results. An
analysis of the survey questions focused on ethics and governance can be found
at Baobao Zhang, Markus Anderljung, Lauren Khan, et al., “Ethics and Governance
of Artificial Intelligence: Evidence from a Survey of Machine Learning
Researchers,” Working Paper, October 2020, https:/ /osf.io/pnvfd/ .
8. These countries were selected based on studies of AI talent that showed they
were the top destinations for AI researchers. Jean-Francois Gagne, “Global AI
Talent Report 2019,” https:/ /jfgagne.ai/talent-2019/ ; Artificial Intelligence Index,
“2019 Annual Report,” https:/ /hai.stanford.edu/sites/g/files/sbiybj10986/f/ai_
index_2019_report.pdf ; Zwetsloot et al., Keeping Top AI Talent in the United
States, Figure 9.
9. Note that these results are not necessarily representative of the views of AI
researchers as a whole. The response rate of the survey was 17 percent, and we
do not know why some contacted researchers filled out the survey and why
others did not, so the results should be interpreted with caution. For example, it
could be that researchers who had unfavorable attitudes toward the United
States or the United Kingdom were less likely to respond to a survey sent out by
The Immigration Preferences of Top AI Researchers:
New Survey Evidence27
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoeuniversities from those countries. Our analysis comparing respondents with
nonrespondents finds little imbalance in terms of demographic characteristics
(Appendix B), but we do not have data that allow us to compare response rates
across attitudinal variables.
10. Country of undergraduate degree is a common and validated proxy for
nationality when actual nationality data is not available, as discussed in Appendix
A.
11. See, e.g., Xueying Han and Richard P. Appelbaum, Will They Stay or Will They
Go? International STEM Students Are Up for Grabs (Kansas City, MO: Ewing
Marion Kauffman Foundation: July 2016), https:/ /eric.ed.gov/?id=ED570660 ;
Mark Musumba, Yanhong H. Jin, and James W. Mjelde, “Factors Influencing Career
Location Preferences of International Graduate Students in the United States,”
Education Economics, 19(5): 501-517, September 2009, https:/ /doi.
org/10.1080/09645290903102902 . For a summary of this research, see Remco
Zwetsloot, James Dunham, Zachary Arnold, and Tina Huang, Keeping Top AI
Talent in the United States: Findings and Policy Options for International Graduate
Student Retention (Washington, D.C.: Center for Security and Emerging
Technology: December 2019), 10-12, https:/ /cset.georgetown.edu/wp-content/
uploads/Keeping-Top-AI-Talent-in-the-United-States.pdf . More potential factors
could of course have been added, but the number of included factors was limited
by the need to avoid overburdening respondents.
12. Fourteen respondents indicated other factors that were important to them in
the open-ended textbox. Factors that were mentioned multiple times included
distance from family and friends (mentioned four times), political issues or
inequality (three times), income and quality of life (twice), and the health care
system (twice).
13. There are at least two ways to interpret the “current immigration difficulties”
finding. It could be that respondents are not encountering difficulties—for
example, because they are already permanent residents or citizens. Or respondents
may be encountering difficulties but not letting those influence their immigration
decisions. Unfortunately, the phrasing of the question does not allow us to
distinguish between these two interpretations.
14. For data and statements, see Zachary Arnold, Roxanne Heston, Remco
Zwetsloot, and Tina Huang, Immigration Policy and the U.S. AI Sector (Washington,
D.C.: Center for Security and Emerging Technology, September 2019), https:/ /
cset.georgetown.edu/research/immigration-policy-and-the-u-s-ai-sector/ ; Tina
Huang and Zachary Arnold, Immigration Policy and the Global Competition for AI
Talent (Washington, D.C.: Center for Security and Emerging Technology, June
2020), https:/ /cset.georgetown.edu/research/immigration-policy-and-the-
global-competition-for-ai-talent/ .
15. For AI researcher statements, see Zachary Arnold, “Misguided Immigration
Policies Are Endangering America’s AI Edge,” Defense One, November 18, 2019,
https:/ /www.defenseone.com/ideas/2019/11/misguided-immigration-policies-
are-endangering-americas-ai-edge/161366/ . The quote comes from Ian
Goodfellow, then a research scientist at Google Brain and currently director for
machine learning at Apple.
16. For a survey of 2,300 STEM researchers whose findings reinforce this takeaway,
see Richard van Noorden, “Global Mobility: Science on the Move,” Nature, October
17, 2012,  https:/ /www.nature.com/news/global-mobility-science-on-the-
move-1.11602.
The Immigration Preferences of Top AI Researchers:
New Survey Evidence28
Zwetsloot, Zhang, Anderljung, Horowitz, Dafoe17. The most likely source of error in using the country of undergraduate degree
as a proxy for nationality is Chinese and Indian students, who study abroad—
mostly for graduate school but also for their bachelor’s—at much higher rates
than students from other countries. This means that some Chinese and Indian
students will be misclassified as, for example, U.S., Canadian, or British nationals.
Past studies have found that error rates are generally 10 to 20 percent at most;
see, for example, Zwetsloot, et al., Keeping Top AI Talent, fn. 24. Moreover, to the
extent that there is an error, the direction of bias is predictable. For example,
because Chinese citizens are more likely to say that they might move to China,
measurement error would likely lead us to overestimate the attractiveness of
China to U.S. or U.K. citizens.
18. The United States is home to 108 (imputed) noncitizens, the United Kingdom
to 26, Switzerland to 17, and Canada to 15.
19. Asian countries present in our dataset are China, South Korea, Japan, India,
and Singapore. Russia is included in Europe.
20. Zhang, et. al.,“Ethics and Governance of Artificial Intelligence: Evidence
from a Survey of Machine Learning Researchers.”
21. Jean-Francois Gagne, “Global AI Talent Report 2019,” https:/ /jfgagne.ai/
talent-2019/ ; Artificial Intelligence Index, “2019 Annual Report,” https:/ /hai.
stanford.edu/sites/g/files/sbiybj10986/f/ai_index_2019_report.pdf ; Zwetsloot,
et al., “Keeping Top AI Talent in the United States,” Figure 9.
22. Including Zhang, et al., “Ethics and Governance of Artificial Intelligence:
Evidence from a Survey of Machine Learning Researchers.”
UNIVERSITY OF PENNSYLVANIA
PERRY WORLD HOUSE
3803 LOCUST WALK,
PHILADELPHIA, PA 19104
215.573.5730

Remco Zwetsloot
Baobao Zhang
Markus Anderljung
Michael C. Horowitz
Allan Dafoe


Should Artificial Intelligence Governance be Centralised?: Design Lessons from History

INTRODUCTION In 2018, Canada and France proposed the International Panel on Artificial Intelligence (IPAI). After being rejected at the G7 in 2019, negotiations shifted to the OECD and are presently ongoing. As the field of AI continues to mature and spark public interest and legislative concern  [41] , the priority of such governance initiatives reflects the growing appreciation that AI has the potential to dramatically change the world for both good and ill  [9] . Research into AI governance needs to keep pace with policy-making and technological change. Choices made today may have long-lasting impacts on policymakers' ability to address numerous AI policy problems  [7] . Effective governance can promote safety, accountability, and responsible behaviour in the research, development, and deployment of AI systems. AI governance research to date has predominantly focused at the national and sub-national levels  [6, 16, 44] . Research into AI global governance remains relatively nascent (though see  [5] ). Kemp et al.  [24]  have called for specialised, centralised intergovernmental agencies to coordinate policy responses globally, and others have called for a centralised 'International Artificial Intelligence Organisation'  [14] . Others favour more decentralised arrangements based around 'Governance Coordinating Committees', global standards, or existing international law instruments  [8, 28, 47] . No one has taken a step back to inquire: what would the history of multilateralism suggest, given the state and trajectory of AI? Should AI governance be centralised or decentralised? 'Centralisation', in this case, refers to the degree to which the coordination, oversight and/or regulation of a set of AI policy issues or technologies are housed under a single (global) institution. This is not a binary choice; it exists across a spectrum. Trade is highly (but not entirely) centralised under the umbrella of the WTO. In contrast, environmental multilateralism is much more decentralised. In this paper, we seek to help the community of researchers, policymakers, and other stakeholders in AI governance understand the advantages and disadvantages of centralisation. This may help set terms and catalyse a much-needed debate to inform governance design decisions. We first outline the international governance challenges of AI, and review early proposed global responses. We then draw on existing literatures on regime fragmentation  [3]  and 'regime complexes'  [36]  to assess considerations in centralising the international governance of AI. We draw on the history of other international regimes  1  to identify considerations that speak in favour or against designing a centralised regime complex for AI. We conclude with two recommendations. First, many trade-offs are contingent on how well-designed a central body would be. An adaptable, powerful institution with a manageable mandate would be beneficial, but a poorly designed body could prove a fate worse than fragmentation. Second, for now there should be structured monitoring of existing efforts to see whether they are they are self-organising or insufficient.
 THE STATE OF AI GOVERNANCE There is debate as to whether AI is a single policy area or a diverse series of issues. Some claim that AI cannot be cohesively regulated as it is a collection of disparate technologies, with different risk profiles across different applications and industries  [45] . This is an important but not entirely convincing objection. The technical field has no settled definition for 'AI', 2 so it should be no surprise that defining a manageable scope for AI governance will be difficult. Yet this challenge is not unique to AI: definitional issues abound in areas such as environment and energy, but have not figured prominently in debates over centralisation. Indeed, energy and environment ministries are common at the domestic level, despite problems in setting the boundaries of natural systems and resources. We contend that there are numerous ways in which a centralised body could be designed for AI governance. For example, a centralised approach could carve out a subset of interlinked AI issues to cover. This could involve focusing on the potentially highrisk applications of AI systems, such as AI-enabled cyberwarfare, lethal autonomous weapons (LAWS), other advanced military applications, or high-level machine intelligence (HLMI).  3  Another approach could govern underlying hardware resources (e.g. largescale compute resources) or software libraries. We are agnostic on the specifics of how centralisation could or should be implemented, and instead focus on the costs and benefits of centralisation in the abstract. The exact advantages and disadvantages of centralisation are likely to vary depending on the institutional design. This is an important area of further study, particularly once more specific proposals are put forward. However, such work must be grounded in a higher-level investigation of trade-offs in centralising AI governance. It is this foundational analysis which we seek to offer. Numerous AI issues could benefit from international cooperation. These include the potentially catastrophic applications mentioned above. It also encompasses more quotidian uses, such as AI-enabled cybercrime; human health applications; safety and regulation of autonomous vehicles and drones; surveillance, privacy and datause; and labour automation. Multilateral coordination could also use AI to tackle other global problems such as climate change  [43] , or help meet the Sustainable Development Goals  [46] . This is an illustrative but not exhaustive list of international AI policy issues. Global regulation across these issues is currently nascent, fragmented, yet evolving. A wide range of UN institutions have begun to undertake some activities on AI  [20] . The bodies covering AI policy issues range across existing organisations including the International Labour Organisation (ILO), International Telecommunication Union (ITU), and UNESCO. This is complemented by budding regulations and working groups across the International Organisation for Standardisation (ISO), International Maritime Organisation (IMO), International Civil Aviation Organisation (ICAO), and other bodies, as well as treaty amendments, such as the updating of the Vienna Convention on Road Traffic to encompass autonomous vehicles  [28] , or the ongoing negotiations at the Convention on Certain Conventional Weapons (CCW) on LAWS. The UN System Chief Executives Board (CEB) for Coordination through the High-Level Committee on Programmes has been empowered to draft a system-wide AI capacity building strategy. The High-level Panel on Digital Cooperation has also sought to gather together common principles and ideas for AI relevant areas  [19] . Whether these initiatives bear fruit, however, remains questionable, as many of the involved international organisations have fragmented membership, were not originally created to address AI issues and lack effective enforcement or compliance mechanisms  [32, p.2] . The trajectory of these initiatives matters. How governance is initially organised can be central to its success. Debates over centralisation and fragmentation are long-lasting and prominent with good reason. How we structure international cooperation can be critical to its success, and most other debates often implicitly hinge on structural debates. Fragmentation and centralisation exist across a spectrum. In a world lacking a global government, some fragmentation will always prevail. But the degree to which it prevails is crucial. We define 'fragmentation' as a patchwork of international organisations and institutions which focus on a particular issue area, but differ in scope, membership and often rules  [3, p.16 ]. We define centralisation as an arrangement in which governance of a particular issue lies under the authority of a single umbrella body. A regime complex is a network of three or more international regimes on a common issue area. These should have overlapping membership and cause potentially problematic interactions  [36, p.29] . These definitions and terms are by nature normatively loaded. For example, some may find 'decentralisation' to be a positive framing, while others may see 'fragmentation' to possess negative connotations. Recognising this, we seek to use these terms in a primarily analytical manner. We will use findings from each of these theoretical areas to inform our discussion of the history of multilateral fragmentation and its implications for AI governance.
 CENTRALISATION CRITERIA: HISTORY OF GOVERNANCE TRADE-OFFS In the following discussion, we explore a series of considerations for AI governance. Political power and efficient participation support centralisation. The breadth vs. depth dilemma, as well as slowness and brittleness support decentralisation. Policy coordination and forum shopping considerations can cut both ways.
 Political Power Regimes embody power in their authority over rules, norms, and knowledge beyond states' exclusive control. A more centralised regime will see this power concentrated among fewer institutions. A centralised, powerful architecture is likely to be more influential against competing international organisations and with constituent states  [36, pp.36-7 ]. An absence of centralised authority to manage regime complexes has presented challenges in the past. Across the proliferation of Multilateral Environmental Agreements (MEAs) there is no requirement to cede responsibility to the UN Environmental Programme in the case of overlap or competition. This has led to turf wars, inefficiencies and even contradictory policies  [3] . One of the most notable examples is that of hydrofluorocarbons (HFCs). HFCs are potent greenhouse gases, and yet their use has been encouraged by the Montreal Protocol since 1987 as a replacement for ozonedepleting substances. This has only recently been resolved via the 2015 Kigali Amendment to the Montreal Protocol, which itself has a prolonged implementation period. Similarly, the internet governance regime complex is diffuse. Multiple venues and norms govern technical standards, cyber crime, human rights, and warfare  [35] . Although the UN Internet Governance Forum (IGF) discusses several cross-cutting issues, it does not have a mandate to consolidate even principles, let alone negotiate new formal agreements  [33] . In contrast, other centralised regimes have supported effective management. For example, under the umbrella of the WTO, norms such as the most-favoured-nation principle (equally treating all WTO member states) principle have become the bedrock of international trade. The power and track-record of the WTO is so formidable that it has created a chilling effect: the fear of colliding with WTO norms and rules has led environmental treaties to self-censor and actively avoid discussing or deploying trade-related measures  [12] . Both the chilling effect and the remarkably powerful application of common trade rules were not a marker of international trade until the establishment of the WTO. The power of these centralised body has stretched beyond influencing states in the domain of trade, to moulding related issues. Political power offers further benefits in governing emerging technologies that are inherently uncertain in both substance and policy impact. Uncertainty in technology and preferences has been associated with some increased centralisation in regimes  [25] . There may also be benefits to housing a foresight capacity within the regime complex, to allow for accelerated or even proactive efforts  [39] . Centralised AI governance would enable an empowered organisation to more effectively use foresight analyses to inform policy responses across the regime complex.
 Supporting Efficiency & Participation Decentralised AI governance may undermine efficiency and inhibit participation. States often create centralised regimes to reduce costs, for instance by eliminating duplicate efforts, yielding economies of scale within secretariats, and simplifying participation  [15] . Conversely, fragmented regimes may force states to spread resources and funding over many distinct institutions, particularly limiting the ability of less well-resourced states or parties to participate fully  [32, p.2] . Historically, decentralised regimes have presented cost and related participation concerns. Hundreds of related and sometimes overlapping international environmental agreements can create 'treaty congestion'  [1] . This complicates participation and implementation for both developed and developing nations  [15] . This includes costs associated with travel to different forums, monitoring and reporting for a range of different bodies, and duplication of effort by different secretariats (ibid.). Similar challenges are already being witnessed in AI governance. Simultaneous and globally distributed meetings pose burdensome participation costs for civil society. Fragmented organisations must duplicatively invest in high-demand machine learning subject matter experts to inform their activities. Centralisation would support institutional efficiency and participation.
 Slowness & Brittleness of Centralised Regimes One potential problem of centralisation lies in the relatively slow process of establishing centralised institutions, which may often be outpaced by the rate of technological change. Another challenge lies in centralised institutions' brittleness after they are established, i.e., their vulnerability to regulatory capture, or failure to react to changes in the problem landscape. Establishing new international institutions is often a slow process. For example, the Kyoto Protocol took three years of negotiations to create and then another eight to enter into force. This becomes even more onerous with higher participation and stakes. Under the GATT, negotiations for a 26% cut in tariffs between 19 countries took 8 months in 1947. The Uruguay round, beginning in 1986, took 91 months to achieve a tariff reduction of 38% between 125 parties  [31] . International law has been quick to respond to technological changes in some cases, and delayed in others  [42, p.184 ]. Decentralised efforts may prove quicker to respond to complex, 'transversal' issues, if they rely more on informal institutions with a smaller but like-minded membership  [32, pp.2-3] . Centralised AI governance may be particularly vulnerable to sparking lengthy negotiations, because progress on centralised regimes for new technologies tends to be hard if a few states hold clearly unequal stakes in the technology, or if there are significant differences in information and expertise among states or between states and private industry  [42, pp.187-94] . Both these conditions closely match the context of AI technology. Moreover, because AI technology develops rapidly, such slow implementation of rules and principles could lead to certain actors taking advantage by setting de facto arrangements or extant state practice. Even after its creation, a centralised regime can be brittle; the very qualities that provide it with political power may exacerbate the adverse effects of regulatory capture; the features that ensure institutional stability, may also mean that the institution cannot adapt quickly to unanticipated outside stressors outside its established mission. The regime might break before it bends. The first potential risk is regulatory capture. Given the high profile of AI issue areas, political independence is paramount. However, as illustrated by numerous cases, including undue corporate influence in the WHO during the 2009 H1N1 pandemic  [11] , no institution is fully immune to regime capture, and centralisation may reduce the costs of lobbying, making capture easier by providing a single locus of influence. On the other hand, a regime complex comprising many parallel institutions could find itself vulnerable to capture by powerful actors, who are better positioned than smaller parties to send representatives to every forum. Moreover, centralised regimes entail higher stakes. Many issues are in a single basket and thus failure is more likely to be severe if it does occur. International institutions can be notoriously pathdependent and thus fail to adjust to changing circumstances, as seen with the ILO's considerable difficulties in reforming its participation and rulemaking processes in the 1990s  [2] . The public failure of a flagship global AI institution or governance effort could have lasting political repercussions. It could strangle subsequent, more well-conceived proposals in the crib, by undermining confidence in multilateral governance generally or capable governance on AI issues specifically. By contrast, for a decentralized regime complex to similarly fail, all of its component institutions would need to simultaneously 'break' or fail to innovate at once.  4  A centralised institution that does not outright collapse, but which remains ineffective, may become a blockade against better efforts. Ultimately, brittleness is not an inherent weakness of centralisationand indeed depends far more on institutional design details. There may be strategies to 'innovation-proof'  [29]  governance regimes. Periodic renegotiation, modular expansion, 'principles based regulation', or sunset clauses can also support ongoing reform  [30, pp.29-30] . Such approaches have often proved successful historically, due partially to decentralisation but, importantly, also to particular designs.
 The Breadth vs. Depth Dilemma Pursuing centralisation may create an overly high threshold that limits participation. All multilateral agreements face a trade-off between having higher participation ('breadth') or stricter rules and greater ambition of commitments ('depth'). The dilemma is particularly evident for centralised institutions that are intended to be powerful and require strong commitments from states. However, the opposite dynamics of sacrificing depth for breadth can also pose risks. The 2015 Paris Agreement on Climate Change was significantly watered down to allow for the legal participation of the US. Anticipated difficulties in ratification through the Senate led to negotiators opting for a 'pledge and review' structure with few legal obligations. Thus, the US could join simply through the approval of the executive  [23] . In this case, inclusion of the US (which at any rate proved temporary) came at the cost of significant cutbacks on the demands which the regime sought to make of all parties. In contrast, decentralisation could allow for major powers to engage in relevant regulatory efforts where they would be deterred from signing up to a more comprehensive package. This has precedence in the history of climate governance. Some claim that the US-led Asia-Pacific Partnership on Clean Development and Climate helped, rather than hindered climate governance, as it bypassed UNFCCC deadlock and secured non-binding commitments from actors not bound by the Kyoto Protocol [49, pp.259-60].  4  We thank Nicolas Moës for this observation. This matters, as buy-in may prove a thorny issue for AI governance. The actors who lead in AI development include powerful states that are potentially most adverse to global regulation in this area. They have thus far proved recalcitrant in the global governance of security issues such as anti-personnel mines or cyberwarfare. In response, some have already recommended a critical-mass governance approach to the military uses of AI. Rather than seeking a comprehensive agreement, devolving and spinning off certain components into separate treaties (e.g. for LAWS testing standards; liability and responsibility; and limits to operational usage) could instead allow for the powerful to ratify and move forward at least a few of those options  [48] . The breadth vs. depth dilemma is a trade-off in multilateralism generally. However, it is a particularly pertinent challenge for centralisation. The key benefit of a centralised body would be to be a powerful anchor that ensures policy coordination and coherence, without suffering fragmentation in membership. This dilemma suggests it is unlikely to have both. It will likely need to restrict membership to have teeth, or lose its teeth to have wide participation. A critical mass approach may be able to deliver the best of both worlds. Nonetheless these dilemma poses a difficult knot for centralisation to unravel.
 Forum Shopping Forum shopping may help or hinder AI governance, depending on the particular circumstances. Fragmentation enables actors to choose where and how to engage. Such 'forum shopping' may take one of several forms: moving venues, abandoning one organisation, creating new venues, and working across multiple organisations to sew competition between them  [4] . Even when there is a natural venue for an issue, actors have reasons to forum-shop. For instance, states may look to maximise their influence, appease domestic pressure  [40]  and placate constituents by shifting to a toothless forum  [18] . The ability to successfully forum-shop depends on an actor's power. Most successful examples of forum-shifting have been led by the US  [4] . Intellectual property rights in trade, for example, was subject to prolonged, contentious forum shopping. Developed states resisted attempts of the UN Conference on Trade and Development (UNCTAD) to address intellectual property rights in trade by trying to push them onto the World Intellectual Property Organization (WIPO) (ibid., 566) and then subsequently to the WTO  [18] , overruling protests from developing states. Outcomes often reflect power, but weak states and non-state actors can also pursue forum shopping strategies in order to challenge the status-quo  [22] . Forum shopping may help or hurt governance. This is evident in current efforts to regulate LAWS. While the Group of Governmental Experts has made some progress, on the whole the CCW has taken slow deliberations on LAWS. In response, frustrated activists have threatened to shift to another forum, as happened with the Ottawa Treaty that banned landmines  [10] . This strategy could catalyse progress, but also brings risks of further forum shopping and weak or unimplemented agreements. Forum shopping may similarly delay, stall, or weaken regulation of time-sensitive AI policy issues, including potential future HLMI development. It is plausible that leading AI firms also have sway when they elect to participate in some venues but not others. The OECD Expert Group on AI included representatives from leading firms, whereas engagement at UN efforts, including the Internet Governance Forum (IGF), do not appear to be similarly prioritised. A decentralised regime will enable forum shopping, though further work is needed to determine whether this will help or hurt governance outcomes on the whole.
 Policy Coordination There are good reasons to believe that either centralisation or fragmentation could enhance coordination. A centralised regime can enable easier coordination both across and within policy issues, acting as a focal point for states. Others argue that this is not always the case, and that fragmentation can mutually supportive and even more creative institutions. Centralisation reduces the occurrence of conflicting mandates and enables communication. These are the ingredients for policy coherence. As noted previously, the WTO has been remarkably successful in ensuring coherent policy and principles across the realm of trade, and even into other areas such as the environment. However, fragmented regimes can often act as complex adaptive systems. Political requests and communication between secretariats often ensures bottom-up coordination even in the absence of centralisation. Multiple organisations have sought to reduce greenhouse gas emissions within their respective remits, often at the behest of the UNFCCC Conference of Parties. When effective, bottom-up coordination can slowly evolve into centralisation. Indeed, this was the case for the GATT and numerous regional, bilateral and sectoral trade treaties, which all coalesced together into the WTO. While this organic self-organisation has occurred, it has taken decades, with forum shopping and inaction prevailing for many years. Indeed, some have argued that decentralisation does not just deliver 'good enough' global governance  [38]  that reflects a demand for diverse principles in a multipolar world. Instead, they argue 'polycentric' governance approaches  [37]  may be more creative and legitimate than centrally coordinated regimes. Arguments in favour of polycentricity include the notion that it enables governance initiatives to begin having impacts at diverse scales, and that it enables experimentation with diverse policies and approaches, learning from experience and best practices (ibid., 552). Consequently, these scholars assume "that the invisible hand of a market of institutions leads to a better distribution of functions and effects"  [50, p.7] . It is unclear if the different bodies covering AI issues will selforganise or collide. Many of the issues are interdependent and will need to be addressed in tandem. Some particular policy-levers, such as regulating computing power or data, will impact almost all use areas, given that AI progress and use is closely tied to such inputs. Numerous initiatives on AI and robotics are displaying loose coordination  [28] , but it remains uncertain whether the virtues of a free market of governance will prevail here. Great powers can exercise monopsony-like influence in forum shopping, and the supply of both computing power and machine learning expertise are highly concentrated. In sum, centralisation can reduce competition and enhance coordination, but it may suffocate the creative selforganisation of more fragmented arrangements over time.
 DISCUSSION: WHAT WOULD HISTORY SUGGEST? 4.1 A Summary of Considerations The multilateral track record and peculiarities of AI yield suggestions and warnings for the future. A centralised regime could lower costs, support participation, and act as a powerful new linchpin within the international system. Yet centralisation presents risks for AI governance. It could simply produce a brittle dinosaur, of symbolic value but with little meaningful impact on underlying political or technological issues. A poorly executed attempt could lock-in a poorly designed centralised body: a fate worse than fragmentation. Accordingly, ongoing efforts at the UN, OECD, and elsewhere could benefit from addressing the considerations presented in this paper.
 The Limitations of 'Centralisation vs. Decentralisation' Debates Structure is not a panacea. Specific provisions such as agendas and decision-making procedures matter greatly, as do the surrounding politics. Underlying political will may be impacted by framing or connecting policy issues  [26, pp.770-1] . The success of a regime is not just a result of fragmentation, but of design details. Moreover, institutions can be dynamic and broaden over time by taking in new members, or deepen in strengthening commitments. Successful multilateral efforts, such as trade and ozone depletion, tend to do both. We are in the early days of global AI governance. Decisions taken early on will constrain and partially determine the future path. This dependency can even take place across regimes. The Kyoto Protocol was largely shaped by the targets and timetables approach of the Montreal Protocol, which in turn drew from the Convention on Long-range Transboundary Air Pollution. This targets and timetables approach continues today in the way that most countries frame their climate pledges to the Paris Agreement. The choices we make on governing short-term AI challenges will likely shape the management of other policy issues in the long term  [7] . On the other hand, committing to centralisation, even if successful, may amount to solving the wrong problem. The problem may not be structural, but geopolitical. Centralisation could even exacerbate the problem by diluting scarce political attention, incurring heavy transaction costs, and shifting discussions away from bodies which have accumulated experience and practice  [21] . For example, the Bretton Woods Institutions of the IMF and World Bank, joined later by the WTO, are centralised regimes that engender power. However, those institutions had the express support of the US and may have simply manifested state power in institutional form. Efforts to ban LAWS and create a cyberwarfare convention have been broadly opposed by states with an established technological superiority in these areas  [13] . A centralised regime may not unpick these power struggles, but just add a layer of complexity.
 LESSONS AND CONCLUSIONS Our framework provides a tool for policy-makers to inform their decisions of whether to join, create, or forgo new institutions that tackle AI policy problems. For instance, the recent choice of whether to support the creation of an independent IPAI involved these considerations. Following the US veto, ongoing negotiations for its replacement at the OECD may similarly benefit from their consideration. For now, it is worth closely monitoring the current landscape of AI governance to see if it exhibits enough policy coordination and political power to effectively deal with mounting AI policy problems. While there are promising initial signs  [28]  there are also already growing governance failures in LAWS, cyberwarfare, and elsewhere. We outline a suggested monitoring method in Table  1 . There are three key areas to monitor: conflict, coordination, and catalyst. First, conflict should measure the extent to which principles, rules, regulations and other outcomes from different bodies in the AI regime complex undermine or contradict each other or are in tension either in their principles or goals. Second, coordination seeks to measure the proactive steps that AI-related regimes take to work with each other. This includes liaison relationships, joint initiatives, as well as the extent to which their rules, outputs and principles tend to reinforce one another. Third, catalyst raises the important question of governance gaps: is the regime complex self-organising to proactively address international AI policy problems? Numerous AI policy problems currently have no clear coverage under international law, including AI-enabled cyber warfare and HLMI. Whether this changes is of vital importance. These areas require investigation through multiple methods. Qualitative surveys of relevant organisations and actors can yield data on expert perceptions of these questions. Surveys can be augmented with quantitative methods, including network analyses of the regime complex relations  [36, p.32] . Natural language processing could be used to examine contradictions and similarities between different regime outputs, e.g., statements, meeting minutes, and more. Monitoring the outcomes of fragmentation can help to determine whether centralisation is needed. One way forward would be to empower the OECD AI Policy Observatory or the UN CEB to regularly review the monitoring outcomes. This could inform a democratic discussion and decision of whether to centralise AI governance further. Our framework and discussion may also be useful for non-state actors. Researchers and leading AI firms can play an important role in sharing technical expertise and informing forecasts of new policy problems on the horizon. The considerations may benefit their decisions of where to engage. Civil society has a key role as participants, watch-dogs, and catalysts. For example, the Campaign to Stop Killer Robots has sought to boost engagement and support for a LAWS ban within the CCW. Given prolonged delays and a pessimistic outlook, some have articulated a strategy of creating an entirely new forum for the ban, inspired by the Ottawa Treaty which outlawed landmines. Our framework can help reveal the potential virtues (allowing for progress while avoiding high-threshold deadlocks) and vices (enabling forum shopping) of such an approach. It could even help inform the structure of a future international institution, such as allowing for a modular, flexible structure with 'critical mass' agreements. One cross-cutting consideration is clear: a fractured regime sees higher participation costs that may threaten to exclude many civil society organisations altogether. The international governance of AI is nascent and fragmented. Centralisation under a well-designed, modular, 'innovation-proof' framework organisation may be a desirable solution. However, such a move must be approached with caution. How to define its scope and mandate is one problem. Ensuring a politically-acceptable and well-designed body is perhaps a more daunting one. It risks cementing in place a fate worse than fragmentation. Monitoring conflict and coordination in the current AI regime complex, and whether governance gaps are filled, is a prudent way of knowing whether the existing structure can suffice. For now we should closely watch the trajectory of both AI technology and its governance initiatives to determine whether centralisation is worth the risk. Table 1 : 1 Regime complex monitoring suggestions Key theme Question
			 A regime is a set of 'implicit or explicit principles, norms, rules and decision-making procedures around which actors' expectations converge in a given area of international relations'[27, p.186]. 2  We define 'AI' as any machine system capable of functioning 'appropriately and with foresight in its environment'[34, p.13]; see too[9, p.5]. 3  'High-level machine intelligence' has been defined as 'unaided machines [that] can accomplish every task better and more cheaply than human workers'[17, p.1].