Pricing Externalities to Balance Public Risks and Benefits of Research

Pricing Externalities to Balance Public Risks
and Beneﬁts of Research
Sebastian Farquhar, Owen Cotton-Barratt, and Andrew Snyder-Beattie
How should scientiﬁc funders evaluate research with public health risks? Some risky work is valuable, but accepting too much
risk may be ethically neglectful. Recent controversy over H5N1 inﬂuenza experiments has highlighted the difﬁculty of this
problem. Advocates of the research claim the work is needed to understand pandemics, while opponents claim that accidents or
misuse could release the very pandemic the work is meant to prevent. In an attempt to resolve the debate, the US governmentsponsored an independent evaluation that successfully prod uced a quantitative estimate of the risks involved, but only a
qualitative estimate of the beneﬁts. Given the difﬁculties of this ‘‘apples-to-oranges’’ risk-beneﬁt analysis, what is the best wayforward? Here we outline a general approach for balancing risks and beneﬁts of research with public risks. Instead of directlycomparing risks and beneﬁts, our approach requires only an estimate of risk, which is then translated into a ﬁnancial price. This
estimate can be obtained either through a centrally commissioned risk assessment or by mandating liability insurance, which
allows private markets to estimate the ﬁnancial burden of risky research. The resulting price can then be included in the cost of theresearch, enabling funders to evaluate grants as usual—comparing the scientiﬁc merits of a project against its full cost to society.This approach has the advantage of aligning incentives by assigning costs to those responsible for risks. It also keeps scientiﬁcfunding decisions in the hands of scientists, while involving the public on questions of values and risk experts on risk evaluation.
In2012, multiple research groups conducted experi-
ments that produced strains of H5N1 avian inﬂuenza that
were airborne transmissible between mammals.1,2Given that
this disease is often fatal in human s, these particular ‘‘gain-of-function’’ experiments sparked wide controversy.* Advocates
of the research argued that the experiments were necessary toimprove our understanding of the virus, thus enabling betterdisease surveillance and vaccine production. Opponents of theresearch argued that a laboratory accident could trigger aglobal pandemic, or that the research could be used by mal-icious actors.
3The debate culminated in a moratorium on US
public funding of research ‘‘that m a yb er e a s o n a b l ya n t i c i p a t e d
to confer attributes to inﬂuenza, MERS, or SARS viruses such
that the virus would have enhanced pathogenicity and/ortransmissibility in mammals vi a the respiratory route.’’
4
Meanwhile, the US government also commissioned an
Sebastian Farquhar, MPhysPhil, is Project Manager; Owen Cotton-Barratt, DPhil, is a Research Fellow; and Andrew Snyder-Beattie,
MS, is Director of Research; all are at the University of Oxford, Future of Humanity Institute, Oxford, England. This article reﬂectsonly the view of the authors. The ERCEA is not responsible for any use that may be made of the information it contains.
ªSebastian Farquhar et al. , 2017; Published by Mary Ann Liebert, Inc. This Open Access article is distributed under the terms of
the Creative Commons License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and re-production in any medium, provided the original work is properly credited.*Gain-of-function experiments encompass a wide range of re-search, and only a small subset of such experiments were subject
to this controversy. In this article, we focus on gain-of-function
research of concern, by which we mean research of the sort af-fected by the funding moratorium, as a case study of the moregeneral problem of making research funding decisions for dual-
use research of concern.Health Security
Volume 15, Number 4, 2017 Mary Ann Liebert, Inc.
DOI: 10.1089/hs.2016.0118
401
independent study from Gryphon Scientiﬁc to weigh the
beneﬁts of the research against the possible biosafety andbiosecurity risks.
5While their research succeeded in producing
a quantitative estimate of bio safety risk, it explained the
beneﬁts of the research only in qualitative terms.
This controversy highlights the difﬁculty of evaluating
research that has the potential to entail signiﬁcant public
risk. Research that is particularly likely to lead to both
positive and negative impacts is especially hard to evaluate.6
These negative consequences might be either the risk of anaccident linked directly to the research itself, or risks ofdeliberate misuse of the resulting technologies. Avoidingresearch that has any theoretical risk at all would meanmissing out on crucial opportunities. But research com-munities that underestimate the potential for a signiﬁcant
accident, or assume that people will use information gained
through research only for good, may be neglecting theirduty to society.
7
The H5N1 ﬂu experiments mark a particularly inter-
esting case study that we use to highlight some problems inevaluating the risks and beneﬁts of dual-use research.
{In
this article, we discuss some of the current difﬁculties ofexisting approaches to evaluating risky research, and we
propose a general framework that avoids many of the
current problems. Within this framework, we outline 2 pos-sible approaches to implementi ng the framework: a market-
led liability insurance approach, and a state-led approach.We conclude by comparing the approaches with each otherand with current policy.
Difﬁculties of Risk-Beneﬁt Analysis for
Scientiﬁc Research
Risk-beneﬁt analysis, the approach used in the Gryphon
Scientiﬁc report, is an analytical tool that has been suc-cessfully applied to a wide range of domains.
8Analysts
outline a range of scenarios with positive and negative
impacts, estimate the probabilities of those scenarios andthe net size of the positive impact, and compute the ex-pected value of the proposal. When attempting to do a risk-beneﬁt analysis of research, determining the scenarios andthe probabilities of each scenario can be difﬁcult. But as itturns out, estimating biosafety risks is far easier than esti-mating the beneﬁts of research. The reason for this is that
one can outline the potential scenarios for biosafety risksfairly well. In order for a laboratory accident to cause aglobal pandemic, a set of events must occur. The proba-bility of each of these steps can then be estimated usinghistorical data on laboratory accidents and models of dis-ease spread. These estimates are highly uncertain, because
of limitations in available data. The data describing exactly
what activities labs engage in and their historical failurerates are incomplete, though some records are kept.
Moreover, assumptions are required to extrapolate past
records into the future. For example, lab safety standards andequipment have improved over the past several decades, butat the same time new labs are being built under the juris-diction of regulators with less of a track record establishing
their biosafety capabilities. Risk estimators must also make
some estimate of the probability distribution of key param-eters that cannot be empirically tested. For example, what isthe probability that a reassortant virus will be sufﬁcientlytransmissible to spread to a wide population? Nevertheless,quantitative estimates are possible, though they lead todisagreement.
9-11Many risks that are in principle very dif-
ﬁcult to estimate for similar reasons are nevertheless quan-
titatively priced by insurers today—such as terrorism,12
cyberattack,13and harm to patients in clinical trials.14
Conversely, estimating the beneﬁts of research is ex-
tremely difﬁcult. Unlike estimating the probability of alaboratory accident scenario, evaluating the beneﬁts of re-search involves uncertainty over what the possible scenarioseven are. Typically, researchers have little informationabout the potential results of research—which is one of the
motivations to do research in the ﬁrst place. Even the most
experienced researchers ﬁnd it difﬁcult to estimate the ex-ante value of new research projects. The work recreatingthe genome of the inﬂuenza virus responsible for the 1918pandemic
15received some criticism for vaguely promising
unspeciﬁed future beneﬁts16but, after some time, has
plausibly created tangible beneﬁts including, perhaps, aid-ing in the management of the 2009 H1N1 pandemic,
according to the authors of the original paper.
17In their
review, Gryphon Scientiﬁc did not even attempt to assessthe beneﬁts of research quantitatively, and it is this part ofthe risk-beneﬁt calculation that typically poses the greatestdifﬁculty.
5,18
Evaluating the risks from malicious activity is more
difﬁcult than estimating the risk of accidents, but easierthan estimating the beneﬁts of research. Ultimately, the
Gryphon Scientiﬁc report included a quantitative assess-
ment of some biosecurity risks (though not the risks posedby the information discovered by the research). Unlike theaccident scenarios, uncertainties around both the intentionsand capabilities of malicious actors mean a wider range ofscenarios is possible. But despite the difﬁculties in quanti-fying such risk, many insurance companies still offer ter-rorism insurance, and models do exist for putting rough
numbers on such risks.
12{In the context of atomic technologies, discussions of dual-use
technology focused on the ability to rapidly convert civilian
technologies for military purposes,8and this trend continued for
much of the 20th century.9Recently, discussions of dual-use
research have broadened to include nonmilitary actors and the
risk of accidents related to research.5,10It is this last, broadest
sense of dual-use research that we focus on in this article, as itcaptures the full spectrum of risks resulting from research activ-
ities. We acknowledge but do not address the fact that the deﬁ-
nition of ‘‘dual-use’’ remains unsettled,
11and that one might
want to consider dual-use research, technology built on that re-search, and artefacts implementing that technology separately.
9BALANCING RISKS AND BENEFITS OF RESEARCH
402 Health Security
When science funders evaluate dual-use research, they
are faced with risks and beneﬁts that are uncertain in dif-ferent ways. Most of the risks are quantitatively estimable,but the parameters are hard to pin down. It is our view that,even when highly uncertain, explicit quantitative risk esti-mates are preferable to implicit ones, because they allowtransparent discussion about disagreements and mitigate
human biases in judgments around small probabilities.
19
The beneﬁts, however, are so uncertain that even specifying
the scenarios is impossible. Fortunately, existing grant-making approaches are already designed to compare proj-ects with nebulous beneﬁts. So, as long as the quantiﬁedrisks are accounted for in the cost of a project, these existingsystems are well suited for evaluating this research.
Existing Approaches and the Scientiﬁc
Grant-Making Process
Grant makers already assess the uncertain beneﬁts of re-
search against their costs and the opportunity costs of un-funded research. Although it is hard to judge quantitatively,reviewers assess the potential for scientiﬁc excellence in
different proposals.
However, the costs that reviewers consider are primarily
direct ﬁnancial costs, rather than externalities such as therisks borne by the public from dual-use research. In prin-ciple, research funders prefer to fund portfolios of researchwith maximal beneﬁts of research for society (includingpositive externalities) and minimal costs (including nega-tive externalities). The concept of scientiﬁc merit already
includes some sense of positive externalities, and the UK, at
least, also considers the social impact of research and itseffect on the research ecosystem.
20Conceptual tools and
intuitions on risk externalities are less well developed.
Reviewers often implicitly evaluate dual-use risk as a matter
of personal or professional ethics21or for the sake of reputa-
tion of the ﬁeld. However, for most research, the dual-usestakes are low. In these situations, researchers have no need to
also become experts in risk assessment and management.
Research institutions rely instead on safety regulations forparticular categories of risk in order to make sure researchprocesses are safe.
22These tools work very well when most of
the risk is local, for accidental releases at the community level,or for health risks to lab researchers themselves. Historically,this is where most of the risk in biological research has lain andis therefore an entirely appropriate default.
However, in the case of risks that carry a small chance of
large catastrophe (eg, research that creates potential pandemicpathogens), these regulations are not enough. Some funders,such as the US government, provide additional decision-process guidelines for funding research involving speciﬁcpathogens, like H5N1,
23as well as institutional guidelines for
best practice for dual-use research.24Others, such as the UK’s
Biotechnology and Biological Sciences Research Council, the
Medical Research Council, and the Wellcome Trust, requiregrant applicants to consider near- and medium-term risks of
misuse but rely heavily on self-governance and a responsibleresearch culture.
25These frameworks put a high burden on
those pursuing dual-use research to become risk assessors.Moreover, public funding guidelines only affect governmentfunding and cooperative private funders. Private dual-useresearch is less constrained.
26
Some additional judgment can in principle be applied at
the stage of publication,27but a 2012 survey of 127 life
sciences journals found that none reported refusing a sub-mission on biosecurity grounds.
28Export controls may also
be able to restrict some kinds of publication29in some
jurisdictions. It may be that under the Australia Group’s‘‘no undercut’’ principle, in its guidelines for transfers ofsensitive chemical or biological weapons, if any participant
nation were to block publication using export controls then
all would be obliged to (though the mechanisms for ap-plying the guidelines to research results is unclear).
30
Pricing and Using Externalities
in Grant Making
Absolute Risk as a Cost of a Grant
Hard-and-fast rules about what research techniques are too
risky, without considering the beneﬁts, may not be re-sponsive enough, because more important research meritsmore risk.
31
Our principal proposal is that one could assess and price
absolute expected risk and explicitly include this cost in
grant proposals. This helps resolve the fact that while riskscan be comparatively easy to assess, the beneﬁts of researchare extremely challenging to quantify. Our proposal allowsgrant reviewers to use existing frameworks to evaluate grantproposals but weigh potential beneﬁts against a fairerreﬂection of their social costs. It also follows calls tostrengthen the evaluation of risks at pre-funding stages of
the research process.
32
For now, we set aside the issues of where an independent
estimate of the cost of externalities comes from and wherethe money to cover this explicit cost is paid. The 2 mainapproaches to setting estimates, which we consider in detailin the next section, are:
1. To establish clear liability in case of catastrophe and
require grant holders to purchase liability insurance as
part of the grant. The strength of this is that it is amarket-led approach, with insurers incentivized to priceexternalities correctly.
2. To centrally commission absolute risk assessments to
price the externalities, and to require a payment to astate or non-state body to cover the expected cost. Thestrength of this approach is that it works even if there
may be no clear liability after the fact, so it could address
biosecurity as well as biosafety risks.FARQUHAR ET AL
Volume 15, Number 4, 2017 403
Beneﬁts of the Approaches
Both approaches have 4 main advantages. First, they give
decision makers greater incentive to use an accurately
priced risk when deciding what research to fund. Currently,the costs of accidental or deliberate harm from gain-of-function research of concern are borne by members of thepublic in any country as well as lab workers. For example,after a foot-and-mouth disease outbreak in Pirbright in2007, caused by poorly maintained drainage,
33prosecutors
were unable to seek damages because there was no de-
monstrable negligence of a speciﬁc party.34Instead, gov-
ernments bore the ﬁnancial cost of compulsory slaughter,farmers bore the costs of damaged and restricted business,and consumers experienced higher food prices. Where therisks of an activity are not borne by those making decisionsabout the activity, economic theory predicts that more riskwill be taken than is socially optimal.
35So long as the risks
are priced correctly and the transaction costs are sufﬁciently
low, research institutions that pay upfront for the risks of
research are more likely to engage in socially optimal risk-taking. If implemented successfully, this would result inboth better grant choices and better use of risk reducingtechniques and equipment.
Second, these approaches keep decisions about what re-
search gets funded in the hands of researchers. One of thelargest difﬁculties in assessing the social value of research is
assessing the intellectual beneﬁts of research. Researchers
themselves are best placed to evaluate the potential forsuccess and the intellectual merit of research because theyhave the deepest knowledge of the area.
Third, these approaches incorporate democratic or public
input on the signiﬁcance of risks. Although researchers haveexpertise in the potential for intellectual success from a grantproposal, they are not necessarily able to speak for the values of
society. Researchers have an obl igation to incorporate public
input on value systems and consider the justice implications oftheir work.
31Our proposal creates ways for the public to express
their values without creating rules allowing or disallowingcertain research techniques or areas of inquiry. In the case of aninsurance mechanism, the legisla tive branch effectively deter-
mines the social cost borne were the risk to materialize, via thejudiciary. In the case of a central body setting the costs, this
body would ultimately be accountable to the electorate.
Fourth, these approaches ensure appropriate risk assess-
ment and governance expertise is brought to bear. Re-searchers submitting grant proposals are often asked toconsider potential dual-use and accident risk. Grant makersdo so similarly. However, neither group tends to includespecialists in risk assessment or biosecurity.
26If absolute
risk is priced separately, the price can be set by expert risk
assessors.
Throughout, our guiding principle is to ensure the right
incentives are in place and that components of the fundingdecision are broken up such that those best placed to assessthem do so.Disadvantages of the Approaches
Although we believe these approaches might improve the
capacity of the research funding system to manage cata-
strophic risk, they introduce some new concerns.
First, research institutions currently receive a subsidy in
the form of an implied guarantee covering the costs ofimposed risks. If they were asked to pay for this guarantee,it would represent an additional unfunded expense. Sincethe aim of this proposal is to distribute research effortwithin, say, the life sciences rather than to change the ab-
solute amount of life science work being done, one might
increase funding for life sciences research to compensate.This would appear to increase explicit expenditure ingovernment research budgets, but in fact moves an implicit,random, and large subsidy from the budgets of disasterresponse agencies, who do not make decisions that deter-mine the risk exposure, to an explicit, predictable, andmoderate payment by those who do make such decisions.
For example, governments and consumers bear a large part
of the cost of escapes of animal pathogens from researchlabs
36but have less inﬂuence over risk exposure than re-
search funding bodies.
Second, penalizing laboratories for reporting accidents
and near misses in a timely way harms biosafety and bio-security in the long run. Increasing reporting would make iteasier to use lessons from mistakes to improve lab design
and improve accident response. Mechanisms for pricing
risk will work best if they avoid creating perverse incentivesaround reporting. Options for facilitating this includesimply not using the information from recent accidents inthe case of central analysis, or forbidding a premium hike inthe case of liability insurance requirements.
Third, these approaches increase bureaucratic overhead.
There are several things that might be done to reduce this
overhead. One could apply our proposal only to areas with
particularly high risk of low-probability but high-impactevents. For example, it might be applied only to the gain-of-function research that has been covered by the recent USmoratorium. Or rather than carrying out a separate riskanalysis for each proposal, one could establish broad cate-gories of similar work and set costs for that category as awhole. For some categories, it might be possible to get
involvement from intelligence communities to incorporate
information not normally available. In the future, scopecould be expanded by addressing new areas or by makingthe cost estimates more granular.
Making Decisions Under Uncertainty
There are things these approaches would not ﬁx. Risk in
many cases of dual-use research is very uncertain. No risk
management or risk governance process can solve this en-tirely. It is hoped that in the case of market-led approaches,the proﬁt motive of companies providing insurance as well asan awareness of ‘‘winners’ curse’’ dynamics will incentivizeBALANCING RISKS AND BENEFITS OF RESEARCH
404 Health Security
appropriate pricing. In the case of a state-led approach, it is
hoped that the system will increase the capacity for risk-assessment by developing a core of expert assessors. Politicalpressures or ‘‘short-termism’’ might cause prices to be inac-curate. However, this is also a feature of existing approachesto dual-use risk management.
Approaches to Risk Pricing
Mandatory Liability Insurance
Laboratories conducting experiments of appropriate kindscould be mandated to purchase insurance against liability
claims arising from damaging accidents from their research.
Insurance companies would then pay out to injured partiesin the public in case of an accident. Some proposals in thisdirection have already been discussed in the UK.
27In some
cases, appropriate use of liability for damages is moreeffective than regulation in encouraging risk-reducingbehavior.
37In most jurisdictions, negligence leading to
catastrophic damage would already establish liability.
Sometimes, the complexity of the research process makes it
hard to establish negligence of a speciﬁc party.34Govern-
ments might therefore need to legislate to establish clearduty of liability in any damages caused by accident, even ifthere was no negligence. Liability would give universitiesstrong incentives to minimize the risk. Framers of thislegislation would need to be quite speciﬁc about which sortsof liability were included (eg, health damage) and which
were not (eg, adversely affected tourism, perhaps, or ter-
rorism). Such an approach might be similar to some aspectsof the 1957 Price-Anderson Act, which placed a strict lia-bility on operators of potentially risky private nuclear fa-cilities and a mandate to purchase the highest commerciallyavailable amount of insurance coverage.
38A similar ap-
proach has been tried and advocated for in environmentalrisk, though it proved hard to maintain a healthy insurance
market.
39,40
Requiring insurance is important because some risk
takers will not be able to compensate victims for cata-strophic harms. Many universities currently self-insureagainst the damage of accidents in their research. Wherethere is a small chance of catastrophic damage, they donot have enough assets to cover the damages—they are‘‘judgment-proof.’’ Insurance providers, with much greater
ﬁnancial liquidity, are able to bear the full costs even when
payouts are very large, which incentivizes them to set pre-miums appropriately.
41For the most catastrophic forms of
damage, such as a global pandemic, costs would be too bigeven for a reinsurer to bear. As a result, an insurance ap-proach could involve an explicitly negotiated liability cap,in which case regulatory bodies tasked with managing themost extreme forms of risk are essential.
Various amendments to the Price-Anderson Act dem-
onstrate the problems facing liability limits. Indemnifyingthose creating the risk from all costs above the privately
insurable can cause excess risk taking.
42The most recent
version of Price-Anderson requires regulated nuclear energyproviders to pool to jointly cover more than $10 billion ofcosts beyond that covered for each individual party by in-surers. While this protects the public by identifying a sourceof funds beforehand, the fact that the individual contri-
butions do not need to be set aside in advance means that
the entire industry carries a large, uninsured, systemic risk.A modern system based on catastrophe bonds might offer asafer way to increase the feasible levels of ﬁnancing.
Advantages of the Liability Approach
There are 2 main advantages to this market-led approach.First, it is a relatively light intervention, requiring littleongoing work from the state (other than maintaining a
healthy insurance market). Second, the insurers would have
a proﬁt incentive to accurately estimate the risk, reducingpossible politicization of the risk assessment process. In asufﬁciently liquid market, competitive pressures shouldbring premiums down. But insurers are aware of a ‘‘win-ners’ curse’’ in the insurance market and would avoid set-ting premiums too low to cover the long-term risk. Ifinsurers adjust their premiums when safer systems and
equipment are adopted, scientists and engineers would be
incentivized to devise effective safety protocols to reducetheir institutions’ insurance costs. This would create a ﬁ-nancial driver for applied biosafety and biosecurity re-search. Imposing liability has improved outcomes in otherdomains such as occupational safety, medicine, and generalrisk management in nonproﬁts and government agencies.
43
Possible Issues with the Liability
Approach
Insurers must be willing to insure against biosafety and
biosecurity risk. While insurers have shown some interest,44
there is no functioning market for the same 2 main reasonsinsurers were unwilling to take on nuclear risk before the1957 Price-Anderson Act:
38
First, the potential risks are simply too large. A cata-
strophic global pandemic could kill hundreds of millions ofpeople, and even the largest reinsurers would be unable toabsorb this cost without bankrupting themselves (costs abovethis level will be implicitly backed by the state or the public
in any case). It is better to be explicit and cap liability at a
speciﬁc industry-wide ﬁgure. If the cap were sufﬁcientlylarge, the effect should be more appropriate risk aversion,even if the tail risk for the insurer were not fully internalized.
Second, the risks are hard to model and the market
would be quite small. Developing models to estimate therisk could be more costly than the expected proﬁt frombeing in the market. However, insurers do already have
models for hard-to-anticipate risks such as terrorism andFARQUHAR ET AL
Volume 15, Number 4, 2017 405
global pandemics. If need be, the development of appro-
priate models to facilitate this insurance could be explicitlysubsidized.
Whether or not insurers are willing to take on the risk,
there are challenges to international adoption. Liability lawsdiffer across jurisdictions, which would particularly affectinternational collaborations. One effective work-around
might be to form an agreement among top-tier research
journals to require proof of suitable liability insurance forsome types of papers.
{
Liability insurance can also increase moral hazard, by
making actors less responsible for the consequences of theiractions. This effect could be reduced if the excess on theinsurance were large enough that research institutions stillhad a direct incentive to avoid risks.
Finally, using a liability approach to capture biosecurity
risks would be difﬁcult. If a terrorist group used informa-tion they gleaned from a paper published by a researchgroup, establishing liability would require lengthy legaldispute. Moreover, done carelessly, the uncertainty addi-tional but unclear liability could create might have an in-appropriate chilling effect on research.
Centrally Commissioned Risk
Assessments
The second approach is to centralize risk assessments.
When an area of potential concern is identiﬁed, a body
commissioned by the state would perform an analysis of the
risks involved. This might be similar to the recent GryphonScientiﬁc analysis, except that it would not attempt to an-alyze the beneﬁts. This absolute risk analysis would presentits outcomes in monetary terms, using value of statisticallife ﬁgures to convert into a cost.
In order to do work of a priced type, laboratories would
have to pay the appropriate charge to a central authority.
This charge could cover part of the risk of biosafety and
biosecurity preparation as well as the cost of risk assessment.
Advantages of the Centrally
Commissioned Risk Assessment
Compared to the market-led approach, centralizing the risk
assessments has 2 main beneﬁts. First, it can be done by ﬁatwithout needing to persuade insurers to enter and stay inthe market. Second, it works well even in cases where therewill be no clear causal chain establishing liability, such as forbiosecurity risks (including risks from the misuse of in-
formation produced from such research). Funds are col-
lected based on the central assessment of the size of the riskposed by the research, with no need to work out after thefact which speciﬁc group is liable. The Gryphon Scientiﬁcreport concluded that the biosecurity risks looked at least as
large as the biosafety risks, so this is a signiﬁcant beneﬁt inthe case of gain-of-function research of concern.
5Third, at
least in principle, avoiding the need for courts to establishliability might speed up payments.
Possible Issues with the Central
Approach
Patchy implementation only in certain jurisdictions might
just move research from nations with good risk manage-ment processes to those without. In practice, however,many researchers are likely to want to remain at top labs,
limiting the risk of research simply moving to jurisdictions
that do not assess the externalities of research. There may bea need for special arrangements for research done in part-nership with labs outside the collecting jurisdiction, such asapplying a requirement for proof of insurance at the stage ofpublishing.
The risk assessment itself would be difﬁcult. The centrally
commissioned assessment would be more likely than an
insurance-led one to need to assess the magnitude of biose-
curity risks, which is particularly hard to model. However,stakeholders might have incentives to put pressure on regu-lators to make certain decisions. This is more likely to besuccessful because of the amount of judgment involved in thecalculations of risk. Unlike the insurance-led solution, therewould be no countervailing proﬁt motive.
Comparisons and Discussion
The 2 approaches discussed would work by aligning theincentives for scientists and for funding bodies more closelywith those of society as a whole. This approach keeps as-sessment of the beneﬁts of scientiﬁc research purely in thehands of scientists, while bringing the right skills to bear atthe right points and managing risk proportionately. We
have explored 2 different ways to achieve this. Each has its
advantages and disadvantages.
The liability approach is more market based. As a result,
the risk assessors have a ﬁnancial incentive to accuratelyestimate risk, and political pressures are diminished. Itmight also be easier to use as a template internationally.Since the risks are global and the potentially risky research isnot being pursued in just 1 country, being able to build
global solutions is extremely valuable.
The main beneﬁt of the centrally commissioned analysis
is it could bypass lack of clarity about who is liable foreventual damages in cases of biosecurity.
These approaches have clear application for gain-of-
function research of concern in the life sciences. However,similar approaches may work in a broader range of ﬁelds.The insurance mechanism is best suited for dual-use work
where risk is predominantly through accident rather than
{We are grateful to Piers Millett for this suggestion.BALANCING RISKS AND BENEFITS OF RESEARCH
406 Health Security
malicious intent. In all cases, pricing the absolute size of
the risk and leaving researchers in charge of decisionsabout how to spend their budgets might offer a moreresponsive and proportionate approach to the challengesof funding research with potential public risks than ex-isting systems.
Acknowledgments
The authors would especially like to thank, for their com-ments and substantive suggestions on drafts, Carrick Flynn,Marc Lipsitch, and Piers Millett. We would additionally
like to thank Anthony Aguirre, Haydn Belﬁeld, Leah
Broad, Kathryn Mecrow, and Stefan Schubert for theircomments on drafts. This project has received fundingfrom the European Research Council (ERC) under theEuropean Union’s Horizon 2020 research and innovationprogramme (grant agreement No 669751).
References
1. Herfst S, Schrauwen EJ, Linster M, et al. Airborne trans-
mission of inﬂuenza A/H5N1 virus between ferrets. Science
2012;336(6088):1534-1541.
2. Imai M, Watanabe T, Hatta M, et al. Experimental adap-
tation of an inﬂuenza H5 HA confers respiratory droplettransmission to a reassortant H5 HA/H1N1 virus in ferrets.
Nature 2012;486(7403):420-428.
3. Duprex WP, Fouchier RA, Imperiale MJ, Lipsitch M, Rel-
man DA. Gain-of-function experiments: time for a real de-bate. Nat Rev Microbiol 2014;13(1):58-64.
4. U.S. government gain-of-function deliberative process and
research funding pause on selected gain-of-function researchinvolving inﬂuenza, MERS, and SARS viruses. 2014. https://
www.phe.gov/s3/d ualuse/Documents/gai n-of-function.pdf.
Accessed July 1, 2017.
5.Risk and Beneﬁt Analysis of Gain of Function Research. Ta-
koma Park, MD: Gryphon Scientiﬁc; 2016. http://www.
gryphonscientiﬁc.com/wp-co ntent/uploads/2016/04/Risk-
and-Beneﬁt-Analysis-of-Gain-o f-Function-Research-Final-
Report.pdf. Accessed July 1, 2017.
6. National Research Council; Policy and Global Affairs; De-
velopment, Security, and Cooperation; Committee on Re-search Standards and Practices to Prevent the Destructive
Application of Biotechnology. Biotechnology Research in an
Age of Terrorism: Confronting the Dual-Use Dilemma. Wash-
ington, DC: National Academies Press; 2004.
7. Kuhlau F, Eriksson S, Evers K, Ho ¨glund AT. Taking due
care: moral obligations in dual use research. Bioethics 2008;
22(9):477-487.
8. Wilson R, Crouch EAC. Risk/Beneﬁt Analysis. 2d ed. Cam-
bridge, MA: Harvard University Press; 2001.
9. Casadevall A, Imperiale MJ. Risks and beneﬁts of gain-of-
function experiments with pathogens of pandemic potential,
such as inﬂuenza virus: a call for a science-based discussion.
MBio 2014;5(4):3-7.10. Lipsitch M, Inglesby T V. Moratorium on research intended to
create novel potential pandemic pathogens. MBio 2014;5(6):1-7.
11. Fouchier RA. Studies on inﬂuenza virus transmission be-
tween ferrets: the public health risks revisited. MBio 2015;
6(1):14-17.
12. Woo G. Quantitative terrorism risk assessment. J Risk Financ
2002;4(1):7-14.
13. Garrie D. Cyber-security insurance: navigating the landscape
of a growing ﬁeld. John Marshall J Inf Technol Priv Law
2014;31(3):379-391.
14. Sagitova BK. Watching clinical trials liability. Allianz Global
Corporate & Specialty. 2009. http://www.agcs.allianz.com/
insights/expert-risk-articles/watching-clinical-trials-liability/.
Accessed July 1, 2017.
15. Taubenberger J, Reid A, Lourens R, Wang R, Jin G, Fan-
ning T. Characterization of the 1918 inﬂuenza virus poly-
merase genes. Nature 2005;437(7060):889-893.
16. van Aken J. When risk outweighs beneﬁt. EMBO Rep
2006;7(S):10-13.
17. Taubenberger JK, Baltimore D, Doherty PC, et al. Re-
construction of the 1918 inﬂuenza virus: unexpected rewardsfrom the past. MBio 2012;3(5):1-5.
18.Risk and Beneﬁt Analysis of Gain of Function Research. Draft
Final Report. Takoma Park, MD: Gryphon Scientiﬁc; De-cember 2015. http:// www.gryphonscientiﬁ c.com/wp-content/
uploads/2015/12/Final-Gain-of-Fu nction-Risk-Beneﬁt-Analysis-
Report-12.14.2015.pdf. Accessed July 1, 2017.
19. Tversky A, Kahneman D. Judgment under uncertainty:
heuristics and biases. Science 1974;185(4157):1124-1131.
20. Research Excellence Framework 2014. Assessment frame-
work and guidance on submissions. 2011.
21. Evans NG, Lipsitch M, Levinson M. The ethics of biosafety
considerations in gain-of-function research resulting in the
creation of potential pandemic pathogens. J Med Ethics
2015;41(11):901-908.
22. Wilson DE, Chosewood LC. Biosafety in Microbiological and
Biomedical Laboratories. 5th ed. Washington, DC: US De-
partment of Health and Human Services; 2009. https://
www.cdc.gov/biosafety/publications/bmbl5/bmbl.pdf. Ac-
cessed July 1, 2017.
23. US Department of Health and Human Services. A frame-
work for guiding US DHHS funding decisions about re-
search proposals with the potential for generating highly
pathogenic avian inﬂuenza H5N1 viruses that are transmis-sible among mammals by respiratory droplets. 2013. https://
www.phe.gov/s3/dualuse/Doc uments/funding -hpai-h5n1.
pdf. Accessed July 1, 2017.
24. United States Government Policy for Institutional Oversight
of Life Sciences Dual Use Research of Concern. 2014.
https://www.phe.gov/s3/dualuse/Documents/oversight-durc.pdf. Accessed July 1, 2017.
25. Biotechnology and Biological Sciences Research Council,
M e d i c a lR e s e a r c hC o u n c i l ,W e l l c o m eT r u s t .B B S R C ,M R Cand Wellcome Trust position st atement on dual use research of
concern and research misuse. 2015. https://wellcome.ac.uk/
sites/default/ﬁles/wtp 059491.pdf. Accessed July 1, 2017.
26. Palmer MJ, Fukuyama F, Relman DA. Science governance: a
more systematic approach to biological risk. Science 2015;
350(6267):1471-1473.
27. Smith M, Revill J. Dual-use biology: how to balance open
science with security. Wilton Park Conference Report.FARQUHAR ET AL
Volume 15, Number 4, 2017 407
WP1260. 2013. https://www.wiltonpark.org.uk/wp-content/
uploads/WP1260-Report.pdf. Accessed July 3, 2017.
28. Patrone D, Resnik D, Chin L. Biosecurity and the review
and publication of dual-use research of concern. Biosecur
Bioterror 2012;10(3):290-298.
29. Council of the European Union. Council Regulation No 428/
2009 of 5 May 2009 setting up a community regime for the
control of exports, transfer, brokering and transit of dual-useitems. 2009. Ofﬁcial Journal of the European Union. http://
eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2009:
134:0001:0269:en:PDF. Accessed July 3, 2017.
30. The Australia Group. Guidelines for transfers of sensitive
chemical or biological items. Australia Group website. June
2015. http://www.australiagroup.net/en/guidelines.html. Ac-cessed July 3, 2017.
31. Selgelid MJ. Gain-of-function research: ethical analysis. Sci
Eng Ethics 2016;22(4):923-964.
32. Lipsitch M, Plotkin JB, Simonsen L, Bloom B. Evolution,
safety, and highly pathog enic inﬂuenza viruses. Science 2012;
336(6088):1529-1531.
33. Anderson I. Foot and Mouth Disease 2007: A Review and
Lessons Learned. London: Stationery Ofﬁce; 2008. https://
www.gov.uk/government/uploads /system/uploads/attachment_
data/ﬁle/2503 63/0312.pdf. Accessed July 3, 2017.
34. No prosecution over Pirbright foot and mouth disease. Times
May 29, 2008. http://www.thetimes.co.uk/tto/news/uk/
article1920260.ece. Accessed July 3, 2017.
35. Arrow KJ. Political and economic evaluation of social effects
and externalities. In: Margolis J, ed. Analysis of Public Out-
put.Cambridge, MA: NBER; 1970.
36. Pendell DL, Marsh TL, Coble KH, Lusk JL, Szmania SC.
Economic assessment of FMDv releases from the NationalBio and Agro Defense Facility. PLoS One 2015;10(6):
e0129134.
37. Shavell S. Liability for harm versus regulation of safety.
J Legal Stud 1984;13:357-374.
38. Berkovitz DM. Price-Anderson Act: model compensation
legislation? The sixty-three million dollar question. Harvard
Environ Law Rev 1989;13(1).
39. Richardson BJ. Mandating environmental liability insurance.
Duke Environ Law Policy Forum 2001;12:293.
40. Katzman MT. Pollution liability insurance and catastrophic
environmental risk. J Risk Insur 1988;55(1):75-100.
41. Jost PJ. Limited liability and the requirement to purchase
insurance. Int Rev Law Econ 1996;16(2):259-276.
42. Rosenthal M. How the Price-Anderson Act failed the nuclear
industry. Columbia J Environ Law 1990;15(121).
43. Schwartz GT. Reality in the economic analysis of tort law: does
tort law really deter? UCLA Law Rev 1994;(42):377-444.
44. Maynard T, Baxter D. Synthetic biology: inﬂuencing de-
velopment. Lloyds Bank Emerging Risk Team. 2009.
Manuscript received December 16, 2016;
revision returned February 9, 2017;
accepted for publication February 27, 2017.
Address correspondence to:
Sebastian Farquhar
University of Oxford
Balliol College
Broad Street
Oxford, England OX1 3BJ
Email: sebastian.farquhar@balliol.ox.ac.ukBALANCING RISKS AND BENEFITS OF RESEARCH
408 Health Security

Sebastian Farquhar
Owen Cotton-Barratt
Andrew Snyder-Beattie