Moral Trade

Moral Trade*
Toby Ord
If people have different resources, tastes, or needs, they may be able to exchange
goods or services such that they each feel they have been made better off. This is
trade. If people have different moral views, then there is another type of trade
that is possible: they can exchange goods or services such that both parties feel
that the world is a better place or that their moral obligations are better satisﬁed.
We can call this moral trade . I introduce the idea of moral trade and explore sev-
eral important theoretical and practical implications.
INTRODUCTION
Our world contains a great diversity of moral views. There are utilitarians,
Kantians, and virtue ethicists. There are Hindus, Buddhists, Christians,
Muslims, and Jews. There are some who believe there is an unalienable
moral right to have an abortion, others who hold it to be the moral
equivalent of murder; some who believe eating meat is the moral equiva-
lent of murder, others who hold it to be completely innocuous; some
who believe there is a moral obligation to share society’s wealth, others who
hold that each person has a right to keep all the fruits of their labor.
People with different moral views often ﬁnd themselves in antago-
nistic relationships. They might ﬁnd cause for a local alliance if their
views agree on a salient issue, such as a utilitarian and a Hindu joining
forces to promote vegetarianism even though they support this practice
* I am very grateful for the comments given by participants of three seminars in which
an early version of this work was presented: the New England Consequentialism Workshop,
the Oxford-Copenhagen Ethics Symposium, and the Uehiro Work in Progress Seminar. I
am especially grateful for the detailed comments of Caspar Hare, Dan Wikler, Roger Crisp,
Karsten Jensen, Frej Thomsen, Owen Schaefer, Owen Cotton-Barratt, Tom Douglas, An-
drew Snyder-Beattie, Dominic Wilkinson, and Daniel Dewey. This work was made possible
by a grant from the Oxford Martin School at the University of Oxford.
118Ethics 126 (October 2015): 118 –138
© 2015 by The University of Chicago. All rights reserved. 0014-1704/2015/12601-0006$10.00
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
for very different reasons.1However, if their views differ on a salient
issue, then the other side is often regarded as the enemy and battle lines
are drawn.
In this essay, I want to show a different way forward. I want to show
how people with conﬂicting moral views can cooperate, creating an out-
come that is better according to each of their moral theories. Consider
the following vignette:
Animals and the Poor: Victoria and Paul are close friends. Victoria is a
vegetarian because she is aware of how much animals suffer in the
modern meat industry and wants to reduce this suffering as much
as possible. Paul is deeply concerned about global poverty and do-
nates 10 percent of his income to organizations that can effectively
improve people’s lives in poor countries. He enjoys many vegetar-
ian dishes, but still eats meat most of the time since he ﬁnds it a bit
more convenient, and, as a contractualist, he doesn’t think that ani-
mals matter from a moral point of view. Victoria can see that Paul is
prepared to make sacriﬁces to help others, but it frustrates her that
he isn’t prepared to be a vegetarian —especially since it would only be
a small sacriﬁce for him. He has similar views about her lack of
serious ﬁnancial support for the world’s poorest people. One day
they decide to make an arrangement. She will start to donate 1 per-
cent of her income to ﬁght global poverty if he agrees to become a
vegetarian. Either of them can end the deal at any time, releasing
them both from any commitment. They could check up on the other
person, but as they trust each other they decide it is sufﬁcient to ask
each other how they are keeping up once each year.
Both Victoria and Paul see this agreement as a good deal. Paul doesn’t
think that animals matter morally and wouldn’t be prepared to forgo
eating meat just to help them. However, with this arrangement his vege-
tarianism causes several hundred dollars per year to go to ﬁghting poverty,
which he thinks is well worth the small sacriﬁce. Victoria values poverty
reduction in other countries and used to donate about $100 per year to
poverty-related charities. But she cares much less about it than she cares
about animal welfare and wouldn’t be prepared to give 1 percent of her
income each year just for this cause. Through their arrangement, however,
she is thrilled at being able to roughly double her impact on preventing
animal suffering at only a modest ﬁnancial cost.
Victoria and Paul have just engaged in what I shall call moral trade .
Different self-regarding tastes allow for gains from trade. For example, if
1. John Rawls calls this sort of instrumental agreement a “modus vivendi,” which he
contrasts with an “overlapping consensus,” where the different parties agree to collaborate
“for the right reasons.” See John Rawls, “The Idea of an Overlapping Consensus,” Oxford
Journal of Legal Studies 7ð1987 Þ:1–25.Ord Moral Trade 119
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
two children would prefer to eat each other’s packed lunches, they can
trade them and both beneﬁt. Similarly, if people have different moral
views, they can potentially realize gains from moral trade, making them
both think that their moral view is better satisﬁed if they make a trade
than if they don’t. For example, consequentialists may think that the
trade produces a better outcome, while deontologists may think that it
better fulﬁls their duties or that it is a supererogatory option. Not all
moral theories will be able to gain from moral trade, but I shall argue later
that many can.
For the moment, I’ll set aside questions about whether a moral trade
could be bad from the perspective of people who weren’t involved in the
trade and whether it could be bad from the perspective of the correct, or
best justiﬁed, moral theory ðif any Þ. These issues, and how one might deal
with them, will be discussed in the ﬁnal section. Until then, we shall just
consider the gains from moral trade from the perspective of each of the
moral views that the trading parties endorse.
In the above example, Victoria and Paul had fairly compatible moral
views. Paul valued poverty reduction and was neutral about animal suf-
fering. Victoria valued both, but valued reducing animal suffering sub-
stantially more. Moral trade is even possible on an issue when the two
parties have diametrically opposed views, so long as there is some area
where they are not diametrically opposed. Consider this second vignette:
Gun Rights and Gun Control: Rebecca and Christopher are coworkers
at a technology start-up. Rebecca is a staunch defender of the right
to bear arms and ﬁercely opposes gun control. Christopher doesn’t
believe that there is such a right and is deeply concerned by the
harm that guns can do to society. A new bill on gun control is being
debated in Congress and has caused many heated arguments at the
water cooler. In one of these conversations, it is revealed that Re-
becca is planning to donate $1,000 to a gun rights charity to fund
their ﬁght against this bill, while Christopher is planning to donate
$1,000 to a gun control charity which is supporting the bill. An on-
looker suggests that their combined behavior is pointless: they are
just ﬁghting a zero-sum battle. She points out that it would clearly be
better if they both donated the money to Oxfam instead —a charity
that Rebecca and Christopher both support. While Rebecca and
Christopher would each rather their own $1,000 went to ﬁghting the
battle for the gun control bill, they accept that the onlooker is right.
They each think that Oxfam is a good charity, and would prefer that
$2,000 is donated to this organization than that $1,000 is given to
help the bill and $1,000 to hinder it. So they decide to make the deal.
Situations like this can be quite common. As well as gun rights and gun
control, there are other pairs of almost diametrically opposed charities,120 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
such as pro-life and pro-choice, or the Republican and Democratic polit-
ical parties. Donations to each of these pairs range from hundreds of mil-
lions of dollars per annum to billions of dollars per annum, so there are
signiﬁcant gains to be made.
Though the numbers in the vignette were carefully balanced to make
the situation clearer, the ability to trade when supporting opposing causes
is quite robust. To begin with, it works even if the opposing charities aren’t
equally efﬁcient at using donations to achieve their aims. For example, it
would work if the gun control charity were twice as effective but Christo-
pher was only planning to give half as much money as Rebecca. It would
also work if they were planning to help by very different amounts. For
example if the charities were equally effective, but Christopher was plan-
ning to donate $10,000, then they could just swap off $1,000 of the do-
nations, leaving Christopher donating $9,000 to the gun control charity. It
could even work if they had different empirical beliefs about the effec-
tiveness of the charities. For example, the trade would still make sense if
they each believed that their own supported charity was somewhat more
effective than the other —so long as they thought that this gap was sufﬁ-
ciently small compared to the value of donating to their compromise
charity.2
In reality, apparently opposed charities will produce some additional
beneﬁts or costs that both parties agree on, such as a more educated pub-
lic, making the existing regulation more nuanced, or creating a more
partisan political culture. However, even if these other effects are net pos-
itive, the perceived value of them is likely to be only a small fraction of
the perceived value of the donation and in many cases there will be a
compromise charity that is judged by both to outweigh these side beneﬁts
of the opposed charities.
VARIETIES OF MORAL TRADE
These examples have focused on a direct clash between two different
moral views, but similar examples arise when a moral view is contrasted
with prudence. For example, Victoria —the vegetarian in the ﬁrst vignette —
could ﬁnd people and simply pay them to become vegetarians. For some
people who are already on the fence abo ut vegetarianism, or who wouldn’t
2. For example, suppose they each thought their campaigners were twice as effective as
those of the other side ðso that $1,000 going to both charities was worth the same as $500 to
their charity Þ. Then they are comparing something equivalent to $500 to their charity with
the option of $2,000 going to Oxfam. If they think that money going Oxfam is more than a
quarter as good as money going to their lobbyists, they should agree to trade. One reason
people might systematically estimate the effectiveness of their campaign to be higher than
their opponents think, is if they think that their arguments are stronger and hence more
likely to gain traction. ðI would like to thank an anonymous referee for this point. ÞOrd Moral Trade 121
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
ﬁnd a vegetarian diet particularly di fﬁcult, there would be a sum of money
that Victoria could offer such that t hey would be prudentially better off
making the trade and Victoria would consider the improvement to the world
to be worth the ﬁnancial sacriﬁce.3
There can also be cases where each party’s gains from trade involve
self-regarding and moral components. Indeed the ﬁrst vignette with Vic-
toria and Paul was already like this, since they were both making internal
trade-offs between prudence and morality. The moral trade allowed them
a new way to gain a lot of what they thought was good ðanimal welfare,
poverty reduction Þfor a relatively small prudential cost ðdonating 1 per-
cent, giving up meat Þ, leading to an outcome that they each judged to be
better, all things considered.
We could summarize these cases by what each person sees as the
change in the moral value and prudential value from making the trade,
compared to the status quo. For Victoria and Paul, it is moral gain plus
small prudential cost versus moral gain plus small prudential cost. For
Victoria just paying someone to become a vegetarian, it is moral gain
plus small prudential cost versus pure prudential gain. For Rebecca and
Christopher, it is simply moral gain versus moral gain. Since the mixed
cases are relevantly similar to the pure cases and it would be useful to
discuss them together, let us deﬁne moral trade so as to include both:
Moral Trade: Trade that is made possible by differences in the par-
ties’ moral views.
I shall use the term regular trade to refer to trade that is not moral trade.
I shall use the term pure moral trade for moral trade where both parties
view the result as morally superior and mixed moral trade for other cases.
Note that there are trades where each person thinks the outcome is
morally superior which don’t count as moral trade on this deﬁnition. For
example, suppose the directors of two charities with different resource
needs trade a car for a website redesign. Even if they have different
moral views, this is a trade that was made possible by the different re-
source needs —it didn’t require a difference in their moral views. Cases of
this sort are much more similar to instances of regular trade than to the
cluster of examples discussed here, and so the deﬁnition of moral trade
was constructed to exclude them.
Note also that as well as substantive differences in moral views, I
include differences due to different indexicals. For example, if Mary and
3. Donations to charities with paid employees could be seen as examples of this kind
of moral-for-prudential trade. A donor effectively pays an employee of the charity to per-
form some good act that the employee would not have otherwise done. This can be more
efﬁcient than the donor doing that good act directly. Such situations could be seen as
examples of specialization or professionalization via ðmixed Þmoral trade.122 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
Susan both hold an agent-relative view which gives a special place to the
agent’s family, then for Mary this will be a special place for Mary’s family
and for Susan it will be for Susan’s family. This could give rise to oppor-
tunities for moral trade. Similarly, a moral theory might give an agent a
strict duty to tell the truth but not require any action concerning whether
others tell the truth. If two people held this theory, it would give them
different duties, which again could give rise to interesting opportunities
for moral trade.
An interesting edge case for the above deﬁnition is the possibility of
intrapersonal moral trade. Many people are guided by prudential consid-
erations and moral considerations in a rather ad hoc mixture. It is pos-
sible that they will notice that a certain pair of behaviors is dominated by
another pair of behaviors, being both prudentially and morally better. For
example, consider the case of Alex, who would dearly like to ﬂy from
Boston to Paris for a cultural holiday, but is deeply concerned about car-
bon emissions and climate change.4He decides that he can’t justify the
prudential beneﬁts at such a moral cost. However, then a friend suggests
that he could ﬂy there and offset twice that amount of carbon at a rela-
tively small cost. He likes this idea, since it allows him to ﬁnally see Paris
despite a small additional cost ðenhancing his overall prudential inter-
estsÞ, while simultaneously helping ﬁght climate change overall ðbetter
fulﬁlling his moral view Þ. Flying with doubly offset carbon emissions is
therefore an improvement over staying in Boston on his two different
value systems and constitutes a kind of internal or intrapersonal moral
trade. I shall not formally include intrapersonal trade in the deﬁnition,
but some of what I say below would apply to these cases as well.
SOME ECONOMIC THEORY OF MORAL TRADE
What can we say about the theory of moral trade? First, it is worth point-
ing out that moral trade is already encompassed within the standard def-
inition of trade in economics. When economists speak of a person’s pref-
erences, they include both the person’s prudential preferences and their
moral preferences5in an undifferentiated set of preferences. So when
it comes to trade, moral trade is mixed in with regular trade. For econ-
omists, moral trade is thus not an additional kind of trade, but a sub-
category.
4. Julia Driver provides a detailed discussion of a case like this in “Moral Bookkeeping
and Carbon Offsets,” in Consequentialism and Environmental Ethics , ed. Avram Hiller, Ra-
mona Ilea, and Leonard Kahn ðLondon: Routledge, 2014 Þ, 164 –73.
5. This raises a metaethical question. I have attempted to neutrally refer to ‘moral
views’ rather than ‘moral beliefs’ or ‘moral preferences’. Does it matter for a theory of
moral trade whether moral disagreements are more like disagreements about beliefs orOrd Moral Trade 123
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
This does not mean that it is well known or extensively studied.
Apart from contexts when they are carefully outlining deﬁnitions, econ-
omists pay relatively little attention to the fact that their deﬁnition of
preferences includes moral preferences. While it is technically covered by
the existing formalisms, a paper on moral trade would be considered sur-
prising in economics.
One way to look at moral trade from the perspective of economics
would be to note that while we have extensive trade based on variation in
prudential preferences, there is strikingly little trade based on variation in
moral preferences. Moreover, there are almost no mechanisms for facil-
itating such trade. A common assumption in economic modeling is a
“complete market” where everything is available for trade,6but in reality
the market for moral trade is very incomplete.
Economic approaches could be used to transform the very simple
examples of moral trade that I give ðwhich we could call moral barter Þinto
more sophisticated systems to realize greater gains. For example, we could
introduce moral trade equivalents of currency ,markets ,bargaining , and pro-
fessionalization . I shall touch on each of these, but a full treatment is beyond
the scope of this essay.
We can also combine techniques from economics with techniques
from normative ethics to get more clarity on the nature of moral trade.
When two parties are in a situation where moral trade could take place,7
there will be a set of options available to them. These include all combi-
nations of them each acting in independent ways as well as all the ways
they could act in concert with each other. Each of these options can be
assessed according to each of the two parties’ moral views. We could asso-
ciate each option with a pair of numbers ðc1,c2Þrepresenting its choice-
worthiness according to the ﬁrst view and according to the second view.8
We could then represent these options as points on a graph, where the
vertical axis represents the choiceworthiness on the ﬁrst party’s moral
view and the horizontal axis represents the choiceworthiness on the sec-
ond party’s moral view ðsee ﬁg. 1 Þ.
6. See, for instance, Mark D. Flood, “An Introduction to Complete Markets,” Federal
Reserve Bank of St. Louis Review 73ð1991 Þ: 32–57.
7. This can be scaled up to any number of parties in a trade, but the diagrams would
become much more complex.about desires? It seems to me that it does not. After all, just as different tastes can give us
mutual reason to perform regular trade, so different beliefs can give us reason to perform
regular trade. For example, a large amount of stock market trade only occurs because
people have different beliefs about the future valuation of the stock.
8. Not all moral theories will have choiceworthiness that behaves exactly like a
number: some theories may allow for incomparability, and for some theories the differ-
ences between numbers may not be meaningful. Figures 1 and 2 only assume comparability
ðand their point could be made less elegantly even without comparability Þ. Figures 3 and 4124 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
I have circled four of the options, because they have a special status.
They are Pareto optimal , meaning that for each of them there is no other
option which is better according to one view and at least as good according
to the other. In contrast, the uncircled options are Pareto dominated . For
each of them there is at least one option that dominates them: being bet-
ter according to one view and at least as good according to the other.
From the perspective of a neutral onlooker, we would hope that they could
agree to choose one of these circled, Pareto optimal, options.
It may look like this diagram builds in an assumption that the parties’
moral views are orthogonal to each other, but it does not. To the extent to
which the views are aligned, the points representing the options will be
FIG. 1.—The available options displayed according to the choiceworthiness each party
assigns them. The Pareto optimal options are circled.
will assume that the differences between the numbers are meaningful: that the theory is
cardinal . None of this analysis assumes that the zero point or the scale is meaningful.Ord Moral Trade 125
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
correlated, being scattered closer to the positive diagonal. Opposed moral
views will create a negative correlation.
In cases of moral trade, there is an additional important element.
One of the options represents a default or status quo: what would hap-
pen if they do not trade.9In ﬁgure 2 I have marked this option with a
cross through it. Two of the Pareto optimal options do not dominate this
default. This means that they are worse than the default according to
one of the parties, so if the trade is being freely made, that party would
refuse to consider that option. This leaves us with the options that lie in
the indicated quadrant: above and to the right of the default, of which a
neutral party would hope that they choose one of the two circled ones.
The question of exactly which option would get chosen depends
upon how the parties negotiate. In economics, this is covered by bar-
gaining theory.10Developing this here would take us too far aﬁeld, but
we can at least note that the ﬁrst party ðmeasured by the vertical axis Þ
would prefer the topmost option and so on down to the default, while
the second party would prefer the rightmost option and so on down to
the default. The distance by which an option is above the default rep-
resents how much “moral surplus” there would be from the ﬁrst party’s
point of view if they agreed on it ðand similarly for the distance to the
right for the second party Þ. Depending on how the parties negotiate it is
possible that a considerable portion of the potential total moral surplus
might be lost.
In some cases, there might be no options that are above and to the
right of the default ðsee ﬁg. 3 where I have removed all such options Þ. If
so, it might look like no trade is possible. For example, suppose that two
people are working out how to spend their Sunday afternoon and were by
default going to work on their individual projects, related to their dif-
ferent moral views. They realize that if they worked together on the same
project, they could achieve signiﬁcantly more gains. The options where
they team up like this are labeled AandB. However, they each think that
the other person’s project is not as morally valuable as their own, such
that joining the other team would be slightly worse than continuing to
work alone on their own project.
9. The analysis of moral trade up to this point is similar to that of moral uncertainty
ðthe theory of how to make decisions when uncertain about moral considerations Þ, with
options being compared in virtue of their choice-worthiness according to different moral
theories and the ideal option to choose being restricted to the Pareto optimal options.
However moral uncertainty does not have an analogue of the default option. For an in-
troduction, see Ted Lockhart, Moral Uncertainty and Its Consequences ðOxford: Oxford Uni-
versity Press, 2000 Þ.
10. See, e.g., Abhinay Muthoo, Bargaining Theory with Applications ðNew York: Cam-
bridge University Press, 1999 Þ.126 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
If they know that this situation will be repeated ðsay, each week Þthen
they could agree to alternate the project they team up on. This shows
the value of considering a wide array of options at once when trading to
increase the gains of moral trade. However, suppose that this case is a one-
off opportunity. What then?
One way out would be to suggest a gamble, for example, to ﬂip a
coin and to team up on the winner’s project. Alternatively, they could
use a randomized method with unequal probabilities. Such randomized
methods will typically be judged to have values that span an arc from A
toB.11For example, a 99.99 percent chance of working on A’s project
would typically be considered by both parties to be almost the same value
FIG. 2.—Adding a restriction that options must be Pareto superior to the default option
11. If both parties are risk neutral, it will form a straight line segment.Ord Moral Trade 127
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
as working on A’s project for certain, and as this chance is lowered bit by
bit, the value will move slowly across the graph until it becomes very close
to the value of B. There will often be some range of probabilities such
that randomizing which project they team up on using that probability
will dominate the default option of working separately. They could then
trade to one of those randomized options, ﬂip the coin ðor run the lot-
teryÞ, and head off to the winning project together.
Another way out would be via side payments, ideally in some more
or less continuous medium. For example, the person whose project they
team up on could give the other person a sum of money. In order to
keep this as an example of pure moral trade, let’s suppose that they do
this by reallocating some of their charitable donations for the year to a
charity of the other person’s choice. This is represented in ﬁgure 4. The
FIG. 3.—The choiceworthiness of new options representing weighted lotteries between
options AandB.128 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
lines that stretch out from Arepresent the results of teaming up on A
combined with a side payment of a certain size. As with the randomized
method, there is reason to expect small differences in side payments to
produce options that are close together in value and so produce a con-
tinuous line. The slope of this line would be set by the party’s relative
moral valuations of the two charities that the donations are being switched
between. If the payments are relatively small, then these lines should be
relatively straight. If the lines have the shape indicated in ﬁgure 4, then the
dominant options would involve teaming up on the second person’s proj-
ect and that person switching some portion of their donations to the ﬁrst
person’s preferred charity. In this particular setup, the options created
by the side-payment approach dominate the options created by the ran-
FIG. 4.—The choiceworthiness of new options representing starting with AorBand
making a side payment to the other party.Ord Moral Trade 129
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
domized approach and this would always happen unless the lines are quite
curved.
CONSEQUENTIALISM, DEONTOLOGY, AND VIRTUE ETHICS
Let us now consider how moral trade ﬁts in with the major branches of
normative ethics. To some extent, it would seem to be a natural ﬁt with
consequentialism. Since most forms of consequentialism are based around
agent-neutral value, consequentialists typically do not care who achieves
a valuable outcome, so long as it is achieved. This means that a conse-
quentialist should see a lot of potential value in moral trades, regardless
of whether the other party is a consequentialist or not.
In contrast, many deontological theories involve agent-relative value
and side constraints. The most obvious effect of side constraints is that
the trade cannot involve the agent doing something she judges to violate
a side constraint. However, this should still allow many opportunities for
trade: for instance, none of the examples above involve the breaking of
side constraints.
There are also more subtle effects. A deontologist who is a vegetar-
ian for moral reasons concerning animals may think that it is imper-
missible for them to eat meat, and that this is much more morally im-
portant than convincing others to not eat meat. They might therefore
see relatively less reason to engage in moral trade. However, it seems
unlikely that they could see it as without value. For example, if they are
a vegetarian because they think it is immoral to inﬂict suffering on ani-
mals, then presumably they are at least somewhat concerned that ani-
mals do not suffer and could see at least some value in convincing other
people to stop causing them suffering.
It might be thought that the central role of the agent’s intentions
in many deontological theories would limit the beneﬁts of moral trade.
For instance, if a person was only donating to help the poor because this
was part of a moral trade and not out of a concern for human welfare,
this might lower or even nullify the moral value of that act. However,
there are reasons to think that there would still be gains to make from
moral trade even in cases of this sort. First, it might be that even if the act
of donating with no real concern for the poor lacks value, the other
party’s act of convincing this person to donate may possess moral value.
Second, it would be puzzling if the theory said there was nothing morally
superior about the situation in which poor people were helped.12And
ﬁnally, society routinely tries to give people nonmoral reasons to do
12. As John Rawls put it: “All ethical doctrines worth our attention take consequences
into account in judging rightness. One which did not would be irrational, crazy.” John
Rawls, A Theory of Justice ðCambridge, MA: Harvard University Press, 1971 Þ, 30.130 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
moral things, such as threatening them with jail if they murder some-
one. Assuming the theories see some value in such ways of avoiding
murder, they should also see value in trades that give people nonmoral
reasons for abstaining from acting in other unethical ways.
Indeed, it is possible that side constraints or agent-relative value
could encourage moral trade. For example, someone might think that
it is impermissible for them to lie in order to avoid some suffering but
that it wouldn’t be impermissibl et oc o n v i n c es o m e o n ee l s et om a k e
this lie in order to avoid the suffering. It could also be that having a
rigid view that lying and breakin gp r o m i s e si si m p e r m i s s i b l ec o u l d
signiﬁcantly help to foster moral t rade. For it could let other parties
see that the deontologist is truthful about their claims and will follow
through with their side of the deal. It could therefore help with the
key trust issues involved in making moral trade work in practice.13
It may seem that virtue ethicists would see much less reason to en-
gage in moral trade than consequentialists or deontologists. For exam-
ple, if a vegetarian virtue ethicist is refraining from eating meat because
it is the compassionate thing to do, then it is not obvious that it is equally
compassionate to convince someone else to refrain from eating meat.
However, theories of virtue ethics often stress the importance of
the promotion of virtue in others. For example, in the Nicomachean Ethics
Aristotle analyzes the sources of virtue and ﬁnds habituation to be par-
ticularly important: we become just by doing what is just, temperate by
doing what is temperate, brave by doing what is brave.14By this he means
that doing what is required by justice, even if for the wrong reasons, leads
us to become just and to go on to do these just acts for the right reasons.
He argues that this justiﬁes moral education to inculcate virtue in others,
and even for states to use coercion to induce habituation and hence
improve virtue in their citizens.15
Moral trade should also be able to create habituation and hence
improve virtue. While trading with a friend to convince them not to eat
meat wouldn’t immediately create the virtue of compassion, it should
habituate them into acting more in accord with compassion, which may in
turn lead them to become more compassionate. It thus seems that virtue
ethics, too, provides reasons for making moral trades.
13. The trust issue could be particularly difﬁcult for consequentialists given that, if
there is no way of enforcing the deal and no potential reputational costs, most forms of
consequentialism would say that it is wrong for the consequentialist to go through with
their side of the deal. Consequentialism is thus more reliant on these practical mechanisms
than the other branches of ethical theory.
14. Aristotle, Nicomachean Ethics , ed. Roger Crisp ðCambridge: Cambridge University
Press, 2000 Þ, bk. 2, chaps. 1 –4.
15. Ibid., bk. 10, chap. 9.Ord Moral Trade 131
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
TRUST
The major practical obstacle to the gains from moral trade is that of in-
sufﬁcient trust between the two parties. This obstacle comes in two parts.
Neither of them are different in principle from regular trade, but in prac-
tice they can be much more difﬁcult to overcome.
The ﬁrst part concerns trust about factual issues, so we shall call it
factual trust . Suppose we are engaged in regular trade. I am exchanging
some of my money for some fruit from your garden. If we were partic-
ularly paranoid, or lived in a particularly lawless society, there may be
difﬁculties in setting up the trade. Do you hand over the fruit ﬁrst, or do
I hand over the money? There might also be some issues of trust after
the trade. Is that a genuine banknote? Will those apples go bad more
quickly than I was led to believe? However, these issues are not too
burdensome and trades like this can happen fairly easily, even between
people who do not know each other.
In comparison, if Victoria agrees to donate 1 percent of her income
in exchange for Paul becoming a vegetarian, how do they know the
other one is continuing with the deal? The continuing nature of this
deal makes trust a big issue. This is particularly so because, unlike most
instances of regular trade, the services are not being delivered in the
presence of the person who bought them. Note that one could have reg-
ular trade with these properties, and one could have moral trade which
is one-off with services delivered in the presence of both parties. For
example, if two parties are trading a vote for one policy with a vote for
another, then they only need to be able to trust each other until both
votes have been cast. However, in practice it seems likely that this prob-
lem of factual trust will be greater for moral trade.
There are various strategies for overcoming this problem in practice.
In cases where the parties know each other, simply trusting one another
might sufﬁce, since they are unlikely to be interested in deceiving each
other.
In other cases, the parties may be able to occasionally check up on
each other and call off the deal if either side has not kept up the arrange-
ment. For instance, someone could ask to see the other party’s receipts
for their donations at the end of the tax year. When this is possible, it
turns the situation from a one-shot prisoner’s dilemma to an iterated
prisoner’s dilemma of unknown duration. If the parties think there are
sufﬁciently many rounds left to come, then they have reason to keep to
their side of the deal so as to be able to continue the arrangement.
It is also possible to arrange penalties for noncompliance, including
legal penalties, just as we have for regular trade. This would be another
way of escaping the prisoner’s dilemma: having society act as leviathan to
lower the payoff for defection, changing the nature of the game. While132 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
one party can’t legally force the other to change their diet even if an
agreement was entered into, there are other possibilities. The most simple
is for each party to put money into escrow, which would be forfeited if
they broke the deal. If enough money was placed in escrow this could give
both parties reason to keep to the deal even if the chance of deception
being discovered was low. This is how contract law works for regular trade:
we don’t force an actor to make a movie just because they signed a con-
tract to do it, instead they risk ﬁnancial losses if they breach the contract.
With regular trade, our societies have established an implicit contract
when you buy something, which overcomes a lot of the transaction cost of
writing contracts. And even at the greengrocer, most of the factual trust
comes from custom rather than from criminal law or contract. For ex-
ample, if the greengrocer refused to give you a dollar of your change, it
would be difﬁcult to appeal to the police to get this back. If moral trade
were to become sufﬁciently popular, customs might evolve to support it,
and the law might be updated so that moral trades also establish implicit
contracts.
The second problem of trust is what one might call counterfactual
trust. For example, Victoria needs to trust that Paul wouldn’t have be-
come a vegetarian without that agreement, and he needs to trust that
she wouldn’t have increased her donations to Oxfam to 1 or more per-
cent without the agreement. This is particularly hard to police. It is even
difﬁcult for each party to determine the truth of their own counterfactual.
Paul might be pretty sure he was not about to become a vegetarian that
year, but who knows about the future?16
In theory this problem of counterfactual trust exists even in regular
trade. How do I know you wouldn’t have given me the fruit from your
garden if I didn’t pay you? Such problems of counterfactual trust are
pretty serious in trade between friends as their interests are often sharply
aligned. To some extent being your friend means that I start to want you to
get the things you desire and vice versa. It is perhaps one of the reasons
that trade between friends can be difﬁcult to set up. In contrast, there is
very little chance that the greengrocer would have given me my vege-
tables if I hadn’t given him some money. Most regular trade therefore
comes with a high degree of counterfactual trust.
Counterfactual trust appears to be a big problem for realizing the
full gains from moral trade. Unlike for factual trust, contract law seems
to be powerless. Playing out the transaction over a long time period seems
to exacerbate the issue, rather than ﬁx it: it is harder for either party to
predict what someone would be doing a long time hence were it not for
this trade. Counterfactual trust is minimized in situations where the ac-
16. This could be a particular challenge if he was known to be interested in moral
trade and if many moral trades concerned vegetarianism.Ord Moral Trade 133
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
tions required by the trade take place over a fairly short timescale ðsuch as
a year Þ, and where the parties are friends or people of known moral
character who can trust each others’ sincere judgments about what they
would have done otherwise.
PRACTICAL CASES
The most well known practical examples of moral trade concern the ex-
changing of votes. For example, consider the role of an independent
senator or leader of a small single-issue party who holds the balance of
power. In such a case, it is a commonplace for them to trade their support
in passing several of the government’s major bills in exchange for them
implementing a new policy favorable to the senator’s ðor small party’s Þ
most strongly held view. The different parties involved in the trade con-
sider their relative bargaining power and how much they think is at stake
in the different cases, then strike a moral trade, trying to make the overall
outcome as good as possible from their own perspectives. This kind of
trade during legislation can allow Pareto superior combinations of poli-
cies to be agreed upon.
This exchange of votes has also occurred in the general public. In the
2000 and 2004 US presidential elections, a number of websites appeared
to enable people in different states to change their votes in such a way
that they each thought the outcome was better than the status quo. For
example, suppose a person in a safe state preferred Al Gore to Ralph
Nader to George W. Bush, while a person in a swing state preferred Ralph
Nader to Al Gore to George W. Bush, and that absent any trade they were
going to vote for their ﬁrst choice candidates. This would produce a vote
for Nader and a vote for Gore, but they would be inefﬁciently distributed:
the vote for Gore would happen in a state where it would have very little
chance of changing the outcome. If they instead agreed to vote for the
other person’s ﬁrst choice, the vote for Gore would have more chance of
electing him and the vote for Nader would be equally good at helping him
win the 5 percent share needed to qualify for federal funding at the next
election.
Websites such as Nader Trader and VotePair helped to facilitate such
trades.17They acted as simple markets for these moral trades, allowing
thousands of people to put up offers and ﬁnd others with matching of-
fers. There were initially serious questions about the legality of such sites
ðleading several to close down in fear of prosecution Þ, but the services
they offer have since been ofﬁcially declared to be legal, with the US 9th
Circuit Court of Appeals declaring, “Whether or not one agrees with these
17. At the time of writing, neither of these is still live, though VotePair is archived at
https://web.archive.org/web/20041220193356/http://votepair.org/.134 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
voters’ tactics, such efforts, when conducted honestly and without money
changing hands, are at the heart of the liberty safeguarded by the First
Amendment.”18
Prior to the 2012 US presidential election, an even more interest-
ing website appeared. Repledge.com was set up to allow people to swap
off their donations to their preferred political parties in much the same
was as my example concerning gun rights and gun control.19Users would
donate a sum of money, list whether they were supporting the Repub-
licans or the Democrats, and choose an alternative charity. At the end of
the process most of these donations would be matched off against each
other and sent to the alternative charities, with the unmatched surplus
for the political party that got the most ﬁnancial support being sent on to
that party.
This would be a clear case of a market for moral trade, helping pairs
of parties to realize exchanges that they both think make the world a
better place. However, the site was taken down before such trades could
be made, presumably due to unresolved legal questions arising from the
fact that it involved political donations, which are subject to additional
rules and regulations.20
Hopefully such moral trade markets will be more successful in the
future. To the best of my knowledge, nothing would stop someone from
setting up a site like Repledge.com dedicated to cancelling out dona-
tions which aren’t election contributions. For example, a site could be
set up to cancel donations between advocates of gun rights and gun con-
trol. Creating such markets would help to remove the moral and eco-
nomic inefﬁciencies in the charity sector that are created by the exis-
tence of charities with opposed missions.
One could even cater for the possibility of different efﬁciencies for
the charities and different beliefs about these efﬁciencies by getting
people to register an additional number when they make their contribu-
tion. Each contribution would say something of the form “I’m willing to
give up to $6,000 to this charity but am prepared to cancel each dollar
against 0.8 dollars pledged to this opposing charity.”21Such contribu-
18. Porter v. Bowen, No. 06 –55517 ð9th Cir. 2007, 9359 Þ.
19. I would like to thank Owen Schaefer for bringing this to my attention. At the time of
writing repledge.com is down but is archived at ht tp://web.archive.org/web/20120625133032
/http://www.repledge.com/.
20. Repledge’s founder, Eric Zolt, made a request to the Federal Election Commission
of the United States to see if it would issue legal advice on some questions related to elec-
tion contributions regulation, but the Commission’s vote was tied 3 –3, so it couldn’t offer
clear advice on the matter.
21. This is equivalent to stock exchanges where people offer to buy or sell some
number of shares at some price and mutually agreeable offers are automatically matched
up.Ord Moral Trade 135
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
tions could then be automatically matched against each other, leaving un-
matched some people who were contri buting to the more popular charity
or who were trying to drive particularly hard bargains. This ﬂexibility would
ultimately be useful, especially when the opposed charities are clearly not
equally efﬁcient, but it does signiﬁcantly complicate things and a ﬁrst effort
would probably do better if kept things simple and only offered one-to-one
exchanges.
If people found success with relatively constrained cases like these,
they could try to extend them to more complete markets. For instance,
one could have a moral-trade version of eBay or Craigslist, with people
offering and requesting a great variety of moral services with associated
prices. Ideally there would be some form of currency to make the trad-
ing more efﬁcient. This could be regular currency such as dollars or
pounds, or it could be some kind of virtual currency, perhaps one that
can only be exchanged for moral services.22There are many possibilities,
so at this stage it is very difﬁcult to predict how a mature market for
moral trade would develop and progress.
And of course there are still the ad hoc arrangements between
friends and colleagues that we started with, like those of Victoria and
Paul, or Rebecca and Christopher. While the overall scale is smaller, the
gains from each individual trade may well be large. In the case of Victoria
and Paul, they each thought that by trading they were increasing their
personal moral impact by a large proportion. These personal arrange-
ments are particularly well placed to avoid the problems of factual and
counterfactual trust. Moreover, one doesn’t have to wait for someone to
set up a larger enterprise but can start realizing the beneﬁts immediately.
THE GAINS FROM MORAL TRADE
The gains from regular trade have been vast. Without some kind of trade,
it is unclear that humans could do much more than merely survive. Our
technology, cities, and culture, could not exist without the ability to spe-
cialize our labor and this has largely been achieved through trade. We each
do the work that other people want done, which gives us a salary to spend
on things that enrich our own lives ðpaying others to do the labor whose
fruits we seek Þ. This simple process has enabled the rise of civilization and
produced great value.
How large are the potential gains from moral trade? It seems doubt-
ful that they could be on that same scale: partly because having one type
of trade is perhaps enough to get the main gains, and partly because of
the additional challenges of acquiring the counterfactual trust needed
22. While it has some drawbacks, the latter might help avoid anxiety about trading
between sacred and nonsacred values.136 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
for moral trade. However, there is at least some reason to think that the
potential gains are large.
How could we quantify these gains? The gains from regular trade
accrue to billions of different individuals, but in each case the gains look
somewhat similar ðimproving someone’s wellbeing Þ, and we have ways of
assessing these gains in aggregate. The gains from moral trade would be
much more difﬁcult to assess, as they are gains in how much better the
world is, or how morally superior things are from the perspectives of
each of the moral views of the trading parties. These are much more
diverse than the prudential preferences of different people and it is not
clear whether there is any sensible way to aggregate them into some-
thing like a total or an average. One could imagine a case where all moral
theories agreed that the outcome would be 10 percent better with moral
trade than without, in which case we would have a kind of deﬁnitive
answer ðthough only expressible as a percentage Þ. However, if they dis-
agreed on these percentages there would be no clear way to weight them
to perform an average.23More problematically, many theories would
completely reject the idea of quantifying the moral difference between
the situations.
However, we can at least make some intuitive progress. For exam-
ple, it seems that moral trade could remove signiﬁcant inefﬁciencies in
the charitable sector, where opposed charities have budgets totaling bil-
lions of dollars per annum. It may also allow us to greatly cut down on
the number of people performing acts that don’t give them much pru-
dential value, but which some people think are morally terrible ðsuch as
purchasing meat that was produced with very bad animal welfare prac-
ticesÞ, as those who hold these moral views could always offer something
of moral or prudential value in exchange.
I should also note that nothing guarantees moral trade will make
things better rather than worse. It only guarantees that it will be better
according to the views of the parties who make the exchange. Just as
regular trade could have negative externalities ðmaking things worse for
people who weren’t involved in the trade Þso moral trade could have
negative ðmoral Þexternalities: making things worse according to the
moral views of people not involved in the trade. If this were sufﬁciently
23. There are two parts to this. First, it is unclear how to weight the different views in
terms of their probability or popularity. Second, even if we could do that, the average of
different percentage gains isn’t always the midpoint, but depends on what bases they are
rising from. This would require a way to compare absolute value differences between moral
theories. This problem also arises in cases of moral uncertainty, where it is known as the
problem of intertheoretical comparison of value. See Edward J. Gracely, “On the Non-
comparability of Judgments Made by Different Ethical Theories,” Metaphilosophy 27ð1996 Þ:
327–32; Ted Lockhart, Moral Uncertainty and Its Consequences ðOxford: Oxford University
Press, 2000 Þ.Ord Moral Trade 137
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions
common it could overcome the beneﬁts, but if the moral trade is sufﬁ-
ciently engaged in by people of all views then I ﬁnd this very unlikely.
Especially considering that each moral trade is likely to have many positive
moral externalities ðmaking things better according to the moral views of
people not involved in the trade Þ.
According to moral realism, moral trade could also have negative
effects if the trades make things worse overall on the objectively correct
moral view —even if no one holds this view. This would also seem to be
unlikely. While it is possible according to moral realism for no one to
have the correct moral view, it seems likely that our moral views at least
cluster near it and that systematically improving the world according to
our moral views would be more likely to improve it according to the
correct view than to make it worse. Indeed if our views on morality are so
misguided that we are systematically pointing away from the objective
moral requirements, then we would presumably have bigger problems
than moral trade, and it would be difﬁcult to blame moral trade for
our situation as in that case practical rationality, moral education, and
strength of will, would all be making things worse.24
Finally, moral trade could incentivize bad behaviors. For example,
someone who thinks that animals have no moral value might start eating
meat in the hope that others will pay him to stop. This problem of perverse
incentives already occurs in other areas: for instance, insurance makes
people more lax about risks since someone else will pick up the bill. If we
don’t have good ways of dealing with such perverse incentives, they could
erode a signiﬁcant portion of the value of moral trade in some areas.
However, there is little fear of such perverse incentives causing moral trade
to make things worse rather than better. For if making offers of moral
trade in some domain were strongly believed to make things worse overall,
then people would simply stop making such offers.
CONCLUSIONS
Trade is everywhere. It is one of the foundations of modern prosperity.
However it is usually restricted to cases of mutual prudential advantage.
There appear to be great potential gains from moral trade, both in terms
of local ad hoc arrangements and larger organized markets. There is also
a lot of potential for further philosophical and economic analysis of
the theoretical underpinnings of moral trade, exploring how much of
the theoretical and practical apparatus of regular trade can be directly
brought across and doing the new work required when it cannot. I hope
that this essay is just the ﬁrst step in a much longer journey.
24. The situation would be similar to blaming rationality in a world where we are
unfortunately situated such that acting rationally on our beliefs leads to woe.138 Ethics October 2015
This content downloaded from 23.235.32.0 on Wed, 21 Oct 2015 09:55:36 AM
All use subject to JSTOR Terms and Conditions


Pascal’s Mugging

Pascal’s mugging
NICKBOSTROM
In some dark alley ...
Mugger : Hey, give me your wallet.
Pascal : Why on Earth would I want to do that?
Mugger : Otherwise I’ll shoot you.
Pascal : But you don’t have a gun.
Mugger : Oops! I knew I had forgotten something .
Pascal : No wallet for you then. Have a nice evening.
Mugger : Wait!
Pascal : Sigh.
Mugger : I’ve got a business proposition for you. ...How about you give me
your wallet now? In return, I promise to come to your house tomorrow and
give you double the value of what’s in the wallet. Not bad, eh? A 200%
return on investment in 24 hours.
Pascal : No way.
Mugger : Ah, you don’t believe that I will be as good as my word? One can’t
be too careful these days. ...Tell you what: give me your wallet, and I come
to your house tomorrow and pay you 10 times its value.
Pascal : Sorry.
Mugger : OK, let me ask you something. Many people are dishonest, but
some people are honest. What probability do you give to the hypothesisthat I will keep my promise?
Pascal : 1 in a 1,000?
Mugger : Great! OK, so give me your wallet, and tomorrow I give you 2,000
times the value of its contents. The expectation value is greatly to your
advantage.
Pascal : There are 10 livres in my wallet. If we made a deal for you to take the
wallet and bring me 10 times the value of its contents tomorrow, then maybe
there’s a 1-in-a-1,000 chance that I would see the 100 livres you owe. But I’drate the chances that you will deliver on a deal to return me 20,000 livres
much lower. I doubt you even have that much money.
Mugger : Your scepticism is understandable, although in this particular case it
happens to be misguided. For you are M. Pascal if I’m altogether not mis-
taken? And I’ve heard that you’re a committed expected-Utility maximizer,
and that your Utility function is aggregative in terms of happy days of life.Is that not so?
Analysis Vol 69 | Number 3 | July 2009 | pp. 443–445 doi:10.1093/analys/anp062
/C223The Author 2009. Published by Oxford University Press on behalf of The Analysis Trust . All rights reserved.
For Permissions, please email: journals.permissions@oxfordjournals.orgpascal’s mugging |443
444 | nick bostrom
Pascal : It is. My Utility function is unbounded. And I deem two days of
happy life twice as good as one such day; and 2,000 days twice as good as
1,000 days. I don’t believe in risk aversion or temporal  discounting.
Mugger : Excellent. I don’t necessarily have to know that you reject risk
aversion and temporal discounting, but it makes things easier. Well, have I
got good news for you! I have magical powers. I can give you any finite
amount of money that you might ask for tonight. What’s more, I can give
you any finite amount of Utility that I choose to promise you tonight.
Pascal : And I should believe you why?
Mugger : Trust me! OK, I realize this does not give you conclusive evidence,
but surely it counts a least a little bit in favour of the truth of what I am
asserting. Honestly, I really do have these powers.
Pascal : Your conduct tonight has not inspired me with confidence in your
honesty.
Mugger: OK, OK, OK, OK. But isn’t it possible that I am telling the truth?
Pascal :It ispossible that you have the magic powers that you claim to have,
but let me tell you, I give that a very, very low probability .
Mugger : That’s fine. But tell me, how low a probability exactly? Remember,
you might think it all seems implausible, but we are all fallible, right? And
you must admit, from what you’ve already seen and heard, that I am a rather
atypical mugger. And look at my pale countenance, my dark eyes; and note
that I’m dressed in black from top to toe. These are some of the telltale signs
of an Operator of the Seventh Dimension. That’s where I come from  and
that’s where the magic work gets done.
Pascal : Gee ... OK, don’t take this personally, but my credence that you have
these magic powers whereof you speak is about one in a quadrillion.
Mugger : Wow, you are pretty confident in your own ability to tell a liar from
an honest man! But no matter. Let me also ask you, what’s your probability
that I not only have magic powers but that I will also use them to deliver on
any promise – however extravagantly generous it may seem  – that I might
make to you tonight?
Pascal : Well, if you really were an Operator from the Seventh Dimension as
you assert, then I suppose it’s not such a stretch to suppose that you might
also be right in this additional claim. So, I’d say one in 10 quadrillion.
Mugger : Good. Now we will do some maths. Let us say that the 10 livres that
you have in your wallet are worth to you the equivalent of one happy day.
Let’s call this quantity of good 1 Util. So I ask you to give up 1 Util. In return,
I could promise to perform the magic tomorrow that will give you an extra
10 quadrillion happy days, i.e. 10 quadrillion Utils. Since you say there is a 1
in 10 quadrillion probability that I will fulfil my promise, this would be a fair
deal. The expected Utility for you would be zero. But I feel generous this
evening, and I will make you a better deal: If you hand me your wallet, I will
perform magic that will give you an extra 1,000 quadrillion happy days
of life .
Pascal : I admit I see no flaw in your mathematics.
Mugger : This is my final offer. You’re not going to pass up a deal that we
have just calculated will give you an expected Utility surplus of nearly 100
Utils, are you? That’s the best offer you are likely to see this year.
Pascal : Is this legitimate? You know, I’ve committed myself to trying to be a
good Christian.
Mugger : Of course it’s legitimate! Think of it as foreign trade. Your currency
is worth a lot in the Seventh Dimension. By agreeing to this transaction, yougive a major boost to our economy. Oh, and did I mention the children?If only you could see the faces of the sweet little orphans who will be made so
much better off if we get this influx of hard currency – and there are so many
of them, so very, very, very many ....
Pascal : I must confess: I’ve been having doubts about the mathematics of
infinity. Infinite values lead to many strange conclusions and paradoxes. You
know the reasoning that has come to be known as ‘Pascal’s Wager’? Between
you and me, some of the critiques I’ve seen have made me wonder whether Imight not be somehow confused about infinities or about the existence ofinfinite values ...
Mugger : I assure you, my powers are strictly finite. The offer before you does
not involve infinite values in any way. But now I really must be off; I have anassignation in the Seventh Dimension that I’d rather not miss. Your wallet,
please!
Pascal hands over his wallet .
Mugger : Pleasure doing business. The magic will be performed tomorrow, as
agreed.
1
Oxford University
Future of Humanity Institute
Faculty of Philosophy & James Martin 21st Century School
Suite 8, Littlegate House, 16/17 St Ebbe’s Street
Oxford OX1 1PT, UK
nick.bostrom@philosophy.ox.ac.uk
1 Related scenarios have recently been discussed informally among various people. Eliezer
Yudkowsky named the problem ‘Pascal’s mugging’ in a post on the Overcoming Bias blog
(http://www.overcomingbias.com/2007/10/pascals-mugging.html ). I am grateful to Toby
Ord and Rebecca Roache for comments.pascal’s mugging |445


CHAI Newsletter #1 2023

17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 1/9        CHAI Newsletter
This newsletter includes CHAI activities from January to April 2023
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 2/9
Events

Supreme Court Brief - 19/01/2023
CHAI's Jonathan Stray joined with Brandie Nonnecke from Berkeley's CITRIS
Policy lab, the Center for Democracy and Technology , and other scholars to ﬁle
a brief for the Supreme Court in an upcoming case that asks whether "targeted
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 3/9recommendations" should be protected under the same law that protects the
"publishing" of third party content. This law, called CDA Section 230, protects
virtually all major apps and platforms that provide access to other people's
information -- from search engines to music streaming to social media. CHAI is
of course very concerned with the personal and societal effects of
recommender systems; however , we believe that creating a legal distinction
around the type of algorithm used will only disincentivize innovative new
algorithms for selecting content -- exactly the type of safe and beneﬁcial
algorithms that we wish to create. Instead, we argued that platforms should be
liable based on whether they contributed to harmful effects, not on what type of
algorithm they use.
Find the link to additional information here.

Topos Institute - 02/27/2023
Nisan Stiennon  gave a talk on "Metagames and imprecise probabili ty" at the
Topos Institute's weekly seminar .

Nobel W eek Dialogues, Stockholm - 12/02/2022
Stuart Russell gave a talk during a day-long symposium preceding the Nobel
Prize award cerem onies on Living with Artiﬁcial Intelligence. He was also part of
two panels: “Living with Technolo gy”, along with Anthony Aguirre, Serge
Haroche, Alice Heiman and Helga Nowotny , and “How to Avoid a Catastrophe”,
with Pamela Björkman, Paola Caselli, Steven Chu and Beatrice Fihn.
Find the link here.

The W orld Economic Forum, Davos - 01/20/2023
Stuart Russel gave two presentations at the annual meeting of the World
Economic Forum in Davos. The ﬁrst talk was on Why Human Perspe ctive is
Essential to Unlock the Transformative Value of AI and the second one was on
The Long V iew for Technology .
Find the link here.

Principles of Intelligent Behavior in Biological and Social Systems
(PIBBSS) - 01/08/2023
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 4/9Gillian Hadﬁeld gave a talk on The Foundations of Cooperative Intellige nce at
the PIBBSS . She explained that models of intelligence begin with individual
agents, seeking to achieve objectives, and approach the question of
cooperation as a strategic challenge , and opportunity , that is layered on when
such individual agents meet up. In this view, the individual is the foundati on and
cooperation is built on top. In this talk, Gillian argued that to build human-like
intelligence (and beyond) we need  to reverse this picture. Coopera ting in
groups is valuable  for many species  but it is foundational for humans: humans
wouldn’t exist but for their evolved capacity to cooperate. She put forward the
idea that cooperative intelligence is the core of human intelligence. And at that
core is the cognitiv e capacity to read and participate in the maintenance  of the
normative environment—the rules that maintain the stability and coordinated
behaviour of a group in distinctive  patterns. Gillian then explored what this
insight could mean for the research agenda on cooperative AI.

WeaveSphere - 1 1/15/2022
In her talk titled Regulation Meets Innovation: A New Governance Economy for
Regulatory Technology at WeaveSphere , Gillian Hadﬁeld tackled the issue of
how governments can develop the resources and expertise to keep up with
lightning-quick developments from Big Tech. The answer requires new
solutions for how we conceive of and implement regulation itself. While AI might
be new territory , regulating the safety and security of technology for the public
good certainly isn’t. In this talk, she provided a birds-eye view of the landscape
of AI governance and explained why building safe machine learning systems
requires not just technical innovatio n but new approaches to policy design as
well. Chief among these new models are regulatory technologies (regtech),
which can automate forms of overs ight to ensure that AI is fair and equitable,
and regulatory markets, which will support an economy that works as
innovatively and rapidly as the technologies that are racing ahead of it. Those
who develop these regulatory tools ﬁrst will not only capture an emergent
market need–they’ll also be building the future of safe and responsible AI.

CAP  | ChatGPT , outils d'intelligence artiﬁcielle et enjeux pédagogiques -
03/09/2023
"ChatGPT , Petite réﬂection sur la technologie et le problème d’aligne ment",
meaning “ChatGPT , pondering on technology and the alignment problem” was
an event organised due to university concerns about the inﬂuence of chatGPT
on education.   Four panel members, among which CHAI’ s Tom Lenaerts, were
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 5/9asked to give their perspective on what chatGPT  does and then it was followed
by a discussion on how to use or not use this tool in education.

TU Delft  - 03/15/2023 to 03/17/2023
Brian Christian gave an opening keynote address at "The Value Connection
Workshop" in Amsterdam, an interdisciplinary workshop on human values,
hosted by TU Delft and sponsored by the Royal Netherlands Academy of Arts
and Sciences and the Delft Design for Values Institute. This event took place
from 03/15 to 03/17/2023.

Weekly CHAI Seminars
Every Wednesday we host our Bene ﬁcial AI Seminar from 10 am to 12 pm PST.
Seminars currently take place remotely through Zoom.  Here  is the schedule for
the Beneﬁcial AI seminars.
If you would like to attend the virtual seminars, please email  chai-
admin@berkeley .edu.
Researcher News

Stuart Russell was appointed as Co-Chair of the World Economic Forum
Council on the Future of AI.
Rachel Freedman  and Oliver Danie ls-Koch won the AI Risk Analysis Award at
the Neur IPS ML Safety Workshop for their paper "The Expe rtise Problem:
Learning from Specialized Feedback ”, and Erik Jenner , Joar Skalse and Adam
Gleave for their paper “ A general framework for reward function distances ”.
Tom Lenaerts was interview on De wereld vandaag, a Belgian Flemish  radio,
and on television on VRT laat, a Belgian Flemish late evening news,
concerning the Future of Life Institute’ s letter .
Brian Christian was interviewed about AI alignment by Kim Hill on Radio New
Zealand’ s "Saturday Morning" program on the 1st of April 2023.
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 6/9Papers

2023 IEEE/RSJ  International Conference on Intelligent Robots and
Systems (IROS 2023)
CHAI’ s Mason Nakamura, Justin Svegliato and Stuart Russell, along with
Samer Nashed and Shlomo Zilberstein, submitted a paper titled Formal
Composition of Robotic Systems as Contract Programs, 03/01/2023.

CHAI’ s Katie Lu, Justin Svegliato and Stuart Russell, along with Samer Nashed
and Shlomo Zilberstein, submitted a paper titled Ethically Compliant
Autonomous Systems under Partial Observability , 03/01/2023.

Journal of Economic Literature
Gillian Hadﬁeld, Legal Markets , Journal of Economic Literature, 12/22/22.
OpenAI
Gillian Hadﬁeld, A Research Agenda for Assessing the Economic Impacts of
Code Generation Models , OpenAI, 03/03/2023.

AI Alignment Forum
Erik Jenner , Research agenda: Formalizing abstractions of computations , AI
Alignment Forum , 02/01/2023 .
Edmund Mills and Scott Emmons, Storytelling Makes GPT-3.5 Deontologist:
Unexpected Effects of Context on LLM Behavior , AI Alignment Forum,
03/14/2023 .
Lawrence Chan, Leon Lang, Erik Jenner , Natural Abstractions: Key claims,
Theorems, and Critiques , AI Alignment Forum, 03/16/2023.

arXiv
Evan Hubinger , Adam Jermyn, Johannes Treutlein, Rubi Hudson, Kate
Woolverton, Conditioning Predictive Models: Risks and Strategies,
arXiv:2302.00805 .
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 7/9The New Yorker
Brian Christian, Can A.I. treat mental illness? ,  The New Yorker , 02/27/2023 .

Last W eek T onight with John Oliver
Brian Christian worked with the producers of Last Week Tonight with John
Oliver for their episode about risks from present -day and near-future AI
systems , on 02/26/2023. His book The Alignment Problem  was featured brieﬂy
on the program.

ACM Digital Library
Gillian Hadﬁeld, Spurious normativity improves the capacity of agents to learn
stable enforcement and compliance behaviors , ACM Digital Library ,
01/22/2022 .
Gillian Hadﬁeld, The Impact of Auditing for Algorithmic Bias , ACM Digital
Library , 01/23/2023.

WIRED
CHAI Jonathan Stray and Gillian Hadﬁeld,  A Uniqu e Experim ent That Could
Make Social Media Better , WIRED, 03/02/2022 .

Work with CHAI

Research Fellowship
CHAI has created a new Research Fellowship and is seeking applications for
the inaugural Fellows. CHAI Research Fellows will work with CHAI faculty
members Stuart Russell, Pieter Abbeel, and Anca Dragan and will also have
the opportunity to collaborate with  CHAI afﬁliate faculty  at Berkeley , other
faculty members in Berkeley AI Research  (where CHAI is located), and many
other institutions. The Fellowship is aimed at training highly qualiﬁed
postdoctoral researchers to carry out research to advance beneﬁcial AI.
17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 8/9Start Date: Flexible
Compensation: Negotiated
Apply: Via this form
Internship Applications
CHAI Internship application is currently closed and will reopen in fall 2023.
Subscribe to our mailing list if you would like to be notiﬁed when the application
opens up.
General enquiries about jobs at CHAI can be sent to chai-info@berkeley .edu .
Find additional information  here.
Donations

If you are interested in supporting CHAI, you can ﬁnd our online donation page
here.
For any inquiries regarding donations or grants, please email chai-
admin@berkeley .edu

To see more, visit us at humancompatible.ai
Copyright © 2018 Center for Human-Compatible AI, All rights reserved.
Our mailing address is:
2121 Berkeley W ay. Ofﬁce #8029. Berkeley , CA. 94704.
Want to change how you receive these emails?
You can update your preferences  or unsubscribe from this list .

17/05/2023, 17:14 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1-13547852 9/9


CHAI Newsletter #3 2022

14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 1/8        CHAI Newsletter
This newsletter includes CHAI activities from mid-September 2022 to
January 2023
Events

Artiﬁcial Intelligence: Promise and Peril
Stuart Russell was invited by Lord McFall, Speaker of the House of Lords in the
UK Parliament, to deliver the Lord Speaker's Lecture on October 18 2022.
Find the link to additional information here.
NSF Convergence Accelerator W orkshop on Provably Safe and Beneﬁcial
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 2/8AI
The Inaugural W orkshop on Provably Safe and Beneﬁcial AI (PSBAI) was held
in-person in Berkeley , California October 7-9, 2022. The workshop focused on
three technical themes: General AI safety: methods for ensuring that AI
systems are safe and beneﬁcial for humans, regardless of how capable they
become; well-founded AI system design: building AI systems from semantically
well-deﬁned components with rigorous compositional properties, with a
particular focus on probabilistic programming as an enabling technology; and
formal methods for veriﬁcation and synthesis: methods providing rigorous
guarantees of correctness for software instantiations of well-founded agent
designs. It is hoped that this workshop will lead to the NSF choosing to fund a
program of relevant research.
Keynote talk "Provably Beneﬁcial Artiﬁcial Intelligence"
Stuart Russell gave the invited keynote talk for a conference in Haifa on
October 19-20 to launch Israel's AI Safety research community , followed by an
extended "ﬁreside chat" with Prof. Emeritus Yoav Shoham (Stanford).
Find the link  here.
Cooperation for Humanity research retreat
The Cooperation for Humanity research retreat was co-organized and
sponsored by the Center for Human-Compatible AI (CHAI) at UC Berkeley , the
Survival and Flourishing Projects (SFP) at Social and Environmental
Entrepreneurs, Encultured AI, PBC., and the Foundations of Cooperative AI
Lab (FOCAL) at CMU. The aim of the retreat was to identify how the attendees
can contribute to civilization-scale cooperation using their background and
experience in computer science, artiﬁcial intelligence, mathematics, and
philosophy .
Alignable Structures
Alignable Structures was an AI safety workshop organized in November 2022
by Ef fective Altruism Philadelphia. Topics included value learning, agency ,
interpretability , ontological crises, learning theory , and probabilistic
programming. Nisan Stiennon gave a talk on open-source game theory .
Find additional information  here.
Assistive T eaching of Motor Control T asks to Humans
CHAI Research Fellow Erdem Bıyık and Stanford PhD student Megha
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 3/8Srivastava presented a poster at NeurIPS 2022, from Nov 28 to Dec 9 2022, on
assistive teaching of motor control tasks. The main idea is to decompose tasks
into lower level skills using expert demonstrations, evaluate which skills the
student human is struggling with, and ﬁnally create individualized drills for them
so that they can practice important skills that they are not good at and the
related transitions between skills.
Learning Preferences for Interactive Autonomy
CHAI Research Fellow Erdem Bıyık was invited to give a talk in November
2022 titled “Learning Preferences for Interactive Autonomy” at Sonoma State
university for their Engineering Colloquium.
Find additional information here.
Weekly CHAI Seminars
Every W ednesday we host our Beneﬁcial AI Seminar from 10 am to 12 pm PST .
Seminars currently take place remotely through Zoom. Here is the schedule for
the Beneﬁcial AI seminars.
If you would like to attend the virtual seminars, please email chai-
admin@berkeley .edu.
Researcher News

Researcher Updates
Brian Christian has been named one of the inaugural recipients of the National
Academies Eric and W endy Schmidt Awards for Excellence in Science
Communication, given by The National Academies of Sciences, Engineering,
and Medicine in partnership with Schmidt Futures, for his book "The Alignment
Problem." He was honored with an event hosted at the National Academy of
Sciences in W ashington, DC in November 2022.
Brian was also interviewed by NPR's Ari Shapiro about 2022's advances in AI
and reasons to be both excited and concerned.
Finally , Brian has been accepted as a PhD student in Experimental Psychology
at Oxford University , where he will complete his current book project on What
Humans W ant.
Rachel Freedman was named a Rising Star in AI Ethics by the W omen in AI
Ethics global initiative.
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 4/8Papers

Conference on Neural Information Processing Systems 2022  - NeurIPS 2022
Joar Skalse, Nikolaus H. R. Howe,  Dmitrii Krasheninnikov and David Krueger ,
Deﬁning and characterizing reward hacking , NeurIPS 2022.
Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, David
Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan
and Sam Devlin, Uni[MASK] Uniﬁed Inference in Sequential Decision
Problems , NeurIPS 2022.
Daniel M. Ziegler , Seraphina Nix, Lawrence Chan, Tim Bauman, Peter Schmidt-
Nielsen, Tao Lin, Adam Scherlis, Noa Nabeshima, Ben W einstein-Raun, Daniel
de Haas, Buck Shlegeris, Nate Thomas, Adversarial Training for High-Stakes
Reliability , NeurIPS 2022.
Cem Anil*, Ashwini Pokle*, Kaiqu Liang*, Johannes Treutlein, Yuhuai W u,
Shaojie Bai, J. Zico Kolter , Roger Grosse,  Path Independent Equilibrium
Models Can Better Exploit Test-T ime Computation , NeurIPS 2022.
Mesut Yang, Micah Carroll and Anca Dragan,  Optimal Behavior Prior:
Improving Human-AI Collaboration Through Generalizable Human Models ,
Human-in-the-loop Learning (HILL) W orkshop, NeurIPS 2022.
David Zhang, Micah Carroll, Andreea Bobu and Anca Dragan, Time-Efﬁcient
Reward Learning via V isually Assisted Cluster Ranking , Human-in-the-loop
Learning (HILL) W orkshop, NeurIPS 2022.
Rachel Freedman and Oliver Daniels-Koch,  The Expertise Problem: Learning
from Specialized Feedback , NeurIPS 2022 ML  Safety W orkshop.
Erik Jenner , Joar Skalse and Adam Gleave, A general framework for reward
function distances , NeurIPS 2022 ML  Safety W orkshop.
arXiv
Richard Ngo, Lawrence Chan and Sören Mindermann,  The alignment problem
from a deep learning perspective , arXiv:2209.00626.
Buck Shlegeris, Fabien Roger , Lawrence Chan, Euan McLean,  Language
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 5/8models are better than humans at next-token prediction , arXiv:2212.1 1281.
Samer , Nashed, Justin Svegliato and Su Lin Blodgett, Fairness and Sequential
Decision Making: Limits, Lessons, and Opportunities , arXiv:2301.05753.
Adam Gleave, Mohammad Taufeeque, Juan Rocamonde, Erik Jenner , Steven
H. W ang, Sam Toyer, Maximilian Ernestus, Nora Belrose, Scott Emmons,
Stuart Russell,  imitation: Clean Imitation Learning Implementations ,
arXiv:221 1.11972.
AAAI 2023 W orkshop on AI Safety
Peter Barnett, Rachel Freedman, Justin Svegliato, Stuart Russell,   Active
Reward Learning from Multiple Teachers , AAAI 2023 W orkshop on AI Safety .

AI Alignment Forum
Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan
Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris and
Nate Thomas, Causal Scrubbing: a method for rigorously testing interpretability
hypotheses , AI Alignment Forum.
Alex Turner ,  Inner and outer alignment decompose one hard problem into two
extremely hard problems , AI Alignment Forum.
Alex Turner and Quintin Pope,  The shard theory of human values , AI Alignment
Forum.
Erik Jenner and Johannes Treutlein, Response to Katja Grace's AI x-risk
counterarguments , AI Alignment Forum.
Johannes Treutlein, Rubi Hudson, Caspar Oesterheld,  Proper scoring rules
don’t guarantee predicting ﬁxed points , AI Alignment Forum.
Rachel Freedman, Adam Gleave,  CIRL  corrigibility is fragile , AI Alignment
Forum.
Nature Scientiﬁc Reports
Tom Lenaerts, Montero-Porras, E., Lenaerts, T., Gallotti, R., & Grujic, J. (2022).
Fast deliberation is related to unconditional behaviour in iterated Prisoners’
Dilemma experiments , Scientiﬁc Reports, 12(1), 1-10.
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 6/8The New York Times
Brian Christian,  How to Use ChatGPT  and Still Be a Good Person , The New
York Times, 12/23/2022.
The Artiﬁcial Intelligence Journal, AIJ
Connor Basich, Justin Svegliato, Kyle H. W ray, Stefan Witwicki, Joydeep
Biswas and Shlomo Zilberstein,  Competence-A ware Systems , Artiﬁcial
Intelligence Journal (AIJ).

The International Joint Conference on Artiﬁcial Intelligence, IJCAI
Siddharth Srivastava and Rushang Karia, Relational Abstractions for
Generalized Reinforcement Learning on Symbolic Problems , IJCAI.
The International Conference on Learning Representations, ICLR
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, Jacob Steinhardt,
Progress measures for grokking via mechanistic interpretability , ICLR 2023
Michael Chang, Alyssa Li Dayan, Franziska Meier , Thomas L. Grifﬁths, Sergey
Levine and Amy Zhang, Hierarchical Abstraction for Combinatorial
Generalization in Object Rearrangement , ICLR 2023.
Other Papers
Thomas Krendl Gilbert, W endy Ju and Jamy Li,  Fleets on the Streets: How
number , afﬁliation and purpose of shared-lane automated vehicle convoys
inﬂuence public acceptance and blame , Transportation Research Part F: Trafﬁc
Psychology and Behavior .
Thomas Krendl Gilbert and Nathaniel Lubin,  Social media is polluting society .
Content moderation alone won't ﬁx the problem , MIT  Tech Review .
George Obaido, Blessing Ogbuokiri, Ibomoiye Domor Mienye, Kehinde Aruleba
and Sydney Mambwe Kasongo et al,  An Interpretable Machine Learning
Approach for Hepatitis B Diagnosis , Applied Sciences (MDPI).
Thomas Krendl Gilbert, Micah Carroll, Jonathan Stray , Smitha Milli and
Anderson Rogers, Trade Regulation Rule on Commercial Surveillance and Data
Security Rulemaking , Federal Trade Commission Call for Public Comment.
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 7/8Work with CHAI

Research Fellowship
CHAI has created a new Research Fellowship and is seeking applications for
the inaugural Fellows. CHAI Research Fellows will work with CHAI faculty
members Stuart Russell, Pieter Abbeel, and Anca Dragan and will also have
the opportunity to collaborate with  CHAI afﬁliate faculty  at Berkeley , other
faculty members in Berkeley AI Research  (where CHAI is located), and many
other institutions. The Fellowship is aimed at training highly qualiﬁed
postdoctoral researchers to carry out research to advance beneﬁcial AI.
Start Date: Flexible
Compensation: Negotiated
Apply: Via this form
Internship Applications
CHAI Internship application is currently closed and will reopen in fall 2023.
Subscribe to our mailing list if you would like to be notiﬁed when the application
opens up.
General enquiries about jobs at CHAI can be sent to chai-info@berkeley .edu .
Find additional information  here.
Donations

If you are interested in supporting CHAI, you can ﬁnd our online donation page
here.
For any inquiries regarding donations or grants, please email chai-
admin@berkeley .edu

To see more, visit us at humancompatible.ai
14/03/2023 09:18 Center for Human-Compatible AI (CHAI) Newsletter
https://mailchi.mp/humancompatible.ai/fwx5lj5be1 8/8Copyright © 2018 Center for Human-Compatible AI, All rights reserved.
Our mailing address is:
2121 Berkeley W ay. Ofﬁce #8029. Berkeley , CA. 94704.
Want to change how you receive these emails?
You can update your preferences  or unsubscribe from this list .