Safe Pareto Improvements for Delegated Game
Playing∗
Caspar Oesterheld†and Vincent Conitzer
October 5, 2021
Abstract
A set of players delegate playing a game to a set of representatives,
one for each player. We imagine that each player trusts their respec-
tive representative's strategic abilities. Thus, we might imagine that
per default, the original players would simply instruct the represen-
tatives to play the original game as best as they can. In this paper,
we ask: are there safe Pareto improvements on this default way of
giving instructions? That is, we imagine that the original players can
coordinate to tell their representatives to only consider some subset of
the available strategies and to assign utilities to outcomes dierently
than the original players. Then can the original players do this in such
a way that the payo is guaranteed to be weakly higher than under
the default instructions for all the original players? In particular, can
they Pareto-improve without probabilistic assumptions about how the
representatives play games? In this paper, we give some examples of
safe Pareto improvements. We prove that the notion of safe Pareto
improvements is closely related to a notion of outcome correspondence
between games. We also show that under some specic assumptions
about how the representatives play games, nding safe Pareto improve-
ments is NP-complete.
Keywords : program equilibrium; delegation; bargaining; Pareto e-
ciency; smart contracts
∗This draft is work-in-progress. Please send any comments to
caspar.oesterheld@duke.edu . A shorter version of this paper has been published in the
Proceedings of AAMAS 2021; see https://users.cs.duke.edu/ ~ocaspar/SPIAAMAS.pdf
for a version of the AAMAS publication with appendices.
†Corresponding author
1
1 Introduction
Between Aliceland and Bobbesia lies a sparsely populated desert. Until
recently, neither of the two countries had any interest in the desert. How-
ever, geologists have recently discovered that it contains large oil reserves.
Now, both Aliceland and Bobbesia would like to annex the desert, but they
worry about a military con
ict that would ensue if both countries insist on
annexing.
Table 1 models this strategic situation as a normal-form game. The
strategy DM (short for \Demand with Military") denotes a military invasion
of the desert, demanding annexation. If both countries send their military
with such an aggressive mission, the countries ght a devastating war. The
strategy RM (for \Refrain with Military") denotes yielding the territory
to the other country, but building defenses to prevent an invasion of one's
current territories. Alternatively, the countries can choose to not raise a
military force at all, while potentially still demanding control of the desert
by sending only its leader (DL, short for \Demand with Leader"). In this
case, if both countries demand the desert, war does not ensue. Finally,
they could neither demand nor build up a military (RL). If one of the two
countries has their military ready and the other does not, the militarized
country will know and will be able to invade the other country. In game-
theoretic terms, militarizing therefore strictly dominates not militarizing.
Instead of making the decision directly, the parliaments of Aliceland and
Bobbesia appoint special commissions for making this strategic decision,
led by Alice and Bob, respectively. The parliaments can instruct these
representatives in various ways. They can explicitly tell them what to do
{ for example, Aliceland could directly tell Alice to play DM. However, we
imagine that the parliaments trust the commissions' judgments more than
they trust their own and hence they might prefer to give an instruction
of the type, \make whatever demands you think are best for our country"
(perhaps contractually guaranteeing a reward in proportion to the utility of
the nal outcome). They might not know what that will entail, i.e., how the
commissions decide what demands to make given that instruction. However
{ based on their trust in their representatives { they might still believe that
this leads to better outcomes than giving an explicit instruction.
We will also imagine these instructions are (or at least can be) given
publicly and that the commissions are bound (as if by a contract) to follow
these instructions. In particular, we imagine that the two commissions can
see each other's instructions. Thus, in instructing their commissions, the
countries play a game with bilateral precommitment. When instructed to
2
play a game as best as they can, we imagine that the commissions play that
game in the usual way, i.e., without further abilities to credibly commit or
to instruct subcommittees and so forth.
It may seem that without having their parliaments ponder equilibrium
selection, Aliceland and Bobbesia cannot do better than leave the game to
their representatives. Unfortunately, in this default equilibrium, war is still
a possibility. Even the brilliant strategists Alice and Bob may not always be
able to resolve the dicult equilibrium selection problem to the same pure
Nash equilibrium.
In the literature on commitment devices and in particular the literature
on program equilibrium, important ideas have been proposed for avoiding
such bad outcomes. Imagine for a moment that Alice and Bob will play a
Prisoner's Dilemma (Table 3) (rather than the Demand Game of Table 1).
Then the default of (Defect, Defect) can be Pareto-improved upon. Both
original players (Aliceland and Bobbesia) can use the following instruction
for their representatives: \If the opponent's instruction is equal to this in-
struction, Cooperate; otherwise Defect." [23, 17, 33, Sect. 10.4, 39] Then
it is a Nash equilibrium for both players to use this instruction. In this
equilibrium, (Cooperate, Cooperate) is played and it is thus Pareto-optimal
and Pareto-better than the default.
In cases like the Demand Game, it is more dicult to apply this ap-
proach to improve upon the default of simply delegating the choice. Of
course, if one could calculate the expected utility of submitting the default
instructions, then one could similarly commit the representatives to follow
some (joint) mix over the Pareto-optimal outcomes ((RM ;DM), (DM;RM),
(RM;RM), (DL;DL), etc.) that Pareto-improves on the default expected
utilities.1However, we will assume that the original players are unable or
unwilling to form probabilistic expectations about how the representatives
play the Demand Game, i.e., about what would happen with the default
instructions. If this is the case, then this type of Pareto improvement on
the default is unappealing.
The goal of this paper is to show and analyze how even without form-
ing probabilistic beliefs about the representatives, the original players can
1One might argue that due to the symmetry of the Demand Game, the original players
should expect the representatives to play the game's unique symmetric equilibrium (in
which both players play DM with probability1=4and RM with probability3=4). Of course,
in general games might be asymmetric. We here consider a symmetric game only for
simplicity. More generally, some games with multiple equilibria might have a single \focal"
equilibrium [35, pp. 54{58] that we expect the representatives to play. However, we
maintain that in many games it is not clear what equilibrium should be played.
3
Player 2
DM RM DL RL
Player 1DM3;32;05;55;5
RM 0;2 1;15;55;5
DL5;55;51;1 2;0
RL5;55;50;2 1;1
Table 1: The Demand Game
Pareto-improve on the default equilibrium. We will call such improvements
safe Pareto improvements (SPIs). We here brie
y give an example in the
Demand Game.
The key idea is for the original players to instruct the representatives to
select only from fDL;RLg, i.e., to not raise a military. Further, they tell
them to disvalue the con
ict outcome without military (DL ;DL) as they
would disvalue the original con
ict outcome of war in the default equilib-
rium. Overall, this means telling them to play the game of Table 2. (Again,
we could imagine that the instructions specify Table 2 to be how Aliceland
and Bobbesia nancially reward Alice and Bob.) Importantly, Aliceland's
instruction to play that game must be conditional on Bobbesia also instruct-
ing their commission to play that game, and vice versa. Otherwise, one of
the countries could prot from deviating by instructing their representative
to always play DM or RM (or to play by the original utility function).
The game of Table 2 is isomorphic to the DM-RM part of the original
Demand Game of Table 1. Of course, the original players know neither
how the original Demand Game nor the game of Table 2 will be played by
the representatives. However, since these games are isomorphic, one should
arguably expect them to be played isomorphically. For example, one should
expect that (RM ;DM) would be played in the original game if and only
if (RL;DL) would be played in the modied game. However, the con
ict
outcome (DM ;DM) is replaced in the new game with the outcome (DL ;DL).
This outcome is harmless (Pareto-optimal) for the original players.
Contributions Our paper generalizes this idea to arbitrary normal-form
games and is organized as follows. In Section 2, we introduce some notation
for games and multivalued functions that we will use throughout this paper.
In Section 3, we introduce the setting of delegated game playing for this pa-
per. We then formally dene and further motivate the concept of safe Pareto
4
Player 2's rep.
DL RL
Player 1's rep.DL3;3 2;0
RL 0;2 1;1
Table 2: A safe Pareto improvement for the Demand Game
improvements. We also dene and give an example of unilateral SPIs. These
are SPIs that require only one of the players to commit their representative
to a new action set and utility function. In Section 4, we brie
y review the
concepts of program games and program equilibrium and show that SPIs can
be implemented as program equilibria. In Section 5, we introduce a notion
of outcome correspondence between games. This relation expresses the orig-
inal players' beliefs about similarities between how the representatives play
dierent games. In our example, the Demand Game of Table 1 (arguably)
corresponds to the game of Table 2 in that the representatives (arguably)
would play (DM ;DM) in the original game if and only if they play (DL ;DL)
in the new game, and so forth. We also show some basic results (re
exivity,
transitivity, etc.) about the outcome correspondence relation on games. In
Section 6 we show that the notion of outcome correspondence is central to
deriving SPIs. In particular, we show that a game sis an SPI on another
game  if and only if there is a Pareto-improving outcome correspondence
relation between sand .
To derive SPIs, we need to make some assumptions about outcome cor-
respondence, i.e., about which games are played in similar ways by repre-
sentatives. We give two very weak assumptions of this type in Section 7.
The rst is that the representatives' play is invariant under the removal of
strictly dominated strategies. For example, we assume that in the Demand
Game the representatives only play DM and RM. Moreover we assume that
we could remove DL and RL from the game and the representatives would
still play the same strategies as in the original Demand Game with certainty.
The second assumption is that the representatives play isomorphic games
isomorphically. For example, once DL and RL are removed for both players
from the Demand Game, the Demand Game is isomorphic to the game in
Table 2 such that we might expect them to be played isomorphically. In
Section 7.5, we derive a few SPIs { including our SPI for the Demand Game
{ using these assumptions. Section 8 shows that determining whether there
exists an SPI based on these assumptions is NP-complete. Section 9 con-
5
siders a dierent setting in which we allow the original players to let the
representatives choose from newly constructed strategies whose correspond-
ing outcomes map arbitrarily onto feasible payo vectors from the original
game. In this new setting, nding SPIs can be done in polynomial time. We
conclude by discussing the problem of selecting between dierent SPIs on a
given game (Section 10) and giving some ideas for directions for future work
(Section 11).
2 Preliminaries
2.1 Games
We here give some basic game-theoretic denitions. We assume the reader
to be familiar with most of these concepts and with game theory more
generally.
Ann-player (normal-form) game is a tuple (A;u) of a setA=A1:::
Anof(pure) strategy proles (or outcomes) and a function u:A!Rnthat
assigns to each outcome a utility for each player. The Prisoner's Dilemma
shown in Table 3 is a classic example of a game. The Demand Game of
Table 1 is another example of a game that we will use throughout this
paper.
Instead of ( A;u) we will also write ( A1;:::;An;u1;:::;un). We also write
Aiforj6=iAi, i.e., for the Cartesian product of the action sets of all players
other than i. We similarly write uiandaifor vectors containing utility
functions and actions, respectively, for all players but i. Ifuiis a utility
function and uiis a vector of utility functions for all players other than i,
then (even if i6= 1) we use ( ui;ui) for the full vector of utility functions
where Player ihas utility function uiand the other players have utility
functions as specied by ui. We use (Ai;Ai) and (ai;ai) analogously.
We say that ai2Aistrictly dominates a0
i2Aiif for allai2Ai,
ui(ai;ai)> ui(a0
i;ai). For example, in the Prisoner's Dilemma, Defect
strictly dominates Cooperate for both players. As noted earlier, DM and
RM strictly dominate DL and RL for both players.
For any given game  = ( A;u), we will call any game 0= (A0;u0) a
subset game of  ifA0
iAifori= 1;:::;n . Note that a subset game may
assign dierent utilities to outcomes than the original game. For example,
the game of Table 2 is a subset game of the Demand Game.
We say that some utility vector y2Rnis a Pareto improvement on
(or is Pareto-better than) y02Rnifyiy0
ifori= 1;:::;n . We will also
denote this by yy0. Note that, contrary to convention, we allow y=y0.
6
Player 2
Cooperate Defect
Player 1Cooperate 3;3 1;4
Defect 4;1 2;2
Table 3: The Prisoner's Dilemma
Whenever we require one of the inequalities to be strict, we will say that yis
a strict Pareto improvement on y0. In a given game, we will also say that an
outcome ais a Pareto improvement on another outcome a0ifu(a)u(a0).
We say that yis Pareto-optimal or Pareto-ecient relative to some SRn
if there is no element of Sthat strictly Pareto-dominates y.
Let  = (A;u) and 0= (A0;u0) be twon-player games. Then we call
ann-tuple of functions  = ( i:Ai!A0
i)i=1;:::;na(game) isomorphism
between  and 0if there are vectors 2Rn
+andc2Rnsuch that
u(a1;:::;an) =u0(1(a1);:::;n(an)) +c
for all a2A. If there is an isomorphism between  and 0, we call  and
0isomorphic . For example, if we let  be the Demand Game and sthe
subset game of Table 2, then ( fDM;RMg;fDM;RMg;u) is isomorphic to
svia the isomorphism  with  i(DM) = DL and  i(RM) = RL and the
constants = (1;1) and c= (0;0).
2.2 Multivalued functions
For setsMandN, amulti-valued function :M(Nis a function which
maps each element m2Mto a set (m)N. For a subset QM, we
dene
(Q):=[
m2Q(m):
Note that ( Q)Nand that (;) =;. For any set M, we dene the
identity function id M:M(M:m!fmg. Also, for two sets MandN,
we dene all M;N:M(N:m7!N. We dene the inverse
1:N(M:n7!fm2Mjn2(m)g:
Note that 1(;) =;for any multi-valued function . For sets M,N
andQand functions : M(Nand :N(Q, we dene the composite
7
 :M(Q:m7! ((m)). As with regular functions, composition
of multi-valued functions is associative. We say that : M(Nissingle-
valued ifj(m)j= 1 for all m2M. Whenever a multi-valued function is
single-valued, we can apply many of the terms for regular functions. For ex-
ample, we will take injectivity, surjectivity, and bijectivity for single-valued
functions to have the usual meaning. We will never apply these notions to
non-single-valued functions.
3 Delegation and safe Pareto improvements
We consider a setting in which a given game  is played through what we
will call representatives . For example, the representatives could be humans
whose behavior is determined or incentivized by some contract  a la the
principal{agent literature [21].
We imagine that one way in which the representatives can be instructed
is to in turn play a subset game s= (As
1A1;:::;As
nAn;us) of the orig-
inal game, without necessarily specifying a strategy or algorithm for solving
such a game . We emphasize, again, that usis allowed to be a vector of
entirely dierent utility functions. For any subset game s, we denote by
(s) the outcome that arises if the representatives play the subset game
sof . Because it is unclear what the right choice is in many games, the
original players might be uncertain about (s). We will therefore model
each (s) as a random variable. We will typically imagine that the rep-
resentatives play  in the usual way, i.e., that they are not able to make
further commitments or delegate again. For example, we imagine that if
is the Prisoner's Dilemma, then () = (Defect ;Defect) with certainty.
The original players trust their representatives to the extent that we
take () to be a default way for the game to played for any . That is,
by default the original players tell their representatives to play the game as
given. For example, in the Demand Game, it is not clear what the right
action is. Thus, if one can simply delegate the decision to someone with
more relevant expertise, that is the rst option one would consider.
We are interested in whether and how the original players can jointly
Pareto-improve on the default. Of course, one option is to compute the
expected utilities in the default ( E[u(())]) and then let the representatives
play a distribution over outcomes whose expected utility exceeds that default
expected utility. However, this is unrealistic if  is a complex game with
potentially many Nash equilibria. For one, the precise point of delegation
is that the original players are unable or unwilling to properly evaluate .
8
Second, there is no widely agreed upon, universal procedure for selecting
an action in the face of equilibrium selection problems. In such cases, the
original players may in practice be unable to form a probability distribution
over (). This type of uncertainty is sometimes referred to as Knightian
uncertainty, following Knight's [19] distinction between the concepts of risk
and uncertainty.
We address this problem in a typical way. Essentially, we require of any
attempted improvement that it incurs no regret in the worst-case. That
is, we are interested in subset games sthat are Pareto improvements with
certainty under weak and purely qualitative assumptions about .2In par-
ticular, in Section 7, we will introduce the assumptions that the represen-
tatives do not play strictly dominated actions and play isomorphic games
isomorphically.
Denition 1. Letsbe a subset game of . We say sis a safe Pareto
improvement (SPI) on ifu((s))u(()) with certainty. We say
that sis a strict SPI if furthermore, there is a player is.t.ui((s))>
ui((s))with positive probability.
For example, in the introduction we have argued that the subset game
in Table 2 is a strict SPI on the Demand Game (Table 1). Less interestingly,
if we let  = ( A;u) be the Prisoner's Dilemma (Table 3), then we might
expect (fCooperateg;fCooperateg;u) to be an SPI on . After all, we
might expect that () = (Defect ;Defect) with certainty, while it must be
(fCooperateg;fCooperateg;u) = (Cooperate ;Cooperate) with certainty,
for lack of alternatives. Both players prefer mutual cooperation over mutual
defection.
3.1 Unilateral SPIs
Both SPIs given above require bothplayers to let their representatives choose
from restricted strategy sets to maximize something other than the original
player's utility function.
2Here is another way of putting this. When one of the players ideliberates whether
she would rather have the representatives play () or (s), we could imagine that the
agent has a number of possible of models of how the representatives () operate. Absent a
probability distribution over models, the only widely accepted circumstance under which
she can make such comparisons is decision-theoretic dominance [30, Sect. 3.1]: she should
prefer (s) ifui((s))ui(()) under allmodels and ui((s))> ui(()) under
at least some model of .
9
Denition 2. We will call a subset game s= (As;us)of = (A;u)
unilateral if for all but one i2f1;:::;ngit holds that As
i=Aiandus
i=ui.
Consequently, if a unilateral subset game sofis also an SPI for , we
callsa unilateral SPI.
We now give an example of a unilateral SPI using the Complicated Temp-
tation Game, formalized as a normal-form game in Table 4. (We give the
not-so complicated Temptation Game { in which we can only give a trivial
example of SPIs { in Section 7.5.) Two players each deploy a robot. Each
of the robots faces two choices in parallel. First, each can choose whether
to work on Project 1 or Project 2. Player 1 values Project 1 higher and
Player 2 values Project 2 higher, but the robots are more eective if they
work on the same project. To complete the task, the two robots need to
share a resource. Robot 2 manages the resource and can choose whether
to control Robot 1's access tightly (e.g., by frequently checking on the re-
source, or requiring Robot 1 to demonstrate a need for the resource) or give
Robot 1 relatively free access. Controlling access tightly decreases the e-
ciency of both robots, though the exact costs depend on which projects the
robots are working on. Robot 1 can choose between using the resource as
intended by Robot 2; or give in to the temptation of trying to steal as much
of the resource as possible to use it for other purposes. Regardless of what
Robot 2 does (in particular, regardless of whether Robot 2 controls access
or not), Player 1 prefers trying to steal. In fact, if Robot 2 controls access
and Robot 1 refrains from theft, they never get anything done. Given that
Robot 1 tries to steal, Player 2 prefers his Robot 2 to control access. As
usual we assume that the original players can instruct their robots to play
arbitrary subset games of  (without specifying an algorithm for solving
such a game) and that they can give such instructions conditional on the
other player providing an analogous instruction.
Player 1 has a unilateral SPI in the Complicated Temptation Game. In
particular, Player 1 can commit her representative to play only from R1
andR2and to assign utilities us
1(R1;F1) =u1(T1;C1) = 2,us
1(R1;F2) =
u1(T1;C2) = 1,us
1(R2;F1) =u1(T2;C1) = 1, andus
1(R2;F2) =u1(T2;C2) =
4; otherwise us
1does not dier from u1. The resulting SPI is given in Table 5.
In this subset game, Player 2's representative { knowing that Player 1's
representative will only play from R1andR2{ will choose from F1andF2
(sinceF1andF2strictly dominate C1andC2in Table 5). Now notice that
the remaining subset game is isomorphic to the ( fT1;T2g;fC1;C2g) subset
game of the original Complicated Temptation Game, where T1maps toR1
andT2maps toR2for both Player 1, and C1maps toF1andC2maps to
10
Player 2
C1 C2 F1 F2
Player 1T1 2;4 1;1 6;0 6;0
T2 1;1 4;2 6;0 6;0
R1 0;0 0;0 3;5 3;2
R2 0;0 0;0 2;2 5;3
Table 4: Complicated Temptation Game
Player 2
C1 C2 F1 F2
Player 1R1 0;0 0;0 2;5 1;2
R2 0;0 0;0 1;2 4;3
Table 5: Safe Pareto improvement for the Complicated Temptation Game
F2for Player 2. Player 1's representative's utilities have been set to be the
same between the two; and Player 2's utilities happen to be the same up to
a constant (1) between the two subset games. Thus, we might expect that
if () = ( T1;C1), then (s) = (R1;F1), and so on. Finally, notice that
u(R1;F1)u(T1;C1) and so on. Hence, Table 5 is indeed an SPI on the
Complicated Temptation Game.
Such unilateral changes are particularly interesting because they only
require one of the players to be able to credibly delegate. That is, it's
enough for a single player to instruct their representative to choose from a
restricted action set to maximize a new utility function. The other players
can simply instruct their representatives to play the game in the normal way
(i.e., maximizing the respective players' original utility functions without
restrictions on the action set). In fact, we may also imagine that only one
playeridelegates at all, while the other players choose an action themselves,
after observing Player i's instruction to her representative.
One may object that in a situation where only one player can commit,
the sole player with commitment power can simply play the meta game as a
standard unilateral commitment (Stackelberg) game [as studied by, e.g., 8,
38, 43] or perhaps as a rst mover in a sequential game (as solved by back-
ward induction), without bothering with any Pareto conditions. For exam-
ple, in the Complicated Temptation Game, Player 1 could simply commit
11
Player 1 Player 2
Instruction 1 Instruction 2
Representative 1 Representative 2
Pl./Rep. 1's strategy Pl./Rep. 2's strategy
Player 1's utility Player 2's utility
Figure 1: A diagram describing the meta-game in the case of two players.
her representative to play R2.
While this is true in many situations, it is easy come up with scenarios
in which it is not and in which SPIs are relevant. For one, we may imag-
ine that only one Player has the ne-grained commitment and delegation
abilities needed for SPIs but that the other players can still credibly com-
mit (or are already credibly committed) against any \commitment trickery"
that clearly leaves them worse o. For instance, many people appear cred-
ibly committed by intuitions about fairness and retributivist instincts and
emotions [see, e.g., 31, Chapter 6, especially the section \The Doomsday
Machine"], but are nonetheless very limited in their abilities to commit.
Second, we may imagine that the players who cannot commit are subject
to reputation eects. Then they might want to build a reputation of re-
sisting coercion. (In contrast, we expect a reputation of playing along with
unilateral SPIs to be benecial in most such situations.)
4 Implementing safe Pareto improvements as pro-
gram equilibria
So far, we have been vague about the details of the strategic situation that
theoriginal players face in instructing their representatives. From what
sets of actions can they choose? How can they jointly let the representatives
play some new subset game s? Are SPIs Nash equilibria of the meta game
12
played by the representatives? If I instruct my representative to play the
SPI of Table 2 in the Demand Game, could my opponent not instruct her
representative to play DM?
In this section, we brie
y describe one way to ll this gap by discussing
the concept of program games and program equilibrium [33, Sect. 10.4, 39,
12, 3, 10, 26]. This section is essential to understanding why SPIs (especially
omnilateral ones) are relevant. However, the remaining technical content of
this paper does not rely on this section and the main ideas presented here
are straightforward from previous work. We therefore only give an informal
exposition. For formal detail, see Appendix A.
For any game  = ( A;u), the program equilibrium literature considers
the following meta game. First, each player ichooses from a set of com-
puter programs. Each program then receives as input a vector containing
everyone else's chosen program. Each player i's program then returns an
action from Ai, playeri's set of actions in . Together these actions then
form an outcome a2Aof the original game. Finally, the utilities u(a) are
realized according to the utility function of . The meta game can be ana-
lyzed like any other game. Its Nash equilibria are called program equilibria .
Importantly, the program equilibria can implement payos not implemented
by any Nash equilibria of  itself. For example, in the Prisoner's Dilemma,
both players can submit a program that says: \If the opponent's chosen
computer program is equal to this computer program, Cooperate; otherwise
Defect." [23, 17, 33, Sect. 10.4, 39] This is a program equilibrium which
implements mutual cooperation.
In the setting for our paper, we similarly imagine that each player ican
choose from a set of programs that in turn choose from Ai. However, the
types of program that we have in mind here are more sophisticated than
those typically considered in the program equilibrium literature. Speci-
cally we imagine that the programs are executed by intelligent representa-
tives who are themselves able to competently choose an action for player i
in any given game s, without the original player having to describe how
this choice is to be made. The original player may not even understand
much about this program other than that it generally plays well. Thus, in
addition to the elementary instructions used in a typical computer program
(branches, comparisons, arithmetic operations, etc.), we allow player ito
use an instruction \Play  i(s)" in the program she submits. To jointly let
the representatives play, e.g., the SPI sof Table 2 on the Demand Game
of Table 1, the representatives can both use an instruction that says, \If
the opponent's chosen program is analogous to this one, play  i(s); oth-
erwise play DM". Assuming some minimal rationality requirements on the
13
representatives (i.e., on how \play  i(s)" is implemented), this is a Nash
equilibrium. Figure 1 illustrates how (in the two-player case) the meta game
between the original players is intended to work.
For illustration consider the following two real-world instantiations of
this setup. First, we might imagine that the original players hire human
representatives. Each player species, e.g., via monetary incentives, how
she wants her representative to act by some contract. For example, a player
might contract her representative to play a particular action; or she might
specify in her contract a function ( us
i) over outcomes according to which she
will pay the representative after an outcome is obtained. Moreover, these
contracts might refer to one another. For example, Player 1's contract with
her representative might specify that if Player 2 and his representative use
an analogous contract, then she will pay her representative according to
Table 2. As a second, more futuristic scenario, you could imagine that the
representatives are software agents whose goals are specied by so-called
smart contracts , i.e., computer programs implemented on a blockchain to be
publicly veriable [6, 34].
To justify our study of SPIs, we prove that every SPI is played in some
program equilibrium:
Theorem 1. Letbe a game and sbe an SPI of . Now consider a
program game on , where each player ican choose from a set of computer
programs that output actions for . In addition to the normal kind of in-
structions, we allow the use of the command \play i(0)" for any subset
game 0of. Finally, assume that () guarantees each player iat least
that player's minimax utility (a.k.a. threat point) in the base game . Then
(s)is played in a program equilibrium, i.e., in a Nash equilibrium of the
program game.
We prove this in Appendix A.
As an alternative to having the original players choose contracts sepa-
rately, we could imagine the use of jointly signed contracts which only come
into eect once signed by all players [cf. 18, 24]. Also compare earlier work
by Sen [37] and Raub [32], which we discuss in Appendix B.
5 Outcome correspondence between games
In this section, we introduce a notion of outcome correspondence, which we
will see is essential to constructing SPIs.
14
Denition 3. Consider two games  = (A1;:::;An;u)and0= (A0
1;:::;A0
n;u0).
We write 0for:A(A0if(0)2(()) with certainty.
Note that 0is a statement about , i.e., about how the represen-
tatives choose. Whether such a statement holds generally depends on the
specic representatives being used. In Section 7, we describe two general
circumstances under which it seems plausible that  0. For example, if
two games  and 0are isomorphic, then one might expect  0, where
 is the isomorphism between the two games.
We now illustrate this notation using our discussion from the Demand
Game. Let  be the Demand Game of Table 1. First, it seems plausible that
 is in some sense equivalent to 0, where 0= (fDM;RM;u) is the game
that results from removing DL and RL for both players from . Again,
strict dominance could be given as an argument. We can now formalize
this as 0, where (a1;a2) =f(a1;a2)gifa1;a22fDM;RMgand
(a1;a2) =;otherwise. Next, it seems plausible that 0 s, where s
is the game of Table 2 and is the isomorphism between 0and s.
We now state some basic facts about the relation , many of which we
will use throughout this paper.
Lemma 2. Let = (A;u),0= (A0;u0),^ = ( ^A;^u)and;:A(A0,
 :A0(^A.
1. Re
exivity: idA, where idA:A(A:a7!fag.
2. Symmetry: If 0, then 01.
3. Transitivity: If 0and0 ^, then  ^.
4. If 0and(a)(a)for all a2A, then 0.
5.allA;A00, where allA;A0:A(A0:a7!A0.
6. If 0and(a) =;, then ()6=awith certainty.
7. If 0and1(a0) =;, then (0)6=a0with certainty.
Proof. 1. By re
exivity of equality, () = () with certainty. Hence,
()2idA(()) by denition of id A. Therefore, idA by deni-
tion of, as claimed.
2. 0means that (0)2(()) with certainty. Thus,
()2fa2Aj(0)2(a)g= 1((0));
15
where equality is by the denition of the inverse of multi-valued func-
tions. We conclude (by denition of ) that 01 as claimed.
3. If 0, 0 ^, then by denition of , (i) (0)2(())
and (ii) ( ^)2 ((0)), both with certainty. The former (i) implies
f(0)g(()). Hence,
 ((0)) = (f(0)g) ((())) :
With ii, it follows that ( ^)2 ((())) with certainty. By deni-
tion,  ^ as claimed.
4. It is
(0)2(())(())
with certainty. Thus, by denition  0.
5. By denition of , it is (0)2A0with certainty. By denition
of allA;A0, it is all A;A0(()) =A0with certainty. Hence, (0)2
allA;A0(()) with certainty. We conclude that  allA;A00as claimed.
6. With certainty, (0)2(()) (by assumption). Also, with cer-
tainty (0)=2;. Hence, (()) 6=;with certainty. We conclude
that ()6=awith certainty.
7. If 0, then by re
exivity of (Lemma 2.1) 01. If
1(a0) =;, then by Lemma 2.6, (0)6=a0with certainty.
Items 1{3 show that has properties resembling those of an equivalence
relation. Note, however, that since is not a binary relationship, itself
cannot be an equivalence relation in the usual sense. We can construct equiv-
alence relations, though, by existentially quantifying over the multivalued
function. For example, we might dene an equivalence relation Ron games,
where (;0)2Rif and only if there is a single-valued bijection  such that
0.3Item 4 states that if we can make an outcome correspondence
claim less precise, it will still hold true. Item 5 states that in the extreme,
3Note that the fact that this is an equivalence relation relies on the following three
facts:
1. For re
exivity of R: The identity function id is a single-valued bijection.
2. For symmetry of R: If  is a single-valued bijection, so is 1.
3. For transitivity of R: If ; are bijections, so is .
16
it is always allA;A00, where all A;A0is the trivial, maximally imprecise
outcome correspondence function that confers no information. Item 6 shows
thatcan be used to express the elimination of outcomes, i.e., the belief
that a particular outcome (or strategy) will never occur.
Besides an equivalence relation, we can also use with quantication
over the respective outcome correspondence function to construct (non-
symmetric) preorders over games, i.e., relations that are transitive and re-

exive (but not symmetric or antisymmetric). Most importantly, we can
construct a preorder on games where  0if 0for a  that
always increases every player's utilities.
6 Safe Pareto improvements through outcome cor-
respondence
We now show that as advertised, outcome correspondence is closely tied to
SPIs. The following theorem shows not only how outcome correspondences
can be used to nd (and prove) SPIs. It also shows that any SPI requires
an outcome correspondence relation via a Pareto-improving correspondence
function.
Denition 4. Let = (A;u)be a game and s= (As;us)be a subset game
of. Further let :A!Asbe such that 0. We call a Pareto-
improving outcome correspondence (function) if u(as)u(a)for all a2A
and all as2(a).
Theorem 3. Let = (A;u)be a game and s= (As;us)be a subset game
of. Then sis an SPI on if and only if there is a Pareto-improving
outcome correspondence from tos.
Proof.(: By denition, (s)2(()) with certainty. Hence, for i=
1;2,
ui((s))2ui((()))
with certainty. Hence, by assumption about , with certainty, ui((s))
ui(()).
): Assume that ui(())ui((s)) with certainty for i= 1;2. We
dene
:A!As:a7!fas2Asju(as)u(a)g:
It is immediately obvious that  is Pareto-improving as required. Also,
whenever () = aand (s) =asfor any a2Aandas2As, it is (by
17
assumption) with certainty u(as)u(a). Thus, by denition of , it holds
thatas2(a). We conclude that  sas claimed.
Note that the theorem concerns weak SPIs and therefore allows the case
where with certainty u(()) = u((s)). To show that some sis astrict
SPI, we need additional information about which outcomes occur with posi-
tive probability. This, too, can be expressed via our outcome correspondence
relation. However, since this is cumbersome, we will not formally address
strictness much to keep things simple.4
We now illustrate how outcome correspondences can be used to derive
the SPI for the Demand Game from the introduction as per Theorem 3.
Of course, at this point we have not made any assumptions about when
games are equivalent. We will introduce some in the following section. Nev-
ertheless, we can already sketch the argument using the specic outcome
correspondences that we have given intuitive arguments for. Let  again be
the Demand Game of Table 1. Then, as we have argued,  0, where
0= (fDM;RMg;fDM;RMg;u) is the game that results from removing DL
and RL for both players; and ( a1;a2) =f(a1;a2)gifa1;a22fDM;RMg
and (a1;a2) =;otherwise. In a second step, 0 s, where sis the
game of Table 2 and is the isomorphism between 0and s. Finally,
transitivity (Lemma 2.3) implies that   s. To see that  is
Pareto-improving for the original utility functions of , notice that  does
not change utilities at all. The correspondence function maps the con
ict
outcome (DM ;DM) onto the outcome (DL ;DL), which is better for both
original players. Other than that, , too, does not change the utilities.
Hence,  is Pareto-improving. By Theorem 3, sis therefore an SPI on
.
In principle, Theorem 3 does not hinge on () and (s) resulting from
playing games. An analogous result holds for any random variables over A
andAs. In particular, this means that Theorem 3 applies also if the rep-
resentatives receive other kinds of instructions (cf. Section 4). However, it
seems hard to establish non-trivial outcome correspondences between ()
and other types of instructions. Still, the use of more complicated instruc-
tions can be used to derive dierent kinds of SPIs. For example, if there are
dierent game SPIs, then the original players could tell their representatives
to randomize between them in a coordinated way.
4For an SPI sto be also be a strict SPI on , there must be aswhich strictly Pareto-
dominates asuch that for all  with  s, it must be as2(a).
18
7 Assumptions about outcome correspondence
To make any claims about how the original players should play the meta-
game, i.e., about what instructions they should submit, we generally need to
make assumptions about how the representatives choose and (by Theorem 3)
about outcome correspondence in particular.5We here make two fairly weak
assumptions.
7.1 Elimination
Our rst is that the representatives never play strictly dominated actions
and that removing them does not aect what the representatives would
choose.
Assumption 1. Let = (A;u)be an arbitrary n-player game where A1;:::;An
are pairwise disjoint, and let ~ai2Aibe strictly dominated by some other
strategy in Ai. Then (Ai;Aif~aig;uj(Ai;Aif~aig)), where for all
ai2Ai,(~ai;ai) =;and(ai;ai) =f(ai;ai)gwheneverai6= ~ai.
Assumption 1 expresses that representatives should never play strictly
dominated strategies. Moreover, it states that we can remove strictly dom-
inated strategies from a game and the resulting game will be played in the
same way by the representatives. For example, this implies that when eval-
uating a strategy ai, the representatives do not take into account how many
other strategies aistrictly dominates. Assumption 1 also allows (via Tran-
sitivity ofas per Lemma 2.3) the iterated removal of strictly dominated
strategies. The notion that we can (iteratively) remove strictly dominated
strategies is common in game theory [28, 20, 27, Section 2.9, Chapter 12]
and has rarely been questioned. It is also implicit in the solution concept
of Nash equilibrium { if a strategy is removed by iterated strict dominance,
that strategy is played in no Nash equilibrium. However, like the concept of
Nash equilibrium, the elimination of strictly dominated strategies becomes
implausible if the game is not played in the usual way. In particular, for
Assumption 1 to hold, we will in most games  have to assume that the
representatives cannot in turn make credible commitments (or delegate to
further subrepresentatives) or play the game iteratively [2].
5There are trivial, uninteresting cases in which no assumptions are needed. In particu-
lar, if a game  has an outcome athat Pareto dominates all other outcomes of the game,
then (by Lemma 2.5 with Theorem 3) any game s= (As=fag;us) is an SPI on .
19
7.2 Isomorphisms
Our second assumption is that the representatives play isomorphic games
isomorphically when those games are fully reduced.
Assumption 2. Let = (A;u)and0= (A0;u0)be two games that do not
contain strictly dominated actions. If and0are isomorphic, then there
exists an isomorphism between and0such that 0.
Similar desiderata have been discussed in the context of equilibrium se-
lection, e.g., by Harsanyi and Selten [16, Chapter 3.4] [cf. 40, for a discussion
in the context of fully cooperative multi-agent reinforcement learning].
Note that if there are multiple game isomorphisms, then we assume
outcome correspondence for only one of them. This is necessary for the
assumption to be satisable in the case of games with action symmetries.
(Of course, such games are not the focus of this paper.) For example, let
be Rock{Paper{Scissors. Then  is isomorphic to itself via the function
that for both players maps Rock to Paper, Paper to Scissors, and Scissors to
Rock. But if it were  , then this would mean that if the representatives
play Rock in Rock{Paper{Scissors, they play Paper in Rock{Paper{Scissors.
Contradiction! We will argue for the consistency of our version of the as-
sumption in Section 7.3. Notice also that we make the assumption only for
reduced games. This relates to the previous point about action-symmetric
games. For example, consider two versions of Rock{Paper{Scissors and
assume that in both versions both players have an additional strictly domi-
nated action that breaks the action symmetries e.g., the action, \resign and
give the opponent $10 if they play Rock/Paper". Then there would only be
one isomorphism between these two games (which maps Rock to Paper, Pa-
per to Scissors, and Scissors to Rock for both players). However, in light of
Assumption 1, it seems problematic to assume that these strictly dominated
actions restrict the outcome correspondences between these two games.6
One might worry that reasoning about the existence of multiple iso-
morphisms renders it intractable to deal with outcome correspondences as
implied by Assumption 2, and in particular that it might make it impossible
to tell whether a particular game is an SPI. However, one can intuitively see
that the dierent isomorphisms between two games do analogous operations.
In particular, it turns out that if one isomorphism is Pareto-improving, then
they all are:
6In fact, depending on formal details we omit throughout this paper { how to quantify
over games in these assumptions, what type of objects actions are, etc. a more general
version of Assumption 2 threatens to yield a contradiction with Assumption 1 again.
20
Lemma 4. Letand be isomorphisms between and0. Ifis (strictly)
Pareto-improving, then so is .
We prove Lemma 4 in Appendix C.
Lemma 4 will allow us to conclude from the existence of a Pareto-
improving isomorphism  that there is a Pareto-improving s.t.   0
by Assumption 2, even if there are multiple isomorphisms between  and
0. In the following, we can therefore aord to be lax about our ignorance
(in some games) about which outcome isomorphism induces outcome equiva-
lence. We will therefore generally write \ 0by Assumption 2" as short
for \ is a game isomorphism between  and 0and hence by Assumption 2
there exists an isomorphism such that   0".
One could criticize Assumption 2 by referring to focal points (introduced
by Schelling [35, pp. 54{58]) as an example where context and labels of
strategies matter. A possible response might be that in games where context
plays a role, that context should be included as additional information and
not be considered part of ( A;u). Assumption 2 would then either not apply
to such games with (relevant) context or would require one to, in some way,
translate the context along with the strategies. However, in this paper we
will not formalize context, and assume that there is no decision-relevant
context.
7.3 Consistency of Assumptions 1 and 2
We will now argue that there exist representatives that indeed satisfy As-
sumptions 1 and 2, both to provide intuition and because our results would
not be valuable if Assumptions 1 and 2 were inconsistent. We will only
sketch the argument informally. To make the argument formal, we would
need to specify in more detail what the set of games looks like and in par-
ticular what the objects of the action sets are.
Imagine that for each player ithere is a book7that on each page describes
a normal-form game that does not have any strictly dominated strategies.
The actions have consecutive integer labels. Importantly, the book contains
no pair of games that are isomorphic to each other. Moreover, for every fully
reduced game, the book contains a game that is isomorphic to this game.
(Unless we strongly restrict the set of games under consideration, the book
must therefore have innitely many pages.) We imagine that each player's
book contains the same set of games. On each page, the book for Player i
7The use of a book as illustration is inspired by Binmore [4, Section 1].
21
recommends one of the actions of Player ito be taken deterministically.8
Each representative owns a potentially dierent version of this book and
uses it as follows to play a given game . First the given game is fully
reduced by iterated strict dominance. They then look up the unique game
in the book that is isomorphic to redand map the action labels in red
onto the integer labels of the game in the book via some isomorphism. If
there are multiple isomorphisms from redto the relevant page in the book,
then all representatives decide between them using the same deterministic
procedure. Finally they choose the action recommended by the book.
It is left to show a pair of representatives  thus specied satises As-
sumptions 1 and 2. We rst argue that Assumption 1 is satised. Let  be
a game and let 0be a game that arises from removing a strictly dominated
action from . By the well known path independence of iterated elimination
of strictly dominated strategies [1, 15, 28], fully reducing  and 0results in
the same game. Hence, the representatives play the same actions in  and
0.
Second, we argue that Assumption 2 is satised. Let us say  and ^
are fully reduced and isomorphic. Then it is easy to see that each player i,
plays  and ^ based on the same page of their book. Let the game on that
book page be ~. Let :A!~Aand 0:A0!~Abe the bijections used by
the representatives to translate actions in  and 0, respectively, to labels
in~. Then if the representatives take actions ain , the actions ( a) are
the ones specied by the book for ~, and hence the actions ^1((a)) are
played in 0. Thus ^1^. It is easy to see that ^1 is a game
isomorphism between  and ^.
7.4 Discussion of alternatives to Assumptions 1 and 2
One could try to use principles other than Assumptions 1 and 2. We here
give some considerations. First, game theorists have also considered the
iterated elimination of weakly dominated strategies [14, 22, Section 4.11].
Unfortunately, the iterated removal of weakly dominated strategies is path-
dependent [20, Section 2.7.B, 5, Section 5.2, 27, Section 12.3]. That is,
for some games, iterated removal of weakly dominated strategies can lead
to dierent subset games, depending on which weakly dominated strategy
8Of course, in many cases (such as Rock{Paper{Scissors) it is implausible for the
players to choose deterministically. The idea is that in such games the randomization was
performed in the process of printing. If the book is intended to be used multiple times, we
may imagine that a sequence of (randomly generated) actions is provided ( a la the RAND
Corporation's book of random numbers).
22
one chooses to eliminate at any stage. A straightforward extension of As-
sumption 1 to allow the elimination of weakly dominated strategies would
therefore be inconsistent. The iterated removal of strictly dominated strate-
gies, on the other hand, is path- independent, and in the 2-player case al-
ways eliminates exactly the non- rationalizable strategies [1, 15, 28]. Many
other dominance concepts have been shown to be path independent. For an
overview, see Apt [1]. We could have used any of these path-independent
dominance concepts.
With Assumptions 1 and 2, all our outcome correspondence functions
are either 1-to-1 or 1-to-0. Other elimination assumptions could involve the
use of many-to-1 or even many-to-many functions. In general, such func-
tions are needed when a strategy ~ aican be eliminated to obtain a strategi-
cally equivalent game, but in the original game ~ aimay still be played. The
simplest example would be the elimination of payo-equivalent strategies.
Imagine that in some game  for all opponent strategies ai2Aiit is the
case that u(~ai;ai) =u(^ai;ai) and that there are no other strategies that
are similarly payo-equivalent to ~ aiand ^ai. Then one would assume that
(Aif~aig;Ai;u), where  maps ~ aiontof^aigand otherwise  is just
the identity function.
7.5 Examples
In this section, we use Assumptions 1 and 2 to formally prove a few SPIs.
Proposition (Example) 5. Letbe the Prisoner's Dilemma (Table 3) and
s= (As
1;As
2;us
1;us
2)be any subset game of withAs
1=As
2=fCooperateg.
Then under Assumption 1, sis a strict SPI on .
Proof. By applying Assumption 1 twice and Transitivity once,  D,
where D= (fDefectg;fDefectg;u) and (Defect ;Defect) =f(Defect;Defect)g
and (a1;a2) =;for all (a1;a2)6= (Defect;Defect). By Lemma 2.5, we fur-
ther obtain Dalls, where sis as described in the proposition. Hence,
by transitivity,  alls. It is easy to verify that the function all  is
Pareto-improving.
Proposition (Example) 6. Letbe the Demand Game of Table 1 and s
be the subset game described in Table 2. Under Assumptions 1 and 2, sis
an SPI on . Further, if P(()=(DM ;DM))>0, then sis a strict SPI.
Proof. Let (A1;A2;u1;u2) = . We can repeatedly apply Assumption 1 to
eliminate from  the strategies DL and RL for both players. We can then ap-
ply Lemma 2.3 (Transitivity) to obtain  ^, where ^ = (fDM;RMg;fDM;RMg;u1;u2)
23
Player 2's representative
C F
Player 1's rep.T 1;2 5;1
R 0;0 4;4
Table 6: Simple Temptation Game
and
(a1;a2) =f(a1;a2)gifa1;a22fDM;RMg
; otherwise:
Next, by Assumption 2, ^ s, where i(DM) = DL and i(RM) = RL
fori= 1;2. We can then apply Lemma 2.3 (Transitivity) again, to infer
 s. It is easy to verify that for all ( a1;a2)2A1A2, it is for all
(as
1;as
2)2 ((s)) the case that u(as
1;as
2)u(a1;a2).
Next, we give two examples of unilateral SPIs. We start with an example
that is trivial in that the original player instructs her resentatives to take a
specic action. We then give the SPI for the Complicated Temptation game
as a non-trivial example.
Consider the Temptation Game given in Table 6. In this game, Player
1'sT(for Temptation) strictly dominates R. OnceRis removed, Player 2
prefersC. Hence, this game is strict-dominance solvable to ( T;C). Player 1
can safely Pareto-improve on this result by telling her representative to play
R, since Player 2's best response to RisFandu(R;F) = (4;4)>(1;2) =
u(T;C). We now show this formally.
Proposition (Example) 7. Let = (A1;A2;u1;u2)be the game of Table 6.
Under Assumption 1, s= (fRg;A2;u1;u2)is a strict SPI on .
Proof. First consider . We can apply Assumption 1 to eliminate Player 1's
Rand then apply Assumption 1 again to the resulting game to also eliminate
Player 2'sR. By transitivity, we nd  0, where 0= (fTg;fCg;u1;u2)
and (T;C) =f(T;C)gand (A1A2f(T;C)g) =;.
Next, consider s. We can apply Assumption 1 to remove Player 2's
strategyCand nd s ^s, where ^s= ((fRg;fFg;u1;u2)) and (R;F) =
f(R;F)gand (R;C) =;.
Third, 0all^sby Lemma 2.5, where all( T;C) =f(R;F)g.
Finally, we can apply transitivity to conclude  s, where  = 1
all. It is easy to verify that ( T;C) = (R;F) and (A1A2f(R;F)g) =
;. Hence,  is Pareto-improving and so by Theorem 3, sis an SPI on .
24
Note that in this example, Player 1 simply commits to a particular strat-
egyRand Player 2 maximizes their utility given Player 1's choice. Hence,
this SPI can be justied with much simpler unilateral commitment setups
[8, 38, 43]. For example, if the Temptation Game was played as a sequential
game in which Player 1 plays rst, its unique subgame-perfect equilibrium
is (R;F).
In Table 4 we give the Complicated Temptation Game, which better
illustrates the features specic to our setup. Roughly, it is an extension of
the simpler Temptation Game of Table 6. In addition to choosing Tversus
RandCversusF, the players also have to make an additional choice (1
versus 2), which is dicult in that it cannot be solved by strict dominance.
As we have argued in Section 3.1, the game in Table 5 is a unilateral SPI
on Table 4. We can now show this formally.
Proposition (Example) 8. Letbe the Complicated Temptation Game
(Table 4) and sbe the subset game in Table 5. Under Assumptions 1 and 2,
sis a unilateral SPI on .
Proof. In , for Player 1, T1 andT2strictly dominate R1 andR2. We can
thus apply Assumption 1 to eliminate Player 1's R1 andR2. In the resulting
game, Player 2's C1andC2strictly dominate F1andF2, so one can apply
Assumption 1 again to the resulting game to also eliminate Player 2's F1and
F2. By transitivity, we nd  0, where 0= (fT1;T2g;fC1;C2g;u1;u2)
and
(a1;a2) =f(a1;a2)gifa12fT1;T2ganda22fC1;C2g
; otherwise:
Next, consider s(Table 5). We can apply Assumption 1 to remove
Player 2's strategies C1andC2and nd s ^s, where ^s= ((fR1;R2g;fF1;F2g;us
1;u2))
and
 (a1;a2) =f(a1;a2)gifa12fR1;R2ganda22fF1;F2g
; otherwise:
Third, 0^sby Assumption 2, where  decomposes into  1and
2, corresponding to the two players, respectively, where  1(Ti) =Riand
2(Ci) =Fifori= 1;2.
Finally, we can apply transitivity and the rule about symmetry and
inverses (Lemma 2.2) to conclude   1s. It is easy to verify that
 1 is Pareto-improving.
25
8 Computing safe Pareto improvements
In this section, we ask how computationally costly it is for the original
players to identify for a given game  a non-trivial SPI s. Of course, the
answer to this question depends on what the original players are willing
to assume about how their representatives act. For example, if only trivial
outcome correspondences (as per Lemma 2.1 and 2.5) are assumed, then the
decision problem is easy. Similarly, if  0for given  is hard to decide
(e.g., because it requires solving for the Nash equilibria of  and 0), then
this could trivially also make the safe Pareto improvement problem hard to
decide. We specically are interested in deciding whether a given game
has a non-trivial SPI that can be proved using only Assumptions 1 and 2,
the general properties of game correspondence (in particular Transitivity
(Lemma 2.3), Symmetry (Lemma 2.2) and Theorem 3).
Denition 5. The SPI decision problem consists in deciding for any given
, whether there is a game sand a sequence of outcome correspondences
1;:::;kand a sequence of subset games 0= ;1;:::;k= sofs.t.:
1. (Non-triviality:) If we fully reduce sandusing iterated strict dom-
inance (Assumption 1), the two resulting games are not equal. (Of
course, they are allowed to be isomorphic.)
2. Fori= 1;:::;k ,i1iiis valid by a single application of either
Assumption 1 or Assumption 2, or an application of Assumption 1 in
reverse via Lemma 2.2.
3. For all a2A, and whenever as2(kk1:::1)(a), it is the
case thatu(as)u(a).
For the strict SPI decision problem, we further require:
(4.) There is a player iand an outcome athat survives iterated elimination
of strictly dominated strategies from s.t.ui((kk1:::1)(a))>
ui(a).
For the unilateral SPI decision problem, we further require:
(5.) For all but one of the players i,ui=us
iandAi=As
i.
Many variants of this problem may be considered. For example, to match
Denition 1, the denition of the strict SPI problem assumes that all out-
comes athat survive iterated elimination occur with positive probability.
Alternatively we could have required that for demonstrating strictness, there
must be a player isuch that for alla2Athat survive iterated elimination,
ui((kk1:::1)(a))> ui(a). Similarly one may wish to nd SPIs
that are strict improvements for allplayers. We may also wish to allow the
use of the elimination of duplicate strategies (as described in Section 7.4)
26
or trivial outcome correspondence steps as per Lemma 2.5. These modi-
cations would not change the computational complexity of the problem,
nor would they require new proof ideas. One may also wish to compute all
SPIs, or { in line with multi-criteria optimization [11, 42] { all SPIs that
cannot in turn be safely improved upon. However, in general there may
exist exponentially many such SPIs. To retain any hope of developing an
ecient algorithm, one would therefore have to rst develop a more ecient
representation scheme [cf. 29, Sect. 16.4].
Theorem 9. The (strict) (unilateral) SPI decision problem is NP-complete,
even for 2-player games.
Proposition 10. For games withjA1j+:::+jAnj=mthat can be reduced
(via iterative application of Assumption 1) to a game 0withjA0
1j+:::+
jA0
nj=l, the (strict) (unilateral) SPI decision problem can be solved in
O(ml).
The full proof is tedious (see Appendix D), but the main idea is simple,
especially for omnilateral SPIs. To nd an omnilateral SPI on  based on As-
sumptions 1 and 2, one has to rst iteratively remove all strictly dominated
actions to obtain a reduced game 0, which the representatives would play
the same as the original game. This can be done in polynomial time. One
then has to map the actions 0onto the original  in such a way that each
outcome in 0is mapped onto a weakly Pareto-better outcome in . Our
proof of NP-hardness works by reducing from the subgraph isomorphism
problem, where the payo matrices of 0and  represent the adjacency
matrices of the graphs.
Besides being about a specic set of assumptions about , note that
Theorem 9 and Proposition 10 also assume that the utility function of the
game is represented explicitly in normal form as a payo matrix. If we
changed the game representation (e.g., to boolean circuits, extensive form
game trees, quantied boolean formulas, or even Turing machines), this can
aect the complexity of the SPI problem. For example, Gabarr o, Garc a,
and Serna [13] show that the game isomorphism problem on normal-form
games is equivalent to the graph isomorphism problem, while it is equivalent
to the (likely computationally harder) boolean circuit isomorphism problem
for a weighted boolean formula game representation. Solving the SPI prob-
lem requires solving a subset game isomorphism problem (see the proof of
Lemma 28 in Appendix D for more detail). We therefore suspect that the
SPI problem analogously increases in computational complexity (perhaps to
27
being p
2-complete) if we treat games in a weighted boolean formula rep-
resentation. In fact, even reducing a game using strict dominance by pure
strategies { which contributes only insignicantly to the complexity of the
SPI problem for normal-form games { is dicult in some game represen-
tations [7, Section 6]. Note, however, that for any game representation to
which 2-player normal-form games can be eciently reduced { such as, for
example, extensive-form games { the hardness result also applies.
9 Safe Pareto improvements under improved co-
ordination
9.1 Setup
In this section, we imagine that the players are able to simply invent new
token strategies with new payos that arise from mixing existing feasible
payos. To dene this formally, we rst dene for any game  = ( A;u),
C():=u((A)) =(X
a2Apau(a)X
a2Apa= 1 and8a2A:pa2[0;1])
to be the set of feasible coordinated payo vectors of , which is exactly
the convex closure of u(A), i.e., of the set of deterministically achievable
utilities of the original game.
For any game , we then imagine that in addition to subset games, the
players can let the representatives play a perfect-coordination token game
(As;us;ue), where for all i,As
i\Ai=;andus
i:As!Rare arbitrary
utility functions to be used by the representatives and ue:As!C() are
the utilities that the original players assign to the token strategies.
The instruction ( As;us;ue) lets the representatives play the game ( As;us)
as usual. However, the strategies Asare imagined to be meaningless token
strategies which do not resolve the given game . Once some token strate-
giesasare selected, these are translated into some probability distribution
overA, i.e., over outcomes of the original game, thus giving rise to (ex-
pected) utilities ue(as)2C(). These distributions and thus utilities are
specied by the original players. We here imagine in our denition of C()
that these distributions over Acould require the representatives to correlate
their choices for the original game for any given as.
Denition 6. Letbe a game. A perfect-coordination SPI for is
a perfect-coordination token game (As;us;ue)fors.t.ue((As;us))
28
u(()) with certainty. We call (As;us;ue)a strict perfect-coordination
SPI if there furthermore is a player ifor whomue
i((As;us))> ui(())
with positive probability.
As an example, imagine that  is just the DM-RM subset game of the
Demand Game of Table 1. Then, intuitively, an SPI under improved co-
ordination could consist of the original players telling the representatives,
\Play as if you were playing the DM-RM subset game of the Demand Game,
but whenever you nd yourself playing (DM ;DM), randomize [according to
some given distribution] between the other (Pareto-optimal) outcomes in-
stead". Formally, As
1=f^D;^RgandAs
2=f^D;^Rgwould then consist of
tokenized versions of the original strategies. The utility functions us
1andus
2
are then simply the same as in the original Demand Game except that they
are applied to the token strategies. For example, us(^D;^R) = (2;0). The util-
ities for the original players remove the con
ict outcome. For example, the
original players might specify ue(^D;^D) = (1;1), representing that the repre-
sentatives are supposed to play (RM ;RM) in the ( ^D;^D) case. For all other
outcomes (^a1;^a2), it must be the case that ue(^a1;^a2) =us(^a1;^a2) because
the other outcomes cannot be Pareto-improved upon. As with our earlier
SPIs for the Demand Game, Assumption 2 implies that  s, where
 maps the original con
ict outcome (DM ;DM) onto the Pareto-optimal
(^D,^D).
Relative to the SPIs considered up until now, these new types of instruc-
tions put signicant additional requirements on how the representatives in-
teract. They now have to engage in a two-round process of rst choosing and
observing one another's token strategies and then playing the corresponding
distribution over outcomes from the original game. Further, it must be the
case that this additional coordination does not aect the payos of the orig-
inal outcomes. The latter may not be the case in, e.g., the Game of Chicken.
That is, we could imagine a Game of Chicken in which coordination is pos-
sible but that the rewards of the game change if the players do coordinate.
After all, the underlying story in the Game of Chicken is that the positive
reward (admiration from peers) is attained precisely for accepting a grave
risk.
9.2 Finding safe Pareto improvement under improved rep-
resentative coordination
With these more powerful ways to instruct representatives, we can now re-
place individual outcomes of the default game ad libitum . For example,
29
in the reduced Demand Game, we singled out the outcome (DM ;DM) as
Pareto-suboptimal and replaced it by a Pareto-optimal outcome, while keep-
ing all other outcomes the same. This allows us to construct SPIs in many
more games then before.
Denition 7. The strict full-coordination SPI decision problem consists in
deciding for any given whether under Assumption 2 there is a perfect-
coordination SPI sfor.
Lemma 11. For a given n-player game and payo vector y2Rn, it can
be decided by linear programming and thus in polynomial time whether yis
Pareto-optimal in C().
For an introduction to linear programming, see, e.g., Schrijver [36]. In
short, a linear program is a specic type of constrained optimization problem
that can be solved eciently.
Proof. Finding a Pareto improvement on a given y2Rncan be formulated
as the following linear program:
Variables: pa2[0;1] for all a2A
MaximizenX
i=1 X
a2Apaui(a)!
yi
s.t.X
a2Apa= 1
X
a2Apaui(a)yifori= 1;:::;n
Based on Lemma 11, Algorithm 1 decides whether there is a strict
perfect-coordination SPI for a given game .
It is easy to see that this algorithm runs in polynomial time (in the size
of, e.g., the normal form representation of the game). It is also correct: if it
returns True, simply replace the Pareto-suboptimal outcome while keeping
all other outcomes the same; if it returns False, then all outcomes are Pareto-
optimal withinC() and so there can be no strict SPI. We summarize this
result in the following proposition.
Proposition 12. Assuming supp(()) is known and that Assumption 2
holds, it can be decided in polynomial time whether there is a strict perfect-
coordination SPI.
30
Algorithm 1: An algorithm for deciding the strict perfect-
coordination SPI problem.
Data: Game , set supp(()
1for a2supp(()) do
2 if u(a)is Pareto-suboptimal within C()then
3 Return True;
4Return False;
9.3 Characterizing safe Pareto improvements under improved
representative coordination
From the problem of deciding whether there are strict SPIs under improved
coordination at all, we move on to the question of what dierent perfect-
coordination SPIs there are. In particular, one might ask what the cost is of
only considering safePareto improvements relative to acting on a probabil-
ity distribution over () and the resulting expected utilities E[u(())].
We start with a lemma that directly provides a characterization. So far, all
the considered perfect-coordination SPIs ( As;us;ue) for a game ( A;u) have
consisted in letting the representatives play a game ( As;us) that is isomor-
phic to the original game, but Pareto-improves (from the original players'
perspectives, i.e., ue) at least one of the outcomes. It turns out that we can
restrict attention to this very simple type of SPI under improved coordina-
tion.
Lemma 13. Let = (fa1
1;:::;al1
1g;:::;fa1
n;:::;alnng;u)be any game. Let 0
be a perfect-coordination SPI on . Then we can dene uewith values in
C()such that under Assumption 2 the game
s=
^A1:=f^a1
1;:::;^al1
1g;:::;^An:=f^a1
n;:::;^alnng;
^u: (^ai1
1;:::;^ainn)7!u(ai1
1;:::;ainn);ue
is also an SPI on , with
E[u((s))j()= a] =E
u((0))()= a
for all a2Aand consequently E[u((s))] =E[u((0))].
Proof. First note that ( ^A;^u) is isomorphic to . Thus by Assumption 2,
there is isomorphism  s.t.  (^A;^u). WLOG assume that  simply
31
mapsai1
1;:::;ainn7!^ai1
1;:::;^ainn. Then dene ueas follows:
ue(^ai1
1;:::;^ainn) =Eh
u0((0))j() = (ai1
1;:::;ainn)i
:
Hereu0describes the utilities that the original players assign to the outcomes
of 0. Since u0maps ontoC() andC() is convex, ueas dened also
maps intoC() as required. Note that for all ai1
1;:::;ainnit is by assumption
u0((0))u(ai1
1;:::;ainn) with certainty. Hence,
ue(^ai1
1;:::;^ainn)) = Eh
u0((0))j() = (ai1
1;:::;ainn)i
u(ai1
1;:::;ainn);
as required.
Because of this result, we will focus on these particular types of SPIs,
which simply create an isomorphic game with dierent (Pareto-better) util-
ities. Note, however, that without assigning exact probabilities to the dis-
tributions of () ;(0), the original players will in general not be able to
construct a sthat satises the expected payo equalities. For this rea-
son, one could still conceive of situations in which a dierent type of SPI
would be chosen by the original players and the original players are unable
to instead choose an SPI of the type described in Lemma 13.
Lemma 13 directly implies a characterization of the expected utilities
that can be achieved with perfect-coordination SPIs. Of course, this charac-
terization depends on the exact distribution of (). We omit the statement
of this result. However, we state the following implication.
Corollary 14. Under Assumption 2, the set of Pareto improvements that
are safely achievable with perfect coordination
fE[u(0)]j0is perfect-coordination SPI on g
is a convex polygon.
Because of this result, one can also eciently optimize convex functions
over the set of perfect-coordination SPIs. Even without referring to the
distribution (), many interesting questions can be answered eciently.
For example, we can eciently identify the perfect-coordination SPI that
maximizes the minimum improvements across players and outcomes a2A.
In the following, we aim to use Lemma 13 and Corollary 14 to give
maximally strong positive results about what Pareto improvements can be
32
safely achieved, without referring to exact probabilities over (). To keep
things simple, we will do this only for the case of two players. To state our
results, we rst need some notation: We use
PF(C):=
y2C69y02C;i2f1;:::;ng:y0y;y0
i>y
to denote the Pareto frontier of a convex polygon C(or more generally
convex, closed set). For any real number x2R, we usei(x;C()) to denote
they02C() which maximizes y0
iunder the constraint y0
i=x. (Recall that
we consider 2-player games, so y0
iis a single real number.) Note that such a
y0exists if and only if xisi's utility in some feasible payo vector. We rst
state our result formally. Afterwards, we will give a graphical explanation
of the result, which we believe is easier to understand.
Theorem 15. Make Assumption 2. Let be a two-player game. Let y2R2
be some potentially unsafe Pareto improvement on E[u(())] . Fori= 1;2,
letxmin=max
i = min=maxui(supp(())) . Then:
A) If there is some element in C() which Pareto-dominates all of
supp(()) and if yis Pareto-dominated by an element of at least one of
the following three sets:
•L1 := the line segment between 1(xmin
1;PF(C()) and
1(xmax
1;PF(C());
•L2:=the segment of the curve PF(C()) between1(xmax
1;PF(C())))
and2(xmax
2;PF(C()))) ;
•L3 := the line segment between 2(xmax
2;PF(C()) and
2(xmin
2;PF(C()).
Then there is an SPI under improved coordination ssuch that
E[u((s))] =y.
B) If there is no element in C()which Pareto-dominates all of supp(())
and if yis Pareto-dominated by an element each of L1andL3as dened
above, then there is a perfect-coordination SPI ssuch that E[u((s))] =
y.
We now illustrate the result graphically. We start with Case A, which
is illustrated in Figure 2. The Pareto-frontier is the solid line in the north
and east. The points marked x indicate outcomes in supp(()). The
point marked by a lled circle indicates the expected value of the default
equilibrium E[u(())]. For some y2R2to be a Pareto improvement,
33
Player 1's utility
E[u1(()))]E[u2(()))]Player 2's utility
Figure 2: This gure illustrates Theorem 15, Case A.
it must be to the north-east of the lled circle. The vertical dashed lines
starting at the two extreme x marks illustrate the application of 1to project
xmin=max
1 onto the Pareto frontier. The dotted line between these two points
isL1. Similarly, the horizontal dashed lines starting at x marks illustrate
the application of 2to projectxmin=max
2 onto the Pareto frontier. The line
segment between these two points is L3. In this case, this line segments
lies on the Pareto frontier. The set L2is simply that part of the Pareto
frontier, which Pareto-dominates all elements of supp(()), i.e., the part
of the Pareto frontier to the north-east between the two intersections with
the northern horizontal dashed line and eastern vertical dashed line.
Case B of Theorem 15 is depicted in Figure 3. Note that here the two
line segments L1andL3intersect. To ensure that a Pareto improvement is
safely achievable, the theorem requires that it is below both of these lines.
For a full proof, see Appendix E. Roughly, Theorem 15 is proven by
re-mapping each of the outcomes of the original game as per Lemma 13.
For example, the projection of the default equilibrium E[u(())] (i.e., the
lled circle) onto L1is obtained as an SPI by projecting all the outcomes
(i.e., all the x marks) onto L1. In Case A, any utility vector y2L2that
Pareto-improves on all outcomes of the original game can be obtained by
re-mapping all outcomes onto y. Other kinds of yare handled similarly.
As a corollary of Theorem 15, we can see that all (potentially unsafe)
Pareto improvements in the DM-RM subset game of the Demand Game of
Table 1 are equivalent to some perfect-coordination SPI. However, this is
not always the case:
34
Player 1's utility
E[u1(())]E[u2(())])Player 2's utility
Figure 3: This gure illustrates Theorem 15, Case B.
Player 2
a b c
Player 1a5;5 4;0 10;100
b 0;4 1;1 10;100
c100;10100;10 3;3
Table 7: An example of a game in which { depending on  { a Pareto
improvement may not be safely achievable.
Proposition 16. There is a game  = (A;u), representatives that satisfy
Assumptions 1 and 2, and an outcome a2As.t.ui(a)>E[ui(())] for
all playersi, but there is no perfect-coordination SPI (As;us;ue)s.t. for all
playersi,E[ue
i((As;us))] =ui(a).
As an example of such a game, consider the game in Table 7. Strategy
ccan be eliminated by strict dominance (Assumption 1) for both players,
leaving a typical Chicken-like payo structure with two pure Nash equilibria
((a;b) and (b;a)), as well as a mixed Nash equilibrium ( 3=8a+5=8b;3=8
a+5=8b).
Now let us say that in the resulting game P(()=(a;b)) =p=
P(()=(b;a)) for some pwith 0< p1=2. Then one (unsafe) Pareto
improvement would be to simply always have the representatives play ( c;c)
for a certain payo of (3 ;3). Unfortunately, there is no safePareto improve-
35
Player 1's utility
1 3 4134
(c;c)Player 2's utility
Figure 4: This gure illustrates the Game of Table 7 as an instance of
Theorem 15, Case B.
ment with the same expected payo. Notice that (3 ;3) is the unique element
ofC() that maximizes the sum of the two players' utilities. By linearity of
expectation and convexity of C(), if for any sit isE[u((s))] = (3;3), it
must be u((s)) = (3;3) with certainty. Unfortunately, in any safe Pareto
improvement the outcomes ( a;b) and (b;a) must corresponds to outcomes
that still gives utilities of (4 ;0) and (0;4), respectively, because these are
Pareto-optimal within the set of feasible payo vectors.
10 The SPI selection problem
In the Demand Game, there happens to be a single non-trivial SPI. However,
in general (even without the type of coordination assumed in Section 9)
there may be multiple SPIs that result in dierent payos for the players.
For example, imagine an extension of the Demand Game imagine that both
players have an additional action DL0, which is like DL, except that under
(DL0;DL0), Aliceland can peacefully annex the desert. Aliceland prefers this
SPI over the original one, while Bobbesia has the opposite preference. In
other cases, it may be unclear to some or all of the players which of two SPIs
they prefer. For example, imagine a version of the Demand Game in which
one SPI mostly improves on (DM ;DM) and another mostly improves on the
other three outcomes, then outcome probabilities are required for comparing
the two. If multiple SPIs are available, the original players would be left
36
with the dicult decision of which SPI to demand in their instruction.9
This diculty of choosing what SPI to demand cannot be denied. How-
ever, we would here like to emphasize that players can prot from the use of
SPIs even without addressing this SPI selection problem. To do so, a player
picks an instruction that is very compliant (\dove-ish") w.r.t. what SPI is
chosen, e.g., one that simply goes with whatever SPI the other players de-
mand as long as that SPI cannot further be safely Pareto-improved upon.10
In many cases, all such SPIs benet all players. For example, optimal SPIs
in bargaining scenarios like the Demand Game remove the con
ict outcome,
which benets all parties. Thus, a player can expect a safe improvement
even under such maximally compliant demands on the selected SPI.
In some cases there may also be natural choices of demands (a l a Schelling
[35, pp. 54{58] or focal points ). If the underlying game is symmetric, a
symmetric safe Pareto improvement may be a natural choice. For example,
the fully reduced version of the Demand Game of Table 1 is symmetric.
Hence, we might expect that even if multiple SPIs were available, the original
players would choose a symmetric one.
11 Conclusion and future directions
Safe Pareto improvements are a promising new idea for delegating strategic
decision making. To conclude this paper, we discuss some ideas for further
research on SPIs.
Straightforward technical questions arise in the context of the complex-
ity results of Section 8. First, what impact on the complexity does varying
the assumptions have? Our NP-completeness proof is easy to generalize at
least to some other types of assumptions. It would be interesting to give a
generic version of the result. We also wonder whether there are plausible
assumptions under which the complexity changes in interesting ways. Sec-
ond, one could ask how the complexity changes if we use more sophisticated
game representations (see the remarks at the end of that section). Third,
one could impose additional restrictions on the sought SPI. For example,
some of the players may be unable to have their representative maximize
arbitrary utility functions. We could then ask whether there is an SPI in
which only a given subset of the players adopt dierent utility functions and
9A second question is what to instruct the representatives to do in case of diering
demands.
10Of course, such an instruction would then also have to specify what happens if all
players submit such an instruction. This appears to be a lesser problem, however.
37
restrictions on the set of available strategies. Fourth, we could restrict the
games under consideration. Are there games in which it becomes easy to
decide whether there is an SPI?
It would also be interesting to see what real-world situations can already
be interpreted as utilizing SPIs, or could be Pareto-improved upon using
SPIs.
Acknowledgments
This work was supported by the National Science Foundation under Award
IIS-1814056. Some early work on this topic was conducted by Caspar Oester-
held while working at the Foundational Research Institute (now the Center
on Long-Term Risk). For valuable comments and discussions, we are grate-
ful to Keerti Anand, Tobias Baumann, Jesse Clifton, Max Daniel, Lukas
Gloor, Adrian Hutter, Vojt ech Kova r k, Anni Leskel a, Brian Tomasik and
Johannes Treutlein, and our wonderful anonymous referees. We also thank
attendees of a 2017 talk at the Future of Humanity Institute at the Uni-
versity of Oxford, a talk at the May 2019 Eective Altruism Foundation
research retreat, and our talk at AAMAS 2021.
38
References
[1] Krzysztof R. Apt. \Uniform Proofs of Order Independence for Various
Strategy Elimination Procedures". In: The B.E. Journal of Theoretical
Economics 4.1 (2004), pp. 1{48. doi:10.2202/1534-5971.1141 .
[2] Robert Axelrod. The Evolution of Cooperation . New York: Basic
Books, 1984.
[3] Mihaly Barasz et al. Robust Cooperation in the Prisoner's Dilemma:
Program Equilibrium via Provability Logic . Jan. 2014. url:https:
//arxiv.org/abs/1401.5577 .
[4] Ken Binmore. Game Theory { A Very Short Introduction . Oxford Uni-
versity Press, 2007.
[5] Tilman B orgers. \Pure Strategy Dominance". In: Econometrica 61.2
(Mar. 1993), pp. 423{430.
[6] Vitalik Buterin. Ethereum White Paper { A Next Generation Smart
Contract & Decentralized Application Platform . Updated version avail-
able at https://github.com/ethereum/wiki/wiki/White-Paper .
2014. url:https : / / cryptorating . eu / whitepapers / Ethereum /
Ethereum\_white\_paper.pdf .
[7] Vincent Conitzer and Tuomas Sandholm. \Complexity of (Iterated)
Dominance". In: Proceedings of the 6th ACM conference on Electronic
commerce . Vancouver, Canada: Association for Computing Machinery,
June 2005, pp. 88{97. doi:10.1145/1064009.1064019 .
[8] Vincent Conitzer and Tuomas Sandholm. \Computing the Optimal
Strategy to Commit to". In: Proceedings of the ACM Conference on
Electronic Commerce (EC) . Ann Arbor, MI, USA: Association for
Computing Machinery, 2006, pp. 82{90.
[9] Stephen A. Cook. \The complexity of theorem-proving procedures".
In:STOC '71: Proceedings of the third annual ACM symposium on
Theory of computing . New York: Association for Computing Machin-
ery, May 1971, pp. 151{158. doi:10.1145/800157.805047 .
[10] Andrew Critch. \A Parametric, Resource-Bounded Generalization
of L ob's Theorem, and a Robust Cooperation Criterion for Open-
Source Game Theory". In: Journal of Symbolic Logic 84.4 (Dec. 2019),
pp. 1368{1381. doi:10.1017/jsl.2017.42 .
[11] Matthias Ehrgott. Multicriteria Optimization . 2nd ed. Berlin:
Springer, 2005.
39
[12] Lance Fortnow. \Program equilibria and discounted computation
time". In: Proceedings of the 12th Conference on Theoretical Aspects
of Rationality and Knowledge (TARK '09) . July 2009, pp. 128{133.
doi:10.1145/1562814.1562833 .
[13] Joaquim Gabarr o, Alina Garc a, and Maria Serna. \The complexity
of game isomorphism". In: Theoretical Computer Science 412.48 (Nov.
2011), pp. 6675{6695. doi:10.1016/j.tcs.2011.07.022 .
[14] David Gale. \A Theory of N-Person Games with Perfect Information".
In:Proceedings of the National Academy of Sciences of the United
States of America 39.6 (June 1953), pp. 496{501. doi:10.1073/pnas.
39.6.496 .
[15] Itzhak Gilboa, Ehud Kalai, and Eitan Zemel. \On the order of elim-
inating dominated strategies". In: Operations Research Letters 9.2
(Mar. 1990), pp. 85{89. doi:10.1016/0167-6377(90)90046-8 .
[16] John C. Harsanyi and Reinhard Selten. A General Theory of Equilib-
rium Selection in Games . Cambridge, MA: The MIT Press, 1988.
[17] J. V. Howard. \Cooperation in the Prisoner's Dilemma". In: Theory
and Decision 24 (May 1988), pp. 203{213. doi:10.1007/BF00148954 .
[18] Adam Tauman Kalai et al. \A commitment folk theorem". In: Games
and Economic Behavior 69 (2010), pp. 127{137. doi:10.1016/j.
geb.2009.09.008 .
[19] Frank H. Knight. Risk, Uncertainty, and Prot . Boston, MA, USA:
Houghton Miin Company, 1921.
[20] Elon Kohlberg and Jean-Francois Mertens. \On the Strategic Stability
of Equilibria". In: Econometrica 54.5 (Sept. 1986), pp. 1003{1037. doi:
10.2307/1912320 .
[21] Jean-Jacques Laont and David Martimort. The Theory of Incentives
{ The Principal-Agent Model . Princeton, NJ: Princeton University
Press, 2002.
[22] R. Duncan Luce and Howard Raia. Games and Decisions. Introduc-
tion and Critical Survey . New York: Dover Publications, 1957.
[23] R. Preston McAfee. \Eective Computability in Economic Deci-
sions". May 1984. url:https : / / www . mcafee . cc / Papers / PDF /
EffectiveComputability.pdf .
40
[24] Dov Monderer and Moshe Tennenholtz. \Strong mediated equilib-
rium". In: Articial Intelligence 173.1 (Jan. 2009), pp. 180{195. doi:
10.1016/j.artint.2008.10.005 .
[25] John von Neumann. \Zur Theorie der Gesellschaftsspiele". In: Math-
ematische Annalen 100 (1928), pp. 295{320. doi:https://doi.org/
10.1007/BF01448847 .
[26] Caspar Oesterheld. \Robust Program Equilibrium". In: Theory and
Decision 86.1 (Feb. 2019), pp. 143{159.
[27] Martin J. Osborne. An Introduction to Game Theory . New York: Ox-
ford University Press, 2004.
[28] David G. Pearce. \Rationalizable Strategic Behavior and the Problem
of Perfection". In: Econometrica 54.4 (July 1984), pp. 1029{1050.
[29] Guillaume Perez. \Decision diagrams: constraints and algorithms".
PhD thesis. Universit e C^ ote d'Azur, 2017. url:https : / / tel .
archives-ouvertes.fr/tel-01677857/document .
[30] Martin Peterson. An Introduction to Decision Theory . Cambridge Uni-
versity Press, 2009.
[31] Steven Pinker. How the Mind Works . W. W. Norton, 1997.
[32] Werner Raub. \A General Game-Theoretic Model of Preference Adap-
tions in Problematic Social Situations". In: Rationality and Society 2.1
(Jan. 1990), pp. 67{93.
[33] Ariel Rubinstein. Modeling Bounded Rationality . Ed. by Karl Gunnar
Persson. Zeuthen Lecture Book Series. The MIT Press, 1998.
[34] Alexander Savelyev. \Contract law 2.0: `Smart' contracts as the be-
ginning of the end of classic contract law". In: Information & Com-
munications Technology Law 26.2 (2017), pp. 116{134. doi:10.1080/
13600834.2017.1301036 .
[35] Thomas C. Schelling. The Strategy of Con
ict . Cambridge, MA: Har-
vard University Press, 1960.
[36] Alexander Schrijver. Theory of Linear and Integer Programming .
Chichester, UK: John Wiley & Sons, 1998.
[37] Amartya Sen. \Choice, orderings and morality". In: Practical Reason .
Ed. by Stephan K orner. New Haven, CT, USA: Basil Blackwell, 1974.
Chap. II, pp. 54{67.
[38] Heinrich von Stackelberg. \Marktform und Gleichgewicht". In: Vi-
enna: Springer, 1934, pp. 58{70.
41
[39] Moshe Tennenholtz. \Program equilibrium". In: Games and Economic
Behavior 49.2 (Nov. 2004), pp. 363{373.
[40] Johannes Treutlein et al. \A New Formalism, Method and Open Is-
sues for Zero-Shot Coordination". In: Proceedings of the Thirty-eighth
International Conference on Machine Learning (ICML'21) . 2021.
[41] Wiebe van der Hoek, Cees Witteveen, and Micheal Wooldridge. \Pro-
gram equilibrium { a program reasoning approach". In: International
Journal of Game Theory 42 (3 Aug. 2013), pp. 639{671.
[42] Luc N. van Wassenhove and Ludo F. Gelders. \Solving a bicriterion
scheduling problem". In: European Journal of Operational Research 4
(1980), pp. 42{48.
[43] Bernhard Von Stengel and Shmuel Zamir. Leadership with commit-
ment to mixed strategies . Tech. rep. LSE-CDAM-2004-01. London
School of Economics, 2004. url:http://www.cdam.lse.ac.uk/
Reports/Files/cdam-2004-01.pdf .
42
A Proof of Theorem 1 { program equilibrium im-
plementations of safe Pareto improvements
This paper considers the meta-game of delegation. SPIs are a proposed
way of playing these games. However, throughout most of this paper, we
do not analyze the meta-game directly as a game using the typical tools
of game theory. We here ll that gap and in particular prove Theorem 1,
which shows that SPIs are played in Nash equilibria of the meta game,
assuming suciently strong contracting abilities. As noted, this result is
essential. However, since it is mostly an application of existing ideas from
the literature on program equilibrium, we left a detailed treatment out of
the main text.
Aprogram game for  = (A;u) is dened via a set PROG = PROG 1
:::PROGnand a non-deterministic mapping exec: PROG 1:::PROGn
A. We obtain a new game with action sets PROG and utility function
U: PROG!Rn:c7!E[u(exec(c))]:
Though this denition is generic, one generally imagines in the program
equilibrium literature that for all i, PROGiconsists of computer programs
in some programming language, such as Lisp, that take as input vectors in
PROG and return an action ai. The function exec on input c2PROG
then executes each player i's program cioncto assignian action. The
denition implicitly assumes that PROG only contains programs that halt
when fed one another as input. A program equilibrium is then simply a Nash
equilibrium of the program game.
For the present paper, we add the following feature to the underlying
programming language. A program can call a \black box subroutine"  i(0)
for any subset game 0of , where  i(0) is a random variable over A0
iand
(0) = ( 1(0);:::;n(0)).
We need one more denition. For any game  and player i, we dene
Playeri'sthreat point (a.k.a. minimax utility) v
ias
v
i= min
i2j6=i(Aj)max
i2(Ai)ui(i;i):
In words,v
iis the minimum utility that the players other than ican force
ontoi, under the assumption that ireacts optimally to their strategy. We
further will use minimax (i;j)2(Aj) to denote the strategy for Player j
that is played in the minimizer iof the above. Of course, in general, there
might be multiple minimizers i. In the following, we will assume that the
43
function minimax breaks such ties in some consistent way, such that for all
i,
(minimax (i;j))j2f1;:::;ngfig2 arg min
i2j6=i(Aj)max
i2(Ai)ui(i;i):
Note that for n= 2, each player's threat point is computable in polyno-
mial time via linear programming; and that by the minimax theorem [25],
the threat point is equal to the maximin utility, i.e.,
v
i= max
i2(Ai)min
i2(Ai)ui(i;i);
sov
iis also the minimum utility that Player ican guarantee for herself
under the assumption that the opponent sees her mixed strategy and reacts
in order to minimize Player i's utility.
Tennenholtz' [39] main result on program games is the following:
Theorem 17 (Tennenholtz 2004 [39]) .Let = (A;u)be a game and let
x2un
i=1(Ai)
be a (feasible) payo vector. If xiv
ifori= 1;:::;n ,
thenxis the utility of some program equilibrium of a program game on .
Throughout the rest of this section, our goal is to use similar ideas as
Tennenholtz did for Theorem 17 to construct for any SPI son , a program
equilibrium that results in the play of (s). As noted in the main text, the
Playeri's instruction to her representative to play the game swill usually be
conditional on the other player telling her representative to also play her part
of sandand vice versa . After all, if Player isimply tells her representative
to maximize us
ifromAs
iregardless of Player i's instruction, then Player
iwill often be able to prot from deviating from the sinstruction. For
example, in the safe Pareto improvement on the Demand Game, each player
would only want their representative to choose from fDL;RLgrather than
fDM;DMgif the other player's representative does the same. It would then
seem that in a program equilibrium in which (s) is played, each program
ciwould have to contain a condition of the type, \if the opponent code plays
as in (s) against me, I also play as I would in (s)." But in a naive
implementation of this, each of the programs would have to call the other,
leading to an innite recursion.
In the literature on program equilibrium, various solutions to this prob-
lem have been discovered. We here use the general scheme proposed by
Tennenholtz [39], because it is the simplest. We could similarly use the
variant proposed by Fortnow [12], techniques based on L ob's theorem [3,
44
10], or-grounded mutual simulation [26] or even (meta) Assurance Game
preferences (see Appendix B).
In our equilibrium, we let each player submit code as sketched in Algo-
rithm 2. Roughly, each player uses a program that says, \if everyone else
submitted the same source code as this one, then play (s). Otherwise,
if there is a player jwho submits a dierent source code, punish player j
by playing her minimax strategy". Note that for convenience, Algorithm 2
receives the player number ias input. This way, every player can use the ex-
act same source code. Otherwise the original players would have to provide
slightly dierent programs and in line 2 of the algorithm, we would have
to use a more complicated comparison, roughly: \if cj6=ciare the same,
except for the player index used".
Algorithm 2: A program equilibrium implementation of an SPI
sof .
Data: Everybody's source code c, my index i
1forj2f1;:::;ngfigdo
2 ifcj6=cithen
3 Play minimax (i;j);
4Play i(s);
Proposition 18. Letbe a game and let sbe an SPI on . Let cbe the
program prole consisting only of Algorithm 2 for each player. Assume that
() guarantees each player at least threat point utility in expectation. Then
cis a program equilibrium and apply (c) = (s).
Proof. By inspection of Algorithm 2, we see that exec(c) = (s). It is
left to show that cis a Nash equilibrium. So let ibe any player and c0
i2
PROGifcig. We need to show that E[ui(exec(ci;c0
i))]E[ui(exec(c))].
Again, by inspection of c,exec(ci;c0
i) is the threat point of Player i. Hence,
E
ui(exec(ci;c0
i))
=vi
E[ui(())]
E[ui((s))]
=E[ui(exec(c))]
as required.
Theorem 1 follows immediately.
45
B A discussion of work by Sen (1974) and Raub
(1990) on preference adaptation games
We here discuss Raub's [32] paper in some detail, which in turn elaborates
on an idea by Sen [37]. Supercially, Raub's setting seems somewhat similar
to ours, but we here argue that it should be thought of as closer to the work
on program equilibrium and bilateral precommitment.
In Sections 1, 3 and 4, we brie
y discuss multilateral commitment games,
which have been discussed before in various forms in the game-theoretic
literature. Our paper extends this setting by allowing instructions that let
the representatives play a game without specifying an algorithm for solving
that game. On rst sight, it appears that Raub pursues a very similar idea.
Translated to our setting, Raub allows that as an instruction, each player i
chooses a new utility function us
i:A!R, whereAis the set of outcomes of
the original game . Given instructions us
1;:::;us
n, the representatives then
play the game ( A;us). In particular, each representative can see what utility
functions all the other representatives have been instructed to maximize.
However, what utility function representative imaximizes is not conditional
on any of the instructions by other players. In other words, the instructions
in Raub's paper are raw utility functions without any surrounding control
structures, etc. Raub then asks for equilibria usof the meta-game that
Pareto-improve on the default outcome.
To better understand how Raub's approach relates to ours, we here give
an example of the kind of instructions Raub has in mind. (Raub uses the
same example in his paper.) As the underlying game , we take the Pris-
oner's Dilemma. Now the main idea of his paper is that the original players
can instruct their representatives to adopt so-called Assurance Game prefer-
ences. In the Prisoner's Dilemma, this means that the representatives prefer
to cooperate if the other representative cooperates, and prefer to defect if
the other player defects. Further, they prefer mutual cooperation over mu-
tual defection. An example of such Assurance Game preferences is given
in Table 8. (Note that this payo matrix resembles the classic Stag Hunt
studied in game theory.)
The Assurance Game preferences have two important properties.
1. If both players tell their representatives to adopt Assurance Game
preferences, (Cooperate, Cooperate) is a Nash equilibrium. (Defect,
Defect) is a Nash equilibrium as well. However, since (Cooperate,
Cooperate) is Pareto-better than (Defect, Defect), the original play-
ers could reasonably expect that the representatives play (Cooperate,
46
Player 2
Cooperate Defect
Player 1Cooperate 4;4 1;3
Defect 3;1 2;2
Table 8: Assurance Game preferences for the Prisoner's Dilemma
Cooperate).
2. Under reasonable assumptions about the rationality of the represen-
tatives, it is a Nash equilibrium of the meta-game for both players to
adopt Assurance Game preferences. If Player 1 tells her representa-
tive to adopt Assurance Game preferences, then Player 2 maximizes
his utility by telling his representative to also maximize Assurance
Game preferences. After all, representative 1 prefers defecting if rep-
resentative 2 defects. Hence, if Player 2 instructs his representative
to adopt preferences that suggest defecting, then he should expect
representative to defect as well.
The rst important dierence between Raub's approach and ours is re-
lated to item 2. We have ignored the issue of making SPIs sNash equilibria
of our meta game. As we have explained in Section 4 and Appendix A, we
imagine that this is taken care of by additional bilateral commitment mech-
anisms that are not the focus of this paper. For Raub's paper, on the other
hand, ensuring mutual cooperation to be stable in the new game sis ar-
guably the key idea. Still, we could pursue the approach of the present
paper even when we limit assumptions to those that consist only of a utility
function.
The second dierence is even more important. Raub assumes that { as
in the PD { the default outcome of the game (() in the formalism of
this paper) is known. (Less signicantly, he also assumes that it is known
how the representatives play under assurance game preferences.) Of course,
the key feature of the setting of this paper is that the underlying game
might be dicult (through equilibrium selection problems) and thus that
the original players might be unable to predict ().
These are the reasons why we cite Raub in our section on bilateral com-
mitment mechanisms. Arguably, Raub's paper could be seen as very early
work on program equilibrium, except that he uses utility functions as a
programming language for representative. In this sense, Raub's Assurance
47
Game preferences are analogous to the program equilibrium schemes of Ten-
nenholtz [39], Oesterheld [39], Barasz et al. [3] and van der Hoek, Witteveen,
and Wooldridge [41], ordered in increasing order of similarity of the main
idea of the scheme.
C Proof of Lemma 4
Lemma 4. Letand be isomorphisms between and0. Ifis (strictly)
Pareto-improving, then so is .
Proof. First, we argue that if  and are isomorphisms, then they are
isomorphisms relative to the same constants andc. For each player i, we
distinguish two cases. First the case where all outcomes ain  have the
same utility for Player iis trivial. Now imagine that the outcomes of  do
not all have the same utility. Then let yminandymaxbe the lowest and
highest utilities, respectively, in . Further, let xminandxmaxbe the lowest
and highest utilities, respectively, in 0. It is easy to see that if is a game
isomorphism, it maps outcomes with utility yminin  onto outcomes with
utilityxminin 0, and outcomes with utility ymaxin  onto outcomes with
utilityxmaxin 0. Thus, if ;iandc ;iare to be the constants for , then
ymin = ;ixmin+c ;i
ymax = ;ixmax+c ;i:
Sincexmin6=xmax, this system of linear equations has a unique solution. By
the same pair of equations, the constants for  are uniquely determined.
It follows that for all a2A,
u(a) = u0( (a)) +c
=u(1( (a)))
u((1( (a))))
=u( (a)):
Furthermore, if  is strictly Pareto-improving for some ~a2A, then by
bijectivity of  ; , there is a2As.t. 1( (a)) = ~a. For this a, the
inequality above is strict and therefore u(a)ui(a0
i;ai)>ui(ai;ai):
Both inequalities are due to the denition of strict dominance. We
conclude that ^ aimust strictly dominate ai.
Case 2: The strategy removed is one other than aiora0
i. Since the set of
strategies of the new game is a subset of the strategies of the old game
it is still for each strategy aiin the new game
ui(a0
i;ai)>ui(ai;ai);
i.e.,a0
istill strictly dominates ai.
The next lemma shows that instead of rst applying Assumption 1 plus
symmetry (Lemma 2.2) to add a strictly dominated action and then applying
Assumption 1 to eliminate a dierent strictly dominated strategy, we could
also rst eliminate the strictly dominated strategy and then add the other
strictly dominated strategy.
49
Lemma 20. Let(red)1^by Assumption 1, where is the reduced
game, and ^~red~by Assumption 1. Then either  =^or there is a game
0s.t.~red0by Assumption 1 and 0(red)1~by Assumption 1.
Proof. By the assumption both ~ and  can be obtained from eliminating a
strictly dominated action from ^. Let these actions be ~ aanda, respectively.
If ~a=a, then  = ~. So for the rest of this proof assume ~ a6=a. Let 0be
the game we obtain by removing ~ afrom . We now show the two outcome
correspondences:
•First we show that  ~red0, i.e., that ~ais strictly dominated in .
For this notice that ~ aandaare both strictly dominated in ^. Now
 is obtained from ^ by removing a. By Lemma 19, ~ ais still strictly
dominated in , as claimed.
•Second we show that 0(red)1~, i.e., that ~red0, i.e., that
ais strictly dominated in ~. Recall again that ~ aandaare both
strictly dominated in ^. Now ~ is obtained from ^ by removing a. By
Lemma 19, ais still strictly dominated in ~, as claimed.
We are ready to state our lemma about the structure of outcome corre-
spondences.
Lemma 21. Let
11:::k1k;
where each outcome correspondence is due to a single application of Assump-
tion Assumption 1, Assumption 1 plus symmetry (Lemma 2.2) or Assump-
tion 2. Then there is a sequence 01;:::;0mwith 01= 1and0m= m,
mkandlsuch that
1 102 203 3::: l10l
all by single applications of Assumption 1, 0land0l+1are fully reduced
games such that 0l l0l+1by a single application of Assumption 2, and
0l+1( l+1)10l+2( 0l+2)1:::( 0m)10m
all by single applications of Assumption 1 with Lemma 2.2.
50
A conciser way to state the consequence is that there must be games
red, s;redand ssuch that redis obtained from  by iterated elimination
of strictly dominated strategies, s;redis isomorphic to s;red, and s;redis
obtained from sby iterated elimination of strictly dominated strategies.
Proof. First divide the given sequence of outcome correspondences up into
periods that are maximally long while containing only correspondences by
Assumption 1 (with or without Lemma 2.2). That is, consider subsequences
of the form qq:::r1rsuch that:
•Each of the correspondences qqq+1, ..., r1r1ris by
applying Assumption 1 with or without Lemma 2.2.
•Eitherq= 1 or the correspondence q1q1qis by Assumption 2.
•Eitherr=kor the correspondence rrr+1is by Assumption 2.
In each such period apply Lemma 20 iteratively to either eliminate or move
to the right all inverted reduction elimination steps.
In all but the rst period, qcontains no strictly dominated actions (by
stipulation of Assumption 2). Hence all but the rst period cannot contain
any non-reversed elimination steps. Similarly, in all but the nal period, r
contains no strictly dominated actions. Hence, in all but the nal period,
there can be no reversed applications of Assumption 1.
Overall, our new sequence of outcome correspondences thus has the fol-
lowing structure: rst there is a sequence of elimination steps via Assump-
tion 1, then there is a sequence of isomorphism steps, and nally there is a
sequence of reverse elimination steps. We can summarize all the applications
of Assumption 2 into a single step applying that assumption to obtain the
claimed structure.
Now notice that that the reverse elimination steps are only relevant for
deriving unilateral SPIs. Using the above concise formulation of the lemma,
we can always simply use s;reditself as an omnilateral SPI { it is not relevant
that there is some subset game sthat reduces to s;red.
Lemma 22. As in Lemma 21, let 11:::k1k, where each outcome
correspondence is due to a single application of Assumption Assumption 1,
Assumption 1 plus symmetry (Lemma 2.2) or Assumption 2. Let 2;:::;k
all be subset games of 1. Moreover, let k1:::1be Pareto improving.
Then there is a sequence of subset games 02;:::;0m;0m+1such that 1 1
02 2::: m10mall by applications of Assumption 1 (without applying
51
symmetry), and 0m0m+1by application of Assumption 2 such that
 m1::: 1is Pareto improving.
Proof. First apply Lemma 21. Then notice that the correspondence func-
tions from applying Assumption 1 with symmetry have no eect on whether
the overall outcome correspondence is Pareto improving.
D.2 Non-deterministic polynomial-time algorithms for the
SPI problem
D.2.1 The omnilateral SPI problem
We now show that the SPI problem is in NP at all. The following algo-
rithm can be used to determine whether there is a safe Pareto improve-
ment: Reduce the given game  until it can be reduced no further to obtain
some subset game 0= (A0;u). Then non-deterministically select injections
i:A0
i!Ai. If  = ( 1;:::;n) is (strictly) Pareto-improving (as required
in Theorem 3), return True with the solution sdened as follows: The set
of action proles is dened as As=ii(A0
i). The utility functions are
us
i:As!R:as7!(ui(1
1(as
1);:::;1
n(as
n)))i=1;:::;n:
Otherwise, return False.
Proposition 23. The above algorithm runs in non-deterministic polynomial
time and returns True if and only if there is a (strict) unilateral SPI.
Proof. It is easy to see that this algorithm runs in non-deterministic polyno-
mial time. Furthermore, with Lemma 4 it is easy to see that if this algorithm
nds a solution s, that solution is indeed a safe Pareto improvement. It
is left to show that if there is a safe Pareto improvement via a sequence of
Assumption 2 and 1 outcome correspondences, then the algorithm indeed
nds a safe Pareto improvement.
Let us say there is a sequence of outcome correspondences as per As-
sumptions 1 and 2 that show  sfor Pareto-improving . Then by
Lemma 22, there is 0such that  red0via applying Assumption 1
iteratively to obtain a fully reduced 0and 0 isosvia a single applica-
tion of Assumption 2. By construction, our algorithm nds (guesses) this
Pareto-improving outcome correspondence.
Overall, we have now shown that our non-deterministic polynomial-time
algorithm is correct and therefore that the SPI problem is in NP. Note
that the correctness of other algorithms can be proven using very similar
52
ideas. For example, instead of rst reducing and then nding an isomor-
phism, one could rst nd an isomorphism, then reduce and then (only
after reducing) test whether the overall outcome correspondence function is
Pareto-improving. One advantage of reducing rst is that there are fewer
isomorphisms to test if the game is smaller. In particular, the number of
possible isomorphisms is exponential in the number of strategies in the re-
duced game 0but polynomial in everything else. Hence, by implementing
our algorithm deterministically, we obtain the following positive result.
Proposition 24. For games withjA1j+:::+jAnj=mthat can be reduced
(via iterative application of Assumption 1) to a game 0withjA0
1j+:::+
jA0
nj=l, the (strict) omnilateral SPI decision problem can be solved in
O(ml).
D.2.2 The unilateral SPI problem
Next we show that the problem of nding unilateral SPIs is also in NP. Here
we need a slightly more complicated algorithm: We are given an n-player
game  and a player i. First reduce the game  fully to obtain some subset
game red. Then non-deterministically select injections  i:Ared
i!Ai. The
resulting candidate SPI game then is
s= ((Ai;i(Ared
i));(ui;us
i));
whereus
i(as) =ui(1
1(as
1);:::;1
n(as
n)) for all as2(Ared), andus
i(as)
is arbitrary for as=2(Ared). Return True if the following conditions are
satised:
1. The correspondence function  must be (strictly) Pareto improving
(as per the utility functions u).
2. For each j2f1;:::;ngfig, there are j2R+andcj2Rsuch that
for all a2Ared, we haveuj(a) =juj((a)) +cj.
3. The game sreduces to the game (( Ared);(ui;us
i)).
Otherwise, return False.
Proposition 25. The above algorithm runs in non-deterministic polynomial
time and returns True if and only if there is a (strict) unilateral SPI.
Proof. First we argue that the algorithm can indeed be implemented in
non-deterministic polynomial time. For this notice that for checking Item 2,
53
the constants can be found by solving nsystems of linear equations of two
variables.
It is left to prove correctness, i.e., that the algorithm returns True if and
only if there exists an SPI. We start by showing that if the algorithm returns
True, then there is an SPI. Specically, we show that if the algorithm returns
True, the game sis indeed an SPI game. Notice that   redfor some
 by iterative application of Assumption 1 with Transitivity (Lemma 2.2);
that red((Ared);(ui;us
i)) by application of Assumption 2. Finally,
((Ared);(ui;us
i))1sfor some  by iterative application of Assump-
tion 1 to s, plus transitivity (Lemma 2.3) with reversal (Lemma 2.2).
It is left to show that if there is an SPI, then the above algorithm will
nd it and return true. To see this, notice that Lemma 21 implies that there
is a sequence of outcome correspondences   reds;reds. We
can assume that s;redand shave the same action sets for Player i. It
is easy to see that in swe could modify the utilities us
i(a) for any athat
is not in s;red, because Player i's utilities do not aect the elimination of
strictly dominated strategies from s.
Proposition 26. For games withjA1j+:::+jAnj=mthat can be reduced
(via iterative application of Assumption 1) to a game 0withjA0
1j+:::+
jA0
nj=l, the (strict) unilateral SPI decision problem can be solved in O(ml).
D.3 The SPI problems are NP-hard
We now proceed to showing that the safe Pareto improvement problem is
NP-hard. We will do this by reducing the subgraph isomorphism problem
to the (two-player) safe Pareto improvement problem. We start by brie
y
describing one version of that problem here.
A(simple, directed) graph is a tuple ( n;a:f1;:::;ngf 1;:::;ng!B),
wheren2NandB:=f0;1g. We callathe adjacency function of the graph.
Since the graph is supposed to be simple and therefore free of self-loops
(edges from one vertex to itself), we take the values a(j;j) forj2f1;:::;ng
to be meaningless.
For given graphs G= (n;a),G0= (n0;a0) a subgraph isomorphism from
GtoG0is an injection :f1;:::;ng!f 1;:::n0gsuch that for all j6=l
a(j;l)a0((j);(l)):
In words, a subgraph isomorphism from GtoG0identies for each node in
Ga node inG0s.t. if there is an edge from node jto nodelinG, there
must also be an edge in the same direction between the corresponding nodes
54
(j);(l) inG0. Another way to say this is that we can remove some set of
(n0n) nodes and some edges from G0to get a graph that is just a relabeled
(isomorphic) version of G.
Denition 8. Given two graphs G;G0, the subgraph isomorphism prob-
lem consists in deciding whether there is a subgraph isomorphism between
G;G0.
The following result is well-known.
Lemma 27 ([9, Theorem 2]) .The subgraph isomorphism problem is NP-
complete.
Lemma 28. The subgraph isomorphism problem is reducible in linear time
with linear increase in problem instance size to the (strict) (unilateral) safe
Pareto improvement problem for two players. As a consequence, the (strict)
(unilateral) safe Pareto improvement problem is NP-hard.
Proof. LetG= (n;a) and ^G= (^n;^a) be graphs. Without loss of generality
assume both graphs have at least 2 vertices, i.e., that n;^n2. For this
proof, we dene [ i]:=f1;:::;igfor anyi2N.
We rst dene two games, one for each graph, and then a third game
that contains the two.
The game for Gis the game  as in Table 9. Formally, let < 1=2n. Then
we let  = ( A1;A2;u1;u2), whereA1=A2= [2n+ 2]. The utility functions
are dened via
u1(i;j) =8
>>>>>>>>>>>>>><
>>>>>>>>>>>>>>:2; ifi=jandi;j2[n]
a(i;j); ifi6=jandi;j2[n]
1; ifi2fn+ 1;:::;2ng;j2[n] andi6=j+n
1; ifi2[n];j2fn+ 1;:::;2ngandi+n6=j
4 + (n+i)ifi2[n] andj=i+n
4 +j ifj2[n] andi=j+n
1 ifi;j2fn+ 1;:::;2ng
3 ifi2f2n+ 1;2n+ 2gandj2[2n]
6 ifi=j= 2n+ 1
 ifj2f2n+ 1;2n+ 2gand noti=j= 2n+ 1
55
1::: n n +1::: 2n 2n+1 2n+2
1 2;2 a;1 4+n+;41;1 0;3 0;3
...............
na;1 2 ;21;1 4+2 n;4 0;3 0;3
n+1 4+;41;1 0;3 0;3
...... 1;1......
2n1;1 4+ n;4 0;3 0;3
2n+ 1 3;0::: 3;0 3;0::: 3;0 6; ;
2n+ 2 3;0::: 3;0 3;0::: 3;0; ; 6
Table 9: The game  constructed to represent the graph G= (n;a).
fDg[2n+ 2]fPg[2^n+ 2]
fRg[2^n+ 2]2;1 ^
fTg[2n+ 2]  10;10
Table 10: The game cas constructed from  and ^.
and
u2(i;j) =8
>>>>>>>>>>>>>><
>>>>>>>>>>>>>>:2;ifi=jandi;j2[n]
1;ifi6=jandi;j2[n]
1;ifi2fn+ 1;:::;2ng;j2[n] andi6=j+n
1;ifi2[n];j2fn+ 1;:::;2ngandi+n6=j
4 ifi2[n] andj=i+n
4 ifj2[n] andi=j+n
1 ifi;j2fn+ 1;:::;2ng
3 ifj2f2n+ 1;2n+ 2gandi2[2n]
6 ifi=j= 2n+ 2
ifi2f2n+ 1;2n+ 2gand noti=j= 2n+ 2:
We dene ^ based on ^Ganalogously, except that in Player 1's utilities we
use 5 instead of 4, 5 + ( n+i)instead of 4 + ( n+i), 5 +jinstead of 4 + j
and 4 instead of 3.
We now dene c= (Ac;uc) from  and ^ as sketched in Table 10. For
the following let
A= (fTg[2n+ 2])(fDg[2n+ 2])
A^= (fRg[2^n+ 2])(fPg[2^n+ 2]);
56
andAc
1=A
1[A^
1andAc
2=A
2[A^
2. Fori;j2[2n+2], let uc((T;i);(D;j))
be the utility of ( i;j) in . For i;j2[2^n+ 2] let uc((R;i);(P;j)) be the
utility of (i;j) in ^. Finally, dene uc((R;i);(D;j)) = (2;1) for all
i2[2^n+ 2] and all j2[2n+ 2]; and uc((T;i);(P;j)) = (10;10) for all
i2[2n+ 2] and all j2[2^n+ 2].
It is easy to show that this reduction can be computed in linear time
and that it also increases the problem instance size only linearly.
To prove our claim, we need to prove the following two propositions:
1. If there is a subgraph isomorphism from Gto^G, then there is a uni-
lateral, strict SPI.
2. If there is any SPI, then there is a subgraph isomorphism from Gto
^G.
1. We start with the rst claim. Assume there is a subgraph isomorphism
fromGtoG0. We construct our SPI as usual: rst we reduce the game cby
iterated elimination of strictly dominated strategies, then we nd a Pareto-
improving outcome equivalence between the reduced game and some subset
game s;redof c. Finally, we show that s;redarises from removing strictly
dominated strategies from subset game sof c. It is easy to see that the
game resulting from iterated elimination of strictly dominated strategies is
just the  part of it. Abusing notation a little, we will in the following just
call this  (even though it has somewhat dierently named action sets).
Next we dene a pair of functions 1=2, which will later form our iso-
morphism. For all i2[n] andb2f1;2g, we dene 1via
 1(T;i) = (R;(i))
 1(T;n+i) = (R;^n+(i))
 1(T;2n+b) = (R;2^n+b)
Dene 2(D;i) = (P;(i)) and so on analogously.
Now dene s;redto be the subset game with action sets 1(A1) and
 2(A2), whereA1andA2are the action sets of ; and with utility functions
us
1: 1(A1) 2(A2)!R:as7!uc
1( 1(as))
anduc
2(as restricted to 1(A1) 2(A2)).
We must now show that is a game isomorphism between  and s;red.
First, it is easy to see that for i= 1;2, iis a bijection between Aiand
 i(Ai). Moreover,
us
1( (a)) =uc
1( 1( (a))) =uc
1(a):
57
For player 2, we need to distinguish the dierent cases of actions. Since each
case is trivial from looking at the denition of  and ^ we omit the detailed
proof.
Next we need to show that is strictly Pareto-improving as judged by
the original players' utility function uc. Again, this is done by distinguishing
a large number of cases of action proles a, all of which are trivial on their
own. The most interesting one is that of (( T;i);(D;j)) fori;j2[n] with
i6=jbecause this is where we use the fact that is a subgraph isomorphism:
uc
1( ((T;i);(D;j))) =uc
1((R;(i));(P;(j)))
= ^a((i);(j))
a(i;j)
=uc
1((T;i);(D;j)):
We omit the other cases.
It is left to construct a unilateral subset game sof csuch that s
reduces to s;redvia iterated elimination of strictly dominated strategies.
Let s= ( 1(A1);Ac
2;us
1;uc
2), where we set us
1(a1;a2) arbitrarily for a22
Ac
2 2(A2).
We now show that sreduces to s;redvia repeated application of As-
sumption 1. So let a22Ac
2 2(A2). We distinguish the following cases:
•Ifa22D[2n+ 2], thena2is strictly dominated by ( P;2^n+ 1) and
by (P;2^n+ 2).
•Ifa2= (P;i) for some i2f1;:::;^ng, then by assumption that a22
Ac
2 (A2) and by construction of , we know that ( R;n+i)=2 1(A1).
From this and inspecting Table 9 we see that ( P;2^n+1) and (P;2^n+2)
strictly dominate ( P;i).
•Ifa2= (P;^n+i) for somei2f1;:::;^ng, then by assumption that a22
Ac
2 2(A2) and by construction of , we know that ( R;i)=2 1(A1).
From this and inspecting Table 9 we see that ( P;2^n+1) and (P;2^n+2)
strictly dominate ( P;^n+i).
Note that ( P;2^n+ 1) and (P;2^n+ 2) are both in 2(A2) by construction of
 .
2. It is left to show that if there is any kind of non-trivial SPI, there is
also a subgraph isomorphism from Gto^G.
By Lemma 21, if there is an SPI, there are bijections 1; 2that are
jointly Pareto-improving from the reduced game  to c. From these func-
tions we will construct a subgame isomorphism. However, to do so (and to
58
see that the resulting function is indeed a subgraph isomorphism), we need
to rst make a few simple observations about the structure of 1and 2.
DeneA;[n]= (fTg[n])(fDg[n]) andA^;[^n]= (fRg[^n])(fPg[^n]).
(a) First we will argue that there is an action a2Aof the reduced game
s.t. ( a)2A^. We prove this by showing the following contrapositive:
if (A) andA^are disjoint, then must, contrary to assumption,
be trivial, i.e., must be the pair of identity functions on A.
From the fact that is Pareto improving, it follows that (( T;2n+
1);(D;2n+1)) = ((T;2n+1);(D;2n+1)), since outside of A^there is
no outcome with utility at least 6 for Player 1. Similarly, (( T;2n+
2);(D;2n+ 2)) = (( T;2n+ 2);(D;2n+ 2)). It then follows that
 ((T;n);(D;2n)) = ((T;n);(D;2n)), since apart from the outcomes
we have already mapped to, no other outcome gives Player 1 a util-
ity of 4 + 2 n. Next it follows that (( T;n1);(D;2n1)) =
((T;n1);(D;2n1)), again because all outcomes with utility at least
4+(2n1)for Player 1 outside of A^are already mapped to. And so
on, until we obtain that (( T;1);(D;n+ 1)) = ((T;1);(D;n+ 1)). By
an analogous line of argument we can show that (( T;2n);(D;n)) =
((T;2n);(D;n));:::; ((T;n+ 1);(D;1)) = ((T;n+ 1);(D;1)). To-
gether these equalities uniquely specify 1= id and 2= id.
(b) We next argue that ( A)A^. We show a contrapositive, specif-
ically that if this were not the case then would not be Pareto-
improving. So assume that ( A)6A^. Then from item a it follows
that there is a2Asuch that neither ( a)2A^nor ( a)2A. Then
eitheruc
1( (a)) =2 and hence uc
1(a)>uc
1( (a)) oru2( (a)) =10
and henceuc
2(a)1 =uc
2((R;j);(P;l)) =uc
2( 1(T;i); 2(R;i))
in contradiction to the assumption that is Pareto improving.
We are ready to construct our subgraph isomorphism. For i2[n], dene
(i) to be the second element of the pair 1(T;i). By Item d, (i) can
equivalently be dened as the second item in the pair 2(D;i). By Item
c,is a function from [ n] to [^n]. By assumption about , is injective.
Further, by construction of cand, as well as the assumption that is
Pareto improving, we infer that for all i;j2[n] withi6=j,
^a((a);(j)) =uc
1((R;(i));(P;(j)))
=uc
1( 1(T;i); 2(D;j))
uc
1((T;i);(D;j))
=a(i;j):
We conclude that is a subgraph isomorphism.
E Proof of Theorem 15
Proof. We will give the proof based on the graphs as well, without giving
all formal details. Further we assume in the following that neither L1nor
L3consist of just a single point, since these cases are easy.
Case A : Note rst that by Corollary 14 it is enough to show that if yis
in any of the listed sets L1;L2;L3, it can be made safe.
It's easy to see that all payo vectors on the curve segment of the Pareto
frontierL2are safely achievable. After all, all payo vectors in this set
Pareto-improve on all outcomes in supp(()). Hence, for each yon the
line segment, one could select the swhere ue=y.
It is left to show that all elements of L1=2are safely achievable. Remem-
ber that not all payo vectors on the line segments are Pareto improvements,
only those that are to the north-east of (Pareto-better than) the default util-
ity. In the following, we will use L0
1andL0
3to denote those elements of L1
andL3, respectively, that are Pareto-improvements on the default.
We now argue that the Pareto improvement yon the line L1for which
y1=E[u1(())] is safely achievable. In other words, yis the projection
60
northward of the default utility, or y=1(E[u(())];L1). This yis
also one of the endpoints of L0
1. To achieve this utility, we construct the
equivalent game as per Lemma 13, where the utility to the original players
of each outcome (^ a1;^a2) of the new game sis similarly the projection
northward onto L1of the utility of the corresponding outcome ( a1;a2) in
s. That is,
ue(^a1;^a2) =1(u(a1;a2);L1):
Note that because C() is convex and the endpoints of the line segment L1
are by denition in C(), it isL1C(). Hence, all values of uethus dened
are feasible. Because all outcomes in the original game lie below the line L1,
1is linear. Hence,
E[ue((s))] = E[1(u(());L1)]
=1(E[u(())];L1)
as required.
We have now shown that one of the endpoints of L0
1is safely achievable.
Since the other endpoint of L0
1is inL2, it is also safely achievable. By
Corollary 14, this implies that all of L0
1is safely achievable.
By an analogous line of reasoning, we can also show that all elements of
L0
3are safely achievable.
Case B : DeneL0
1;L0
3as before as those elements of L1;L3respectively
that Pareto improve on the default E[u(())]. By a similar argument as
before, one can show that the utilities i(E[u(())];L0
j) is safely achievable
both fori= 1;j= 1 and for i= 2;j= 3. Call these points E1andE3,
respectively.
We now proceed in two steps. First, we will show that there is a third
safely achievable utility point E2, which is above both L1andL3. Then we
will show the claim using that point.
To construct E2, we again construct an SPI sas per Lemma 13. For
each (a1;a2)2A1A2we will set the utility ue(^a1;^a2) of the corresponding
(^a1;^a2)2^A1^A2to be above or on both L1andL3, i.e., on or above a set
which we will refer to as max( L1;L3). Formally, max( L1;L3) is the set of
outcomes in L1[L3that are not strictly Pareto dominated by some other
element of L0
1[L0
3. Note that by denition every outcome in supp(())
is Pareto-dominated by some outcome in either L1orL3. Hence, by tran-
sitivity of Pareto dominance, each outcome is Pareto-dominated by some
outcome in max( L1;L3). Hence, the described ueis indeed feasible.
61
Now note that the set of feasible payos of  is convex. Further, the
curve max( L1;L3) is concave. Because the area above a concave curve is
convex and because the intersection of convex sets is convex, the set of
feasible payos on or above max( L1;L3) is also convex. By denition of
convexity,E2=E[ue((s))] is therefore also in the set of feasible payos
on or above max( L1;L3) and therefore above both L1andL3as desired.
In our second step, we now use E1;E2;E3to prove the claim. Because of
convexity of the set of safely achievable payo vectors as per Corollary 14,
all utilities below the curve consisting of the line segments from E1toE2
and fromE2toE3are safely achievable. The line that goes through E1;E2
intersects the line that contains L1atE1, by denition. Since non-parallel
lines intersect each other exactly once and parallel lines that intersect each
other are equal and because E2is above or on L1, the line segment from
E1toE2lies entirely on or above L1. Similarly, it can be shown that the
line segment from E2toE3lies entirely on or above L3. It follows that
theE1E2E3curve lies entirely above or on min( L1;L3). Now take
any Pareto improvement that lies below both L0
1andL0
3. Then this Pareto
improvement lies below min( L0
1;L0
3) and therefore below the E1E2E3
curve. Hence, it is safely achievable.
62