Safe Pareto Improvements for Delegated Game Playing
Caspar Oesterheld
Department of Computer Science, Duke University
caspar.oesterheld@duke.eduVincent Conitzer
Department of Computer Science, Duke University
conitzer@cs.duke.edu
ABSTRACT
A set of players delegate playing a game to a set of representatives,
one for each player. We imagine that each player trusts their respec-
tive representative’s strategic abilities. Thus, we might imagine that
per default, the original players would simply instruct the represen-
tatives to play the original game as best as they can. In this paper,
we ask: are there safe Pareto improvements on this default way of
giving instructions? That is, we imagine that the original players
can coordinate to tell their representatives to only consider some
subset of the available strategies and to assign utilities to outcomes
differently than the original players. Then can the original players
do this in such a way that the payoff is guaranteed to be weakly
higher than under the default instructions for all the original play-
ers? In particular, can they Pareto-improve without probabilistic
assumptions about how the representatives play games? In this
paper, we give some examples of safe Pareto improvements. We
prove that the notion of safe Pareto improvements is closely re-
lated to a notion of outcome correspondence between games. We
also show that under some specific assumptions about how the
representatives play games, finding safe Pareto improvements is
NP-complete.
KEYWORDS
program equilibrium; delegation; bargaining; Pareto efficiency;
smart contracts
ACM Reference Format:
Caspar Oesterheld and Vincent Conitzer. 2021. Safe Pareto Improvements
for Delegated Game Playing. In Proc. of the 20th International Conference
on Autonomous Agents and Multiagent Systems (AAMAS 2021), Online, May
3–7, 2021 , IFAAMAS, 17 pages.
1 INTRODUCTION
Between Aliceland and Bobbesia lies a sparsely populated desert.
Until recently, neither of the two countries had any interest in the
desert. However, geologists have recently discovered that it contains
large oil reserves. Now, both Aliceland and Bobbesia would like
to annex the desert, but they worry about a military conflict that
would ensue if both countries insist on annexing.
Table 1 models this strategic situation as a normal-form game.
The strategy DM(short for “Demand with Military”) denotes a
military invasion of the desert, demanding annexation. If both
countries send their military with such an aggressive mission, the
countries fight a devastating war. The strategy RM(for “Refrain
with Mility”) denotes yielding the territory to the other country, but
building defenses to prevent an invasion of one’s original territories.
Alternatively, the countries can choose to not raise a military force
Proc. of the 20th International Conference on Autonomous Agents and Multiagent Systems
(AAMAS 2021), U. Endriss, A. Nowé, F. Dignum, A. Lomuscio (eds.), May 3–7, 2021, Online .
©2021 International Foundation for Autonomous Agents and Multiagent Systems
(www.ifaamas.org). All rights reserved.at all, while potentially still demanding control of the desert by
sending only its leader ( DL, short for “Demand with Leader”). In
this case, if both countries demand the desert, war does not ensue.
Finally, they could neither demand nor build up a mility ( RL). If one
of the two countries has their military ready and the other does
not, the militarized country will know and will be able to invade
the other country. In game-theoretical terms, militarizing therefore
strictly dominates not militarizing.
Instead of making the decision directly, the parliaments of Al-
iceland and Bobbesia appoint special commissions for making this
strategic decision, led by Alice and Bob, respectively. The parlia-
ments can instruct these representatives in various ways. They can
explicitly tell them what to do – for example, Aliceland could di-
rectly tell Alice to play DM. However, we imagine that the par-
liaments trust the commissions’ judgments more than they trust
their own and hence they might prefer to give an instruction of the
type, “make whatever demands you think are best for our coun-
try” (perhaps contractually guaranteeing a reward in proportion
to the utility of the final outcome). They might not know what
that will entail, i.e., how the commissions decide what demands
to make given that instruction. However – based on their trust in
their representatives – they might still believe that this leads to
better outcomes than giving an explicit instruction.
We will also imagine these instructions are (or at least can be)
given publicly and that the commissions are bound (as if by a
contract) to follow these instructions. In particular, we imagine
that the two commissions can see each other’s instructions. Thus,
in instructing their commissions, the countries play a game with
bilateral precommitment. When instructed to play a game as best
as they can, we imagine that the commissions play that game in
the usual way, i.e., without further abilities to credibly commit or
to instruct subcommittees and so forth.
It may seem that without having their parliaments ponder equi-
librium selection, Aliceland and Bobbesia cannot do better than
leave the game to their representatives. Unfortunately, in this de-
fault equilibrium, war is still a possibility. Even the brilliant strate-
gists Alice and Bob may not always be able to resolve the difficult
equilibrium selection problem to the same pure Nash equilibrium.
In the literature on commitment devices and in particular the
literature on program equilibrium, important ideas have been pro-
posed for avoiding such bad outcomes. Imagine for a moment that
Alice and Bob will play a Prisoner’s Dilemma (rather than the De-
mand Game of Table 1). Then the default of (Defect, Defect) can be
Pareto-improved upon. Both original players (Aliceland and Bobbe-
sia) can use the following instruction for their representatives: “If
the opponent’s instruction is equal to this instruction, Cooperate;
otherwise Defect.” [ 11,15,25] Then it is a Nash equilibrium for
both players to use this instruction. In this equilibrium, (Cooperate,
Cooperate) is played and it is thus Pareto-optimal and Pareto-better
than the default.
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
Player 2
DM RM DL RL
Player 1DM−5,−5 2,0 5,−55,−5
RM 0,2 1,1 5,−55,−5
DL−5,5−5,5 1,1 2,0
RL−5,5−5,5 0,2 1,1
Table 1: The Demand Game
In cases like the Demand Game, it is more difficult to apply this
approach to improve upon the default of simply delegating the
choice. Of course, if one could calculate the expected utility of sub-
mitting the default instructions, then one could similarly commit the
representatives to follow some (joint) mix over the Pareto-optimal
outcomes ((RM,DM),(DM,RM),(RM,RM),(DL,DL), etc.) that
Pareto-improves on the default expected utilities. However, we
will assume that the original players are unable or unwilling to
form probabilistic expectations about how the representatives play
the Demand Game, i.e., about what would happen with the default
instructions. If this is the case, then this type of Pareto-improvement
on the default is unappealing.
The goal of this paper is to show and analyze how even without
forming probabilistic beliefs about the representatives, the original
players can Pareto-improve on the default equilibrium. We will call
such improvements safe Pareto improvements (SPIs). We here briefly
give an example in the Demand Game.
The key idea is for the original players to instruct the repre-
sentatives to select only from {DL,RL}, i.e., to not raise a military.
Further, they tell them to disvalue the conflict outcome (DL,DL)as
they would disvalue the original conflict outcome of war in the de-
fault equilibrium. Overall, this means telling them to play the game
of Table 2. (Again, we could imagine that the instructions specify
Table 2 to be how Aliceland and Bobbesia financially reward Alice
and Bob.) Importantly, Aliceland’s instruction to play that game
must be conditional on Bobbesia also instructing their commission
to play that game, and vice versa. Otherwise, one of the countries
could profit from deviating by instructing their representative to
always play DMorRM(or to play by the original utility function).
The game of Table 2 is isomorphic to the DM-RMpart of the orig-
inal Demand Game of Table 1. Of course, the original players know
neither how the original Demand Game nor the game of Table 2 will
be played by the representatives. However, since these games are
isomorphic, one should arguably expect them to be played isomor-
phically. For example, one should expect that (RM,DM)would be
played in the original game if and only if (RL,DL)would be played
in the modified game. However, the conflict outcome (DM,DM)is
replaced in the new game with the outcome (DL,DL). This outcome
is harmless (Pareto-optimal) for the original players.
Contributions. Our paper generalizes this idea to arbitrary normal-
form games and is organized as follows. In Section 2, we introduce
some notation for games and multivalued functions that we will
use throughout this paper. In Section 3, we introduce the setting of
delegated game playing for this paper and define and motivate the
concept of safe Pareto improvements in more detail. In Section 4,Player 2’s rep.
DL RL
Player 1’s rep.DL−5,−5 2,0
RL 0,2 1,1
Table 2: A safe Pareto improvement for the Demand Game
we briefly review the the concepts of program games and program
equilibrium and show that SPIs can be implemented as program
equilibria. In Section 5, we introduce a notion of outcome corre-
spondence between games. This relation expresses the original
players’ beliefs about similarities between how the representatives
play different games. For example, in our example, the Demand
Game of Table 1 (arguably) corresponds to the game of Table 2 in
that the representatives (arguably) would play (DM,DM)in the
original game if and only if they play (DL,DL)in the new game,
and so forth. We also show some basic results (reflexivity, transitiv-
ity, etc.) about the outcome correspondence relation on games. In
Section 6 we show that the notion of outcome correspondence is
central to deriving SPIs. In particular, we show that a game Γ𝑠is an
SPI on another game Γif and only if there is a Pareto-improving
outcome correspondence relation between Γ𝑠andΓ.
To derive SPIs, we need to make some assumptions about out-
come correspondence, i.e., about which games are played in similar
ways by representatives. We give two very weak assumptions of
this type in Section 7. The first is that the representatives play
isomorphic games isomorphically. The second is that the represen-
tatives’ play is invariant under the removal of strictly dominated
strategies. For example, we assume that in the Demand Game the
representatives only play DMandRM. Moreover we assume that
we could remove DLandRLfrom the game and the representatives
would still play the same strategies as in the original Demand Game
with certainty. Our SPI for the Demand Game can be proven using
these assumptions. Section 8 shows that determining whether there
exists an SPI based on these assumptions is NP-complete. Section 9
considers a different setting in which we allow the original players
to let the representatives choose from newly constructed strategies
whose corresponding outcomes map arbitrarily onto feasible payoff
vectors from the original game. In this new setting, finding SPIs can
be done in polynomial time. We conclude by discussing the problem
of selecting between different SPIs on a given game (Section 10)
and giving some ideas for directions for future work (Section 11).
2 PRELIMINARIES
2.1 Games
We here recall some basic definitions from game theory. An 𝑛-player
game is a tuple(𝐴,u)of a set𝐴=𝐴1×...×𝐴𝑛of(pure) strategy
profiles (or outcomes) and a function u:𝐴→R𝑛that assigns to
each outcome a utility for each player. Instead of, (𝐴,u)will also
write(𝐴1,...,𝐴𝑛,𝑢1,...,𝑢𝑛). We say that 𝑎𝑖∈𝐴𝑖strictly dominates
𝑎′
𝑖∈𝐴𝑖if for all𝑎−𝑖∈𝐴−𝑖,𝑢𝑖(𝑎𝑖,𝑎−𝑖)>𝑢𝑖(𝑎′
𝑖,𝑎−𝑖). For any given
game Γ=(𝐴,u), we will call any game Γ′=(𝐴′,u′)asubset game
ofΓif𝐴′
𝑖⊆𝐴𝑖for𝑖=1,...,𝑛 . Note that a subset game may assign
different utilities to outcomes than the original game. For any set of
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
strategies𝑆, we denote by Γ−𝑆B((𝐴1−𝑆)×....,(𝐴𝑛−𝑆),u)the
game that arises from Γby removing the strategies 𝑆for all players.
We say that some utility vector y∈R𝑛is a Pareto-improvement
on (or is Pareto-better than) y′∈R𝑛if𝑦𝑖≥𝑦′
𝑖for𝑖=1,...,𝑛 . We
will also denote this by y≥y′. Note that, contrary to convention,
we allow y=y′. Whenever we require one of the inequalities to
be strict, we will say that yis a strict Pareto improvement on y′.
In a given game, we will also say that an outcome ais a Pareto-
improvement on another outcome a′ifu(a)≥u(a′). We say that
yis Pareto-optimal or Pareto-efficient relative to some 𝑆⊂R𝑛if
there is no element of 𝑆that strictly Pareto-dominates y.
The Demand Game of Table 1 is an example of a game that we
will use throughout this paper. As noted earlier, DMandRMstrictly
dominate DLandRL. The game of Table 2 is a subset game of the
Demand Game.
2.2 Multivalued functions
For sets𝑀and𝑁, amulti-valued function Φ:𝑀⊸𝑁is a function
which maps each element 𝑚∈𝑀to a set Φ(𝑚)⊆𝑁. For a subset
𝑄⊆𝑀, we define Φ(𝑄)BÐ
𝑚∈𝑄Φ(𝑚). Note that Φ(𝑄) ⊆𝑁
and that Φ(∅)=∅. For any set 𝑀, we define the identity function
id𝑀:𝑀⊸𝑀:𝑚→ {𝑚}. Also, for two sets 𝑀,𝑁 , we define
all𝑀,𝑁:𝑀⊸𝑁:𝑚↦→𝑁. We define the inverse Φ−1:𝑁⊸
𝑀:𝑛↦→{𝑚∈𝑀|𝑛∈Φ(𝑚)}. Note that Φ−1(∅)=∅for any
multi-valued function Φ. For sets𝑀,𝑁,𝑄 and functions Φ:𝑀⊸
𝑁,Ψ:𝑁⊸𝑄, we define the composite Ψ◦Φ:𝑀⊸𝑄:𝑚↦→
Ψ(Φ(𝑚)). As with regular functions, composition of multi-valued
functions is associative. We say that Φ:𝑀⊸𝑁issingle-valued
if|Φ(𝑚)|=1for all𝑚∈𝑀. Whenever a multi-valued function is
single-valued, we can apply many of the terms for regular functions.
For example, we will take injectivity, surjectivity, and bijectivity for
single-valued functions to have the usual meaning. We will never
apply these notions to non-single-valued functions.
3 DELEGATION AND SAFE PARETO
IMPROVEMENTS
We consider a setting in which a given game Γis played through
what we will call representatives . For example, the representatives
could be humans whose behavior is determined or incentivized by
some contract à la the principal–agent literature [14].
We imagine that one way in which the representatives can be
instructed is to in turn play a subset game Γ𝑠=(𝐴𝑠
1⊆𝐴1,...,𝐴𝑠𝑛⊆
𝐴𝑛,u𝑠)of the original game, without necessarily specifying a strategy
or algorithm for solving such a game . We emphasize, again, that u𝑠
is allowed to be a vector of entirely different utility functions. For
any subset game Γ𝑠, we denote by Π(Γ𝑠)the outcome that arises
if the representatives play the subset game Γ𝑠ofΓ. Because in
many games, it is not clear what the right choice is, the original
players might be uncertain about Π(Γ𝑠)for many games Γ𝑠. We
will therefore model each Π(Γ𝑠)as a random variable.
The original players trust their representatives to the extent that
we take Π(Γ)to be a default way for the game to played for any Γ.
For example, in the Game of Chicken, it is not clear what the right
action is. Thus, if one can simply delegate the decision to someone
with more relevant expertise, that is the first option one would
consider.We are interested in whether and how the original players can
jointly Pareto-improve on the default. Of course, one option is to
compute the expected utilities in the default ( E[u(Π(Γ))]) and then
let the representatives play a distribution over outcomes whose
expected utility exceeds that default expected utility. However, this
is unrealistic if Γis a complex game with multiple Nash equilibria.
For one, the precise point of delegation is that the original players
are unable or unwilling to properly evaluate Γ. Second, there is no
widely agreed upon, universal procedure for selecting an action in
the face of equilibrium selection problems.
We address this problem in a typical way. Essentially, we require
of any attempted improvement that it incurs no regret in the worst-
case. That is, we are interested in subset games Γ𝑠that are Pareto
improvements with certainty under weak and purely qualitative
assumptions about Π.
Definition 3.1. LetΓ𝑠be a subset game of Γ. We say Γ𝑠is a
safe Pareto improvement (SPI) onΓifu(Π(Γ𝑠))≥ u(Π(Γ))with
certainty. We say that Γ𝑠is astrict SPI if furthermore, there is a
player𝑖s.t.𝑢𝑖(Π(Γ𝑠))>𝑢𝑖(Π(Γ𝑠))with positive probability.
4 PROGRAM EQUILIBRIUM
So far, we have been vague about the details of the strategic situation
that the original players face in instructing their representatives.
From what set of actions can they choose? How can they jointly
let the representatives play some new subset game Γ𝑠? Are SPIs
Nash equilibria of the meta game played by the representatives?
In this section, we briefly describe one way to fill this gap by dis-
cussing the concept of program games and program equilibrium
[3,6,8,17,25]. This section is essential to understanding why
SPIs are relevant. However, the remaining technical content of this
paper does not rely on this section and the main ideas presented
here are straightforward from previous work. For more detail, see
Appendix A.
For any game Γ=(𝐴,u), the program equilibrium literature
considers the following meta game. First, each player 𝑖chooses
from a set of computer programs. Each program then receives as
input a vector containing everyone else’s chosen program. Each
player𝑖’s program then returns an action from 𝐴𝑖, player𝑖’s set of
actions in Γ. Together these actions then form an outcome a∈𝐴of
the original game. Finally, the utilities u(a)are realized according
to the utility function of Γ. The meta game can be analyzed like
any other game. Its Nash equilibria are called program equilibria .
Importantly, the program equilibria can implement payoffs not
implemented by any Nash equilibria of Γitself. For example, in
the Prisoner’s Dilemma, both players can submit a program that
says: “If the opponent’s chosen computer program is equal to this
computer program, Cooperate; otherwise Defect.” [ 11,15,25] This
is a program equilibrium which implements mutual cooperation.
In the setting for our paper, we similarly imagine that each player
𝑖can choose from a set of programs that in turn choose from 𝐴𝑖.
However, the types of program that we have in mind here are more
sophisticated than those typically considered in the program equi-
librium literature. Specifically we imagine that the programs are
executed by intelligent representatives who are themselves able to
competently choose an action for player 𝑖in any given game Γ𝑠,
without the original player having to describe how this choice is
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
to be made. The original player may not even understand much
about this program other than that it generally plays well. Thus, in
addition to the elementary instructions used in a typical computer
program (branches, comparisons, arithmetic operations, etc.), we
allow player 𝑖to use an instruction “Play Π𝑖(Γ𝑠)” in the program
she submits. To jointly let the representatives play, e.g., the SPI Γ𝑠
of Table 2 on the Demand Game of Table 1, the representatives
can both use an instruction that says, “If the opponent’s chosen
program is analogous to this one, play Π𝑖(Γ𝑠); otherwise play DM”.
Assuming some minimal rationality requirements on the represen-
tatives (i.e., on how “play Π𝑖(Γ𝑠)” is implemented), this is a Nash
equilibrium. More generally, we can prove the following result (see
Appendix A for a proof).
Theorem 4.1. LetΓbe a game and Γ𝑠be an SPI of Γ. Now consider
a program game on Γ, where each player 𝑖can choose from a set of
computer programs that output actions for Γ. In addition to the normal
kind of instructions, we allow the use of the command “play Π𝑖(Γ′)”
for any subset game Γ′ofΓ. Finally, assume that Π(Γ)guarantees
each player𝑖at least that player’s minimax utility (a.k.a. threat point)
in the base game Γ. Then Π(Γ𝑠)is played in a program equilibrium,
i.e., in a Nash equilibrium of the program game.
As an alternative to having the original players choose contracts
separately, we could imagine the use of jointly signed contracts
which only come into effect once signed by all players [cf. 12,16].
Also compare earlier work by Sen [ 24] and [ 21], which we discuss
in Appendix B.
5 OUTCOME CORRESPONDENCE BETWEEN
GAMES
In this section, we introduce a notion of outcome correspondence,
which we will see is essential to constructing SPIs.
Definition 5.1. Consider two games Γ=(𝐴1,...,𝐴𝑛,u)andΓ′=
(𝐴′
1,...,𝐴′𝑛,u′). We write Γ∼ΦΓ′forΦ:𝐴⊸𝐴′ifΠ(Γ′) ∈
Φ(Π(Γ))with certainty.
Note that Γ∼ΦΓ′is a statement about Π, i.e., about how the
representatives choose. Whether such a statement holds generally
depends on the specific representatives being used. In Section 7, we
describe two general circumstances under which it seems plausible
thatΓ∼ΦΓ′. For example, if two games ΓandΓ′are isomorphic,
then one might expect Γ∼ΦΓ′, where Φis constructed from the 𝑛
isomorphisms of the particular action spaces.
We now state some basic facts about the relation ∼, many of
which we will use throughout this paper.
Lemma 5.2. LetΓ=(𝐴,u),Γ′=(𝐴′,u′),ˆΓ=(ˆ𝐴,ˆu)andΦ,Ξ:𝐴⊸
𝐴′,Ψ:𝐴′⊸ˆ𝐴.
(1) Reflexivity: Γ∼id𝐴Γ, where id𝐴:𝐴⊸𝐴:a↦→{a}.
(2) Symmetry: If Γ∼ΦΓ′, then Γ′∼Φ−1Γ.
(3) Transitivity: If Γ∼ΦΓ′andΓ′∼ΨˆΓ, then Γ∼Ψ◦ΦˆΓ.
(4) If Γ∼ΦΓ′andΦ(a)⊆Ξ(a)for all a∈𝐴, then Γ∼ΞΓ′.
(5)Γ∼all𝐴,𝐴′Γ′, where all𝐴,𝐴′:𝐴⊸𝐴′:a↦→𝐴′.
(6) If Γ∼ΦΓ′andΦ(a)=∅, then Π(Γ)≠awith certainty.
For completness, we prove these in Appendix C. Items 1–3 show
that∼has properties resembling those of an equivalence relation.Note, however, that since ∼is not a binary relationship, ∼itself
cannot be an equivalence relation in the usual sense. Item 4 states
that we can make an outcome correspondence claim less precise
and it will still hold true. Item 5 states that in the extreme, it is
always Γ∼all𝐴,𝐴′Γ′, where all𝐴,𝐴′is the trivial, maximally impre-
cise outcome correspondence function that confers no information.
Item 6 shows that∼can be used to express the elimination of out-
comes, i.e., the belief that a particular outcome (or strategy) will
never occur.
6 SAFE PARETO IMPROVEMENTS THROUGH
OUTCOME CORRESPONDENCE
We now show that as advertised, outcome correspondence is closely
tied to SPIs. The following theorem shows not only how outcome
correspondences can be used to find (and prove) SPIs. It also shows
that any SPI requires an outcome correspondence relation with
what we will call a Pareto-improving correspondence function.
Theorem 6.1. LetΓ=(𝐴,u)be a game and Γ𝑠=(𝐴𝑠,u𝑠)be a
subset game of Γ. Then Γ𝑠is an SPI on Γif and only if there is Φsuch
thatΓ∼ΦΓ𝑠and for all a∈𝐴it is for all a𝑠∈Φ(a)the case that
u(a𝑠)≥u(a).
Proof.⇐: By definition, Π(Γ𝑠)∈Φ(Π(Γ))with certainty. Hence,
for𝑖=1,2,𝑢𝑖(Π(Γ𝑠))∈𝑢𝑖(Φ(Π(Γ)))with certainty. Hence, by as-
sumption about Φ, with certainty, 𝑢𝑖(Π(Γ𝑠))≥𝑢𝑖(Π(Γ)).
⇒: Assume that 𝑢𝑖(Π(Γ))≥𝑢𝑖(Π(Γ𝑠))with certainty for 𝑖=
1,2. We define Φ:𝐴→𝐴𝑠:a↦→{a𝑠∈𝐴𝑠|u(a𝑠)≥u(a)}. It is
immediately obvious that Φis Pareto-improving as required. Also,
whenever Π(Γ)=aandΠ(Γ𝑠)=a𝑠for any a∈𝐴anda𝑠∈𝐴𝑠, it is
(by assumption) with certainty u(a𝑠)≥u(a). Thus, by definition of
Φ, it holds that a𝑠∈Φ(a). We conclude that Γ∼ΦΓ𝑠as claimed. □
Note that the theorem concerns weak SPIs and therefore allows
the case where with certainty u(Π(Γ))=u(Π(Γ𝑠)). To show that
someΓ𝑠is astrict SPI, we need additional information about which
outcomes occur with positive probability.
We now illustrate how outcome correspondences can be used
to derive the SPI for the Demand Game from the introduction
as per Theorem 6.1. Of course, at this point we do not have any
assumptions about when games are equivalent. We will introduce
some in the following section. Nevertheless, we can already sketch
the argument. Let Γbe the Demand Game of Table 1. First, it seems
plausible that Γis in some sense equivalent to Γ′, where Γ′=
Γ−{DL,RL}is the game that results from removing DLandRL
for both players from Γ. Again, strict dominance could be given as
an argument. We can formalize this as Γ∼ΦΓ′, where Φ(𝑎1,𝑎2)=
{(𝑎1,𝑎2)}if𝑎1,𝑎2∈{DM,RM}andΦ(𝑎1,𝑎2)=∅otherwise. In a
second step, it seems plausible that Γ′∼ΨΓ𝑠, where Γ𝑠is the game
of Table 2 and Ψis the isomorphism between Γ′andΓ𝑠. Finally, we
can use transitivity to obtain Γ∼Ψ◦ΦΓ𝑠. To see that Ψ◦Φis Pareto-
improving for the original utility functions of Γ, notice that Φdoes
not change utilities at all. Ψmaps the conflict outcome (DM,DM)
onto the outcome(DL,DL), which is better for both original players.
Other than that, Ψ, too, does not change the utilities. Hence, Ψ◦Φ
is Pareto-improving. By Theorem 6.1, Γ𝑠is therefore an SPI on Γ.
In principle, Theorem 6.1 does not hinge on Π(Γ)andΠ(Γ𝑠)
resulting from playing games. An analogous result holds for any
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
random variables over 𝐴and𝐴𝑠. In particular, this means that
Theorem 6.1 applies also if the representatives receive other kinds of
instructions (cf. Section 4). However, it seems hard to establish non-
trivial outcome correspondences between Π(Γ)and other types
of instructions. Still, the use of more complicated instructions can
be used to derive different kinds of SPIs. For example, if there
are different game SPIs, then the original players could tell their
representatives to randomize between them in a coordinated way.
7 ASSUMPTIONS ABOUT OUTCOME
CORRESPONDENCE
To make any claims about how the original players should play the
meta-game, i.e., about what instructions they should submit, we
have to make assumptions about how the representatives choose
and (by Theorem 6.1) about outcome correspondence in particular.
We here make two fairly weak assumptions. The first is that the
representatives play two isomorphic games isomorphically.
Assumption 7.1. LetΓ=(𝐴,u)andΓ′=(𝐴′,u′)be two games
for which there are single-valued bijections Φ𝑖:𝐴𝑖→𝐴′
𝑖for𝑖=
1,...,𝑛 such that u(𝑎1,...,𝑎𝑛)=u′(Φ1(𝑎1),...,Φ𝑛(𝑎𝑛))for all a∈𝐴.
Then for one tuple Φof such bijections, Γ∼ΦΓ′.
Similar desiderata have been discussed in the context of equi-
librium selection, e.g., by Harsanyi and Selten [ 10, Chapter 3.4].
In fact, they consider a generalization in which the utilities are
allowed to be linear transformations of each other. Although this
generalization is extremely plausible, we omit it here for simplicity.
One could criticize Assumption 7.1 by referring to focal points
(introduced by Schelling [ 22, pp. 54–58]) as an example where con-
text and labels of strategies matter. A possible response might be
that in games where context plays a role, that context should be
included as additional information and not be considered part of
(𝐴,u). Assumption 7.1 would then either not apply to such games
with (relevant) context or would require one to, in some way, trans-
late the context along with the strategies. However, in this paper
we will not formalize context, and assume that there is no decision-
relevant context.
Assumption 7.2. LetΓ=(𝐴,u)be an arbitrary 𝑛-player game
where𝐴1,...,𝐴𝑛are pairwise disjoint, and ˜𝑎𝑖∈𝐴𝑖be strictly domi-
nated by some other strategy ˆ𝑎𝑖∈𝐴𝑖. Then Γ∼ΦΓ−{˜𝑎𝑖}, where for
all𝑎−𝑖∈𝐴−𝑖,Φ(˜𝑎𝑖,𝑎−𝑖)=∅andΦ(𝑎𝑖,𝑎−𝑖)={(𝑎𝑖,𝑎−𝑖)}whenever
𝑎𝑖≠˜𝑎𝑖.
Assumption 7.2 expresses that representatives should never play
strictly dominated strategies. Moreover, it states that we can re-
move strictly dominated strategies from a game and the resulting
game will be played in the same way by the representatives. For
example, this implies that when evaluating a strategy 𝑎𝑖, the repre-
sentatives do not take into account how many other strategies 𝑎𝑖
strictly dominates. Assumption 7.2 also allows (via Transitivity of
∼as per Lemma 5.2.3) the iterated removal of strictly dominated
strategies. The notion that we can (iteratively) remove strictly dom-
inated strategies is common in game theory [ 13,18,19, Section 2.9,
Chapter 12] and has rarely been questioned. It is also implicit in
the solution concept of Nash equilibrium – if a strategy is removed
by iterated strict dominance, that strategy is played in no Nash
equilibrium. However, like the concept of Nash equilibrium, theelimination of strictly dominated strategies becomes implausible if
the game is not played in the usual way. In particular, for Assump-
tion 7.2 to hold, we will in most games Γhave to assume that the
representatives cannot in turn make credible precommitments (or
delegate to further subrepresentatives) or play the game iteratively
[2].
With Assumptions 7.1 and 7.2 we can finally state our example
SPIs formally: Our proofs are in Appendix D.
Proposition (Example) 7.1. LetΓbe the Prisoner’s Dilemma
andΓ𝑠=(𝐴𝑠
1,𝐴𝑠
2,𝑢𝑠
1,𝑢𝑠
2)be any subset game of Γwith𝐴𝑠
1=𝐴𝑠
2=
{Cooperate}. Then under Assumption 7.2, Γ𝑠is a strict SPI on Γ.
Proposition (Example) 7.2. LetΓbe the Demand Game of Table 1
andΓ𝑠be the subset game described in Table 2. Under Assumptions 7.1
and 7.2, Γ𝑠is an SPI on Γ. Further, if𝑃(Π(Γ)=(DM,DM))>0,Γ𝑠is
a strict SPI.
8 COMPUTING SAFE PARETO
IMPROVEMENTS
In this section, we ask how computationally costly it is for the
original players to identify for a given game Γa non-trivial SPI Γ𝑠.
In particular, we ask whether a given game Γhas a non-trivial SPI
that can be proved using only Assumptions 7.1 and 7.2, Transitivity
(Lemma 5.2.3) and Theorem 6.1. Formally:
Definition 8.1. TheSPI decision problem consists in deciding for
any given Γ, whether there is a sequence of outcome correspon-
dences Φ1,...,Φ𝑘and a sequence of subset games Γ0=Γ,Γ1,...,Γ𝑘
ofΓs.t.:
(1)(Non-triviality:) If we fully reduce Γ𝑘andΓusing iterated
strict dominance (Assumption 7.2), the two resulting games
are not equal. (Of course, they are allowed to be isomorphic.)
(2)For𝑖=1,...,𝑘 ,Γ𝑖−1∼Φ𝑖Γ𝑖is valid by a single application of
either Assumption 7.1 or Assumption 7.2.
(3)For all a∈𝐴, and whenever a𝑠∈(Φ𝑘◦Φ𝑘−1◦...◦Φ1)(a),
it is the case that 𝑢(a𝑠)≥u(a).
For the strict SPI decision problem, we further require:
(4.)There is a player 𝑖and an outcome athat survives iter-
ated elimination of strictly dominated strategies from Γs.t.
𝑢𝑖((Φ𝑘◦Φ𝑘−1◦...◦Φ1)(a))>𝑢𝑖(a).
Many variants of this problem may be considered. For example,
we might generalize it to allow imposing additional properties on
the SPI. This will generally not change the computational complex-
ity of the problem. One may also wish to compute all SPIs, or – in
line with multi-criteria optimization [ 7,27] – all SPIs that cannot in
turn be safely improved upon. However, in general there may exist
exponentially many such SPIs. To retain any hope of developing
an efficient algorithm, one would therefore have to first develop a
more efficient representation scheme [cf. 20, Sect. 16.4].
Theorem 8.2. The (strict) SPI decision problem is NP-complete,
even for 2-player games.
Proposition 8.3. For games Γwith|𝐴1|+...+|𝐴𝑛|=𝑚that can
be reduced (via iterative application of Assumption 7.2) to a game
Γ′with|𝐴′
1|+...+|𝐴′𝑛|=𝑙, the (strict) SPI decision problem can be
solved in𝑂(𝑚𝑙).
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
The full proof is tedious (see Appendix E), but the main idea
is simple. To find an SPI on Γbased on Assumptions 7.1 and 7.2,
one has to first iteratively remove all strictly dominated actions to
obtain a reduced game Γ′, which the representatives would play
the same as the original game. This can be done in polynomial time.
One then has to map the actions Γ′onto the original Γin such a
way that each outcome in Γ′is mapped onto a weakly Pareto-better
outcome in Γ. Our proof of NP-hardness works by reducing from
the subgraph isomorphism problem, where the payoff matrices of
Γ′,Γrepresent the adjacency matrices of the graphs.
Besides being about a specific set of assumptions about ∼, note
that Theorem 8.2 and Proposition 8.3 also assume that the utility
function of the game is represented explicitly in normal form as
a payoff matrix. If we changed the game representation (e.g., to
boolean circuits, extensive form game trees, quantified boolean
formulas, or even Turing machines), this can affect the complexity
of the SPI problem [cf. 9]. In fact, even reducing a game using strict
dominance by pure strategies – which contributes only insignifi-
cantly to the complexity of the SPI problem for normal-form games
– is difficult in some game representations [4, Section 6].
9 SAFE PARETO IMPROVEMENTS UNDER
IMPROVED COORDINATION
In this section, we imagine that the players are able to simply
invent new token strategies with new payoffs that arise from mixing
existing feasible payoffs. To define this formally, we first define for
any game Γ=(𝐴,u),
C(Γ)Bu(Δ(𝐴))=(Õ
a∈𝐴𝑝au(a)∀a∈𝐴:𝑝a∈[0,1],Õ
a∈𝐴𝑝a=1)
to be the set of feasible coordinated payoff vectors of Γ, which is
exactly the convex closure of u(𝐴), i.e., of the deterministically
achievable utilities of the original game.
For any game Γ, we then imagine that in addition to subset games,
the players can let the representatives play a perfect-coordination
token game(𝐴𝑠,u𝑠,u𝑒), where for all 𝑖,𝐴𝑠
𝑖∩𝐴𝑖=∅and𝑢𝑠
𝑖:𝐴𝑠→R
are arbitrary utility functions to be used by the representatives and
u𝑒:𝐴𝑠→C( Γ)are the utilities that the original players assign to
the token strategies.
The instruction(𝐴𝑠,u𝑠,u𝑒)lets the representatives play the
game(𝐴𝑠,u𝑠)as usual. However, the strategies 𝐴𝑠are imagined
to be meaningless token strategies which do not resolve the given
game Γ. Once some token strategies a𝑠are selected, these are
translated into some probability distribution over 𝐴, i.e., over out-
comes of the original game, thus giving rise to (expected) utilities
u𝑒(a𝑠)∈C( Γ). These distributions and thus utilities are specified
by the original players. We here imagine in our definition of C(Γ)
that these distributions over 𝐴could require the representatives to
correlate their choices for the original game for any given a𝑠.
Definition 9.1. LetΓbe a game. A perfect-coordination SPI
forΓis a perfect-coordination token game (𝐴𝑠,u𝑠,u𝑒)forΓs.t.
u𝑒(Π(𝐴𝑠,𝑢𝑠)) ≥ u(Π(Γ))with certainty. We call (𝐴𝑠,u𝑠,u𝑒)a
strict perfect-coordination SPI if there furthermore is a player 𝑖
for whom𝑢𝑒
𝑖(Π(𝐴𝑠,𝑢𝑠))>𝑢𝑖(Π(Γ))with positive probability.As an example, imagine that Γis just the DM-RMsubset game
of the Demand Game of Table 1. Then, intuitively, an SPI under
improved coordination could consist of the original players telling
the representatives, “Play as if you were playing the DM-RMsubset
game of the Demand Game, but whenever you find yourself play-
ing(DM,DM), randomize [according to some given distribution]
between the other (Pareto-optimal) outcomes instead”. Formally,
𝐴𝑠
1={ˆ𝐷,ˆ𝑅},𝐴𝑠
2={ˆ𝐷,ˆ𝑅}would then consist of tokenized versions
of the original strategies. The utility functions 𝑢𝑠
1,𝑢𝑠
2are then sim-
ply the same as in the original Demand Game except that they are
applied to the token strategies. E.g., u𝑠(ˆ𝐷,ˆ𝑅)=(2,0). The utilities
for the original players remove the conflict outcome. For example,
the original players might specify u𝑒(ˆ𝐷,ˆ𝐷)=(1,1), represent-
ing that the representatives are supposed to play (RM,RM)in the
(ˆ𝐷,ˆ𝐷)case. For all other outcomes (ˆ𝑎1,ˆ𝑎2), it must be the case
that u𝑒(ˆ𝑎1,ˆ𝑎2)=u𝑠(ˆ𝑎1,ˆ𝑎2)because the other outcomes cannot be
Pareto-improved upon. As with our earlier SPIs for the Demand
Game, Assumption 7.1 implies that Γ∼ΦΓ𝑠, where Φmaps the
original conflict outcome (DM,DM)onto the Pareto-optimal ( ˆ𝐷,ˆ𝐷).
Relative to the SPIs considered up until now, these new types
of instructions put significant additional requirements on how the
representatives interact. They now have to engage in a two-round
process of first choosing and observing one another’s token strate-
gies and then playing the corresponding distribution over outcomes
from the original game. Further, it must be the case that this ad-
ditional coordination does not affect the payoffs of the original
outcomes. The latter may not be the case in, e.g., the Game of
Chicken. That is, we could imagine a Game of Chicken in which
coordination is possible but that the rewards of the game change
if the players do coordinate. After all, the underlying story in the
Game of Chicken is that the positive reward (admiration from peers)
is attained precisely for accepting a grave risk.
With these more powerful ways to instruct representatives, we
can now replace individual outcomes of the default game ad libi-
tum. For example, in the reduced Demand Game, we singled out
the outcome(DM,DM)as Pareto-suboptimal and replaced it by
a Pareto-optimal outcome, while keeping all other outcomes the
same. This allows us to construct SPIs in many more games then
before.
Definition 9.2. The strict full-coordination SPI decision problem
consists in deciding for any given Γwhether under Assumption 7.1
there is a perfect-coordination SPI Γ𝑠forΓ.
Lemma 9.3. For a given𝑛-player game Γand payoff vector y∈R𝑛,
it can be decided by linear programming and thus in polynomial time
whether yis Pareto-optimal in C(Γ).
For completeness the linear program is given in Appendix F.
Based on Lemma 9.3, Algorithm 1 decides whether there is a strict
perfect-coordination SPI for a given game Γ.
It is easy to see that this algorithm runs in polynomial time (in
the size of, e.g., the normal form representation of the game). It is
also correct: if it returns True, simply replace the Pareto-suboptimal
outcome while keeping all other outcomes the same; if it returns
False, then all outcomes are Pareto-optimal within C(Γ)and so
there can be no strict SPI. We summarize this result in the following
proposition.
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
Algorithm 1: An algorithm for deciding the strict perfect-
coordination SPI problem.
Data: Game Γ, setsupp(Π(Γ)
1fora∈supp(Π(Γ))do
2 ifu(a)is Pareto-suboptimal within C(Γ)then
3 Return True;
4Return False;
Proposition 9.4. Assuming supp(Π(Γ))is known and that As-
sumption 7.1 holds, it can be decided in polynomial time whether
there is a strict perfect-coordination SPI.
From the problem of deciding whether there are strict SPIs un-
der improved coordination at all, we move on to the question of
what different perfect-coordination SPIs there are. In particular, one
might ask what the cost is of only considering safePareto improve-
ments relative to acting on a probability distribution over Π(Γ)and
the resulting expected utilities E[u(Π(Γ))]. We start with a lemma
that directly provides a characterization. So far, all the considered
perfect-coordination SPIs (𝐴𝑠,u𝑠,u𝑒)for a game(𝐴,u)have con-
sisted in letting the representatives play a game (𝐴𝑠,u𝑠)that is
isomorphic to the original game, but Pareto-improves (from the
original players’ perspectives, i.e., u𝑒) at least one of the outcomes.
It turns out that we can restrict attention to this very simple type
of SPI under improved coordination.
Lemma 9.5. LetΓ=({𝑎1
1,...,𝑎𝑙1
1},...,{𝑎1𝑛,...,𝑎𝑙𝑛𝑛},u)be any game.
LetΓ′be a perfect-coordination SPI on Γ. Then we can define u𝑒with
values inC(Γ)such that under Assumption 7.1 the game
Γ𝑠=
ˆ𝐴1B{ˆ𝑎1
1,...,ˆ𝑎𝑙1
1},...,ˆ𝐴𝑛B{ˆ𝑎1
𝑛,...,ˆ𝑎𝑙𝑛𝑛},
ˆu:(ˆ𝑎𝑖1
1,...,ˆ𝑎𝑖𝑛𝑛)↦→ u(𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛),u𝑒
is also an SPI on Γ, with
E
u(Π(Γ𝑠))Π(Γ)=a
=E
u(Π(Γ′))Π(Γ)=a
for all a∈𝐴and consequently E[u(Π(Γ𝑠))]=E[u(Π(Γ′))].
We prove this in Appendix G. Because of this result, we will
focus on these particular types of SPIs, which simply create an
isomorphic game with different (Pareto-better) utilities. Note, how-
ever, that without assigning exact probabilities to the distributions
ofΠ(Γ),Π(Γ′), the original players will in general not be able to
construct aΓ𝑠that satisfies the expected payoff equalities. For this
reason, one could still conceive of situations in which a different
type of SPI would be chosen by the original players and the original
players are unable to instead choose an SPI of the type described
in Lemma 9.5.
Lemma 9.5 directly implies a characterization of the expected
utilities that can be achieved with perfect-coordination SPIs. Of
course, this characterization depends on the exact distribution of
Π(Γ). We omit the statement of this result. However, we state the
following implication.
Corollary 9.6. Under Assumption 7.1, the set of Pareto improve-
ments that are safely achievable with perfect coordination {E[u(Γ′)]|
Γ′is perfect-coordination SPI on Γ}is a convex polygon.Because of this result, one can also efficiently optimize convex
functions over the set of perfect-coordination SPIs. Even without
referring to the distribution Π(Γ), many interesting questions can
be answered efficiently. For example, we can efficiently identify the
perfect-coordination SPI that maximizes the minimum improve-
ments across players and outcomes a∈𝐴.
In the following, we aim to use Lemma 9.5 and Corollary 9.6 to
give maximally strong positive results about what Pareto improve-
ments can be safely achieved, without referring to exact probabili-
ties over Π(Γ). To keep things simple, we will do this only for the
case of two players. To state our results, we first need some notation:
We use PF(C)B
y∈C∄y′∈C,𝑖∈{1,...,𝑛}:y′≥y,𝑦′
𝑖>𝑦 to
denote the Pareto frontier of a convex polygon C(or more generally
convex, closed set). For any real number 𝑥∈R, we use𝜋𝑖(𝑥,C(Γ))
to denote the y′∈C(Γ)which maximizes 𝑦′
−𝑖under the constraint
𝑦′
𝑖=𝑥. (Recall that we consider 2-player games, so 𝑦′
−𝑖is a single
real number.) Note that such a y′exists if and only if 𝑥is𝑖’s utility
in some feasible payoff vector. We first state our result formally. Af-
terwards, we will give a graphical explanation of the result, which
we believe is easier to understand.
Theorem 9.7. Make Assumption 7.1. Let Γbe a two-player game.
Lety∈R2be some potentially unsafe Pareto improvement on
E[u(Π(Γ))]. For𝑖=1,2, let𝑥min/max
𝑖=min/max𝑢𝑖(supp(Π(Γ))).
Then:
A) If there is some element in C(Γ)which Pareto-dominates all of
supp(Π(Γ))and if yis Pareto-dominated by an element of at least
one of the following three sets:
•𝐿1Bthe line segment between 𝜋1(𝑥min
1,PF(C(Γ))and
𝜋1(𝑥max
1,PF(C(Γ));
•𝐿2Bthe segment of the curve PF(C(Γ))between
𝜋1(𝑥max
1,PF(C(Γ))))and𝜋2(𝑥max
2,PF(C(Γ))));
•𝐿3Bthe line segment between 𝜋2(𝑥max
2,PF(C(Γ))and
𝜋2(𝑥min
2,PF(C(Γ)).
Then there is an SPI under improved coordination Γ𝑠such that
E[u(Π(Γ𝑠))]=y.
B) If there is no element in C(Γ)which Pareto-dominates all of
supp(Π(Γ))and if yis Pareto-dominated by an element each of 𝐿1
and𝐿3as defined above, then there is a perfect-coordination SPI Γ𝑠
such that E[u(Π(Γ𝑠))]=y.
We now illustrate the result graphically. We start with Case A,
which is illustrated in Figure 1. The Pareto-frontier is the solid
line in the north and east. The points marked x indicate outcomes
insupp(Π(Γ)). The point marked by a filled circle indicates the
expected value of the default equilibrium E[u(Π(Γ))]. For some
y∈R2to be a Pareto-improvement, it must be to the north-east of
the filled circle. The vertical dashed lines starting at the two extreme
x marks illustrate the application of 𝜋1to project𝑥min/max
1onto
the Pareto frontier. The dotted line between these two points is 𝐿1.
Similarly, the horizontal dashed lines starting at x marks illustrate
the application of 𝜋2to project𝑥min/max
2onto the Pareto frontier.
The line segment between these two points is 𝐿3. In this case, this
line segments lies on the Pareto frontier. The set 𝐿2is simply that
part of the Pareto frontier, which Pareto-dominates all elements of
supp(Π(Γ)), i.e., the part of the Pareto frontier to the north-east
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
E[𝑢1(Π(Γ𝑠))]
E[𝑢1(Π(Γ)))]E[𝑢2(Π(Γ)))]E[𝑢2(Π(Γ𝑠))]
Figure 1: This figure illustrates Theorem 9.7, Case A.
E[𝑢1(Π(Γ𝑠))]
E[𝑢1(Π(Γ))]E[𝑢2(Π(Γ))])E[𝑢2(Π(Γ))]
Figure 2: This figure illustrates Theorem 9.7, Case B.
between the two intersections with the northern horizontal dashed
line and eastern vertical dashed line.
Case B of Theorem 9.7 is depicted in Figure 2. Note that here
the two line segments 𝐿1and𝐿3intersect. To ensure that a Pareto
improvement is safely achievable, the theorem requires that it is
below both of these lines.
Theorem 9.7 is proven by re-mapping each of the outcomes of
the original game as per Lemma 9.5. For example, the projection
of the default equilibrium E[u(Π(Γ))](i.e., the filled circle) onto
𝐿1is obtained as an SPI by projecting all the outcomes (i.e., all the
x marks) onto 𝐿1. In Case A, any utility vector y∈𝐿2that Pareto-
improves on all outcomes of the original game can be obtained
by re-mapping all outcomes onto y. Other kinds of yare handled
similarly. For a full proof, see Appendix H.
As a corollary of Theorem 9.7, we can see that all (potentially
unsafe) Pareto improvements in the DM-RMsubset game of the De-
mand Game of Table 1 are equivalent to some perfect-coordination
SPI. However, this is not always the case:
Proposition 9.8. There is a game Γ=(𝐴,u), representatives
Πthat satisfy Assumptions 7.1 and 7.2, and an outcome a∈𝐴
s.t.𝑢𝑖(a)>E[𝑢𝑖(Π(Γ))]for all players 𝑖, but there is no perfect-
coordination SPI(𝐴𝑠,u𝑠,u𝑒)s.t. for all players 𝑖,E
𝑢𝑒
𝑖(Π(𝐴𝑠,u𝑠))
=
𝑢𝑖(a).
We prove this in Appendix I.10 THE SPI SELECTION PROBLEM
In the Demand Game, there happens to be a single non-trivial
SPI. However, in general (even without the type of coordination
assumed in Section 9) there may be multiple incomparable SPIs
that result in different payoffs for the players. If multiple SPIs
are available, the original players would be left with the difficult
decision of which SPI to demand in their instruction.
This difficulty of choosing what SPI to demand cannot be denied.
However, we would here like to emphasize that players can profit
from the use of SPIs even without addressing this SPI selection
problem. To do so, a player picks an instruction that is very compli-
ant (“dove-ish”) w.r.t. what SPI is chosen, e.g., one that simply goes
with whatever SPI the other players demand as long as that SPI
cannot further be safely Pareto-improved upon. In many cases, all
such SPIs benefit both players. For example, SPIs in bargaining sce-
narios like the Demand Game remove the conflict outcome, which
benefits all parties. Thus, a player can expect a safe improvement
even under such maximally compliant demands on the selected
SPI.
11 CONCLUSION AND FUTURE DIRECTIONS
Safe Pareto improvements are a promising new idea for delegating
strategic decision making. To conclude this paper, we discuss some
ideas for further research on SPIs.
Straightforward technical questions arise in the context of the
complexity results of Section 8. First, what impact on the complexity
does varying the assumptions have? Our NP-completeness proof is
easy to generalize at least to some other types of assumptions. It
would be interesting to give a generic version of the result. We also
wonder whether there are plausible assumptions under which the
complexity changes in interesting ways. Second, one could ask how
the complexity changes if we use more sophisticated game repre-
sentations (see the remarks at the end of that section). Third, one
could impose additional restrictions on the sought SPI. For example,
some of the players may be unable to have their representative
maximize arbitrary utility functions. We could then ask whether
there is an SPI in which only a given subset of the players adopt
different utility functions and restrictions on the set of available
strategies. Fourth, we could restrict the games under consideration.
Are there games in which it becomes easy to decide whether there
is an SPI?
It would also be interesting to see what real-world situations can
already be interpreted as utilizing SPIs, or could be Pareto-improved
upon using SPIs.
ACKNOWLEDGMENTS
This work was supported by NSF under Award IIS-1814056 and the
Foundational Research Institute (now the Center on Long-Term
Risk). For valuable comments and discussions, we thank our anony-
mous referees, Keerti Anand, Tobias Baumann, Jesse Clifton, Max
Daniel, Lukas Gloor, Adrian Hutter, Anni Leskelä, Brian Tomasik
and Johannes Treutlein.
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
REFERENCES
[1]Krzysztof R. Apt. 2004. Uniform Proofs of Order Independence for Various
Strategy Elimination Procedures. The B.E. Journal of Theoretical Economics 4, 1
(2004), 1–48. https://doi.org/10.2202/1534-5971.1141
[2] Robert Axelrod. 1984. The Evolution of Cooperation . Basic Books, New York.
[3]Mihaly Barasz, Paul Christiano, Benja Fallenstein, Marcello Herreshoff, Patrick
LaVictoire, and Eliezer Yudkowsky. 2014. Robust Cooperation in the Prisoner’s
Dilemma: Program Equilibrium via Provability Logic. https://arxiv.org/abs/1401.
5577
[4]Vincent Conitzer and Tuomas Sandholm. 2005. Complexity of (Iterated) Dom-
inance. In Proceedings of the 6th ACM conference on Electronic commerce .
Association for Computing Machinery, Vancouver, Canada, 88–97. https:
//doi.org/10.1145/1064009.1064019
[5]Stephen A. Cook. 1971. The complexity of theorem-proving procedures. In STOC
’71: Proceedings of the third annual ACM symposium on Theory of computing .
Association for Computing Machinery, New York, 151–158. https://doi.org/10.
1145/800157.805047
[6]Andrew Critch. 2019. A Parametric, Resource-Bounded Generalization of Löb’s
Theorem, and a Robust Cooperation Criterion for Open-Source Game Theory.
Journal of Symbolic Logic 84, 4 (12 2019), 1368–1381. https://doi.org/10.1017/jsl.
2017.42
[7] Matthias Ehrgott. 2005. Multicriteria Optimization (2 ed.). Springer, Berlin.
[8]Lance Fortnow. 2008. Program equilibria and discounted ComputationTime . Tech-
nical Report 1473. Kellogg School of Management – Center for Mathemati-
cal Studies in Economics and Management Science, Northwestern University.
https://www.econstor.eu/bitstream/10419/31197/1/587667478.PDF
[9]Joaquim Gabarró, Alina García, and Maria Serna. 2011. The complexity of game
isomorphism. Theoretical Computer Science 412, 48 (11 2011), 6675–6695. https:
//doi.org/10.1016/j.tcs.2011.07.022
[10] John C. Harsanyi and Reinhard Selten. 1988. A General Theory of Equilibrium
Selection in Games . The MIT Press, Cambridge, MA.
[11] J. V. Howard. 1988. Cooperation in the Prisoner’s Dilemma. Theory and Decision
24 (5 1988), 203–213. https://doi.org/10.1007/BF00148954
[12] Adam Tauman Kalai, Ehud Kalai, Ehud Lehrer, and Dov Samet. 2010. A com-
mitment folk theorem. Games and Economic Behavior 69 (2010), 127–137.
https://doi.org/10.1016/j.geb.2009.09.008[13] Elon Kohlberg and Jean-Francois Mertens. 1986. On the Strategic Stability of
Equilibria. Econometrica 54, 5 (9 1986), 1003–1037. https://doi.org/10.2307/
1912320
[14] Jean-Jacques Laffont and David Martimort. 2002. The Theory of Incentives – The
Principal-Agent Model . Princeton University Press, Princeton, NJ.
[15] R. Preston McAfee. 1984. Effective Computability in Economic Decisions. (5
1984). https://www.mcafee.cc/Papers/PDF/EffectiveComputability.pdf
[16] Dov Monderer and Moshe Tennenholtz. 2009. Strong mediated equilibrium.
Artificial Intelligence 173, 1 (1 2009), 180–195. https://doi.org/10.1016/j.artint.
2008.10.005
[17] Caspar Oesterheld. 2019. Robust Program Equilibrium. Theory and Decision 86, 1
(2 2019), 143–159.
[18] Martin J. Osborne. 2004. An Introduction to Game Theory . Oxford University
Press, New York.
[19] David G. Pearce. 1984. Rationalizable Strategic Behavior and the Problem of
Perfection. Econometrica 54, 4 (7 1984), 1029–1050.
[20] Guillaume Perez. 2017. Decision diagrams: constraints and algorithms . Ph.D. Dis-
sertation. Université Côte d’Azur. https://tel.archives-ouvertes.fr/tel-01677857/
document
[21] Werner Raub. 1990. A General Game-Theoretic Model of Preference Adaptions
in Problematic Social Situations. Rationality and Society 2, 1 (1 1990), 67–93.
[22] Thomas C. Schelling. 1960. The Strategy of Conflict . Harvard University Press,
Cambridge, MA.
[23] Alexander Schrijver. 1998. Theory of Linear and Integer Programming . John Wiley
& Sons, Chichester, UK.
[24] Amartya Sen. 1974. Choice, orderings and morality. In Practical Reason , Stephan
Körner (Ed.). Basil Blackwell, New Haven, CT, USA, Chapter II, 54–67.
[25] Moshe Tennenholtz. 2004. Program equilibrium. Games and Economic Behavior
49, 2 (11 2004), 363–373.
[26] Wiebe van der Hoek, Cees Witteveen, and Micheal Wooldridge. 2013. Program
equilibrium – a program reasoning approach. International Journal of Game
Theory 42 (8 2013), 639–671. Issue 3.
[27] Luc N. van Wassenhove and Ludo F. Gelders. 1980. Solving a bicriterion schedul-
ing problem. European Journal of Operational Research 4 (1980), 42–48.
[28] John von Neumann. 1928. Zur Theorie der Gesellschaftsspiele. Math. Ann. 100
(1928), 295–320. https://doi.org/10.1007/BF01448847
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
A PROOF OF THEOREM 4.1 – PROGRAM
EQUILIBRIUM IMPLEMENTATIONS OF
SAFE PARETO IMPROVEMENTS
This paper considers the meta-game of delegation. SPIs are a pro-
posed way of playing these games. However, throughout most of
this paper, we do not analyze the meta-game directly as a game
using the typical tools of game theory. We here fill that gap and
in particular prove Theorem 4.1, which shows that SPIs are played
in Nash equilibria of the meta game, assuming sufficiently strong
contracting abilities. As noted, this result is essential. However,
since it is mostly an application of existing ideas from the literature
on program equilibrium, we left a detailed treatment out of the
main text.
Aprogram game forΓ=(𝐴,u)is defined via a set PROG =
PROG 1×...×PROG𝑛and a non-deterministic mapping exec: PROG 1×
...×PROG𝑛⇝𝐴. We obtain a new game with action sets PROG
and utility function
𝑈: PROG→R𝑛:c↦→E[u(exec(c))]. (1)
Though this definition is generic, one generally imagines in
the program equilibrium literature that for all 𝑖,PROG𝑖consists
of computer programs in some programming language, such as
Lisp, that take as input vectors in PROG and return an action 𝑎𝑖.
The function exec on input c∈PROG then executes each player
𝑖’s program𝑐𝑖oncto assign𝑖an action. The definition implicitly
assumes that PROG only contains programs that halt when fed
one another as input. A program equilibrium is then simply a Nash
equilibrium of the program game.
For the present paper, we add the following feature to the un-
derlying programming language. A program can call a “black box
subroutine” Π𝑖(Γ′)for any subset game Γ′ofΓ, where Π𝑖(Γ′)is a
random variable over 𝐴′
𝑖andΠ(Γ′)=(Π1(Γ′),...,Π𝑛(Γ′)).
We need one more definition. For any game Γand player𝑖, we
define Player 𝑖’sthreat point (a.k.a. minimax utility) 𝑣Γ
𝑖as
𝑣Γ
𝑖= min
𝝈−𝑖∈>
𝑗≠𝑖Δ(𝐴𝑗)max
𝜎𝑖∈Δ(𝐴𝑖)𝑢𝑖(𝜎𝑖,𝝈−𝑖). (2)
In words,𝑣Γ
𝑖is the minimum utility that the players other than 𝑖
can force onto 𝑖, under the assumption that 𝑖reacts optimally to
their strategy. We further will use minimax(𝑖,𝑗)∈Δ(𝐴𝑗)to denote
the strategy for Player 𝑗that is played in the minimizer 𝜎−𝑖of the
above. Of course, in general, there might be multiple minimizers
𝜎−𝑖. In the following, we will assume that the function minimax
breaks such ties in some consistent way, such that for all 𝑖,
(minimax(𝑖,𝑗))𝑗∈{1,...,𝑛}−{𝑖}∈ arg min
𝝈−𝑖∈>
𝑗≠𝑖Δ(𝐴𝑗)max
𝜎𝑖∈Δ(𝐴𝑖)𝑢𝑖(𝜎𝑖,𝝈−𝑖).
(3)
Note that for 𝑛=2, each player’s threat point is computable in
polynomial time via linear programming; and that by the minimax
theorem [28], the threat point is equal to the maximin utility, i.e.,
𝑣Γ
𝑖=max
𝜎𝑖∈Δ(𝐴𝑖)min
𝜎−𝑖∈Δ(𝐴−𝑖)𝑢𝑖(𝜎𝑖,𝜎−𝑖), (4)
so𝑣Γ
𝑖is also the minimum utility that Player 𝑖can guarantee for
herself under the assumption that the opponent sees her mixed
strategy and reacts in order to minimize Player 𝑖’s utility.
Tennenholtz’ [ 25] main result on program games is the following:Theorem A.1 (Tennenholtz 2004 [ 25]).LetΓ=(𝐴,u)be a
game and let x∈u>𝑛
𝑖=1Δ(𝐴𝑖)be a (feasible) payoff vector. If 𝑥𝑖≥
𝑣Γ
𝑖for𝑖=1,...,𝑛 , then xis the utility of some program equilibrium of
a program game on Γ.
Throughout the rest of this section, our goal is to use similar
ideas as Tennenholtz did for Theorem A.1 to construct for any SPI
Γ𝑠onΓ, a program equilibrium that results in the play of Π(Γ𝑠).
As noted in the main text, the Player 𝑖’s instruction to her represen-
tative to play the game Γ𝑠will usually be conditional on the other
player telling her representative to also play her part of Γ𝑠andand
vice versa . After all, if Player 𝑖simply tells her representative to
maximize𝑢𝑠
𝑖from𝐴𝑠
𝑖regardless of Player −𝑖’s instruction, then
Player−𝑖will often be able to profit from deviating from the Γ𝑠
instruction. For example, in the safe Pareto improvement on the
Demand Game, each player would only want their representative to
choose from{DL,RL}rather than{DM,DM}if the other player’s
representative does the same. It would then seem that in a program
equilibrium in which Π(Γ𝑠)is played, each program 𝑐𝑖would have
to contain a condition of the type, “if the opponent code plays as in
Π(Γ𝑠)against me, I also play as I would in Π(Γ𝑠).” But in a naive
implementation of this, each of the programs would have to call
the other, leading to an infinite recursion.
In the literature on program equilibrium, various solutions to
this problem have been discovered. We here use the general scheme
proposed by Tennenholtz [ 25], because it is the simplest. We could
similarly use the variant proposed by Fortnow [ 8], techniques based
on Löb’s theorem [ 3,6], or𝜖-grounded mutual simulation [ 17] or
even (meta) Assurance Game preferences (see Appendix B).
In our equilibrium, we let each player submit code as sketched
in Algorithm 2. Roughly, each player uses a program that says, “if
everyone else submitted the same source code as this one, then
playΠ(Γ𝑠). Otherwise, if there is a player 𝑗who submits a different
source code, punish player 𝑗by playing her minimax strategy”.
Note that for convenience, Algorithm 2 receives the player number
𝑖as input. This way, every player can use the exact same source
code. Otherwise the original players would have to provide slightly
different programs and in line 2 of the algorithm, we would have
to use a more complicated comparison, roughly: “if 𝑐𝑗≠𝑐𝑖are the
same, except for the player index used”.
Algorithm 2: A program equilibrium implementation of
an SPI Γ𝑠ofΓ.
Data: Everybody’s source code c, my index𝑖
1for𝑗∈{1,...,𝑛}−{𝑖}do
2 if𝑐𝑗≠𝑐𝑖then
3 Play minimax(𝑖,𝑗);
4PlayΠ𝑖(Γ𝑠);
Proposition A.2. LetΓbe a game and let Γ𝑠be an SPI on Γ. Let c
be the program profile consisting only of Algorithm 2 for each player.
Assume that Π(Γ)guarantees each player at least threat point utility
in expectation. Then cis a program equilibrium and apply(c)=
Π(Γ𝑠).
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
Player 2
Cooperate Defect
Player 1Cooperate 4,4 1,3
Defect 3,1 2,2
Table 3: Assurance Game preferences for the Prisoner’s
Dilemma
Proof. By inspection of Algorithm 2, we see that exec(c)=
Π(Γ𝑠). It is left to show that cis a Nash equilibrium. So let 𝑖
be any player and 𝑐′
𝑖∈PROG𝑖−{𝑐𝑖}. We need to show that
E
𝑢𝑖(exec(c−𝑖,𝑐′
𝑖))
≤E[𝑢𝑖(exec(c))]. Again, by inspection of c,
exec(c−𝑖,𝑐′
𝑖)is the threat point of Player 𝑖. Hence,
E
𝑢𝑖(exec(c−𝑖,𝑐′
𝑖))
=𝑣𝑖
≤E[𝑢𝑖(Π(Γ))]
≤E
𝑢𝑖(Π(Γ𝑠))
=E[𝑢𝑖(exec(c))]
as required. □
Theorem 4.1 follows immediately.
B A DISCUSSION OF WORK BY SEN (1974)
AND RAUB (1990) ON PREFERENCE
ADAPTATION GAMES
We here discuss Raub’s [ 21] paper in some detail, which in turn
elaborates on an idea by Sen [ 24]. Superficially, Raub’s setting
seems somewhat similar to ours, but we here argue that it should
be thought of as closer to the work on program equilibrium and
bilateral precommitment.
In Sections 1, 3 and 4, we briefly discuss multilateral commit-
ment games, which have been discussed before in various forms
in the game-theoretic literature. Our paper extends this setting
by allowing instructions that let the representatives play a game
without specifying an algorithm for solving that game. On first
sight, it appears that Raub pursues a very similar idea. Translated
to our setting, Raub allows that as an instruction, each player 𝑖
chooses a new utility function 𝑢𝑠
𝑖:𝐴→R, where𝐴is the set of
outcomes of the original game Γ. Given instructions 𝑢𝑠
1,...,𝑢𝑠𝑛, the
representatives then play the game (𝐴,u𝑠). In particular, each repre-
sentative can see what utility functions all the other representatives
have been instructed to maximize. However, what utility function
representative 𝑖maximizes is not conditional on any of the instruc-
tions by other players. In other words, the instructions in Raub’s
paper are raw utility functions without any surrounding control
structures, etc. Raub then asks for equilibria u𝑠of the meta-game
that Pareto-improve on the default outcome.
To better understand how Raub’s approach relates to ours, we
here give an example of the kind of instructions Raub has in mind.
(Raub uses the same example in his paper.) As the underlying game
Γ, we take the Prisoner’s Dilemma. Now the main idea of his pa-
per is that the original players can instruct their representatives
to adopt so-called Assurance Game preferences. In the Prisoner’s
Dilemma, this means that the representatives prefer to cooperate
if the other representative cooperates, and prefer to defect if theother player defects. Further, they prefer mutual cooperation over
mutual defection. An example of such Assurance Game preferences
is given in Table 3. (Note that this payoff matrix resembles the
classic Stag Hunt studied in game theory.)
The Assurance Game preferences have two important properties.
(1)If both players tell their representatives to adopt Assurance
Game preferences, (Cooperate, Cooperate) is a Nash equilib-
rium. (Defect, Defect) is a Nash equilibrium as well. However,
since (Cooperate, Cooperate) is Pareto-better than (Defect,
Defect), the original players could reasonably expect that
the representatives play (Cooperate, Cooperate).
(2)Under reasonable assumptions about the rationality of the
representatives, it is a Nash equilibrium of the meta-game for
both players to adopt Assurance Game preferences. If Player
1 tells her representative to adopt Assurance Game prefer-
ences, then Player 2 maximizes his utility by telling his rep-
resentative to also maximize Assurance Game preferences.
After all, representative 1 prefers defecting if representative
2 defects. Hence, if Player 2 instructs his representative to
adopt preferences that suggest defecting, then he should
expect representative to defect as well.
The first important difference between Raub’s approach and
ours is related to item 2. We have ignored the issue of making
SPIsΓ𝑠Nash equilibria of our meta game. As we have explained in
Section 4 and Appendix A, we imagine that this is taken care of by
additional bilateral commitment mechanisms that are not the focus
of this paper. For Raub’s paper, on the other hand, ensuring mutual
cooperation to be stable in the new game Γ𝑠is arguably the key
idea. Still, we could pursue the approach of the present paper even
when we limit assumptions to those that consist only of a utility
function.
The second difference is even more important. Raub assumes
that – as in the PD – the default outcome of the game ( Π(Γ)in
the formalism of this paper) is known. (Less significantly, he also
assumes that it is known how the representatives play under assur-
ance game preferences.) Of course, the key feature of the setting of
this paper is that the underlying game Γmight be difficult (through
equilibrium selection problems) and thus that the original players
might be unable to predict Π(Γ).
These are the reasons why we cite Raub in our section on bilateral
commitment mechanisms. Arguably, Raub’s paper could be seen as
very early work on program equilibrium, except that he uses utility
functions as a programming language for representative. In this
sense, Raub’s Assurance Game preferences are analogous to the
program equilibrium schemes of Tennenholtz [ 25], Oesterheld [ 25],
Barasz et al .[3] and van der Hoek et al .[26], ordered in increasing
order of similarity of the main idea of the scheme.
C PROOF OF PROPOSITION 5.2
Lemma 5.2. LetΓ=(𝐴,u),Γ′=(𝐴′,u′),ˆΓ=(ˆ𝐴,ˆu)andΦ,Ξ:𝐴⊸
𝐴′,Ψ:𝐴′⊸ˆ𝐴.
(1) Reflexivity: Γ∼id𝐴Γ, where id𝐴:𝐴⊸𝐴:a↦→{a}.
(2) Symmetry: If Γ∼ΦΓ′, then Γ′∼Φ−1Γ.
(3) Transitivity: If Γ∼ΦΓ′andΓ′∼ΨˆΓ, then Γ∼Ψ◦ΦˆΓ.
(4) If Γ∼ΦΓ′andΦ(a)⊆Ξ(a)for all a∈𝐴, then Γ∼ΞΓ′.
(5)Γ∼all𝐴,𝐴′Γ′, where all𝐴,𝐴′:𝐴⊸𝐴′:a↦→𝐴′.
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
(6) If Γ∼ΦΓ′andΦ(a)=∅, then Π(Γ)≠awith certainty.
Proof. 1.By reflexivity of equality, Π(Γ)=Π(Γ)with cer-
tainty. Hence, Π(Γ)∈id𝐴(Π(Γ))by definition of id𝐴. There-
fore,Γ∼id𝐴Γby definition of∼, as claimed.
2.Γ∼ΦΓ′means that Π(Γ′)∈Φ(Π(Γ))with certainty. Thus,
Π(Γ)∈{ a∈𝐴|Π(Γ′)∈Φ(a)}=Φ−1(Π(Γ′)),
where equality is by the definition of the inverse of multi-
valued functions. We conclude (by definition of ∼) thatΓ′∼Φ−1
Γas claimed.
3.IfΓ∼ΦΓ′,Γ′∼ΨˆΓ, then by definition of ∼, (i)Π(Γ′)∈
Φ(Π(Γ))and (ii) Π(ˆΓ)∈Ψ(Π(Γ′)), both with certainty. The
former (i) implies{Π(Γ′)}⊆Φ(Π(Γ)). Hence,
Ψ(Π(Γ′))=Ψ({Π(Γ′)})⊆ Ψ(Φ(Π(Γ))).
With ii, it follows that Π(ˆΓ)∈Ψ(Φ(Π(Γ)))with certainty.
By definition, Γ∼Ψ◦ΦˆΓas claimed.
4. It is
Π(Γ′)∈Φ(Π(Γ))⊆Ξ(Π(Γ))
with certainty. Thus, by definition Γ∼ΞΓ′.
5.By definition of Π, it is Π(Γ′)∈𝐴′with certainty. By def-
inition of all𝐴,𝐴′, it is all𝐴,𝐴′(Π(Γ))=𝐴′with certainty.
Hence, Π(Γ′)∈all𝐴,𝐴′(Π(Γ))with certainty. We conclude
thatΓ∼all𝐴,𝐴′Γ′as claimed.
6.With certainty, Π(Γ′) ∈Φ(Π(Γ))(by assumption). Also,
with certainty Π(Γ′)∉∅. Hence, Φ(Π(Γ))≠∅with cer-
tainty. We conclude that Π(Γ)≠awith certainty.
□
D EXAMPLES
D.1 Proof of Proposition (Example) 7.1
Proposition (Example) 7.1. LetΓbe the Prisoner’s Dilemma
andΓ𝑠=(𝐴𝑠
1,𝐴𝑠
2,𝑢𝑠
1,𝑢𝑠
2)be any subset game of Γwith𝐴𝑠
1=𝐴𝑠
2=
{Cooperate}. Then under Assumption 7.2, Γ𝑠is a strict SPI on Γ.
Proof. By applying Assumption 7.2 twice and Transitivity once,
Γ∼ΦΓ−{Cooperate}, where Φ(Defect,Defect)={(Defect,Defect)}
andΦ(𝑎1,𝑎2)=∅for all(𝑎1,𝑎2)≠(Defect,Defect). By Lemma 5.2.5,
we further obtain Γ−{Cooperate}∼allΓ𝑠, where Γ𝑠is as described
in the proposition. Hence, by transitivity, Γ∼all◦ΦΓ𝑠. It is easy to
verify that the function all◦Φis Pareto-improving. □
D.2 Proof of Proposition (Example) 7.2
Proposition (Example) 7.2. LetΓbe the Demand Game of Table 1
andΓ𝑠be the subset game described in Table 2. Under Assumptions 7.1
and 7.2, Γ𝑠is an SPI on Γ. Further, if𝑃(Π(Γ)=(DM,DM))>0,Γ𝑠is
a strict SPI.
Proof. Let(𝐴1,𝐴2,𝑢1,𝑢2)=Γ. We can repeatedly apply As-
sumption 7.2 to eliminate from Γthe strategies DLandRLfor both
players. We can then apply Lemma 5.2.3 (Transitivity) to obtain
𝐺∼Φˆ𝐺=({DM,RM},{DM,RM},𝑢1,𝑢2), where
Φ(𝑎1,𝑎2)={(𝑎1,𝑎2)} if𝑎1,𝑎2∈{DM,RM}
∅ otherwise. (5)Next, by Assumption 7.1, ˆΓ∼ΨΓ𝑠, where Ψ𝑖(DM)=DLand
Ψ𝑖(RM)=RLfor𝑖=1,2. We can then apply Lemma 5.2.3 (Tran-
sitivity) again, to infer Γ∼Ψ◦ΦΓ𝑠. It is easy to verify that for all
(𝑎1,𝑎2)∈𝐴1×𝐴2, it is for all(𝑎𝑠
1,𝑎𝑠
2)∈Ψ(Φ(Γ𝑠))the case that
u(𝑎𝑠
1,𝑎𝑠
2)≥u(𝑎1,𝑎2). □
E PROOF OF THEOREM 8.2
We here prove Theorem 8.2.
Throughout this section, we use the following lemma.
Lemma E.1. LetΦ,Ψbe isomorphisms between Γ,Γ′. IfΦis (strictly)
Pareto-improving, then so is Ψ.
This will allow us to conclude from the existence of a Pareto-
improving isomorphism Φthat there is Pareto-improving Ψs.t.
Γ∼ΨΓ′by Assumption 7.1, even if there are multiple isomorphisms
between Γ,Γ′.
Proof. LetΓ=(𝐴,u),Γ′=(𝐴′,u′). Then for all a∈𝐴,
u(a)=u′(Ψ(a))
=u(Φ−1(Ψ(a))
≤u(Φ(Φ−1(Ψ(a))))
=u(Ψ(a)).
Furthermore, if Φis strictly Pareto-improving for some ˜a∈𝐴, then
by bijectivity of Φ,Ψ, there is a∈𝐴s.t.Φ−1(Ψ(a))=˜a. For this a,
the inequality above is strict and therefore u(a)
𝑖Φ𝑖(𝐴′
𝑖). The utility functions
are
𝑢𝑠
𝑖:𝐴𝑠→R:a𝑠↦→(𝑢𝑖(Φ−1
1(𝑎𝑠
1),...,Φ−1
𝑛(𝑎𝑠
𝑛)))𝑖=1,...,𝑛.(6)
Otherwise, return False.
It is easy to see that this algorithm runs in non-deterministic
polynomial time. Furthermore, with Lemma E.1 it is easy to see that
if this algorithm finds a solution Γ𝑠, that solution is indeed a safe
Pareto improvement. It is left to show that if there is a safe Pareto
improvement via a sequence of Assumption 7.1 and 7.2 outcome
correspondences, then the algorithm indeed finds a safe Pareto
improvement. To prove this fact, we prove a few simple lemmata.
First, one might worry that the algorithm only ever finds se-
quences of outcome correspondences that start with a number of
reductions and end with a single isomorphism step. Perhaps some
safe Pareto improvements can only be found by considering very
different sequences? The following two lemmata show that this is
not an issue, i.e., that it is sufficient to consider sequences that start
with a number of reductions and end in a single isomorphism step.
Lemma E.2. LetΓ∼ΦisoˆΓby Assumption 7.1 and ˆΓ∼Φred˜Γ
by Assumption 7.2. Then there are Γ′,Ψred,Ψisos.t.Γ∼ΨredΓ′by
Assumption 7.2, Γ′∼Ψiso˜Γby Assumption 7.1 and Ψiso◦Ψred=
Φred◦Φiso.
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
Intuitively, this means that isomorphism steps as per Assump-
tion 7.1 and reduction steps as per Assumption 7.2 commute. Instead
of first applying Assumption 7.1 and then Assumption 7.2 to a game,
we can also apply Assumption 7.2 first and then Assumption 7.1 to
obtain the same game ˜Γin both cases.
Proof. We construct Γ′,Ψred,Ψisoas follows. First, Γ′=
(𝐴′
1,...,𝐴′𝑛,u′), where𝐴′
𝑖=(Φiso
𝑖)−1(˜𝐴𝑖)and𝑢′
𝑖=𝑢𝑖|𝐴′. Next, we
define
Ψred=(Φiso)−1◦Φred◦Φiso(7)
and
Ψiso=Φiso|𝐴′
1×𝐴′
2. (8)
We now need to show that these satisfy the consequents of the
lemma.
First, it is
Ψiso◦Ψred=Φiso|𝐴′
1×𝐴′
2◦
(Φiso)−1◦Φred◦Φiso
=
Φiso|𝐴′
1×𝐴′
2◦(Φiso)−1
◦Φred◦Φiso
=Φred◦Φiso
as claimed. Note that the second step uses the associativity of ◦on
multivalued functions. The third step uses the fact that Φisois a
single-valued bijection, which means that Φiso◦
Φiso−1
=id. Of
course, Φisois here restricted to 𝐴′
1×𝐴′
2, but this is not a problem,
because𝐴′
𝑖=(Φiso
𝑖)−1(˜𝐴𝑖)and ˜𝐴is the codomain of Φred◦Φiso.
Hence, the restriction is inconsequential.
Second, we need to show that it is indeed Γ′∼Ψiso˜Γby Assump-
tion 7.1. (Note that because Ψiso◦Ψred=Φred◦Φiso, it really must
beΓ′∼Ψiso˜Γrather than Γ′∼˜Ψiso˜Γfor some different isomorphism
˜Ψiso≠Ψiso.) First, it is easy to show that Ψisodecomposes into
Ψiso
1,...,Ψiso𝑛as required because Φisodecomposes. Further, Ψisois
a single-valued injection because Φisois a single-valued bijection.
It is surjective because its codomain is defined as the image of its
domain.
Now let(𝑎′
1,𝑎′
2)∈𝐴′
1×𝐴′
2. It is
u′(a′)=u(a′)
=ˆu(Φiso(a′))
=˜u(Φiso(a′))
=˜u(Ψiso(a′)),
as required.
Finally, we have to show that Γ∼ΨredΓ′by Assumption 7.2. We
leave this as an exercise to the reader. □
Lemma E.3. Let
Γ1∼Φ1...∼Φ𝑘−1Γ𝑘, (9)
where each outcome correspondence is due to a single application
of Assumption 7.1 or 7.2. Then there is a sequence Γ′2,...,Γ′𝑚with
𝑚≤𝑘−1such that
Γ1∼Ψ1Γ′2∼Ψ2Γ′3∼Ψ3...∼Ψ𝑚−1Γ′𝑚(10)
all by single applications of Assumption 7.2, Γ′𝑚∼ΞΓ𝑘by a single
application of Assumption 7.1, and
Φ𝑘−1◦Φ𝑘−2◦...◦Φ1=Ξ◦Ψ𝑚−1◦...◦Ψ1. (11)Proof. Start with the initial sequence of line 9. We can iter-
atively apply Lemma E.2 to obtain a new sequence of the same
length in which one first applies only Assumption 7.2 and then
only Assumption 7.1 while obtaining the same composite outcome
correspondence function. We can summarize all the applications of
Assumption 7.1 into a single step applying that assumption. □
A second potential worry about our algorithm is that it reduces
the game completely and only then looks for a Pareto-improving
isomorphism step. Perhaps in some cases one has to only partially
reduce and then look for a Pareto-improving isomorphism step?
The next lemma shows that the answer to this is no and that one
can restrict oneself to sequences that fully reduce.
Lemma E.4. LetΓ=(𝐴,u),ˆΓ𝑎=(ˆ𝐴𝑎,u),ˆΓ𝑏=(ˆ𝐴𝑏,u)such that
ˆ𝐴𝑏
𝑖⊆ˆ𝐴𝑎
𝑖⊆𝐴𝑖for𝑖=1,...,𝑛 . If there is a subset game ˜Γ𝑎=(˜𝐴𝑎,˜u𝑎)
ofΓsuch that ˆΓ𝑎∼Φ˜Γ𝑎by Assumption 7.1, then ˆΓ𝑏∼Φ|ˆ𝐴𝑏˜Γ𝑏,
where ˜Γ𝑏=(Φ1(ˆ𝐴𝑏
1),...,Φ𝑛(ˆ𝐴𝑏𝑛),˜u𝑎). Note that if the correspondence
function Φis Pareto-improving, so is Φ|ˆ𝐴𝑏.
Lemma E.4 shows that it is enough to consider isomorphism
steps from fully reduced versions of Γ. A third worry might be that
even so, elimination via Assumption 7.2 might be path-dependent
and therefore we have to consider the resulting games from multiple
paths. However, iterated elimination of strictly dominated strategies
is known to be path-independent [1, 19].
Proposition E.5. If there is a safe Pareto improvement for a given
game, then the above algorithm applied to that game returns True.
Proof. Let us say there is a sequence of outcome correspon-
dences as per Assumptions 7.1 and 7.2 that show Γ∼ΦΓ𝑠for
Pareto-improving Φ. Then by Lemma E.3, there is Γ′such that
Γ∼ΨredΓ′via an arbitrary number of applications of Assump-
tion 7.2 and Γ′∼ΨisoΓ𝑠via a single application of Assumption 7.1.
Because of the path-independence of iterated removal of strictly
dominated strategies, Γ′contains (as a subset game with equal util-
ity functions) the unique Γ𝑟arising from full iterated removal as
per Assumption 7.2. By Lemma E.4, there is a Pareto-improving
outcome correspondence Γ𝑟∼˜ΨisoΓ𝑠𝑟as per Assumption 7.2. By
construction, our algorithm finds (guesses) this Pareto-improving
outcome correspondence. □
Overall, we have now shown that our non-deterministic
polynomial-time algorithm is correct and therefore that the SPI
problem is in NP. Note that the correctness of other algorithms
can be proven using very similar ideas. For example, instead of
first reducing and then finding an isomorphism, one could first find
an isomorphism, then reduce and then (only after reducing) test
whether the overall outcome correspondence function is Pareto-
improving. One advantage of reducing first is that there are fewer
isomorphisms to test if the game is smaller. In particular, the number
of possible isomorphisms is exponential in the number of strategies
in the reduced game Γ′but polynomial in everything else. Hence,
by implementing our algorithm deterministically, we obtain the
following positive result.
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
Proposition 8.3. For games Γwith|𝐴1|+...+|𝐴𝑛|=𝑚that can
be reduced (via iterative application of Assumption 7.2) to a game
Γ′with|𝐴′
1|+...+|𝐴′𝑛|=𝑙, the (strict) SPI decision problem can be
solved in𝑂(𝑚𝑙).
We now proceed to showing that the safe Pareto improvement
problem is NP-hard. We will do this by reducing the subgraph
isomorphism problem to the (two-player) safe Pareto improvement
problem. We start by briefly describing one version of that problem
here.
A(simple, directed) graph is a tuple(𝑛,𝑎:{1,...,𝑛}×{ 1,...,𝑛}→
B), where𝑛∈NandBB{0,1}. We call𝑎the adjacency function
of the graph. Since the graph is supposed to be simple and therefore
free of self-loops (edges from one vertex to itself), we take the
values𝑎(𝑗,𝑗)for𝑗∈{1,...,𝑛}to be meaningless.
For given graphs 𝐺=(𝑛,𝑎),𝐺′=(𝑛′,𝑎′)a subgraph isomor-
phism from 𝐺to𝐺′is an injection 𝜙:{1,...,𝑛}→{ 1,...𝑛′}such
that for all𝑗≠𝑙
𝑎(𝑗,𝑙)≤𝑎′(𝜙(𝑗),𝜙(𝑙)). (12)
In words, a subgraph isomorphism from 𝐺to𝐺′identifies for each
node in𝐺a node in𝐺′s.t. if there is an edge from node 𝑗to node𝑙
in𝐺, there must also be an edge in the same direction between the
corresponding nodes 𝜙(𝑗),𝜙(𝑙)in𝐺′. Another way to say this is
that we can remove some set of ( 𝑛′−𝑛) nodes and some edges from
𝐺′to get a graph that is just a relabeled (isomorphic) version of 𝐺.
Given two graphs 𝐺,𝐺′, the subgraph isomorphism problem
consists in deciding whether there is a subgraph isomorphism 𝜙
between𝐺,𝐺′. The problem is well-known to be NP-complete [ 5,
Theorem 2].
Lemma E.6. The subgraph isomorphism problem is reducible in lin-
ear time with linear increase in problem instance size to the safe Pareto
improvement problem. As a consequence, the safe Pareto improvement
problem is NP-hard.
Proof. We conduct our proof only for the strict safe Pareto
improvement problem. Reducing to the non-strict safe Pareto im-
provement problem is a little easier and can be done using a subset
of the ideas in this proof.
So take graphs 𝐺=(𝑛,𝑎)and ˆ𝐺=(ˆ𝑛,ˆ𝑎). We will transform
these step-wise into a single game.
First, we define the games Γ𝑎=(𝐴1,𝐴2,𝑢1,𝑢2)and ˆΓ𝑎=
(ˆ𝐴1,ˆ𝐴2,ˆ𝑢1,ˆ𝑢2), where𝐴1=𝐴2={1,...,𝑛},ˆ𝐴1=ˆ𝐴2={1,...,ˆ𝑛},
𝑢1(𝑗,𝑙)=𝑢2(𝑗,𝑙)=𝑎(𝑗,𝑙)for all𝑗,𝑙∈{1,...,𝑛}with𝑗≠𝑙and
𝑢1(𝑗,𝑗)=𝑢2(𝑗,𝑗)=2. We analogously define ˆ𝑢1=ˆ𝑢2based
onˆ𝑎. Setting the utility functions is the main idea of the entire
proof, of course, and will become clearer below. Setting the utilities
𝑢1(𝑗,𝑗)=𝑢2(𝑗,𝑗)=2is to ensure that Pareto-improving mappings
Φbetween Γ𝑎and ˆΓ𝑎satisfy Φ1(𝑗)=Φ2(𝑗)for all𝑗, and thus
directly relate to subgraph isomorphisms.
Next, we add dummy strategies to Γ𝑎,ˆΓ𝑎, to obtain two purposes.
We want to remove exact equivalences and allow only strict Pareto
improvements ; and we want to remove the possibility of reducing
either of these games via Assumption 7.2. In particular, we consider
Γ𝑏as follows:𝐴𝑏
𝑖={1,...,2𝑛};u𝑏(𝑗,𝑙)=u𝑎(𝑗,𝑙)if𝑗,𝑙∈{1,...,𝑛},
u𝑏(𝑗,𝑛+𝑗)=u𝑏(𝑛+𝑗,𝑛)=(3,3)for𝑗∈{1,..,𝑛},u𝑏(𝑗,𝑙)=(−1,−1)
otherwise. We define ˆΓ𝑏analogously, except that utilities of (3,3)
are to be replaced by (4,4).Finally, we construct from Γ𝑏,ˆΓ𝑏a single game Γ𝑐. Roughly,
the idea is for Γ𝑐to contain as subset games both Γ𝑏and ˆΓ𝑏, but
to reduce to Γ𝑏via Assumption 7.2. We construct Γ𝑐thus:𝐴𝑐
𝑖=
({𝐷}×𝐴𝑏
𝑖)∪({𝐶}×ˆ𝐴𝑏
𝑖)and
u𝑐((𝐶,ˆ𝑎1),(𝐶,ˆ𝑎2))=ˆu𝑏(ˆ𝑎1,ˆ𝑎2)for all ˆ𝑎1∈ˆ𝐴𝑏
1,ˆ𝑎2∈ˆ𝐴𝑏
2
u𝑐((𝐷,𝑎 1),(𝐷,𝑎 2))=u𝑏(𝑎1,𝑎2)for all𝑎1∈𝐴𝑏
1,𝑎2∈𝐴𝑏
2
𝑢𝑐
𝑖((𝐷,𝑎𝑖),(𝐶,ˆ𝑎−𝑖))=5for all𝑎𝑖∈𝐴𝑏
𝑖,ˆ𝑎−𝑖∈ˆ𝐴𝑏
2
𝑢𝑐
−𝑖((𝐷,𝑎𝑖),(𝐶,ˆ𝑎−𝑖))=−5for all𝑎𝑖∈𝐴𝑏
𝑖,ˆ𝑎−𝑖∈ˆ𝐴𝑏
2.
It is easy to show that this reduction can be computed in linear
time and that it also increases the problem instance size only linearly.
It is left to prove the correctness of the reduction.
We start by showing that if there is a subgraph isomorphism
from𝐺toˆ𝐺, then there is also a safe (strict) Pareto improvement via
a sequence of outcome correspondence as per Assumptions 7.1 and
7.2. So let𝜙be that subgraph isomorphism. Then we need to con-
struct a series of outcome correspondences as per Assumptions 7.1
and 7.2.
First notice that Γ𝑐∼ΞΓ𝑐,𝐷by Assumption 7.2, where Γ𝑐,𝐷is
the subset game of Γ𝑐that contains only the strategies of type (𝐷,𝑗)
for both players. We now show that Ψis a Pareto-improving iso-
morphism between Γ𝑐,𝐷and ˜Γ=(˜𝐴1,˜𝐴2,˜𝑢1,˜𝑢2), where we define
˜𝐴1=˜𝐴2={𝐶}×(𝜙({1,...,𝑛})∪{ ˆ𝑛+𝜙(𝑗)|𝑗=1,...,𝑛}),(13)
for𝑖=1,2asΨ𝑖(𝐷,𝑗)=(𝐶,𝜙(𝑗))if𝑗∈{1,...,𝑛}andΨ𝑖(𝐷,𝑗)=
(𝐶,ˆ𝑛+𝜙(𝑗−𝑛))otherwise, and the utility function is simply defined
as˜𝑢𝑖(˜𝑎1,˜𝑎2)=𝑢𝑖(Ψ−1
𝑖(˜𝑎1,˜𝑎2)). Again, it will then follow from
Lemma E.1 that in particular it is Γ𝑐,𝐷∼Ψ′˜Γvia Assumption 7.1
for some potentially different, but still Pareto-improving Ψ′. It is
easy to see that Ψ𝑖is surjective for 𝑖=1,2. From the fact that 𝜙is
injective and that 𝜙never returns values greater than ˆ𝑛, it follows
thatΨ𝑖is also injective for 𝑖=1,2. Finally, Ψmaintains utilities by
definition:
˜𝑢𝑖(Ψ(𝑎1,𝑎2))=𝑢𝑖(Ψ−1(Ψ(𝑎1,𝑎2)))=𝑢𝑖(𝑎1,𝑎2). (14)
With Transitivity, it is left to show that Ψis Pareto-improving
for the original players, i.e., for u𝑐. For this we distinguish a number
of different cases. If 𝑗,𝑙∈{1,...,𝑛}and𝑗≠𝑙, then for𝑖=1,2
𝑢𝑐
𝑖((𝐷,𝑗),(𝐷,𝑙))=𝑎(𝑗,𝑙)
≤ ˆ𝑎(𝜙(𝑗),𝜙(𝑙))
=ˆ𝑢𝑐
𝑖((𝐶,𝜙(𝑗)),(𝐶,𝜙(𝑙)))
=ˆ𝑢𝑐
𝑖(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑙)).
For𝑗∈{1,...,𝑛}, it is
u𝑐((𝐷,𝑗),(𝐷,𝑗))=(2,2)
=ˆu𝑐((𝐶,𝜙(𝑗)),(𝐶,𝜙(𝑗)))
=ˆu𝑐(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑗)).
For𝑗,𝑙∈{𝑛+1,...,2𝑛}, it is
u𝑐((𝐷,𝑗),(𝐷,𝑙))=u𝑏(𝑗,𝑙)
=(−1,−1)
=ˆu𝑏(ˆ𝑛+𝜙(𝑗−𝑛),ˆ𝑛+𝜙(𝑙−𝑛))
=u𝑐(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑙)).
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
If(𝑗,𝑙)∈{ 1,...,𝑛}×{𝑛+1,...,2𝑛}and𝑙=𝑗+𝑛, then
u𝑐((𝐷,𝑗),(𝐷,𝑙))=(2,2)
<(3,3)
=u𝑐((𝐶,Φ(𝑗)),(𝐶,Φ(𝑗)+ˆ𝑛))
=u𝑐(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑙)).
If(𝑗,𝑙)∈{ 1,...,𝑛}×{𝑛+1,...,2𝑛}but𝑙≠𝑗+𝑛, then
u𝑐((𝐷,𝑗),(𝐷,𝑙))=u𝑏(𝑗,𝑙)
=(−1,−1)
=ˆu𝑏(𝜙(𝑗),ˆ𝑛+𝜙(𝑙−𝑛))
=u𝑐(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑙)).
The cases(𝑗,𝑙)∈{𝑛+1,...,2𝑛}×{ 1,...,𝑛}work analogously.
This concludes our proof that if a graph isomorphism exists,
there also exists a strict SPI as per Assumptions 7.1 and 7.2.
It is left to prove that if there is a safe Pareto improvement for Γ𝑐,
then there also exists a graph isomorphism. So let Γ𝑐∼Ξ˜Γfor some
˜Γ, via some Pareto-improving outcome correspondence function Ξ.
By our earlier results (Proposition E.5), this means that there is a
sequence of outcome correspondences that first fully reduces Γto
Γ𝑐,𝐷and then applies Assumption 7.1 to get Γ𝑐,𝐷∼Ψ˜Γvia some
Pareto-improving Ψ.
To construct a subgraph isomorphism, we must now realize
some facts about the structure of Ψthat all follow from Ψbeing
utility-increasing:
(1)Ψ({𝐷}×{ 1,...,2𝑛})2⊆({𝐶}×{ 1,...,2ˆ𝑛})2: For Ψto
be strict, there has to be some overlap, i.e., there has to be
(𝐷,𝑗)s.t.Ψ𝑖(𝐷,𝑗)∈{𝐶}×{ 1,...,2ˆ𝑛}. But for Ψto be Pareto-
improving for 𝑖, it has to be for all (𝐷,𝑙)with𝑙=1,...,2𝑛
the case that Ψ−𝑖(𝐷,𝑙)∈{𝐶}×{ 1,...,2ˆ𝑛}since otherwise it
would be
𝑢𝑖(Ψ𝑖(𝐷,𝑗),Ψ−𝑖(𝐷,𝑙))=−5
<𝑢𝑖((𝐷,𝑗),(𝐷,𝑙)).
By an analogous argument it can further be shown that
Ψ𝑖(𝐷,𝑗)∈{𝐶}×{ 1,...,2ˆ𝑛}for all𝑗=1,...,2𝑛.
(2)For every𝑗∈ {1,...,𝑛}, it is Ψ𝑖(𝐷,𝑗) ∈ {𝐶}×{ 1,...,ˆ𝑛}
for𝑖=1,2. That is, Ψ𝑖maps the “non-dummy” strategies
(which correspond to nodes in the original graph) onto “non-
dummy” strategies. We will show this by showing the con-
trapositive, i.e., that if this were not the case, then Ψwould
not be Pareto-improving.
So assume there is 𝑗∈{1,...,𝑛}and𝑖∈{1,2}withΨ𝑖(𝐷,𝑗)∈
{𝐶}×{ ˆ𝑛+1,...,2ˆ𝑛}. Because Ψ−𝑖is injective, there is an
𝑙∈{1,...,𝑛}s.t.Ψ−𝑖(𝐷,𝑙)≠Ψ𝑖(𝐷,𝑗)−ˆ𝑛, where we define
(𝐷,𝑘)−ˆ𝑛B(𝐷,𝑘−ˆ𝑛). Then for that 𝑙it is
u(Ψ𝑖(𝐷,𝑗),Ψ−𝑖(𝐷,𝑙))=(−1,−1)
<(0,0)
≤u𝑐((𝐷,𝑗),(𝐷,𝑙)).
(3)For all𝑗∈{1,...,𝑛}and𝑖∈{1,...,𝑛}it isΨ𝑖(𝐷,𝑗+𝑛)=
Ψ−𝑖(𝐷,𝑗)+ˆ𝑛, where addition is defined to operate only onthe second entry like the subtraction defined above. We again
prove the contrapositive, i.e., if this is not true then Ψis not
Pareto-improving. So let us assume that there is 𝑗∈{1,...,𝑛}
and𝑖∈{1,...,𝑛}s.t.Ψ𝑖(𝐷,𝑗+𝑛)≠Ψ−𝑖(𝐷,𝑗). Then it would
be
u𝑐(Ψ𝑖(𝐷,𝑗+𝑛),Ψ−𝑖(𝐷,𝑗)) ≤ ( 2,2)
<(3,3)
=u𝑐((𝐶,𝑗+𝑛),(𝐶,𝑗)).
(4)Finally, we prove that Ψ1=Ψ2. We first show that for
𝑗∈{1,...,𝑛}it isΨ1(𝐷,𝑗)=Ψ2(𝐷,𝑗). We do this again by
showing the contrapositive. So assume Ψ1(𝐷,𝑗)≠Ψ2(𝐷,𝑗).
Recall that by item 2, it is Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑗)∈{𝐶}×{1,...,ˆ𝑛}.
Hence,
u(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑗)) ≤ ( 1,1)
<(3,3)
=u((𝐷,𝑗),(𝐷,𝑗)),
contradicting the assumption that Ψis Pareto-improving.
Finally, for𝑗∈{1,...,𝑛}it is
Ψ1(𝐷,𝑛+𝑗)=
Item 3ˆ𝑛+Ψ2(𝐷,𝑗)
= ˆ𝑛+Ψ1(𝐷,𝑗)
=
Item 3Ψ2(𝐷,𝑛+𝑗),
where the middle equality is due to the equality we have
already proven.
Given these, we can define our graph isomorphism as
𝜙:{1,...,𝑛}→{ 1,...,ˆ𝑛}:𝑗↦→𝜋2(Ψ1(𝐷,𝑗)), (15)
where𝜋2just maps pairs(𝐶,𝑗)onto the second entry 𝑗. This is
well-defined because of item 2. Note that because Ψis an injection,
so is𝜙.
For all𝑗,𝑙∈{1,...,𝑛}with𝑗≠𝑙it is
ˆ𝑎(𝜙(𝑗),𝜙(𝑙)) = ˆ𝑢𝑎
1(𝜙(𝑗),𝜙(𝑙))
= ˆ𝑢𝑐
1((𝐶,𝜙(𝑗)),(𝐶,(𝜙(𝑙))))
=
Item 1ˆ𝑢𝑐
1(Ψ1(𝐷,𝑗),Ψ1(𝐷,𝑙))
=
Item 4ˆ𝑢𝑐
1(Ψ1(𝐷,𝑗),Ψ2(𝐷,𝑙))
≥𝑢𝑐
1((𝐷,𝑗),(𝐷,𝑙))
=𝑢𝑎
1(𝑗,𝑙)
=𝑎(𝑗,𝑙).
Hence,𝜙is a subgraph isomorphism as desired. □
F PROOF OF LEMMA 9.3
Lemma 9.3. For a given𝑛-player game Γand payoff vector y∈R𝑛,
it can be decided by linear programming and thus in polynomial time
whether yis Pareto-optimal in C(Γ).
For an introduction to linear programming, see, e.g., Schrijver
[23]. In short, a linear program is a specific type of constrained
optimization problem that can be solved efficiently.
AAMAS ’21, May 3–7, 2021, Online Caspar Oesterheld and Vincent Conitzer
Proof. Finding a Pareto-improvement on a given y∈R𝑛can
be formulated as the following linear program:
Variables: 𝑝a∈[0,1]for all a∈𝐴
Maximize𝑛Õ
𝑖=1 Õ
a∈𝐴𝑝a𝑢𝑖(a)!
−𝑦𝑖
s.t.Õ
a∈𝐴𝑝a=1
Õ
a∈𝐴𝑝a𝑢𝑖(a)≥𝑦𝑖for𝑖=1,...,𝑛
□
G PROOF OF LEMMA 9.5
Lemma 9.5. LetΓ=({𝑎1
1,...,𝑎𝑙1
1},...,{𝑎1𝑛,...,𝑎𝑙𝑛𝑛},u)be any game.
LetΓ′be a perfect-coordination SPI on Γ. Then we can define u𝑒with
values inC(Γ)such that under Assumption 7.1 the game
Γ𝑠=
ˆ𝐴1B{ˆ𝑎1
1,...,ˆ𝑎𝑙1
1},...,ˆ𝐴𝑛B{ˆ𝑎1
𝑛,...,ˆ𝑎𝑙𝑛𝑛},
ˆu:(ˆ𝑎𝑖1
1,...,ˆ𝑎𝑖𝑛𝑛)↦→ u(𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛),u𝑒
is also an SPI on Γ, with
E
u(Π(Γ𝑠))Π(Γ)=a
=E
u(Π(Γ′))Π(Γ)=a
for all a∈𝐴and consequently E[u(Π(Γ𝑠))]=E[u(Π(Γ′))].
Proof. First note that(ˆ𝐴,ˆu)is isomorphic to Γ. Thus by Assump-
tion 7.1, there is isomorphism Φs.t.Γ∼Φ(ˆ𝐴,ˆu). WLOG assume that
Φsimply maps 𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛↦→ˆ𝑎𝑖1
1,...,ˆ𝑎𝑖𝑛𝑛. Then define u𝑒as follows:
u𝑒(ˆ𝑎𝑖1
1,...,ˆ𝑎𝑖𝑛𝑛)=Eh
u′(Π(Γ′))|Π(Γ)=(𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛)i
. (16)
Here u′describes the utilities that the original players assign to
the outcomes of Γ′. Since u′maps ontoC(Γ)andC(Γ)is con-
vex, u𝑒as defined also maps into C(Γ)as required. Note that for
all𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛it is by assumption u′(Π(Γ′))≥ u(𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛)with
certainty. Hence,
𝑢𝑒(ˆ𝑎𝑖1
1,...,ˆ𝑎𝑖𝑛𝑛))=Eh
u′(Π(Γ′))|Π(Γ)=(𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛)i
≥u(𝑎𝑖1
1,...,𝑎𝑖𝑛𝑛),
as required. □
H PROOF OF THEOREM 9.7
Proof. We will give the proof based on the graphs as well, with-
out giving all formal details. Further we assume in the following
that neither 𝐿1nor𝐿3consist of just a single point, since these cases
are easy.
Case A : Note first that by Corollary 9.6 it is enough to show that
ifyis in any of the listed sets 𝐿1,𝐿2,𝐿3, it can be made safe.
It’s easy to see that all payoff vectors on the curve segment of
the Pareto frontier 𝐿2are safely achievable. After all, all payoff
vectors in this set Pareto-improve on all outcomes in supp(Π(Γ)).
Hence, for each yon the line segment, one could select the Γ𝑠where
u𝑒=y.
It is left to show that all elements of 𝐿1/2are safely achievable.
Remember that not all payoff vectors on the line segments are Paretoimprovements, only those that are to the north-east of (Pareto-
better than) the default utility. In the following, we will use 𝐿′
1and
𝐿′
3to denote those elements of 𝐿1and𝐿3, respectively, that are
Pareto-improvements on the default.
We now argue that the Pareto improvement yon the line
𝐿1for which𝑦1=E[𝑢1(Π(Γ))]is safely achievable. In other
words, yis the projection northward of the default utility, or
y=𝜋1(E[u(Π(Γ))],𝐿1). This yis also one of the endpoints of
𝐿′
1. To achieve this utility, we construct the equivalent game as per
Lemma 9.5, where the utility to the original players of each outcome
(ˆ𝑎1,ˆ𝑎2)of the new game Γ𝑠is similarly the projection northward
onto𝐿1of the utility of the corresponding outcome (𝑎1,𝑎2)inΓ𝑠.
That is,
u𝑒(ˆ𝑎1,ˆ𝑎2)=𝜋1(u(𝑎1,𝑎2),𝐿1). (17)
Note that because C(Γ)is convex and the endpoints of the line
segment𝐿1are by definition in C(Γ), it is𝐿1⊆C(Γ). Hence, all
values of u𝑒as defined in Eq. 17 are feasible. Because all outcomes
in the original game lie below the line 𝐿1,𝜋1is linear. Hence,
E
u𝑒(Π(Γ𝑠))
=E[𝜋1(u(Π(Γ)),𝐿1)] (18)
=𝜋1(E[u(Π(Γ))],𝐿1) (19)
as required.
We have now shown that one of the endpoints of 𝐿′
1is safely
achievable. Since the other endpoint of 𝐿′
1is in𝐿2, it is also safely
achievable. By Corollary 9.6, this implies that all of 𝐿′
1is safely
achievable.
By an analogous line of reasoning, we can also show that all
elements of 𝐿′
3are safely achievable.
Case B : Define𝐿′
1,𝐿′
3as before as those elements of 𝐿1,𝐿3re-
spectively that Pareto improve on the default E[u(Π(Γ))]. By
a similar argument as before, one can show that the utilities
𝜋𝑖(E[u(Π(Γ))],𝐿′
𝑗)is safely achievable both for 𝑖=1,𝑗=1and
for𝑖=2,𝑗=3. Call these points 𝐸1and𝐸3, respectively.
We now proceed in two steps. First, we will show that there is a
third safely achievable utility point 𝐸2, which is above both 𝐿1and
𝐿3. Then we will show the claim using that point.
To construct 𝐸2, we again construct an SPI Γ𝑠as per Lemma 9.5.
For each(𝑎1,𝑎2)∈𝐴1×𝐴2we will set the utility 𝑢𝑒(ˆ𝑎1,ˆ𝑎2)of the
corresponding(ˆ𝑎1,ˆ𝑎2)∈ˆ𝐴1×ˆ𝐴2to be above or on both 𝐿1and𝐿3,
i.e., on or above a set which we will refer to as max(𝐿1,𝐿3). Formally,
max(𝐿1,𝐿3)is the set of outcomes in 𝐿1∪𝐿3that are not strictly
Pareto dominated by some other element of 𝐿′
1∪𝐿′
3. Note that by
definition every outcome in supp(Π(Γ))is Pareto-dominated by
some outcome in either 𝐿1or𝐿3. Hence, by transitivity of Pareto
dominance, each outcome is Pareto-dominated by some outcome
inmax(𝐿1,𝐿3). Hence, the described u𝑒is indeed feasible.
Now note that the set of feasible payoffs of Γis convex. Further,
the curve max(𝐿1,𝐿3)is concave. Because the area above a concave
curve is convex and because the intersection of convex sets is
convex, the set of feasible payoffs on or above max(𝐿1,𝐿3)is also
convex. By definition of convexity, 𝐸2=E[u𝑒(Π(Γ𝑠))]is therefore
also in the set of feasible payoffs on or above max(𝐿1,𝐿3)and
therefore above both 𝐿1and𝐿3as desired.
In our second step, we now use 𝐸1,𝐸2,𝐸3to prove the claim.
Because of convexity of the set of safely achievable payoff vectors
Safe Pareto Improvements for Delegated Game Playing AAMAS ’21, May 3–7, 2021, Online
Player 2
𝑎 𝑏 𝑐
Player 1𝑎−5,−5 4,0 10,−100
𝑏 0,4 1,1 10,−100
𝑐−100,10−100,10 3,3
Table 4: An example of a game in which – depending on Π–
a Pareto improvement may not be safely achievable.
as per Corollary 9.6, all utilities below the curve consisting of the
line segments from 𝐸1to𝐸2and from𝐸2to𝐸3are safely achievable.
The line that goes through 𝐸1,𝐸2intersects the line that contains
𝐿1at𝐸1, by definition. Since non-parallel lines intersect each other
exactly once and parallel lines that intersect each other are equal
and because 𝐸2is above or on 𝐿1, the line segment from 𝐸1to𝐸2
lies entirely on or above 𝐿1. Similarly, it can be shown that the
line segment from 𝐸2to𝐸3lies entirely on or above 𝐿3. It follows
that the𝐸1−𝐸2−𝐸3curve lies entirely above or on min(𝐿1,𝐿3).
Now take any Pareto improvement that lies below both 𝐿′
1and𝐿′
3.
Then this Pareto improvement lies below min(𝐿′
1,𝐿′
3)and therefore
below the𝐸1−𝐸2−𝐸3curve. Hence, it is safely achievable. □I PROOF OF PROPOSITION 9.8
Proposition 9.8. There is a game Γ=(𝐴,u), representatives
Πthat satisfy Assumptions 7.1 and 7.2, and an outcome a∈𝐴
s.t.𝑢𝑖(a)>E[𝑢𝑖(Π(Γ))]for all players 𝑖, but there is no perfect-
coordination SPI(𝐴𝑠,u𝑠,u𝑒)s.t. for all players 𝑖,E
𝑢𝑒
𝑖(Π(𝐴𝑠,u𝑠))
=
𝑢𝑖(a).
Proof. Consider the game in Table 4. Strategy 𝑐can be elimi-
nated by strict dominance (Assumption 7.2) for both players, leav-
ing a typical Chicken-like payoff structure with two pure Nash
equilibria ((𝑎,𝑏)and(𝑏,𝑎)), as well as a mixed Nash equilibrium
(3/8∗𝑎+5/8∗𝑏,3/8∗𝑎+5/8∗𝑏).
Now let us say that in the resulting game 𝑃(Π(Γ)=(𝑎,𝑏))=𝑝=
𝑃(Π(Γ)=(𝑏,𝑎))for some𝑝with 0<𝑝≤1/2. Then one (unsafe)
Pareto-improvement would be to simply always have the repre-
sentatives play(𝑐,𝑐)for a certain payoff of (3,3). Unfortunately,
there is no safePareto improvement with the same expected payoff.
Notice that(3,3)is the unique element of C(Γ)that maximizes the
sum of the two players’ utilities. By linearity of expectation and con-
vexity ofC(Γ), if for any Γ𝑠it isE[u(Π(Γ𝑠))]=(3,3), it must be
u(Π(Γ𝑠))=(3,3)with certainty. Unfortunately, in any safe Pareto
improvement the outcomes (𝑎,𝑏)and(𝑏,𝑎)must corresponds to
outcomes that still gives utilities of (4,0)and(0,4), respectively,
because these are Pareto-optimal within the set of feasible payoff
vectors. □