The truth behind the myth of the folk theorem.

The truth behind the myth of the Folk theorem[☆](#aep-article-footnote-id1)
===========================================================================

Abstract
--------

We study the problem of computing an *ϵ*-Nash equilibrium in repeated games. Earlier work by [Borgs et al. (2010)](#br0050) suggests that this problem is intractable. We show that if we make a slight change to their model—modeling the players as polynomial-time Turing machines that maintain state—and make a standard cryptographic assumption (that public-key cryptography can carried out), the problem can actually be solved in polynomial time. Our algorithm works not only for games with a finite number of players, but also for constant-degree *graphical games* (where, roughly speaking, which players' actions a given player's utility depends on are characterized by a graph, typically of bounded degree). As [Nash equilibrium](/topics/economics-econometrics-and-finance/nash-equilibrium "Learn more about Nash equilibrium from ScienceDirect's AI-generated Topic Pages") is a weak solution concept for extensive-form games, we additionally define and study an appropriate notion of subgame-perfect equilibrium for computationally bounded players, and show how to efficiently find such an equilibrium in repeated games (again, assuming public-key cryptography).

Introduction
------------

The complexity of finding a Nash equilibrium (NE) is a fundamental question at the interface of game theory and computer science. A celebrated sequence of results showed that the complexity of finding a NE in a normal-form game is PPAD-complete [Chen and Deng (2006), Daskalakis et al. (2006)], even for 2-player games. Less restrictive concepts, such as *ϵ*-NE for an inverse-polynomial *ϵ*, are just as hard [Chen et al. (2006)]. This suggests that these problems are computationally intractable.

There was some hope that the situation would be better in infinitely-repeated games. The *Folk Theorem* (see Osborne and Rubinstein (1994) for a review) informally states that in an infinitely-repeated game *G*, for any payoff profile that is *individually rational*, in that all players get more than1 their minimax payoff (the highest payoff that a player can guarantee himself, no matter what the other players do) and is the outcome of some correlated strategy in *G*, there is a Nash equilibrium of *G* with this payoff profile. With such a large set of equilibria, the hope was that finding one would be less difficult. Indeed, Littman and Stone (2005) showed that ideas from the proof of the Folk Theorem can be used to design an algorithm for finding a NE in a two-player repeated game. Recently, Andersen and Conitzer (2013) used a folk-theorem–like strategy to design an algorithm that with high probability finds a NE in a restricted family of repeated games with more than two players and with the limit-of-means payoff criterion.

Despite these positive results, the hope of being able to compute a NE efficiently in repeated games was apparently dashed by Borgs et al. (2010) (BC+ from now on), who proved results suggesting that, for more than two players, it was as difficult to find a NE in infinitely-repeated games as it was in a single-shot normal-form game. Before explaining the results of BC+ in more detail and our new results, we need to consider more carefully what it means to compute a NE in a repeated game. In a single-shot normal-form game, a strategy for a player is simply a probability distribution over actions, and thus the algorithm that computes a NE can simply output an appropriate distribution. On the other hand, in a repeated game (or, more generally, in an extensive-form game) a strategy is a function from all histories to probability distributions over actions. A naive description of such a function can be quite large (or even infinite); thus, we can't expect an algorithm to output such a description. We instead need to consider a concise representation of the strategy. For example, Littman and Stone (2005) describe strategies using finite automata; the output of their algorithm is a description of such an automaton.

BC+ represent strategies using what they called *strategy machines*. A strategy machine is a Turing Machine (TM) that takes as input the history of the game so far and outputs a probability distribution over actions. Such a representation gives rise to another subtlety. If we do not restrict the TM in some way, there is a trivial algorithm that finds strategy machines that are a NE. The algorithm simply outputs a TM for each player that, at each round, first computes a NE of the underlying game and simply plays according to that player's probability distribution in the NE. Each player's TM might take exponential time to compute its next action, but its description is short (polynomial in the description of the underlying game) and can be computed efficiently. Intuitively, such a representation allows the algorithm to transfer the complexity of the problem to the players. This does not seem reasonable. The interest in being able to compute a NE efficiently largely stems from the fact that we think of players as being resource bounded. As pointed out by Roughgarden (2010), “participants in a repeated game face two types of complexity: that of formulating a strategy, and that of executing a strategy”. It seems inappropriate to limit one type of complexity without also limiting the other. We capture the complexity of formulating the strategy by restricting to polynomial-time algorithms for finding a NE. It seems reasonable to further require the strategy machines themselves to be polynomial-time TMs.2 With this requirement, the strategies that the algorithm outputs are no more “powerful” than the algorithm itself.

This is exactly what BC+ do. They show that the problem of finding a NE (or even an *ϵ*-NE for an inverse-polynomial *ϵ*) represented by a polynomial-time TM in an infinitely repeated game with three or more players where there is a discount factor bounded away from 1 by an inverse polynomial is PPAD-hard. They prove this by showing that, given an arbitrary normal-form game *G* with c≥2 players, there is a game G′ with c+1 players such that an algorithm can use an ϵ/8c-NE for the repeated game based on G′ represented by polynomial-time TMs to compute an *ϵ*-NE for *G*.

While their proof is indeed correct, in this paper, we challenge their conclusion. While BC+ restrict to strategies that are polynomial-time TMs, we do not think that this restriction suffices. As is standard when considering NE, BC+ allow players to deviate in arbitrary ways. However, we believe that polynomial-time players should also be restricted to using only polynomial-time deviations. It seems inconsistent to restrict players to using polynomial-time TMs in equilibrium, but then to allow them to deviate in ways that might not be implementable by a polynomial-time TM. Roughly speaking, we show that if we restrict to polynomial-time deviations, then we can compute an *ϵ*-NE in a repeated game in polynomial time.

Formally, we view players as probabilistic3 polynomial-time Turing machines (PPT TMs). We differ from BC+ in two key respects. First, as we suggested above, we restrict the deviations that can be made in equilibrium to those that can be computed by a PPT TM. Second, BC+ implicitly assume that players have no memory: they cannot remember computation from earlier rounds. By way of contrast, we allow players to have a bounded (polynomial) amount of memory. This allows players to remember, for example, the results of a few coin tosses from earlier rounds, and correlate in a non-trivial way moves in later rounds with these coin tosses. We call such TMs *stateful*, and the TMs used by BC+ *stateless*. We note that if we do not restrict to polynomial-time TMs, then there is no real difference between stateful TMs and stateless TMs in our setting (since a player with unbounded computational power can recreate the necessary state).4 These two assumptions mean that the players can use some cryptography (making some standard cryptographic assumptions) to coordinate the action of different players. We stress that this coordination happens in the process of the game play, not through communication. That is, there are no side channels; the only form of “communication” is by making moves in the game. With these assumptions (and the remaining assumptions of the BC+ model), we show that in fact an *ϵ*-NE in an infinitely-repeated game can be found in polynomial time.

Our equilibrium strategy uses threats and punishment in much the same way that they are used in the Folk Theorem. However, since players are computationally bounded, we can use cryptography (we assume the existence of a secure public key encryption scheme) to secretly correlate the punishing players. This allows us to overcome the difficulties raised by BC+, who show that computing an *uncorrelated* minimax punishment is NP-hard. We instead use the secret correlation to implement a correlated punishment that is easy to compute. Roughly speaking, the *ϵ*-NE can be described as proceeding in three stages. In the first stage, the players play a predefined sequence of actions repeatedly. If some player deviates from the sequence, the second stage begins. In this stage, the non-deviating players use their actions to secretly exchange a random seed, through the use of public-key encryption. In the third stage, the players use a correlated minimax strategy to punish the deviator forever. To achieve this correlation, the players use the secret random seed as the seed of a *pseudorandom* function (i.e., a function that a polynomial-time player cannot distinguish from a truly random function), and use the outputs of the pseudorandom function as the source of randomness for the correlated strategy. Since the existence of public-key encryption implies the existence of pseudorandom functions, the only cryptographic assumption needed is the existence of public-key encryptions—one of the most basic cryptographic assumptions.

The use of modern cryptography in game theory goes back to the works of Urbano and Vila, 2002, Urbano and Vila, 2004 and Gossner (1998); more recently, it has been used by, for example, Dodis, Halevi, and Rabin (2000). The work of Gossner, 1998, Gossner, 2000 is perhaps the application of cryptography to game theory that is most closely related to ours. Gossner (1998) uses cryptographic techniques to show how any payoff profile that is above the players' *correlated* minimax value can be achieved in a NE of a repeated game with public communication played by computationally bounded players. In Gossner (2000), a strategy similar to the one that we use is used to prove that, even without communication, the same result holds. Gossner's results apply only to infinitely-repeated games with 3 players and no discounting; Gossner claims that his results do not hold for games with discounting (which is what we consider here). We note that under the limit-of-means criterion there is a trivial solution to our problem. The algorithm can simply output TMs that first spend some rounds computing the NE of the underlying game and, after finding it, play it repeatedly. Since computing the NE can be done in finite time (although perhaps exponential in the size of the game), the players will eventually find it. Under the limit-of-means criterion, their payoff with this strategy is the equilibrium payoff; thus, the players do not care about the rounds wasted in finding the NE. It is worth noting that the same argument applies to the case where the discount factor is bounded away from 1 by only an inverse exponential.

While not the main emphasis of this paper, our results can be seen informally as showing that, like other folk theorem results, if players are computationally bounded and there exists a public-key encryption scheme, then a utility profile in which each player gets a utility that is above her utility in the correlated minimax can be supported (up to some inverse polynomial) by a computational *ϵ*-NE in games with an inverse-polynomial discount factor and a constant number of players (and, as discussed below, by a subgame-perfect equilibrium and in *graphical games*, where, roughly speaking, which players' actions a given player's utility depends on are characterized by a graph, typically of bounded degree), thus generalizing Gossner's results to these cases.

The idea of using the structure of the game as a means of correlation has also been suggested before. For example, Lehrer (1991) uses this idea to show an equivalence between NE and correlated equilibrium in certain repeated games with nonstandard information structures. In our setting, there is an additional complication: we need the correlation to be hidden from a specific player. The problem of hiding correlation was also considered in two recent papers. Bavly and Neyman (2014) show that concealment is possible if players have bounded recall, and the set of players trying to hide the correlation includes at least one player that has longer recall than any of the opponents. They also show that the same result holds if the players are represented as finite automata, and one of the players trying to hide the correlation is an automaton with more states than any of the opponents. Peretz (2013) shows that under appropriate conditions, if players have bounded recall, even 2 weak players can conceal their correlation from stronger players (although Bavly and Peretz (2019) show that this is no longer the case if the stronger players who recall capacity is linearly larger than that of the weaker players). The earlier results do not help in our setting. The assumptions made by Peretz do not hold in our setting, and to prove that a NE exists, it does not help to assume that players have asymmetric abilities; the stronger player could be the one that defects.

In the second part of the paper we show how to extend this result to a more refined solution concept. While NE has some attractive features, it allows some unreasonable solutions. In particular, the equilibrium might be obtained by what are arguably empty threats. This actually happens in our proposed NE (and in the basic version of the folk theorem). Specifically, players are required to punish a deviating player, even though that might hurt their payoff. If a deviation occurs, it might not be the best response of the players to follow their strategy and punish; thus, such a punishment is actually an empty threat.

To deal with this (well known) problem, a number of refinements of NE have been considered. The one typically used in dynamic games of perfect information is *subgame-perfect equilibrium*, suggested by Selten (1965). A strategy profile is a subgame-perfect equilibrium if it is a NE at every subgame of the original game. Informally, this means that at any history of the game (even those that are not on any equilibrium path), if all the players follow their strategy from that point on, then no player has an incentive to deviate. In the context of repeated games where players' moves are observed (so that it is a game of perfect information), the folk theorem continues to hold even if the solution concept used is subgame-perfect equilibrium [Aumann and Shapley (1994), Fudenberg and Maskin (1986), Rubinstein (1979)].

We define a computational analogue of subgame-perfect equilibrium that we call *computational subgame-perfect ϵ-equilibrium*, where the strategies involved are polynomial-time, and deviating players are again restricted to using polynomial-time strategies. There are a number of subtleties that arise in defining this notion. While we assume that all actions in the underlying repeated game are observable, we allow our TMs to also have memory, which means that the action of a TM does not depend only on the public history. Like subgame-perfect equilibrium, our computational solution concept is intended to capture the intuition that the strategies are in equilibrium after any possible deviation. This means that in a computational subgame-perfect equilibrium, at each history for player *i*, player *i* must make a (possibly approximate) best response, no matter what his and the other players' memory states are.

To compute a computational subgame-perfect *ϵ*-equilibrium, we use the same basic strategy as for NE, but, as often done to get a subgame-perfect equilibrium (for example see Fudenberg and Maskin (1986)), we limit the punishment phase length, so that it is incentive compatible for players to punish deviations. However, to prove our result, we need to overcome one more significant hurdle. When using cryptographic protocols, it is often the case (and, specifically, is the case in the protocol used for NE) that player *i* chooses a secret (e.g., a secret key for a public-key encryption scheme) as the result of some randomization, and then releases some public information which is a function of the secret (e.g., a public key). After that public information has been released, another party *j* typically has a profitable deviation by switching to the TM *M* that can break the protocol—for every valid public information, there always *exists* some TM *M* that has the secret “hardwired” into it (although there may not be an efficient way of finding *M* given the information). We deal with this problem by doing what is often done in practice: we do not use any key for too long, so that *j* cannot gain too much by knowing any one key.

A second challenge we face is that in order to prove that our new proposed strategies are even an *ϵ*-NE, we need to show that the payoff of the *best response* to this strategy is not much greater than that of playing the strategy. However, since for any polynomial-time TM there is always a better polynomial-time TM that has just a slightly longer running time, this natural approach fails. This instead leads us to characterize a class of TMs that we can analyze, and show that any other TM can be converted to a TM in this class that has at least the same payoff. While such an argument might seem simple in the traditional setting, since we only allow for polynomial time TMs, in our setting this turns out to require a surprisingly delicate construction and analysis to make sure this converted TM does indeed has the correct size and running time.

There are a few recent papers that investigate solution concepts for extensive-form games involving computationally bounded players [Gradwohl et al. (2013), Halpern and Pass (2013), Kol and Naor (2008)]; some of these focus on cryptographic protocols [Gradwohl et al. (2013), Kol and Naor (2008)]. Kol and Naor (2008) discuss refinements of NE in the context of cryptographic protocols, but their solution concept requires only that on each history on the equilibrium path, the strategies from that point on form a NE. Our requirement for the computational subgame-perfect equilibrium is much stronger. Gradwohl, Livne and Rosen (2013) also consider this scenario and offer a solution concept different from ours; they try to define when an empty threat occurs, and look for strategy profiles where no empty threats are made. Again, our solution concept is much stronger.

The rest of this paper is organized as follows. In Section 2, we review the relevant definitions from game theory and cryptography. In Section 3, we define our notion of computational *ϵ*-NE and show how find it efficiently for repeated games. In Section 4, we consider computational subgame-perfect *ϵ*-equilibrium and show that it too can be found efficiently.

Section snippets
----------------

One-shot games
--------------

We define a game *G* to be a triple ([c],A,u→), where [c]={1,…,c} is the set of players, Ai is the set of possible actions for player *i*, A=A1×…×Ac is the set of action profiles, and u→:A→Rc is the utility function (u→i(a→) is the utility of player *i*). A (mixed) *strategy* σi for player *i* is a probability distribution over Ai, that is, an element of Δ(Ai) (where, as usual, we denote by Δ(X) the set of probability distributions over the set *X*). We use the standard notation x→−i to denote vector x→

Computing a computational NE
----------------------------

As we discussed in the introduction, we are interested in designing an algorithm that, given a game *G*, outputs a profile (M1,…,Mn) of (possibly probabilistic) polynomial-time TMs, where Mi computes player *i*'s strategy in G∞. Each TM Mi gets as input in each round of the game the history of the game so far, and uses that and its internal memory state (recall that the TMs considered in BC+ did not have internal memory) to compute its next action. By polynomial-time TMs we mean that at round *t* the

Motivation and definition
-------------------------

In this section we would like to define a notion similar to subgame-perfect equilibrium, where for all histories *h* in the game tree (even ones not on the equilibrium path), playing σ→ restricted to the subtree starting at *h* forms a NE. This means that a player does not have any incentive to deviate, no matter where he finds himself in the game tree.

As we suggested in the introduction, there are a number of issues that need to be addressed in formalizing this intuition in our computational

Acknowledgments
---------------

Joseph Halpern and Lior Seeman are supported in part by NSF grants IIS-0911036 and CCF-1214844, by AFOSR grant FA9550-08-1-0266, by ARO grant W911NF-14-1-0017, and by the Multidisciplinary University Research Initiative (MURI) program administered by the AFOSR under grant FA9550-12-1-0040. Lior Seeman is partially supported by a grant from the Simons Foundation #315783. Rafael Pass is supported in part by an Alfred P. Sloan Fellowship, a Microsoft Research Faculty Fellowship, NSF Awards

References (34)
---------------

* G. Bavly *et al.*### [Online concealed correlation and bounded rationality](/science/article/pii/S0899825614001249)

### Games Econ. Behav.

(2014)
* G. Bavly *et al.*### [Limits of correlation in repeated games with bounded memory](/science/article/pii/S0899825619300387)

### Games Econ. Behav.

(2019)
* C. Borgs *et al.*### [The myth of the folk theorem](/science/article/pii/S0899825609000785)

### Games Econ. Behav.

(2010)
* X. Chen *et al.*### [Settling the complexity of two-player Nash equilibrium](/science/article/pii/S1056871906000177)
* S. Goldwasser *et al.*### [Probabilistic encryption](/science/article/pii/0022000084900709)

### J. Comput. Syst. Sci.

(1984)
* M.L. Littman *et al.*### [A polynomial-time Nash equilibrium algorithm for repeated games](/science/article/pii/S0167923604001733)

### Decis. Support Syst.

(2005)
* A. Rubinstein### [Equilibrium in supergames with the overtaking criterion](/science/article/pii/0022053179900024)

### J. Econ. Theory

(1979)
* A. Urbano *et al.*### [Unmediated communication in repeated games with imperfect monitoring](/science/article/pii/S0899825603001167)

### Games Econ. Behav.

(2004)
* G. Andersen *et al.*### Fast equilibrium computation for infinitely repeated games
* R. Aumann *et al.*### Long-term competition—a game-theoretic analysis

- X. Chen *et al.*### Computing Nash equilibria: approximation and smoothed complexity
- C. Daskalakis *et al.*### The complexity of computing a Nash equilibrium
- W. Diffie *et al.*### New directions in cryptography

### IEEE Trans. Inf. Theory

(1976)
- Y. Dodis *et al.*### A cryptographic solution to a game theoretic problem
- D. Fudenberg *et al.*### The folk theorem in repeated games with discounting or with incomplete information

### Econometrica

(1986)
- O. Goldreich### Foundation of Cryptography. Basic Tools, vol. I

(2001)
- O. Goldreich *et al.*### How to construct random functions

### J. ACM

(1986)
View more referencesCited by (2)
------------

* ### [Discounted Repeated Games Having Computable Strategies with No Computable Best Response under Subgame-Perfect Equilibria](https://doi.org/10.1145/3505585)

2022, ACM Transactions on Economics and Computation
* ### [A Complete Characterization of Infinitely Repeated Two-Player Games having Computable Strategies with no Computable Best Response under Limit-of-Means Payoff](http://www.scopus.com/scopus/inward/record.url?partnerID=10&rel=3.0.0&view=basic&eid=2-s2.0-85093274857&md5=5db1ee30c21778b093a9a17a9cd731)

2020, arXiv
Recommended articles (6)
------------------------

* Research article### [Stable sharing rules and participation in pools of essential patents](/science/article/pii/S089982561930079X)

Games and Economic Behavior, Volume 117, 2019, pp. 40-58Show abstractFor pools of essential patents I study whether a pool's sharing rule is stable against arbitrage, so that the pool's members have no incentive to trade patents. I show that the only stable rule is the numeric proportional rule, which gives each member a share of the pool's profit equal to its share of the pool's patents. I study how the stable rule affects firms' incentives to participate, and I show that firms with few patents tend to remain outside the pool. I look at the trade off between stability and participation, and I show that as trade dilutes their shares, members prefer the stable rule. I consider individual licenses, stand-alone patents, integration, and R&D. The results suggest that pools use the stable rule to maximize their founding members' worth. They help us understand why the numeric proportional rule is so common among pools of essential patents.
* Research article### [Positive eigenfunctions for nonlocal dispersal problems](/science/article/pii/S0893965919303325)

Applied Mathematics Letters, Volume 99, 2020, Article 106008Show abstractIn this paper, we investigate the positive eigenfunctions for a class of weighted nonlocal eigenvalue problems. Using some nonlocal estimates and a nonlinear functional, we obtain the existence of a principal eigenvalue and its variational characterization.
* Research article### [An experimental study on sequential auctions with privately known capacities](/science/article/pii/S0899825619300855)

Games and Economic Behavior, Volume 117, 2019, pp. 289-315Show abstractWe experimentally study sequential procurement auctions where bidders' capacity constraints are private information. Our experiment involves two first-price auctions with a belief elicitation stage at the end of the first. Our results show that (i) observed behavior in the second auction is overall consistent with sequential rationality; (ii) first auction bids are decreasing in the capacity of the bidder, but (iii) stated beliefs are inconsistent with the actual play. Hence, subjects seem to be aware of the opportunity cost of early bids (which leads capacity constrained bidders to bid more cautiously than unconstrained ones); on the other hand, since they do not recognize the informative content of bids, the potential signaling cost associated with early bids does not come into play.
* Research article### [Interdependent value auctions with insider information: Theory and experiment](/science/article/pii/S0899825619300958)

Games and Economic Behavior, Volume 117, 2019, pp. 218-237Show abstractWe develop a model of interdependent value auctions in which two types of bidders compete: insiders, who are perfectly informed about their value, and outsiders, who are informed only about the private component of their value. Because the bidding strategy of outsiders cannot efficiently adjust with that of insiders, the second-price auction is inefficient. The English auction has an equilibrium in which the information outsiders infer from the history of drop-out prices enables them to bid toward attaining efficiency. The presence of insiders has positive impacts on the seller's revenue. A laboratory experiment confirms key theoretical predictions, despite evidence of naive bidding.
* Research article### [Teams promise but do not deliver](/science/article/pii/S0899825619301113)

Games and Economic Behavior, Volume 117, 2019, pp. 420-432Show abstractIndividuals and two-person teams play a hidden-action trust game with pre-play communication. We replicate previous results for individuals that non-binding promises increase cooperation rates, but this does not extend to teams. While teams promise to cooperate at the same rate as individuals, they consistently renege on those promises. Additional treatments begin to explore the basis for team behavior. We rule out explanations hypothesizing that concern for partner's payoff drives team outcomes, as absent within-team communication, promise fulfillment rates *increase* compared to individuals. Rather, the results are consistent with the idea that communication between teammates provides support for self-serving behavior.
* Research article### [Group behaviour in tacit coordination games with focal points – an experimental investigation](/science/article/pii/S0899825619301125)

Games and Economic Behavior, Volume 117, 2019, pp. 461-478Show abstractThis paper reports an experimental investigation of Schelling's theory of focal points that compares group and individual behaviour. We find that, when players' interests are perfectly aligned, groups more often choose the salient option and achieve higher coordination success than individuals. However, in games with conflicts of interest, groups do not always perform better than individuals, especially when the degree of conflict is substantial. We also find that groups outperform individuals when identifying the solution to the coordination problem requires some level of cognitive sophistication. Finally, players that successfully identify the solution to this game also achieve greater coordination rates than other players in games with a low degree of conflict. This result raises the question of whether finding the focal point is more a matter of logic rather than imagination as Schelling argued.
[☆](#baep-article-footnote-id1)Preliminary versions of this work appeared in “The truth behind the myth of the folk theorem”, *Proceedings of the 5th Conference on Innovations in Theoretical Computer Science (ITCS)*, 2014 and “Not just an empty threat: subgame-perfect equilibrium in repeated games played by computationally bounded players”, *Proceedings of the 10th Conference on Web and Internet Economics (WINE)*, 2014.

[View full text](/science/article/pii/S0899825619300582)

Joseph Y
Halpern
Rafael Pass
Lior Seeman


Deception in finitely repeated security games.

Deception in Finitely Repeated Security Games
Thanh H. Nguyen1, Yongzhao Wang2, Arunesh Sinha2, Michael P. Wellman2
1University of Oregon, thanhhng@cs.uoregon.edu
2University of Michigan, fwangyzh,arunesh,wellman g@umich.edu
Abstract
Allocating resources to defend targets from attack is often
complicated by uncertainty about the attacker’s capabilities,
objectives, or other underlying characteristics. In a repeated
interaction setting, the defender can collect attack data over
time to reduce this uncertainty and learn an effective defense.
However, a clever attacker can manipulate the attack data to
mislead the defender, inﬂuencing the learning process toward
its own beneﬁt. We investigate strategic deception on the part
of an attacker with private type information, who interacts
repeatedly with a defender. We present a detailed computa-
tion and analysis of both players’ optimal strategies given the
attacker may play deceptively. Computational experiments
illuminate conditions conducive to strategic deception, and
quantify beneﬁts to the attacker. By taking into account the
attacker’s deception capacity, the defender can signiﬁcantly
mitigate loss from misleading attack actions.
Introduction
Real-world security domains are often characterized by im-
perfect information : uncertainty (particularly on the de-
fender’s part) about actions taken or underlying characteris-
tics of the opposing agent. Experience observed through re-
peated interaction in such domains provides an opportunity
for the defender to learn about the behaviors and character-
istics of attacker(s) [Kar et al., 2017, Gholami et al., 2017,
Haghtalab et al., 2016, Nguyen et al., 2016, Xu et al., 2016,
Balcan et al., 2015, Blum et al., 2014, Marecki et al., 2012,
Letchford et al., 2009]. For example, in wildlife protection
[Fang et al., 2016], repeated interaction with poachers al-
lows the defense authorities to observe poaching signs and
patterns over time. From these observations, the defender
may infer features of the poacher’s capabilities and prefer-
ences, and thus design more effective patrolling strategies.
To the extent that the defender relies on data, however,
the attacker may choose to modify its behavior to mislead
the defender. That is, in a particular interaction the attacker
may select an action that does not actually yield the best im-
mediate reward, to avoid revealing sensitive private informa-
tion. Such deceptive behavior could manipulate the outcome
of learning to the long-term beneﬁt of the attacker. A savvy
defender, therefore, would take into account the attacker’s
Copyright c
2019, Association for the Advancement of Artiﬁcial
Intelligence (www.aaai.org). All rights reserved.manipulative strategy in designing his own strategy. We la-
bel an attacker strategy as non-manipulative if it chooses an
action without regard to the defender’s learning, for exam-
ple if its behavior in each stage is a myopic best response.
Previous work on learning in security games has generally
treated the attacker as non-manipulative in this sense [Blum
et al., 2014, Marecki et al., 2012].
We study the strategic deployment of attacker deception
in ﬁnitely repeated security games. We adopt an incomplete-
information model, where the defender has underlying un-
certainty about the attacker’s type. At each time step, the
defender updates his belief on the attacker’s type based on
attack data collected at previous steps. Based on the updated
belief, the defender chooses an action to play. The attacker
decides its own action, aware that the defender is collecting
attack data to infer about the attacker’s type. The ultimate
goal of both players is to maximize expected utility accu-
mulated over the whole time horizon. A pair of strategies
that best-respond to each other, accounting for observations,
constitutes a Perfect Bayesian Nash Equilibrium (PBNE).
The paper includes four main contributions. First, we
present a non-linear optimization program to ﬁnd a PBNE
of the ﬁnitely repeated simultaneous-move game. Second,
we present a result that provides an easy technique to ﬁnd
a sequential equilibrium of the game based on a computed
PBNE. Third, we show that there exists a PBNE in which
players’ equilibrium strategies depend only on histories of
the attacker’s actions. This allows us to represent both play-
ers’ strategies in a compact form, which helps in signiﬁ-
cantly speeding up the equilibrium computation of the game.
Fourth, we provide a preliminary extension to the Stackel-
berg game (sequential move) setting.
Finally, we present a detailed experimental analysis of
strategic deception, showing how various game factors af-
fect the tendency for the attacker to deviate from myopic
best responses to mislead the defender. Our results show that
the defender and attacker receive signiﬁcant loss and beneﬁt
respectively if the defender does not address the attacker’s
deception. By taking into account deceptive attacks, such
loss and beneﬁt is reduced drastically.
Related Work
Learning in security games. Most existing work on learn-
ing in security games follows a Stackelberg model and as-
sumes the attacker plays myopically at every time step [Kar
et al., 2017, Gholami et al., 2017, Haghtalab et al., 2016,
Nguyen et al., 2016, Blum et al., 2014, Marecki et al.,
2012, Letchford et al., 2009]. Balcan et al. [2015] and Xu
et al. [2016] study the problem of learning with no prior
knowledge of the attacker’s behavior. They take a regret-
minimization approach to determine the defender’s strate-
gies at each time step.
Secrecy and deception in security games. Previous work
studies security scenarios in which information available to
the defender and attacker is asymmetric [Guo et al., 2017,
Xu et al., 2015, Rabinovich et al., 2015, Hendricks and
McAfee, 2006, Brown et al., 2005, Farrell and Rabin, 1996,
Zhuang et al., 2010]. The defender can exploit that infor-
mation asymmetricity to strategically reveal or disguise his
information to the attacker. This results in responses of the
attacker which are in favor of the defender. For example, in
the model of Guo et al. [2017], the defender can disguise de-
fense resources to deceive the attacker about the defender’s
type. We study an opposite scenario in which the attacker
acts deceptively to mislead the defender.
Repeated games with incomplete information. Previous
work has studied inﬁnitely repeated games with incomplete
information [Sorin, 2002, Aumann and Maschler, 1995, Jor-
dan, 1995, Zamir, 1992, Forges, 1988]. These studies ana-
lyze properties and the convergence of players’ strategies in
an inﬁnitely repeated game setting. We study the problem
of one-sided incomplete information (i.e., uncertainty in the
attacker’s type) in ﬁnitely repeated security games.
Adversarial machine learning. There have been several
studies on adversarial machine learning, attempting to inves-
tigate different attack scenarios on machine learning algo-
rithms [Br ¨uckner et al., 2012, Br ¨uckner and Scheffer, 2011,
Barreno et al., 2010, 2006, Lowd and Meek, 2005]. For ex-
ample, causative attacks alters the training process by in-
ﬂuencing the training data or exploratory attacks attempts
to discover information about the learner and its training
data. Different machine learning algorithms are then pro-
posed which can resist these sophisticated attacks. Our work
focuses on a causative attack scenario in security games.
We aim at obtaining effective defense strategies which min-
imizes the damage of deceptive attacks in security games,
given some learning outcome of attack data.
Game Model
In a ﬁnitely repeated simultaneous-move security game,
there is a set of Ntargets, denoted by N=f1;:::;Ng. A
defender attempts to protect these targets by allocating lim-
ited security resources over these targets. Conversely, an at-
tacker aims at attacking these targets. We denote by K <N
the number of the defender’s security resources. At each
time steptin a ﬁnite time horizon T=f1;:::;Tg, both
the defender and the attacker has to decide on which action
to take. An action of the defender, s, is an allocation of K
resources over N. We denote by Sthe set of all feasible ac-
tions of the defender. An action of the attacker is a target
to attack. There is a set of attacker types =f1;:::;Lg.Each type2has a prior probability p2(0;1)such
thatP
p= 1. At the beginning, Nature randomly draws
a type to play the game according to a prior distribution
fpg. The attacker knows its type while the defender does
not. The defender is aware of fpg.
Player payoffs. Each targeti2Nis associated with re-
wards and penalties of the defender,
Rd(i);Pd(i)
, and the
attacker,
R(i);P(i)
, for every type 2. When the at-
tacker of type 2attacksi, if the defender is protecting
i, the attacker receives a penalty P(i)while the defender
obtains a reward Rd(i). Conversely, if the defender is not
protecting target i, the attacker gets R(i)> P(i)while
the defender receives Pd(i)<Rd(i).
Player observations. Att+ 12T, both play-
ers observe their actions at previous time steps ht=
f(s1;i1);:::; (st;it)gwhere st0andit0are the defender and
the attacker actions respectively at time step t0. We denote by
Htthe set of all possible histories of length tandH=fHtg
(wheret= 0;:::;T1) the set of all histories. In particular,
H0=;. We denote by ha
t=fi1;:::;itga history of the
attacker’s actions and Ha
tthe set of all these attack histories.
Behavioral strategies. At each step t+ 1, given a history
ht2Ht, a behavioral strategy of the defender is a proba-
bility distribution x(ht) =fx(sjht) :P
sx(sjht) =
1;x(sjht)2[0;1];8s2Sgover the defender’s action
setS.x(sjht)is the probability the defender takes action
s2Sgiven the history ht. Similarly, a behavioral strat-
egy of the attacker of type is a probability distribution
y(ht) =fy(ijht) :P
iy(ijht) = 1;y(ijht)2
[0;1];8i2Ngover the attacker’s actions N.y(ijht)is
the probability the attacker of type attacks target igiven
ht. We denote by x=fx(ht)gandy=fy(ht)gstrate-
gies of the defender and attacker of type over all ht2H
respectively. Finally, XandY=fYgdenote the sets of
all strategies xandyrespectively.
Player expected utilities. Letxandy=fygbe the de-
fender and attacker’s behavioral strategies respectively. At
eacht+ 1, the defender can update his belief on the attacker
types using the Bayes rule, which is formulated as:
p(jht)/pYt
t0=1y(it0jht01)
where h0=;;ht0=fht01;it0g. Let:
EUd
i(x;ht)=hX
s:i2sx(sjht)i
Rd(i)Pd(i)
+Pd(i)
EU
i(x;ht)=hX
s:i2sx(sjht)i
P(i)R(i)
+R(i)
be immediate expected utilities of the defender and the at-
tacker of type respectively at target iat stept+1given the
defender plays x(ht). Based on the immediate expected util-
ities at every target, the players’ total expected utilities over
Tcan be computed using backward induction as follows.
At the last time step T, the total expected utilities of the
defender and the attacker with respect to history hT12
HT1is equal to their immediate expected utilities at hT1:
Ud
T(x;yjhT1)=X
;ip(jhT1)y(ijhT1)EUd
i(x;hT1)
U
T(x;yjhT1) =X
iy(ijhT1)EU
i(x;hT1);8
At time step t+ 1< T , the total expected utilities of
both players with respect to history ht2Htconsists of
(i) the immediate expected utility at t+ 1; and (ii) the future
expected utility after t+ 1. These utilities are formulated as:
Ud
t+1(x;yjht) =X
;ip(jht)y(ijht)EUd
i(x;ht)
+X
s;;ix(sjht)p(jht)y(ijht)Ud
t+2(x;yjht;(s;i))
U
t+1(x;yjht) =X
iy(ijht)EU
i(x;ht)
+X
s;ix(sjht)y(ijht)U
t+2(x;yjht;(s;i));8
Player’s goals. Given any history ht, both players aim at
choosing strategies x(ht);fy(ht)gthat maximize their to-
tal expected utility at ht. In this scenario, the attacker is no
longer myopic; it has to reason about all future possibilities
to decide on which behavioral strategy to play at each ht.
Such attack strategies (which may not be myopically opti-
mal) are chosen to mislead the defender about the attacker’s
type, ultimately beneﬁting the attacker in future steps. These
optimal behavioral strategies of players form a PBNE.
Game Equilibria
Deﬁnition 1 (PBNE) .Behavioral strategies of the defender
xand attacker yform a PBNE of the game if and only if
for every ht2Htthat occurs, we have:
xis the best response of the defender:
Ud
t+1(x;yjht)Ud
t+1(x;yjht);8x2X
y;is the best response of the attacker type :
U
t+1(x;yjht)U
t+1(x;yjht);8y2Y
Since the action sets of both players are ﬁnite, there al-
ways exists a PBNE of the game. Our ﬁrst result extends a
given PBNE to a reﬁned sequential equilibrium.
Theorem 1. For each PBNE, there is a sequential equilib-
rium in which players’ strategies are identical to the ones in
the PBNE at histories that occur with a positive probability.
Proof. We denote by (x;y)a PBNE of the game. We fol-
low the trembling-hand approach to ﬁnd a corresponding se-
quential equilibrium of (x;y). Letrd(ht)andr(ht)be the
ratios of the number of zero probabilities to the number of
non-zero probabilities in x(ht)andy(ht)respectively. For
each>0, we construct a new fully mixed behavioral strat-
egy of the defender and the attacker, (x;y), as follows:
x(sjht) =;ifx(sjht) = 0
x(sjht) =x(sjht)rd(ht);ifx(sjht)>0
y
(ijht) =;ify(ijht) = 0
y
(ijht) =y(ijht)r(ht);ify(ijht)>0
is chosen to be small enough such that all resulting prob-
abilities are positive. We use the breath-ﬁrst search (accord-
ing to the time horizon T) to examine the whole history set.When encountering a history with a zero probability of oc-
currencep(ht) = 0 according to (x;y), we construct a new
belief of the defender over attacker types at htas follows:
p0(jht)= lim
!0p(jht)= lim
!0pQ
t0y
(it0jht01)P
0p0Q
t0y0
(it0jht01)
We ﬁnd a PBNE of the corresponding sub-game starting
from this history htwith this new belief. We then replace the
strategies of the sub-game in (x;y)with these new equilib-
rium strategies. We also update (x;y)accordingly with the
updated strategies of the sub-game. This process will con-
tinue until all histories are examined. The resulting strategies
(x0;y0)with belieffp0(jht)gbelong to a sequential equi-
librium of the game. Indeed, it is straightforward to prove:
x0(sjht) = lim
!0x(sjht)
y;0(ijht) = lim
!0y
(ijht)
p0(jht) = lim
!0p(jht)(by deﬁnition)
Furthermore, the updating process only replaces strategies in
(x;y)at histories htwith a zero-probability of occurrence
by a PBNE of the sub-game at htrespective to the belief
p0(jht). Therefore, (x0;y0)is the best response of the
players at every htaccording to p0(jht).
Next, we present a result that enables a compact repre-
sentation of the game. We denote by Xaa subset of behav-
ioral strategies of the defender in which all the strategies are
independent of histories of the defender’s actions. In other
words, for all x2Xa,x(sjht) =x(sjha
t)for every
history htwhere ha
tis the corresponding history of attacker
actions. Similarly, Yais a subset of behavioral strategies of
the attacker.
Theorem 2. There exists a PBNE of the game in which the
equilibrium strategies of the players only depend on the his-
tories of the attacker’s actions.
Proof. We use Brouwer’s ﬁxed-point theorem and a back-
ward induction method. We are going to show that there ex-
ists a PBNE x2Xaandy2Yasuch that:8ht
Ud
t+1(x;yjht)Ud
t+1(x;yjht);8x2X (1)
U
t+1(x;yjht)U
t+1(x;yjht);8y2Y (2)
We denote by xfx(ht) sgthe defender strategy obtained
by replacing x(ht)inxby a defense action s. Similarly,
yfy(ht) igis the attacker strategy of type obtained
by replacing y(ht)inyby an attack action i. We deﬁne:
d
s(x;yjht)=maxf0;Ud
t+1(xfx(ht) sg;yjht)Ud
t+1(x;yjht)g

i(x;yjht)=maxf0;U
t+1(x;yfy(ht) ig)jht)U
t+1(x;yjht)g
which are non-negative continuous functions in (x;y). We
deﬁne a function F: (Xa;Ya)!(Xa;Ya)as follows:
F(x;y) = (x0;y0)where
x0(sjht) =x(sjht) +d
s(x;yjht)P
s0x(s0jht) +d
s0(x;yjht)
y;0(ijht) =y(ijht) +
i(x;yjht)P
jy(jjht) +
j(x;yjht)
SinceFis continuous over a convex and compact set
(Xa;Ya), there exists (x;y)such thatF(x;y) =
(x;y)according to the Brouwer’s ﬁxed point theorem.
On the other hand, according to the linearity of expecta-
tion, there must be an action ssuch thatUd
t+1(xfx(ht)
sg;yjht)Ud
t+1(x;yjht)0, meaning that d
s(x;yj
ht) = 0 . Therefore, we have:
x(sjht) =x(sjht)
1 +P
s0d
s0(x;yjht)
This implies d
s0(x;yjht) = 0;8s0. Similarly,

i(x;yjht) = 0;8i. As a result, we obtain:
Ud
t+1(xfx(ht) sg;yjht)Ud
t+1(x;yjht)(3)
U
t+1(x;yfy;(ht) ig)jht)U
t+1(x;yjht)(4)
for all sandiandht. While the above result holds for any
convex and compact (Xa;Ya), showing that the ﬁxed point
proﬁle provides more utility than any deviation to XorY
requires (Xa;Ya)to depend on the attacker’s past actions.
This dependence is due to the dependence of the posterior
belief on the attacker’s past actions, as can be seen in the
next steps of the proof. Based on the above inequalities, we
show that (x;y)satisfy (1–2) using backward induction.
At last time step T, for every x2X, we have:
Ud
T(x;yjhT1)
X
sx(sjhT1)Ud
T(xfx(hT1) sg;yjhT1)
=Ud
T(x;yjhT1)
Therefore, xis the defender best response against the at-
tacker’s strategy yat time stepT.
At time step t+ 1< T, suppose that (1–2) hold true for
allt0>t+ 1, then for every x2X, we have:
Ud
t+1(x;yjht)
X
sx(sjht)Ud
t+1(xfx(ht) sg;yjht)
=X
;ip(jht)y;(ijht)EUd
i(x;ht) +X
sx(sjht)
X
i;p(jht)y;(ijht)Ud
t+2(x;yjht;(s;i))
X
;ip(jht)y;(ijht)EUd
i(x;ht) +X
sx(sjht)
X
i;p(jht)y;(ijht)Ud
t+2(x;yjht;(s;i))
=Ud
t+1(x;yjht)
Therefore, xis the defender best response against the at-
tacker’s strategy yatt+ 1. Similarly, yis the attacker’s
best response against xat all time steps.
Equilibrium Computation
Based on Theorem 2, in computing a PBNE, we only need
to search over the strategy sets (Xa;Ya). We also only
need to consider attack histories fha
tg. We can now repre-
sent the defender behavioral strategies as compact marginalcoverage probabilities over targets. We overload the nota-
tionx(ha
t) =fx(ijha
t)gwherex(ijha
t)is the de-
fender’s coverage probability at target iat history ha
tsuch
thatP
ix(ijha
t)Kandx(ijha
t)2[0;1]for alli2N.
In particular, x(ijha
t) =P
s:i2sx(sjha
t). The players’
immediate and total expected utilities can be reformulated
accordingly as follows:
EUd
i(x;ha
t) =x(ijha
t)(Rd(i)Pd(i)) +Pd(i) (5)
EU
i(x;ha
t) =x(ijha
t)(P(i)R(i)) +R(i) (6)
Ud
t+1(x;yjha
t) =X
;ip(jha
t)y(ijha
t)EUd
i(x;ha
t)
+X
i;p(jha
t)y(ijha
t)Ud
t+2(x;yjha
t;i)(7)
U
t+1(x;yjha
t) =X
iy(ijha
t)EU
i(x;ha
t)
+X
iy(ijha
t)U
t+2(x;yjha
t;i) (8)
Note that the total expected utilities of players at last time
stepUd
T(x;yjha
T1)andU
T(x;yjha
T1)do not have the
second term (which represents the future expected utility) as
in Equations (7–8). In the following, we present a backward-
induction based method to ﬁnd the attacker (defender) best
response against a ﬁxed behavioral strategy of the defender
(attacker). We then introduce a program to compute a PBNE
based on these best-response solutions.
Attacker best response
Given a defender’s strategy x, we can compute a best re-
sponse of the attacker of type using backward induction:
At last time step T.Given a history ha
T1, ﬁnding a best
response of the attacker type against xis formulated as the
following linear program:
max
y(ha
T1)X
iy(ijha
T1)EU
i(x;ha
T1) (9)
s.t.X
iy(ijha
T1) = 1;y(ijha
T1)0;8i(10)
which maximizes the attacker’s total expected utility at
ha
T1. Its corresponding dual program:
min
v(ha
T1)v(ha
T1) (11)
s.t.v(ha
T1)EU
i(x;ha
T1);8i2N:(12)
According to complementary slackness, any optimal primal
and dual solutions (y;(ha
T1);v
(ha
T1))satisﬁes:8i
y;(ijha
T1)
v
(ha
T1)EU
i(x;ha
T1)
= 0 (13)
At time step t+ 1<T.Given a history ha
t, ﬁnding a best
response of the attacker type is formulated as:
max
y(ha
t)X
iy(ijha
t)
EU
i(x;ha
t) +v
(ha
t;i)
(14)
s.t.X
iy(ijha
t) = 1;y(ijha
t)0;8i2N: (15)
which maximizes the attacker’s total expected utility at ha
t
wherev
(ha
t;i)is the attacker’s optimal total expected util-
ity at (ha
t;i). Its corresponding dual program:
min
v(ha
t)v(ha
t) (16)
s.t.v(ha
t)EU
i(x;ha
t) +v
(ha
t;i);8i (17)
According to complementary slackness: any optimal solu-
tion(y;(ha
t);v
(ha
t))must satisfy:8i:
y;(ijht)
v
(ht)EU
i(x;ht)v
(ht;i)
=0 (18)
Defender best response
Given an attack strategy y, we denote by:
p(jha
t) =pYt
t0=1y(it0jha
t01)
Then the defender’s belief on attacker types at each history
ha
tcan be computed as follows:
p(jha
t) =p(jha
t)P
0p(0jha
t)(19)
Similar to the computation of a best response of attacker,
we can compute a best response of the defender against an
attack strategy yusing backward induction as follows:
At last time step T.Given a history ha
T1, ﬁnding a best
response x(ha
T1)can be formulated as:
maxX
p(jha
T1)X
iy(ijhT1)EUd
i(x;ha
T1)(20)
s.t.X
ix(ijhT1)K;x(ijhT1)2[0;1];8i: (21)
Proposition 1. For every attack history ha
T1, we denote
byvd
(ha
T1)the defender’s optimal total expected utility
against the attacker’s strategy yatha
T1. Then:
vd
(ha
T1) =vd
(ha
T1)P
0p(0jha
T1)
The proof of Proposition 1 is in the appendix.1Here,
vd
(ha
T1)the optimal objective of (20–21). By removing
the constant Pd(i)inEUd
i(x;ha
T1)(Equation 5) and tak-
ing the dualty, we obtain the corresponding dual program:
minKvd(0jha
T1) +X
ivd(ijha
T1) (22)
s.t.vd(0jha
T1)0;vd(ijha
T1)0;8i (23)
vd(0jha
T1) + vd(ijha
T1) (24)
X
p(jha
T1)y(ijha
T1)
Rd(i)Pd(i)
;8i:
According to complementary slackness, any optimal solu-
tions (x(ha
T1);fvd
(ijha
T1)g;vd
(0jha
T1))satisﬁes:8i
x(ijha
T1)[vd
(0jha
T1) + vd
(ijha
T1) (25)
X
p(jha
T1)y(ijha
T1)
Rd(i)Pd(i)
] = 0
vd
(ijha
T1)
x(ijha
T1)1
= 0 (26)
vd
(0jha
T1)hX
jx(jjha
T1)Ki
= 0 (27)
1Link: https://ix.cs.uoregon.edu/ ˜thanhhng/
publications/ConfPaper/AAAI19_Appendix.pdfAt time step t+ 1< T .Given a history ha
t, ﬁnding an
optimal behavioral strategy x(ha
t)can be formulated as the
following program:
max
x(ha
t)X
;ip(jha
t)y(ijha
t)EUd
i(x;ha
t) (28)
+X
ivd
(ha
t;i)
s.t.X
ix(ijha
t)K;x(ijha
t)2[0;1];8i (29)
We denote by vd
(ha
t)the optimal objective of (28–29) at
ht. In (28), vd
(ha
t;i)is the optimal objective of this primal
program (28–29) but with respect to the history (ha
t;i).
Proposition 2. For every attack history ha
t, we denote by
vd
(ha
t)the defender’s optimal total expected utility against
the attacker’s strategy yatha
t. Then:
vd
(ha
t) =vd
(ha
t)P
0p(0jha
t)
The proof of Proposition 2 is in the appendix. In (28), the
termP
ivd
(ha
t;i)and the term Pd(i)inEUd
i(x;ha
t)are
constant. By removing these constants and taking the dual,
we obtain the corresponding dual program:
minKvd(0jha
t) +X
ivd(ijha
t) (30)
s.t.vd(0jha
t)0;vd(ijha
t)0;8i2N (31)
vd(0jha
t) + vd(ijha
t) (32)
X
p(jht)y(ijha
t)
Rd(i)Pd(i)
;8i:
According to complementary slackness, any optimal solu-
tion(x(ha
t);fvd
(ijha
t)g;vd
(0jha
t))satisﬁes:8i:
x(ijha
t)
vd
(0jha
t) + vd
(ijha
t) (33)
X
p(jha
t)y(ijha
t)
Rd(i)Pd(i)
] = 0
vd
(ijha
t) [x(ijha
t)1] = 0 (34)
vd
(0jha
t)hX
ix(ijha
t)Ki
= 0 (35)
Equilibrium computation program
Based on the computation of players’ best responses,
a pair of behavioral strategies (x;y)forms a PBNE if
and only if these strategies satisfy (i) the feasibility con-
straints (21,23,24,29,31,32) and (10,12,15,17); and (ii) the
complementary slackness constraints (25–27, 33–35) and
(13, 18). Since ﬁnding strategies which satisfy these slack-
ness constraints is not straightforward, we convert the prob-
lem of ﬁnding a PBNE into the following program:
minsuch that8i;ha
t: (36)
y(ijha
t)
v(ha
t)EU
i(x;ha
t)v(ha
t;i)
(37)
x(ijha
t)
vd(0jha
t) + vd(ijha
t) (38)
X
p(jha
t)y(ijha
t)
Rd(i)Pd(i)
]
vd(ijha
t) [x(ijha
t)1] (39)
vd(0jha
t)hX
ix(ijha
t)Ki
(40)
Constraints (21,23,24,29,31,32), (10,12,15,17) (41)
wherev(ha
t;i) = 0 ift=T1. Constraints (37) and (38–
40) correspond to the complementary slackness constraints
of the attacker and defender respectively. Note that any equi-
librium of the game is a feasible solution of the program
(36–41) which returns an objective value of = 0. On the
other hand, the right-hand side of constraints (37–40) is al-
ways non-negative due to constraint (41). Thus, 0for all
feasible solutions of the program (36–41). It means that any
equilibrium of the game is an optimal solution of (36–41).
In addition, since the optimal objective value = 0, any op-
timal solution of (36–41) returns a value of zero for all the
right-hand sides of (37–40). Therefore, any optimal solution
of this program is a PBNE.
Extension to Stackelberg Setting
In the Stackelberg game model, a mixed strategy of the de-
fender is deﬁned as a probability distribution m=fm(s) :P
sm(s) = 1;m(s)2[0;1]gover the action set S. We de-
note by Mthe set of all mixed strategies of the defender.
At each time step, the defender commits to a mixed strategy.
The attacker is aware of that mixed strategy and then decides
which target to attack. Therefore, in ﬁnitely repeated Stack-
elberg games, at each time step t+ 1, an observation of the
defender is a history ht=f(m1;i1);:::; (mt;it)gwhile
an observation of the attacker is a history (ht;mt+1).
The behavioral strategy of the defender at htis a proba-
bility distribution x(ht) =fx(mjht) :P
mx(mjht) =
1;x(mjht)2[0;1]gover the set of mixed strategies of
the defender. On the other hand, a behavioral strategy of
the attacker of type at(ht;mt+1)is a probability distri-
bution y(ht;mt+1) =fy(ijht;mt+1) :P
iy(ij
ht;mt+1) = 1;y(ijht;mt+1)2[0;1]g. A PBNE
of Stackelberg security games is then deﬁned similarly as
simultaneous-move games. Since the set of mixed strategies
of the defender is inﬁnite, the existence of a PBNE in Stack-
elberg security games is an open research question. Never-
theless, we can compute an -PBNE by discretizing this set
of defense mixed strategies and applying the same backward
induction method as in the simultaneous case.
We speciﬁcally analyze the deception of the attacker in
ﬁnitely repeated Stackelberg security games with jNj= 2,
jj= 2, andK= 1. We adopt the tradition in Stackel-
berg security game that rewards and penalties are strictly
positive and negative respectively for both players. We con-
sider a game scenario in which the defender only plays a
pure behavioral strategy in Xpure=fx:x(mjht) =
1;for some m2M;8htg.
Theorem 3. In a ﬁnitely repeated Stackelberg security game
withjNj= 2,jj= 2, andK= 1, if the defender only
plays a pure behavioral strategy in Xpureand the rewards
and penalties are strictly positive and negative respectively
for both the players, there exists a PBNE of the game in
which the attacker plays a myopic best response at every
history ( ht;mt+1).
One signiﬁcance of this preliminary result is that the
assumption about a myopic attacker in previous work on
ﬁnitely repeated Stackelberg security games is justiﬁed (atleast in the simple setting of this result) even when the at-
tackers care about future expected utility. In future research,
we aim to generalize this special case and explore the de-
ception patterns for multiple targets and multiple types in
the Stackelberg setting.
Experiments
We focus on the attacker’s strategic use of deception. In our
experiments, the players’ rewards and penalties are gener-
ated uniformly at random in the range [1;10]and[10;1]
respectively.
Analysis of attacker deception
The purpose of deception is to shift the defender’s belief
away from the attacker’s true type. Any action on part of
the attacker toward this purpose must take into account sim-
ilar reasoning by other attacker types. Further, shaping the
belief of the defender is beneﬁcial only if it results in a later
gain for the attacker. In the following, we present our results
with respect to an attacker of type 1. The behavior for other
attacker types is symmetric.
In our ﬁrst experiment, we analyze games with num-
ber of attacker types: jj= 2, number of targets: jNj 2
f4;6;8;10;12g, and number of time steps jTj 2 f 2;3g.
Results are shown in Figure 1(a)(b). The x-axis is the prior
probability of attacker type 1. The y-axis is the probability a
type-1 attacker attacks a myopically non-optimal target (i.e.,
probability of deceptive action, or “lie” for short) at time step
t= 1 ort= 2 (for3-step games). Each data point is aver-
aged over 220 game instances. Figure 1(a) shows results for
2-step games; each curve corresponds to a number of tar-
gets. Overall, the attacker’s tendency to deceive is roughly
concave in the prior probability of its true type. This makes
sense, as deception has relatively less power to change the
beliefs of a defender when they start near extremes. We also
see an increase in deception with the number of targets. This
reﬂects the growth in options for deception, as well as in-
creased potential beneﬁt for misleading the defender.
Results for 3-step games are shown in Figure 1(b). We
present deception probabilities for the attacker of type 1 at:
(1) Step 1; (ii) Step 2; (iii) Step 2-lie (step 2 conditioned that
the attacker lied at step 1); and (iv) Step 2-not lie (step 2
conditioned that the attacker did not lie at step 1). In this
ﬁgure,jNj= 4. As for the 2-step game, the probability of
deception in each case is roughly concave in the prior. The
probability of deception at step 1 is somewhat elevated in the
3-step game, since the attacker accrues longer-term beneﬁt
from misleading the defender. Moreover, the peak is shifted
to the right, reﬂecting increased chance for successful decep-
tion given its opportunity to repeat the lie over two periods.
Indeed, given that the attacker lies at step 1, the attacker
lies with roughly proportional probability at step 2 (blue
curve versus yellow curve). On the other hand, when the at-
tacker does not lie at step 1, its pattern of deception at step 2
(purple curve) is qualitatively different. Switching to be de-
ceptive at step 2 is more promising at low priors (where the
act has some chance of misleading), and very unlikely at
high priors where there is little chance to mislead the de-
fender if it had not already started in step 1.
0.1 0.3 0.5 0.7 0.9
Prior prob of type 100.20.40.60.81Prob of lie4 T
6 T8 T
10 T12 T(a) 2-type, 2-step games
0.1 0.3 0.5 0.7 0.9
Prior prob of type 100.20.40.60.81Prob of lieStep 1
Step 2Step 2-lie
Step 2-not lie (b) 2-type, 3-step games
0 0.2 0.4 0.6 0.8 1
Prior prob of type 100.20.40.60.81Prior prob of type 2
 00.20.40.60.8 (c) 3-type, 2-step games
0.1 0.3 0.5 0.7 0.9
Prior prob of type 11234567#Def resources
00.20.40.60.8 (d) 2-type, 2-step games
Figure 1: Attacker deception analysis, attacker type 1.
0 0.2 0.4 0.6 0.8 1
Prior prob of type 11591317Att utilityPerfect Bayesian
Myopic w/ Learn
Deceptive w/ Learn
(a) Attacker type 1
0 0.2 0.4 0.6 0.8 1
Prior prob of type 11591317Att utilityPerfect Bayesian
Myopic w/ Learn
Deceptive w/ Learn (b) Attacker type 2
0 0.2 0.4 0.6 0.8 1
Prior prob of type 1-8-6-4-2 Def UtilityPerfect Bayesian
Myopic w/ Learn
Deceitful w/ Learn (c) Defender
3 6 9 12
#Target0481216202428Runtime (mins)2 Step
3 Step (d) Runtime
Figure 2: Solution comparison and runtime performance.
In our second experiment, we analyze deceptive strategies
of the attacker type 1 in 2-step games with the number of
attacker types isjj= 3. In these games,jNj= 8. The re-
sult is shown in Figure 1(c). The x-axis and y-axis represent
prior probabilities of types 1 and 2 respectively. Figure 1(c)
shows that the attacker deception tendency is unimodal with
respect the prior of its type 1, and less sensitive ot the distri-
bution across other types.
In our third experiment, we vary the number of defender
resources in 2-step games with 2 attacker types. The result
is shown in Figure 1(d). When the number of defender re-
sources is high (close to the number of the targets), the de-
fender can provide a high coverage probability at all targets.
Specially, when K=jNj, the defender protects all targets
all the time. As a result, the attacker may not achieve any
beneﬁt by lying. Therefore, the attacker lies less when K
gets closer tojNj.
Solution quality and runtime performance
In our last experiment, we compare the players’ utilities for
playing strategies computed in three scenarios:
1. Perfect Bayesian. The attacker is rationally deceptive and
the defender takes into account the potential deceit.
2. Myopic w/ Learn. The attacker is myopic and the de-
fender also assumes so.
3. Deceptive w/ Learn. The attacker is rationally deceptive
while the defender assumes the attacker is myopic.
The defender performs a Bayesian update on his belief about
the attacker’s type in all three cases. Results are shown in
Figures 2(a)(b)(c), averaging over 2203-step game instanceswith two attacker types and ﬁve targets. The x-axis is the
prior probability of type 1. The y-axis is the attacker util-
ity of each type or the defender utility on average. Fig-
ures 2(a)(b)(c) show if the defender does not account for
deception, the rationally deceptive attacker achieves a signif-
icant gain while the defender suffers a signiﬁcant loss (yel-
low versus red curves). When the defender accounts for the
prospect of deception of the attacker, such gains and losses
are drastically reduced (blue versus red).
Finally, we display in Figure 2(d) the runtime perfor-
mance of our equilibrium-ﬁnding algorithm. The x-axis is
the number of targets and the y-axis is runtime in minutes.
For 2-step games, the runtime remains modest for up to ten
targets. ForjTj= 3, the runtime grows quickly and exceeds
24 minutes when the number of targets is ﬁve.
Summary
We study the problem of deception in ﬁnitely repeated secu-
rity games. In these games, the defender collects attack data
over time to learn about the attacker type while the attacker
plays deceptively to mislead the defender. We present a de-
tailed analysis and computation of ﬁnding optimal strate-
gies of players in the games. We then show through com-
putational experiments that the attacker (defender) receives
a great beneﬁt (loss) when the defender does not take into
account deceptive attacks. Conversely, such beneﬁt (loss) is
reduced signiﬁcantly when the defender addresses the at-
tacker’s deception.
Acknowledgment This work was supported in part by
MURI grant W911NF-13-1-0421 from the US Army Re-
search Ofﬁce.
References
R. J. Aumann and M. Maschler. Repeated Games with In-
complete Information . MIT Press, 1995.
M.-F. Balcan, A. Blum, N. Haghtalab, and A. D. Procaccia.
Commitment without regrets: Online learning in Stackel-
berg security games. In 16th ACM Conference on Eco-
nomics and Computation , pages 61–78, 2015.
M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Ty-
gar. Can machine learning be secure? In ACM Symposium
on Information, Computer and Communications Security ,
pages 16–25, 2006.
M. Barreno, B. Nelson, A. D. Joseph, and J. D. Tygar. The
security of machine learning. Machine Learning , 81(2):
121–148, 2010.
A. Blum, N. Haghtalab, and A. D. Procaccia. Learning op-
timal commitment to overcome insecurity. In Advances
in Neural Information Processing Systems , pages 1826–
1834, 2014.
G. Brown, M. Carlyle, D. Diehl, J. Kline, and K. Wood.
A two-sided optimization for theater ballistic missile de-
fense. Operations Research , 53(5):745–763, 2005.
M. Br ¨uckner and T. Scheffer. Stackelberg games for ad-
versarial prediction problems. In ACM SIGKDD Inter-
national Conference on Knowledge Discovery and Data
Mining , pages 547–555, 2011.
M. Br ¨uckner, C. Kanzow, and T. Scheffer. Static prediction
games for adversarial learning problems. Journal of Ma-
chine Learning Research , 13:2617–2654, 2012.
F. Fang, T. H. Nguyen, R. Pickles, W. Y . Lam, G. R.
Clements, B. An, A. Singh, M. Tambe, and A. Lemieux.
Deploying PAWS: Field optimization of the protection as-
sistant for wildlife security. In 30th AAAI Conference on
Artiﬁcial Intelligence , pages 3966–3973, 2016.
J. Farrell and M. Rabin. Cheap talk. Journal of Economic
Perspectives , 10(3):103–118, 1996.
F. Forges. Repeated games of incomplete information: non-
zero-sum. Technical report, Universit ´e Catholique de
Louvain, Center for Operations Research and Economet-
rics (CORE), 1988.
S. Gholami, B. Ford, F. Fang, A. Plumptre, M. Tambe,
M. Driciru, F. Wanyama, A. Rwetsiba, M. Nsubaga, and
J. Mabonga. Taking it for a test drive: a hybrid spatio-
temporal model for wildlife poaching prediction evalu-
ated through a controlled ﬁeld test. In European Con-
ference on Machine Learning & Principles and Practice
of Knowledge Discovery in Databases , 2017.
Q. Guo, B. An, B. Bosansky, and C. Kiekintveld. Comparing
strategic secrecy and Stackelberg commitment in security
games. In 26th International Joint Conference on Artiﬁ-
cial Intelligence , 2017.
N. Haghtalab, F. Fang, T. H. Nguyen, A. Sinha, A. D. Pro-
caccia, and M. Tambe. Three strategies to success: Learn-
ing adversary models in security games. In 25th Interna-
tional Joint Conference on Artiﬁcial Intelligence , pages
308–314, 2016.K. Hendricks and R. P. McAfee. Feints. Journal of Eco-
nomics & Management Strategy , 15(2):431–456, 2006.
J. S. Jordan. Bayesian learning in repeated games. Games
and Economic Behavior , 9(1):8–20, 1995.
D. Kar, B. Ford, S. Gholami, F. Fang, A. Plumptre,
M. Tambe, M. Driciru, F. Wanyama, A. Rwetsiba,
M. Nsubaga, et al. Cloudy with a chance of poaching:
Adversary behavior modeling and forecasting with real-
world poaching data. In 16th International Conference
on Autonomous Agents and Multi-Agent Systems , pages
159–167, 2017.
J. Letchford, V . Conitzer, and K. Munagala. Learning and
approximating the optimal strategy to commit to. In Inter-
national Symposium on Algorithmic Game Theory , pages
250–262, 2009.
D. Lowd and C. Meek. Adversarial learning. In ACM
SIGKDD International Conference on Knowledge Dis-
covery in Data Mining , pages 641–647, 2005.
J. Marecki, G. Tesauro, and R. Segal. Playing repeated
Stackelberg games with unknown opponents. In 11th In-
ternational Conference on Autonomous Agents and Mul-
tiagent Systems , pages 821–828, 2012.
T. H. Nguyen, A. Sinha, S. Gholami, A. Plumptre, L. Joppa,
M. Tambe, M. Driciru, F. Wanyama, A. Rwetsiba,
R. Critchlow, et al. Capture: A new predictive anti-
poaching tool for wildlife protection. In 15th Inter-
national Conference on Autonomous Agents and Multi-
Agent Systems , pages 767–775, 2016.
Z. Rabinovich, A. X. Jiang, M. Jain, and H. Xu. Information
disclosure as a means to security. In 14th International
Conference on Autonomous Agents and Multi-Agent Sys-
tems, pages 645–653, 2015.
S. Sorin. A ﬁrst course on zero-sum repeated games , vol-
ume 37. Springer Science & Business Media, 2002.
H. Xu, Z. Rabinovich, S. Dughmi, and M. Tambe. Explor-
ing information asymmetry in two-stage security games.
In29th AAAI Conference on Artiﬁcial Intelligence , pages
1057–1063, 2015.
H. Xu, L. Tran-Thanh, and N. R. Jennings. Playing repeated
security games with no prior knowledge. In 15th Inter-
national Conference on Autonomous Agents and Multi-
Agent Systems , pages 104–112, 2016.
S. Zamir. Repeated games of incomplete information: Zero-
sum. Handbook of Game Theory with Economic Applica-
tions , 1:109–154, 1992.
J. Zhuang, V . M. Bier, and O. Alagoz. Modeling secrecy
and deception in a multi-period attacker-defender signal-
ing game. European Journal of Operational Research ,
203:409–418, 2010.
Appendix
Proof of Propositions 1&2
We are going to show that: for all ha
t
vd
(ha
t) =vd
(ha
t)P
0p(0jha
t)(42)
We follow the backward induction approach:
At last time step T, for every defender strategy x(ha
T1),
according to Equation 15, the objective in (16) can be
reformulated as follows:
vd(ha
T1) =X
p(jha
T1)X
iy(ijhT1)EUd
i(x;ha
T1)
=hX
0p(0jha
T1)i

X
;ip(jha
T1)y(ijhT1)EUd
i(x;ha
T1)
=X
0p(0jha
T1)
Ud
T(x;yjha
T1)
Therefore, Equation 42 is true for all ha
T1.
At time step t+ 1, suppose that Equation (42) holds
true for all ht0witht0t+ 1, we are going to show that it
is true for all history ht. Indeed, for every defense strategy
x(ha
t), the objective (24) can be reformulated as follows:
vd(ha
t) =X
;ip(jha
t)y(ijha
t)EUd
i(x;ha
t)
+X
ivd
(ha
t;i)
= X
0p(0jha
t)!X
;ip(jha
t)y(ijha
t)EUd
i(x;ha
t)
+X
ihX
p(jha
t;i)i
vd
(ha
t;i)
The second term of RHS can be rewritten as follows:X
ihX
p(jha
t;i)i
vd
(ha
t;i)
=X
ihX
p(jha
t)y(ijha
t)i
vd
(ha
t;i)
=X
i"X
 X
0p(0jha
t)!
p(jha
t)y(ijha
t)#
vd
(ha
t;i)
= X
0p(0jha
t)!X
i"X
p(jha
t)y(ijha
t)#
vd
(ha
t;i)
Therefore, we obtain:
vd
(ha
t) = X
0p(0jha
t)!
vd
(ha
t)
which means Equation 42 is true for all ha
t.
Stackelberg Game Model
Our Stackelberg game model equilibrium computation is
formulated as optimization problems in a setting with T=
2,jNj= 2, andK= 1. At the ﬁrst time step, both players
maximize their overall utility. At the second time step, they
maximize their myopic utility. Speciﬁcally, their optimiza-
tion problems are formed as follows.For the attacker:
At last time step T= 2, given an observation m2, an opti-
mal behavioral strategy fy(ijh1;m2)gof the attacker of
typeis formulated as the following linear optimization
problem:
max
fy(ijh1;m2)gX
iy(ijh1;m2)[m2(i)
P(i)R(i)
+R(i)]
s.t.X
iy(ijh1;m2) = 1
y(ijh1;m2)0;8i2N:
Since at the last time step the attacker always plays best
response, its behavioral strategy only depends on the de-
fender’s strategy m2he observed.
At time step t= 1, given an observation m1, an optimal
behavioral strategy fy(ijm1)gof the attacker of type is
formulated as the following linear optimization problem:
max
fy(ijm1)gX
iy(ijm1)[m1(i)
P(i)R(i)
+R(i)] +X
iy(ijm1)v
(f(i;m1))
s.t.X
iy(ijm1) = 1
y(ijm1)0;8i2N:
wherev
(m2)is the unscaled future optimal utility of the
attacker type given the observation m2.fis a function
that chooses strategy for the defender given the observa-
tioni.
For the defender:
At last time step T= 2, given observation h1, an optimal
mixed strategy m2is formulated as the following linear
optimization problem:
max
m2X
p(jh1)EUd;
m2;fy(ijh1;m2)g
s.t.X
im2(i) = 1
m2(i)0;8i2N
where
EUd;
m2;fy(ijh1;m2)g
=X
iy(ijh1;m2)
m2(i)
Rd(i)Pd(i)
+Pd(i)
At time step t= 1, an optimal mixed strategy m1is for-
mulated as the following linear optimization problem:
max
m1X
pEUd;
m1;fy(ijm1)g
+X
ivd
(m1;i)
s.t.X
im1(i) = 1
m1(i)0;8i2N
wherevd
(m1;i)P
0p(0jm1;i)is the optimal expected utility of the
defender given observation (m1;i)and
EUd;
m1;fy(ijm1)g
=X
iy(ijm1)
m1(i)
Rd(i)Pd(i)
+Pd(i)
If there exists a Bayesian equilibrium, the strategies in the
equilibrium should satisfy those optimization problems
simultaneously.
Proof of Theorem 3
Before proving Theorem 3, we present two preliminary re-
sults.
Lemma 1. In a ﬁnitely repeated Stackelberg security game
withjNj= 2,jj= 2, andK= 1, given that both types
of attackers play pure behavioral strategies, the set of the
possible defender’s equilibrium strategies at time step 2 is
ﬁnite and strategies in this set are given by speciﬁc formulas.
At the second time step, both types of attackers always
play pure strategies. We denote Ui
as the typeattacker’s
utility by attacking target iat the second time step, given
the defender’s strategy. We denote by M() = (M();1
M())the defender’s strategy that makes type attacker in-
different over the two targets at the second time step where
M()is the coverage probability on target 1. We call de-
fender’s strategy M()the ”indifferent strategy” for the type
attacker. We denote mmax (mmin) as defender’s strategy
that makes the type of attackers who have larger (smaller)
M()indifferent over the two targets. Denote mt(i)as the
coverage probability of defender’s strategy on target iat time
stept.
Speciﬁcally, we have
U1
1=m2(1)
P1(1)R1(1)
+R1(1)
U2
1=m2(2)
P1(2)R1(2)
+R1(2)
U1
2=m2(1)
P2(1)R2(1)
+R2(1)
U2
2=m2(2)
P2(2)R2(2)
+R2(2)
M() =R(1)P(2)
R(1)P(2)+R(2)P(1)
mmax= (max
M();1max
M())
mmin= (min
M();1min
M())Proof. First, we list all possible combinations of the attack-
ers’ best responses and the corresponding utility relationship
without equality. We will discuss the equality scenarios later.
Cases are indexed and these indices are consistent in the fol-
lowing tables.
Table 1: Attackers’ best response
idxy1(1jm2)y1(2jm2)y2(1jm2)y2(2jm2)
1 0 1 0 1
2 1 0 1 0
3 1 0 0 1
4 0 1 1 0
Table 2: Utility Relationship
idx Type 1 Type 2
1U1
1U2
1<0U1
2U2
2<0
2U1
1U2
1>0U1
2U2
2>0
3U1
1U2
1>0U1
2U2
2<0
4U1
1U2
1<0U1
2U2
2>0
We obtain the defender’s strategy space(Table 3) by solv-
ing the inequalities in Table 2. For example, in the case 1,
we have
U1
1U2
1<0
m2(1)
P1(1)R1(1)
+R1(1)
m2(2)
P1(2)R1(2)
R1(2)<0
m2(1)
P1(1)R1(1) +P1(2)R1(2)
+R1(1)P1(2)<0
m2(1)
P1(1)+R1(1)P1(2)+R1(2)
>
R1(1)P1(2)
)m2(1)>R1(1)P1(2)
P1(1)+R1(1)P1(2)+R1(2)=M(1)
Thus, m2(1)> M (1). Similarly, m2(1)> M (2). So we
havem2(1)2(mmax(1);1]. Same reasoning process can
be applied for other cases.
Table 3: Defender’s strategy space
idx Action Spaces Condition
1 m2(1)2(mmax(1);1]
2 m2(1)2[0,mmin(1))
3m2(1)2(mmin( 1 ),mmax( 1 ))M(1)>M(2)
4m2(1)2(mmin( 1 ),mmax( 1 ))M(1 )<M(2)
Notice that for case 3 and case 4, only one of them could
happen, which depends on the relationship between M(1)
andM(2).
Second, we discuss the possible equilibrium strategies of
the defender in each strategy space and show the defender’s
indifferent strategies for two types of attackers are those
equilibrium strategies.
In an equilibrium, any player who deviates from the equi-
librium on its own has nothing to gain. Based on this prop-
erty, we disturb the defender’s strategy and explore the exis-
tence of equilibrium strategies in each strategy space.
At the second time step, the defender’s utility is given byP
p(jh1)EUd;
m2;fy(ijm2)g
as shown in Stackel-
berg model.
Action space 1
X
p(jh1)EUd;
m2;fy(ijm2)g
=p(1jh1)X
iy1(ijm2)[m2(i)(Rd(i)Pd(i))+Pd(i)]
+p(2jh1)X
iy2(ijm2)[m2(j)(Rd(i)Pd(i))+Pd(i)]
=p(1jh1)[m2(2)(Rd(2)Pd(2))+Pd(2)]
+ p(2jh1)[m2(2)(Rd(2)Pd(2))+Pd(2)]
=[p(1jh1)+ p(2jh1)][m2(2)(Rd(2)Pd(2)) +Pd(2)]
=[p(1jh1)+ p(2jh1)][Rd(2)m2(1)(Rd(2)Pd(2))]
The above is decreasing with m2(1). Given the above
holds for m2(1)2(mmax(1);1], we can conclude that
the equilibrium does not lie in m2(1)2(mmax(1);1],
otherwise it is always possible to decrease m2(1)to ob-
tain a better utility. So m2(1)will converge to mmax(1).
Action space 2
By the same reasoning process, the defender’s utility at
the second time step is increasing with m2(1). Given
the above holds for m2(1)2[0;mmin(1)), we can
conclude that the equilibrium does not lie in m2(1)2
[0;mmin(1)), otherwise it is always possible to increase
m2(1)to obtain a better utility. So m2(1)will converge
tommin(1).
Action space 3
X
p(jh1)EUd;
m2;fy(ijm2)g
=p(1jh1)X
iy1(ijm2)[m2(i)(Rd(i)Pd(i))+Pd(i)]
+p(2jh1)X
iy2(ijm2)[m2(i)(Rd(i)Pd(i))+Pd(i)]
=p(1jh1)[m2(1)(Rd(1)Pd(1))+Pd(1)]
+ p(2jh1)[m2(2)(Rd(2)Pd(2))+Pd(2)]
Assume m2= (m2(1);m2(2)) is an equilibrium strat-
egy of the defender in action space 3. First disturb
m2(1) by2[0;mmax(1)m2(1)) and adjust
m2(2)accordingly and we denote the new strategy as
m0
2= (m0
2(1);m0
2(2)), where m0
2(1) = m2(1) +and
m0
2(2) = m2(2). Since m2is an equilibrium strategy,
the defender’s utility obtained by playing m2, denoted asUd(m2), should be no less than that obtained by playing
m0
2, denoted as Ud(m0
2). So we have
Ud(m0
2)Ud(m2)
= p(1jh1)[(m2(1) +)(Rd(1)Pd(1))]
+ p(2jh1)[(m2(2))(Rd(2)Pd(2))]
fp(1jh1)[m2(1)(Rd(1)Pd(1))]
+ p(2jh1)[m2(2)(Rd(2)Pd(2))]g
= p(1jh1)[(Rd(1)Pd(1))]
+ p(2jh1)[(Rd(2)Pd(2))]
0
) p(1jh1) [Rd(1)Pd( 1 ) ]p(2jh1) [Rd(2)Pd( 2 ) ]
Similarly, disturb m2(1)by2[0;m2(1)mmin(1)) in
the opposite direction and adjust m2(2)accordingly and
we denote the new strategy as m0
2= (m0
2(1);m0
2(2)),
where m0
2(1) = m2(1)andm0
2(2) = m2(2)+. Then
we have p(1jh1) [Rd(1)Pd( 1 )]p(2jh1) [Rd(2)Pd( 2 )].
Thus, we have the following equation
p(1jh1) [Rd(1)Pd( 1 )] = p(2jh1) [Rd(2)Pd( 2 )]
Scale both sides by dividing p(1jh1)+ p(2jh1)to replace
the unscaled belief with the true belief. Then we have
p(1jh1) [Rd(1)Pd( 1 )] =p(2jh1) [Rd(2)Pd( 2 )]
Sincep(1jh1) +p(2jh1) = 1 , we have
p(1jh1) =Rd(2)Pd(2)
Rd(1)Pd(1) +Rd(2)Pd(2)
p(2jh1) =Rd(1)Pd(1)
Rd(1)Pd(1) +Rd(2)Pd(2)
which are only related to the reward and penalty of the
defender on different targets and unrelated to the speciﬁc
form of m2.
If these two equalities hold, any such disturbance on m2
would not decrease the defender’s utility. Therefore, any
strategy in this strategy space leads to the same utility
for the defender so any strategy is a potential equilib-
rium strategy. In addition, it can be easily proved that
bothmminandmmax results in the same utility as other
strategies in this space. Intuitively, we can always disturb
one strategy and make the new strategy converge to either
mminormmax without change of utility. So we assume
in this case the defender chooses either mminormmax
as his strategy.
If these two equalities do not hold together, it means either
we have p(1jh1) [Rd(1)Pd( 1 )]>p(2jh1) [Rd(2)Pd( 2 )]
or have p(1jh1) [Rd(1)Pd( 1 )]<p(2jh1) [Rd(2)Pd( 2 )].
So we can always disturb one strategy in certain direction
to get a better utility for the defender. Thus, the strategy
will converge to either mminormmax whichever gives
higher utility.
Action space 4
Similar to action space 3, if p(1jh1) =
Rd(1)Pd(1)
Rd(1)Pd(1)+Rd(2)Pd(2)andp(2jh1) =
Rd(2)Pd(2)
Rd(1)Pd(1)+Rd(2)Pd(2), any strategy in this strat-
egy space leads to the same utility for the defender.
Otherwise, the strategy will converge to either mminor
mmax.
In summary, for two targets and two types of attackers, given
that both types of attackers play pure behavioral strategies,
either mminormmax leads to the highest utility for the de-
fender at the second time step. In fact, for any one shot game,
by replacing the posterior belief with the prior belief, we ob-
tain the same result.
Lemma 2. At the second time step, when the defender com-
mits to either mmin ormmax, one of the two types of at-
tackers will be indifferent among the two targets.
Proof. Without loss of generality, we assume mmax =
(M(1);1M(1)) andmmin= (M(2);1M(2)). If the
the defender commits to mmax, then we have
U1
1=M(1)
P1(1)R1(1)
+R1(1)
=R1(1)R1(2)P1(1)P1(2)
R1(1)P1(2) +R1(2)P1(1)
U2
1= [1M(1)]
P1(2)R1(2)
+R1(2)
=R1(1)R1(2)P1(1)P1(2)
R1(1)P1(2) +R1(2)P1(1)
So we haveU1
1=U2
1, which means type 1 attacker is indif-
ferent over both targets. Similarly, if the the defender com-
mits to mmin, we haveU1
2=U2
2.
We name the attacker who has no preference over both
targets “indifferent attacker”.
Proof of Theorem 3. We claim that at both time steps, the
defender choosing either mmin ormmax whichever gives
higher myopic utility (let the higher utility be given by m)
and the two types of attackers playing best responses in ev-
ery time step is a PBNE. Notice that since the defender does
not know the true type of the attacker, he has to analyze both
types of attackers’ behavior. According to Lemma 2, there
exists an indifferent attacker at the both time steps on the
equilibrium path (since all types of adversary play best re-
sponse to mmin ormmax). At the ﬁrst time step, the in-
different attacker would behave exactly the same as another
type of attackers. This imitation helps to hide its true type
and results in no update of the defender’s belief at the sec-
ond time step, which beneﬁts itself the most since the un-
certainty of his type for the defender remains unchanged. At
the second time step, the indifferent attacker could choose
whatever target to attack because the game ends after its at-
tack. At probability zero histories (deviation from mby the
defender in ﬁrst time step or deviation by adversary in ﬁrsttime step), we ﬁx the defender’s belief at the second time
step to be the prior.
Now we prove the optimality of this equilibrium. We want
to show that no one individually wants to deviate from its
equilibrium strategy at any time step.
At the ﬁrst time step, the defender needs to consider his
myopic utility and the future expected utility simultane-
ously. Since there is no belief update for any outcome af-
ter ﬁrst time step, the future expected utility of the defender
is a constant and is unrelated to the defender’s strategy at
the ﬁrst time step. So the defender at the ﬁrst time step only
needs to choose the strategy that gives the highest myopic
utility. Since mis his myopic best response, he would not
deviate from this strategy.
For the two types of attackers at the ﬁrst time step on
the equilibrium path, if the true attacker is the indifferent
attacker, any action would be its best response and it would
just mimic the other attacker type’s behavior to hide his type.
So it would not deviate to preserve the largest amount of un-
certainty of his type. If the true attacker is not the indifferent
attacker, then whichever target it attacks would not update
the defender’s belief due to the imitation by the indifferent
attacker. So it will just play the myopic best response and
would not deviate from that.
For the two types of attackers at the ﬁrst time step off the
equilibrium path (zero probability history), the defender’s
belief is unaffected by its actions, hence it will play the my-
opic best response.
At the second time step, since the defender’s belief will
not be updated, the reasoning process is exactly the same
as that for the ﬁrst time step. The only difference is at the
second time step the indifferent attacker does not need to
imitate and he can attack any target. To sum up, no one wants
to deviate from its equilibrium strategy alone for two-time-
step game.
Now we extend two-time-step scenario to multiple time
steps. For multiple time steps, at the last time step T, the
case is same as the case of the second time step in two-time-
step setting. From time t= 1 toT1, the case is same as
the case of the ﬁrst time step in two-time-step setting since
the indifferent attacker exists at each time step and would
always imitate to hide his type. The set of strategies in this
equilibrium for any subgame is still an equilibrium in that
the strategies of the defender and the attacker at each time
step fromt= 1toT1are the same. Since belief is updated
based on Bayes rule and players are sequentially rational, the
equilibrium we claim is a PBNE.
Therefore, we conclude that for two targets and two types
of attackers, there exists a PBNE in which both types of at-
tackers would always play their myopic best responses.

Thanh H
Nguyen
Yongzhao Wang
Arunesh Sinha
Michael P
Wellman