---
id: FOUND-012
title: "Game Theory and Decision Theory"
alternative_names:
  - "Rational Choice Theory"
  - "Expected Utility Theory and Strategic Interaction"
category: "Foundations → Mathematics"
volume: I
chapter: 3
entry_type: "Standard Article"
level: 3
status: "Draft"
last_reviewed: 2026-10-15
version: "0.1.0"
prerequisites:
  - "FOUND-008"
related_entries:
  - "FOUND-002"
  - "FOUND-006"
  - "TECH-RL-001"
  - "TECH-MARL-001"
  - "SAFE-ALIGN-001"
---

# Game Theory and Decision Theory

## 1. Concise Definition
**Decision theory** studies how an agent should choose among actions whose outcomes are uncertain, typically by maximizing expected utility. **Game theory** extends the analysis to settings with several decision-makers whose outcomes depend on one another's choices. Together they define the standard formal model of a rational agent used in AI, and the basis for analysing cooperation, competition and strategic behaviour among AI systems.

## 2. Overview & Significance
The textbook definition of an AI system as a rational agent that acts to maximize expected performance (Russell & Norvig, 2020) is a decision-theoretic definition. Reinforcement learning, planning under uncertainty and preference-based training all assume an objective (utility or reward) that the agent optimizes ([[TECH-RL-001] Reinforcement Learning](../vol03_technical/TECH-RL-001_reinforcement_learning.md)). Probability supplies the uncertainty model ([[FOUND-008] Probability, Statistics and Bayesian Inference](FOUND-008_probability_statistics_and_bayesian_inference.md)); utility supplies the preferences.

Game theory matters wherever agents interact: adversarial settings such as poker and negotiation, adversarial training objectives, multi-agent systems, and the strategic behaviour of AI developers and states. It also frames safety questions. Specifying an objective for an agent is a decision-theoretic design problem ([[SAFE-ALIGN-001] AI Alignment](../vol08_safety_governance/SAFE-ALIGN-001_ai_alignment.md)), and some formulations of alignment cast it as a game between a human and an AI (Hadfield-Menell et al., 2016).

A limit of the framework deserves stating at the outset. Expected-utility theory is a normative model of rational choice and a convenient design abstraction; observed human behaviour deviates from it in systematic ways, and real AI systems seldom have an explicit utility function.

## 3. Historical Development
Modern game theory is generally dated to von Neumann and Morgenstern's *Theory of Games and Economic Behavior* (1944), which developed the theory of zero-sum games and gave an axiomatic derivation of expected-utility maximization from preferences over lotteries (von Neumann & Morgenstern, 1944/2007). Nash (1950) proved that finite games have equilibrium points in mixed strategies, extending the analysis to non-zero-sum settings.

The empirical and descriptive limits of expected utility were identified early. Ellsberg (1961) described choices under ambiguity that violate the Savage axioms of subjective expected utility. Kahneman and Tversky (1979) proposed prospect theory to describe observed choices under risk.

Evolutionary and cooperative themes emerged with Axelrod and Hamilton (1981), who analysed the iterated Prisoner's Dilemma, including the results of a computer tournament, and showed how cooperation based on reciprocity can arise and persist under their model's assumptions.

Computational game theory began to matter for AI when complexity was understood. Daskalakis, Goldberg and Papadimitriou (2009) showed that computing a Nash equilibrium is complete for the complexity class PPAD, which gives evidence that there is no efficient general algorithm. In AI practice, algorithms for imperfect-information games reached professional-level poker (Moravčík et al., 2017; Brown & Sandholm, 2018), then six-player poker (Brown & Sandholm, 2019), and an agent combining language models with planning achieved strong results in the negotiation game Diplomacy (Meta Fundamental AI Research Diplomacy Team (FAIR) et al., 2022). Dafoe et al. (2020) set out a research agenda for cooperative AI.

## 4. Technical / Conceptual Explanation

### 4.1 Expected utility
Given actions $a$, states of the world $s$ with probabilities $p(s)$, and a utility function $u(a, s)$ over outcomes, the expected utility of an action is
$$EU(a) = \sum_s p(s)\,u(a, s),$$
and a rational agent chooses $a^* = \arg\max_a EU(a)$. Von Neumann and Morgenstern (1944/2007) showed that if a preference relation over lotteries satisfies certain axioms (completeness, transitivity, continuity and independence), there is a utility function, unique up to positive affine transformation, whose expectation represents the preferences. The axioms are normative assumptions; they can be and are questioned (Ellsberg, 1961; Kahneman & Tversky, 1979).

### 4.2 Sequential decisions
In a Markov decision process, an agent seeks a policy $\pi$ maximizing expected discounted return
$$J(\pi) = \mathbb{E}_\pi\left[\sum_{t=0}^{\infty}\gamma^t r_t\right], \qquad 0 \le \gamma < 1.$$
This is the objective of reinforcement learning ([[TECH-RL-001] Reinforcement Learning](../vol03_technical/TECH-RL-001_reinforcement_learning.md)).

### 4.3 Normal-form games and equilibrium
A game specifies players, their strategy sets and payoff functions. A strategy profile is a **Nash equilibrium** if no player can improve their payoff by unilaterally changing strategy. Nash (1950) proved that every finite game has at least one equilibrium in mixed (randomized) strategies. A strategy is **dominant** if it is best regardless of others' choices.

The Prisoner's Dilemma illustrates a conflict between individual and collective rationality: mutual defection is the unique equilibrium of the one-shot game although mutual cooperation yields higher payoffs to both. In the repeated game, cooperative outcomes can be sustained, as studied by Axelrod and Hamilton (1981).

### 4.4 Zero-sum games and minimax
In a two-player zero-sum game with payoff matrix $A$, von Neumann's minimax theorem states that
$$\max_{x}\min_{y}\, x^\top A y = \min_{y}\max_{x}\, x^\top A y$$
over mixed strategies $x, y$ (see von Neumann & Morgenstern, 1944/2007). This value can be computed by linear programming. The generative adversarial network objective is a two-player minimax game between a generator and a discriminator (Goodfellow et al., 2014), although training dynamics in practice need not converge to an equilibrium.

### 4.5 Imperfect information and complexity
In games with hidden information, such as poker, players must reason about information sets, and the game tree is very large. Algorithms combining abstraction, equilibrium computation by iterative self-play, and real-time subgame solving were used in DeepStack (Moravčík et al., 2017) and Libratus (Brown & Sandholm, 2018). Daskalakis et al. (2009) showed that finding a Nash equilibrium is PPAD-complete, with the SICOMP abstract stating the result for three-player games; hardness of exact computation motivates the use of approximate or special-case methods. [VERIFICATION NEEDED: citation for the extension of PPAD-completeness to two-player games, not cited here]

### 4.6 Cooperative inverse reinforcement learning
Hadfield-Menell et al. (2016) formalize value alignment as a cooperative game in which a human knows the reward function and a robot does not; the robot must infer it from the human's behaviour while both are rewarded by the human's reward. In that model, optimal behaviour includes the human teaching and the robot seeking clarification, in contrast to a robot that optimizes a fixed specified reward.

## 5. Types, Components & Classifications
- **Decision problems**: under certainty, under risk (known probabilities), under uncertainty or ambiguity (unknown probabilities; Ellsberg, 1961).
- **Normative versus descriptive theories**: expected utility (normative) versus prospect theory (descriptive; Kahneman & Tversky, 1979).
- **Games by structure**: simultaneous versus sequential; zero-sum versus general-sum; perfect versus imperfect information; cooperative versus non-cooperative; one-shot versus repeated; two-player versus multi-player.
- **Solution concepts**: dominant strategy, Nash equilibrium, minimax value, and refinements (not covered in the sources cited here).
- **Evolutionary game theory**: strategies compete by relative success (Axelrod & Hamilton, 1981).
- **Mechanism design**: designing the rules of a game to induce desired behaviour (not covered by a source here).

## 6. Applications & Concrete Examples
- **Game-playing AI.** DeepStack and Libratus defeated professional players in heads-up no-limit Texas hold'em (Moravčík et al., 2017; Brown & Sandholm, 2018). Pluribus was reported to perform better than elite professionals in six-player no-limit hold'em over 10,000 hands (Brown & Sandholm, 2019). Cicero achieved more than double the average score of human players in 40 games of an anonymous online Diplomacy league and ranked in the top 10% of participants who played more than one game (Meta FAIR Diplomacy Team et al., 2022). These are results in specified games under specified conditions and do not demonstrate general strategic competence.
- **Adversarial machine learning.** The GAN objective (Goodfellow et al., 2014).
- **Preference learning and alignment.** Cooperative inverse reinforcement learning is a research formalization (Hadfield-Menell et al., 2016), not a deployed method.
- **Multi-agent AI and governance.** Cooperation problems among AI agents and among AI developers are framed in game-theoretic terms (Dafoe et al., 2020); competitive dynamics among developers is analysed in [[GOV-LAW-001] AI Governance](../vol08_safety_governance/GOV-LAW-001_ai_governance.md) and [[ECON-COMP-001] Compute Economics](../vol09_economy/ECON-COMP-001_compute_economics.md).

## 7. Advantages, Limitations & Unresolved Questions
**Strengths.** Precise, mathematically developed models of choice and interaction; clear solution concepts; applicable to learning and planning algorithms.

**Limitations.**
- *Descriptive failure.* People systematically depart from expected-utility axioms (Ellsberg, 1961; Kahneman & Tversky, 1979).
- *Computational intractability.* Equilibrium computation is PPAD-hard (Daskalakis et al., 2009), and large games need approximation.
- *Equilibrium selection.* Games often have several equilibria, and theory does not always indicate which will be played.
- *Utility specification.* Utility or reward functions must be specified; mis-specified objectives lead to unintended behaviour. Neither the von Neumann-Morgenstern theorem nor the formalism supplies the content of the utility function.
- *Idealization of agents.* Real learning agents are not guaranteed to be expected-utility maximizers, and whether advanced AI systems will behave as such is an open question, not a result.

**Open questions.** How to model learning agents in games (convergence and equilibrium selection under learning); how to design agents that cooperate with humans and other agents in mixed-motive settings (Dafoe et al., 2020); and how to represent uncertain or evolving preferences (Hadfield-Menell et al., 2016).

## 8. Current State of Knowledge
As of 2026-10-15:
- The expected-utility representation theorem, the existence of Nash equilibria in finite games, and the PPAD-completeness of equilibrium computation are **Established** results.
- The claim that human decision-making deviates from expected utility is **Well Supported** empirically (Kahneman & Tversky, 1979; Ellsberg, 1961).
- The game-playing results cited are **Well Supported** as published demonstrations in their respective games; their scope is limited to those games.
- Whether advanced AI agents can be usefully modelled as expected-utility maximizers is **Contested** and bears on arguments in the safety literature; it is not asserted here.
- Cooperative AI as a research agenda (Dafoe et al., 2020) and the cooperative-game formulation of alignment (Hadfield-Menell et al., 2016) are **Preliminary**.

## 9. Future Trajectory & Strategic Outlook
No registered forecast is made. Observed directions: multi-agent learning in mixed-motive games; evaluation of cooperation among language-model agents; and incorporation of preference uncertainty into agent design. Near-term work is likely to emphasize empirical evaluation in simulated games. Statements about long-term strategic dynamics among advanced AI systems and states are speculative scenarios and are treated in [[SAFE-XRISK-001] Existential Risk from AI](../vol08_safety_governance/SAFE-XRISK-001_existential_risk.md).

## 10. Related Entries & Knowledge Graph
- **Foundational Concepts**:
  - [[FOUND-008] Probability, Statistics and Bayesian Inference](FOUND-008_probability_statistics_and_bayesian_inference.md): probability in expected utility.
  - [[FOUND-002] Definitions of Intelligence](FOUND-002_definitions_of_intelligence.md): rationality as a definition.
- **Related Methods & Technologies**:
  - [[TECH-RL-001] Reinforcement Learning](../vol03_technical/TECH-RL-001_reinforcement_learning.md): sequential decision-making.
  - [[TECH-MARL-001] Multi-Agent Systems](../vol03_technical/TECH-MARL-001_multi_agent_systems.md): interacting learning agents.
  - [[FOUND-006] Collective and Animal Intelligence](FOUND-006_collective_and_animal_intelligence.md): cooperation among agents in biology.
  - [[FOUND-010] Optimization and Gradient Descent](FOUND-010_optimization_and_gradient_descent.md): optimizing objectives, including minimax objectives.
- **Advanced Implications & Future Trajectories**:
  - [[SAFE-ALIGN-001] AI Alignment](../vol08_safety_governance/SAFE-ALIGN-001_ai_alignment.md): objective specification.
  - [[GOV-LAW-001] AI Governance](../vol08_safety_governance/GOV-LAW-001_ai_governance.md): strategic interaction among developers and states.

## 11. References & Foundational Literature
1. von Neumann, J., & Morgenstern, O. (1944/2007). *Theory of Games and Economic Behavior* (60th anniversary ed.). Princeton University Press. ISBN 978-0691130613. [Founding text of game theory and the expected-utility representation. Edition and ISBN checked against Open Library; theorem statements are standard.]
2. Nash, J. F. (1950). Equilibrium points in n-person games. *Proceedings of the National Academy of Sciences*, 36(1), 48-49. https://doi.org/10.1073/pnas.36.1.48. [Existence of equilibrium in finite games. Checked against Crossref.]
3. Ellsberg, D. (1961). Risk, ambiguity, and the Savage axioms. *The Quarterly Journal of Economics*, 75(4), 643-669. https://doi.org/10.2307/1884324. [Choices under ambiguity that violate subjective expected utility axioms. Checked against Crossref (start page listed; end page from author knowledge).]
4. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. *Econometrica*, 47(2), 263-291. https://doi.org/10.2307/1914185. [Descriptive theory of risky choice. Checked against Crossref (start page listed; end page from author knowledge).]
5. Axelrod, R., & Hamilton, W. D. (1981). The evolution of cooperation. *Science*, 211(4489), 1390-1396. https://doi.org/10.1126/science.7466396. [Iterated Prisoner's Dilemma and the emergence of reciprocal cooperation. Checked against Crossref and Europe PMC abstract.]
6. Daskalakis, C., Goldberg, P. W., & Papadimitriou, C. H. (2009). The complexity of computing a Nash equilibrium. *SIAM Journal on Computing*, 39(1), 195-259. https://doi.org/10.1137/070699652. [PPAD-completeness of Nash equilibrium computation (abstract states three-player games). Checked against Crossref abstract.]
7. Moravčík, M., Schmid, M., Burch, N., Lisý, V., Morrill, D., Bard, N., Davis, T., Waugh, K., Johanson, M., & Bowling, M. (2017). DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. *Science*, 356(6337), 508-513. https://doi.org/10.1126/science.aam6960. [Poker AI defeating professionals in a 44,000-hand study. Authors and abstract checked against Crossref and Europe PMC; volume and pages from author knowledge.]
8. Brown, N., & Sandholm, T. (2018). Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. *Science*, 359(6374), 418-424. https://doi.org/10.1126/science.aao1733. [Libratus defeating four professionals over 120,000 hands. Checked against Crossref and Europe PMC.]
9. Brown, N., & Sandholm, T. (2019). Superhuman AI for multiplayer poker. *Science*, 365(6456), 885-890. https://doi.org/10.1126/science.aay2400. [Pluribus in six-player no-limit hold'em. Checked against Crossref abstract.]
10. Meta Fundamental AI Research Diplomacy Team (FAIR), et al. (2022). Human-level play in the game of Diplomacy by combining language models with strategic reasoning. *Science*, 378(6624), 1067-1074. https://doi.org/10.1126/science.ade9097. [Cicero's results in an online Diplomacy league. Checked against Crossref abstract (27 listed authors including the team).]
11. Goodfellow, I. J., et al. (2014). Generative adversarial nets. *Advances in Neural Information Processing Systems (NeurIPS 2014)*. arXiv:1406.2661. [GAN as a two-player minimax game. Eight authors confirmed on arXiv; venue from author knowledge.]
12. Hadfield-Menell, D., Dragan, A., Abbeel, P., & Russell, S. (2016). Cooperative inverse reinforcement learning. *Advances in Neural Information Processing Systems (NeurIPS 2016)*. arXiv:1606.03137. [Alignment as a cooperative game. Four authors confirmed on arXiv; venue from author knowledge.]
13. Dafoe, A., et al. (2020). Open problems in cooperative AI. arXiv:2012.08630 (preprint). [Research agenda for cooperative AI. Eight authors confirmed on arXiv.]
14. Russell, S., & Norvig, P. (2020). *Artificial Intelligence: A Modern Approach* (4th ed.). Pearson. ISBN 978-0134610993. [Rational-agent definition. Checked against Open Library.]

## 12. Evidential Classification Matrix
| Section / Claim | Evidence Category | Supporting Source | Confidence / Status |
| :--- | :--- | :--- | :--- |
| Expected-utility representation theorem under the axioms | Established | von Neumann & Morgenstern (1944/2007) | High |
| Existence of mixed-strategy Nash equilibrium in finite games | Established | Nash (1950) | High |
| Computing Nash equilibrium is PPAD-complete | Established | Daskalakis et al. (2009) | High |
| Human choices deviate from expected utility | Well Supported | Ellsberg (1961); Kahneman & Tversky (1979) | High |
| Reciprocity can sustain cooperation in iterated Prisoner's Dilemma | Established (model result) | Axelrod & Hamilton (1981) | High within model assumptions |
| Poker and Diplomacy AI results | Well Supported (per game) | Moravčík et al. (2017); Brown & Sandholm (2018, 2019); Meta FAIR Diplomacy Team et al. (2022) | Moderate to High |
| Alignment as a cooperative game | Preliminary | Hadfield-Menell et al. (2016) | Moderate |
| Advanced AI systems behave as expected-utility maximizers | Contested | None cited | Low |

## 13. Revision History
- **v0.1.0** (2026-10-15): Initial draft. Citations checked against arXiv/Crossref/publisher pages where marked; not independently fact-checked.