---
id: HIST-009
title: "Computer Game Playing: Chess to Go"
alternative_names:
  - "Game-Playing AI"
  - "From Deep Blue to AlphaGo"
category: "History → Milestones in Game Playing"
volume: II
chapter: 8, 12
entry_type: "Historical Record"
level: 3
status: "Draft"
last_reviewed: 2026-10-15
version: "0.1.0"
prerequisites:
  - "HIST-003"
  - "TECH-RL-001"
related_entries:
  - "HIST-002"
  - "HIST-005"
  - "TECH-RL-001"
  - "FOUND-012"
---

# Computer Game Playing: Chess to Go

## 1. Concise Definition
**Computer game playing** is the study and construction of programs that play board and card games, treated in AI as a source of well-defined problems with measurable outcomes. Its history runs from the first proposals for chess programs in 1950 through the defeat of a reigning chess world champion in 1997 to programs that learn Go, chess and shogi through self-play.

## 2. Overview & Significance
Games provide closed environments with explicit rules, clear win conditions and human reference opponents, so progress can be measured directly. Early AI researchers regarded chess as a test of machine intelligence; later milestones, such as AlphaGo, were used to demonstrate new learning methods. Game results are easy to communicate, but they are also narrow: a program that is superhuman at a game does not thereby possess general intelligence, a point stressed in the articles on general-purpose systems ([[AGI-001] Artificial General Intelligence](../vol05_agi/AGI-001_artificial_general_intelligence.md)).

The history shows two successive technical regimes. In the first, strength came from hand-designed evaluation functions and brute-force search with specialised hardware. In the second, strength came from learned evaluation and policy functions, trained with reinforcement learning, combined with search. The second regime connects to [[TECH-RL-001] Reinforcement Learning](../vol03_technical/TECH-RL-001_reinforcement_learning.md) and to the deep-learning developments in [[HIST-005] The Deep Learning Breakthrough (2012)](./HIST-005_deep_learning_breakthrough.md).

## 3. Historical Development

### 3.1 Foundations (1950-1960s)
Shannon (1950) described how a computer could be programmed to play chess: represent the position, define an evaluation function that scores positions, and search a game tree of possible moves using the minimax principle. He distinguished searching all moves to a fixed depth from selective search, and discussed the limits imposed by the size of the game tree. Samuel (1959) reported a checkers program that improved its play through machine learning, using stored-position learning ("rote learning") and adjustment of evaluation parameters through self-play. It is a frequently cited early demonstration of a program that learned to play better than the person who wrote it. [VERIFICATION NEEDED: the exact strength claims for Samuel's program were not checked against the paper's text and are not asserted here.]

### 3.2 Chess programs and specialised hardware
Through the 1970s to the 1990s, chess programs improved with faster hardware, deeper search and refined evaluation functions. Campbell, Hoane, and Hsu (2002) describe the IBM Deep Blue system, which combined a parallel architecture with custom chess chips and a search guided by an expert-designed evaluation function. According to IBM's historical account, Deep Blue lost a six-game match to world champion Garry Kasparov 4-2 in Philadelphia in February 1996, winning one game, and won a rematch in New York in May 1997 by 3.5-2.5 (IBM, n.d.). IBM's account describes the 1997 result as a win over the reigning world champion in a six-game match; this article does not assert additional priority claims. The match remains discussed: Kasparov disputed aspects of it, and the sources cited here describe the system and the result but do not adjudicate those disputes. Deep Blue was a special-purpose system and did not learn from games.

### 3.3 Solving games
Schaeffer et al. (2007) reported that checkers is solved: with perfect play by both sides the game is a draw. The team had worked on the problem since 1989, and the paper gives the size of the game as about $5 \times 10^{20}$ positions. Solving a game replaces heuristics with a proof of the game-theoretic value. Chess and Go have not been solved.

### 3.4 Learning from self-play: backgammon
Tesauro (1995) described TD-Gammon, a neural network trained by temporal-difference learning from self-play to evaluate backgammon positions. It showed that a learned evaluation function, without human-written strategy knowledge, could reach high levels of play and influenced later work in reinforcement learning. [VERIFICATION NEEDED: the strength level reported for TD-Gammon was not checked against the source text.]

### 3.5 Monte Carlo tree search and Go
Go resisted the methods that worked for chess because its branching factor is larger and positions are hard to evaluate. Browne et al. (2012) survey Monte Carlo tree search (MCTS), a family of algorithms that grow a search tree guided by random simulations of games to the end; MCTS programs raised the strength of computer Go substantially in the years before deep learning was added.

### 3.6 AlphaGo and successors
Silver et al. (2016) introduced AlphaGo, which combined "policy networks" to select moves and "value networks" to evaluate positions with MCTS. The networks were trained by supervised learning from human expert games and then by reinforcement learning from self-play. The paper reports that AlphaGo won 99.8% of its games against other Go programs and defeated the European champion (a professional player) 5 games to 0, the first time a program had defeated a human professional in full-sized Go. In March 2016 AlphaGo played Lee Sedol in Seoul and won 4-1, according to DeepMind (DeepMind, n.d.); that match is not reported in the 2016 paper. Silver et al. (2017) then reported AlphaGo Zero, trained by reinforcement learning from self-play alone with no human game data. Silver et al. (2018) generalised the method as AlphaZero, which, given only the rules and starting from random play, "convincingly defeated a world champion program" in chess, shogi and Go. Schrittwieser et al. (2020) described MuZero, which plans with a learned model and reached the performance of AlphaZero on Go, chess and shogi without being given the game rules, and achieved state-of-the-art results on 57 Atari games.

### 3.7 Imperfect information
Poker presents hidden information, which breaks perfect-information search. Brown and Sandholm (2018) reported Libratus, which defeated four top professionals over 120,000 hands of heads-up no-limit Texas hold'em; Brown and Sandholm (2019) reported Pluribus, shown to be stronger than top human professionals in six-player no-limit hold'em.

## 4. Technical / Conceptual Explanation

### 4.1 Minimax and evaluation
For a two-player zero-sum game, the value of a position $s$ to the player to move, searched to depth $d$ with evaluation function $e$, is

$$V(s,d) = \begin{cases} e(s) & d=0 \text{ or } s \text{ terminal},\\ \max_{a} \; -V(\operatorname{next}(s,a),\,d-1) & \text{otherwise (negamax form)}.\end{cases}$$

Shannon (1950) set out this scheme. Practical programs add alpha-beta pruning, which skips branches that cannot affect the result, to search deeper with the same effort. [VERIFICATION NEEDED: alpha-beta pruning is standard textbook material but is not specifically sourced in this article's reference list.]

### 4.2 Temporal-difference learning
In TD learning, a value estimate $V_\theta(s_t)$ is updated towards the next estimate: $\theta \leftarrow \theta + \alpha\,[r_{t+1} + V_\theta(s_{t+1}) - V_\theta(s_t)]\,\nabla_\theta V_\theta(s_t)$. Self-play generates training data without human labels (Tesauro, 1995).

### 4.3 Monte Carlo tree search with learned networks
MCTS repeats four steps: select a path through the tree, expand a node, evaluate it (by a rollout or a value network), and back up the result. In AlphaGo the policy network biases which moves are explored and the value network evaluates leaves (Silver et al., 2016; Browne et al., 2012). In AlphaGo Zero and AlphaZero a single network supplies both and is trained to match the search's improved move choices and the game outcomes (Silver et al., 2017; Silver et al., 2018).

## 5. Types, Components & Classifications
| Category | Examples | Key method |
| :--- | :--- | :--- |
| Perfect-information, search-based | Deep Blue | Special-purpose hardware, minimax-style search, hand-designed evaluation (Campbell et al., 2002) |
| Solved games | Checkers | Exhaustive proof search (Schaeffer et al., 2007) |
| Learning from self-play | TD-Gammon, AlphaZero | TD learning; reinforcement learning with search (Tesauro, 1995; Silver et al., 2018) |
| Learned-model planning | MuZero | Search with a learned dynamics model (Schrittwieser et al., 2020) |
| Imperfect-information | Libratus, Pluribus | Game-theoretic equilibrium approximation (Brown & Sandholm, 2018, 2019) |

## 6. Applications & Concrete Examples
Game-playing methods were developed for games, and this article does not assert that the specific systems were deployed elsewhere. The techniques (search, reinforcement learning, self-play, learned value functions) have influenced other research areas; for example, reinforcement learning from human feedback and agent research draw on this lineage, as discussed in [[TECH-RL-001] Reinforcement Learning](../vol03_technical/TECH-RL-001_reinforcement_learning.md).

## 7. Advantages, Limitations & Unresolved Questions
**Strengths of games as benchmarks.** Clear objectives, measurable results and the possibility of unlimited self-play data.

**Limitations.**
- Narrowness: each system is specialised to one game or family (AlphaZero and MuZero extend the approach to several games but not to open-ended tasks; Silver et al., 2018; Schrittwieser et al., 2020).
- Sample cost: the self-play systems cited used very large amounts of computation; the papers report this and exact figures are in the primary sources and not repeated here.
- Brute-force chess programs offered little insight into human cognition; whether game success tells us about general intelligence is disputed.
- The interpretation of milestones: "defeated a world champion" should be read with the match conditions in mind. Silver et al. (2016) defeated a professional European champion, not the world champion; the Lee Sedol match is the one against a top world-class player and is sourced only to DeepMind's own page here, a developer claim.

**Open questions.** Games with many players, open-ended rules, natural language negotiation or very long horizons remain harder; the sources here do not cover them in depth.

## 8. Current State of Knowledge
By the mid-2020s, learned game-playing systems are widely regarded as superhuman at chess, shogi and Go, and strong poker programs have beaten professionals (Silver et al., 2018; Brown & Sandholm, 2018, 2019). This article does not report current rating figures or engine rankings, which change and are better held in the Living Reference ([[FACT-001] Factbook](../reference/factbook/FACT-001_factbook_historical_dates_and_milestones.md)). See [[HIST-010] The Transformer and Foundation Model Era, 2017-2022](./HIST-010_the_transformer_and_foundation_model_era_2017_2022.md) for the shift in research attention from games to language models.

## 9. Future Trajectory & Strategic Outlook
No forecasts are registered in this article. Observed trend: the self-play and search ideas developed for games have reappeared in work on reasoning models that use additional compute at inference time; see [[TECH-REASON-001] Reasoning Models and Test-Time Compute](../vol03_technical/TECH-REASON-001_reasoning_models_and_test_time_compute.md). For registered forecasts consult [[FCST-REG-001] Forecast Register](../reference/factbook/FCST-REG-001_forecast_register.md).

## 10. Related Entries & Knowledge Graph
- **Foundational Concepts**:
  - [[HIST-003] Early Symbolic AI & The Perceptron](./HIST-003_symbolic_ai_perceptron.md): search and early game programs.
  - [[FOUND-012] Game Theory and Decision Theory](../vol01_foundations/FOUND-012_game_theory_and_decision_theory.md): minimax and equilibrium concepts.
- **Related Methods & Technologies**:
  - [[TECH-RL-001] Reinforcement Learning](../vol03_technical/TECH-RL-001_reinforcement_learning.md): TD learning and policy optimisation.
  - [[TECH-REASON-001] Reasoning Models and Test-Time Compute](../vol03_technical/TECH-REASON-001_reasoning_models_and_test_time_compute.md): later search-and-learning ideas.
- **Advanced Implications & Future Trajectories**:
  - [[HIST-005] The Deep Learning Breakthrough (2012)](./HIST-005_deep_learning_breakthrough.md): the deep learning context of AlphaGo.
  - [[AGI-001] Artificial General Intelligence](../vol05_agi/AGI-001_artificial_general_intelligence.md): the contrast between narrow and general ability.

## 11. References & Foundational Literature
1. Shannon, C. E. (1950). Programming a computer for playing chess. *Philosophical Magazine*, 41(314), 256-275. https://doi.org/10.1080/14786445008521796. [Proposes evaluation functions and minimax search for chess.]
2. Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. *IBM Journal of Research and Development*, 3(3), 210-229. https://doi.org/10.1147/rd.33.0210. [Early machine-learning checkers program.]
3. Campbell, M., Hoane, A. J., Jr., & Hsu, F.-h. (2002). Deep Blue. *Artificial Intelligence*, 134(1-2), 57-83. https://doi.org/10.1016/S0004-3702(01)00129-1. [Technical description of the Deep Blue system.]
4. IBM. (n.d.). *Deep Blue* (IBM history). https://www.ibm.com/history/deep-blue. [Developer account of the 1996 and 1997 matches against Kasparov; accessed 2026-10-15.]
5. Schaeffer, J., Burch, N., Björnsson, Y., Kishimoto, A., Müller, M., Lake, R., Lu, P., & Sutphen, S. (2007). Checkers is solved. *Science*, 317(5844), 1518-1522. https://doi.org/10.1126/science.1144079. [Shows that perfect play in checkers is a draw.]
6. Tesauro, G. (1995). Temporal difference learning and TD-Gammon. *Communications of the ACM*, 38(3), 58-68. https://doi.org/10.1145/203330.203343. [Self-play temporal-difference learning for backgammon.]
7. Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012). A survey of Monte Carlo tree search methods. *IEEE Transactions on Computational Intelligence and AI in Games*, 4(1), 1-43. https://doi.org/10.1109/TCIAIG.2012.2186810. [Survey of MCTS.]
8. Silver, D., Huang, A., Maddison, C. J., et al. (2016). Mastering the game of Go with deep neural networks and tree search. *Nature*, 529(7587), 484-489. https://doi.org/10.1038/nature16961. [Introduces AlphaGo; reports 99.8% win rate against programs and 5-0 against the European champion.]
9. DeepMind. (n.d.). *AlphaGo* (research breakthroughs page). https://deepmind.google/research/breakthroughs/alphago/. [Developer account of the 2016 Lee Sedol match (4-1); accessed 2026-10-15.]
10. Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017). Mastering the game of Go without human knowledge. *Nature*, 550(7676), 354-359. https://doi.org/10.1038/nature24270. [AlphaGo Zero: reinforcement learning from self-play without human data.]
11. Silver, D., Hubert, T., Schrittwieser, J., et al. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. *Science*, 362(6419), 1140-1144. https://doi.org/10.1126/science.aar6404. [AlphaZero.]
12. Schrittwieser, J., Antonoglou, I., Hubert, T., et al. (2020). Mastering Atari, Go, chess and shogi by planning with a learned model. *Nature*, 588(7839), 604-609. https://doi.org/10.1038/s41586-020-03051-4. arXiv:1911.08265. [MuZero.]
13. Brown, N., & Sandholm, T. (2018). Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. *Science*, 359(6374), 418-424. https://doi.org/10.1126/science.aao1733. [Libratus.]
14. Brown, N., & Sandholm, T. (2019). Superhuman AI for multiplayer poker. *Science*, 365(6456), 885-890. https://doi.org/10.1126/science.aay2400. [Pluribus.]

## 12. Evidential Classification Matrix
| Section / Claim | Evidence Category | Supporting Source | Confidence / Status |
| :--- | :--- | :--- | :--- |
| Minimax-with-evaluation framework for chess | Established | Shannon (1950) | High |
| Deep Blue defeated Kasparov 3.5-2.5 in May 1997 | Well Supported (developer account plus technical paper) | IBM (n.d.); Campbell et al. (2002) | High for the score; match circumstances were disputed |
| Checkers is a draw under perfect play | Established | Schaeffer et al. (2007) | High |
| AlphaGo defeated a European champion 5-0 and won 99.8% against programs | Well Supported (peer-reviewed, developer-reported) | Silver et al. (2016) | High |
| AlphaGo defeated Lee Sedol 4-1 (March 2016) | Well Supported (developer-reported; widely documented) | DeepMind (n.d.) | Moderate to high |
| AlphaZero defeated a world-champion program in chess, shogi and Go | Well Supported (peer-reviewed, developer-reported) | Silver et al. (2018) | High |
| MuZero matches AlphaZero without given rules | Well Supported (developer-reported) | Schrittwieser et al. (2020) | High |
| Poker AIs beat top professionals | Well Supported | Brown & Sandholm (2018, 2019) | High |
| Game success implies progress toward general intelligence | Contested | Silver et al. (2018) (for the narrow claim only) | Low |

## 13. Revision History
- **v0.1.0** (2026-10-15): Initial draft. Citations checked against arXiv/Crossref/publisher pages where marked; not independently fact-checked.