Richard Sutton

Richard S. Sutton is a computer scientist, professor of computing science at the University of Alberta, who with Andrew Barto developed the conceptual and algorithmic foundations of reinforcement learning, including temporal-difference learning, and co-authored the standard textbook on the subject. They shared the 2024 ACM A.M. Turing Award.

Reinforcement learning (RL) is the study of agents that learn to act by interacting with an environment and receiving rewards (see [TECH-RL-001]). Sutton's work supplied core algorithms and the textbook framing used in the field, and underlies systems such as game-playing agents and the preference-based fine-tuning of language models. His 2019 essay "The Bitter Lesson" is also widely cited in debates about scaling.

Sutton's birth year is reported inconsistently (1957 or 1958) in the secondary source, which gives his birthplace as Toledo, Ohio; this entry does not assert either year. He received a BA in psychology from Stanford in 1978 and an MS (1980) and PhD (1984) in computer science from the University of Massachusetts Amherst, where Andrew Barto advised him (Wikipedia contributors, 2026k). He worked at GTE Laboratories (1985–1994) and AT&T Labs (1998–2002), and has been a professor of computing science at the University of Alberta since 2003 and a chief scientific advisor at Amii, the Alberta Machine Intelligence Institute (Wikipedia contributors, 2026k). From 2017 to 2023 he was a distinguished research scientist at DeepMind (Wikipedia contributors, 2026k). The secondary source further reports that he joined John Carmack's Keen Technologies after DeepMind (a partnership announced in 2023) and that in 2026 he left, with former student Khurram Javed, to launch Oak Lab, a start-up building agents that learn continually through reinforcement learning. Trade-press reports found by search (July 2026) agree on the founders, the Keen background and the continual-learning focus; they also describe a non-profit, the Openmind Research Institute, run alongside it. These later-career details rest on the secondary source and trade press, not on a company, University of Alberta or Amii page, and his current role is not asserted here.

Temporal-difference learning. Sutton (1988) analysed methods that learn to predict by updating an estimate toward a later estimate rather than waiting for the final outcome. In the simplest tabular form, the value estimate is updated as

Integrated planning and learning. Sutton's Dyna architecture (1991) combined learning from real experience with planning from a learned model (Wikipedia contributors, 2026k).

Policy gradients and options. Papers listed in the secondary source (not checked against publisher pages) include the 1999 work on temporally abstract actions with Precup and Singh and the 2000 policy-gradient paper with McAllester, Singh and Mansour (Wikipedia contributors, 2026k).

Textbook. Sutton and Barto's Reinforcement Learning: An Introduction (first edition 1998; second edition 2018) is the standard text (Sutton & Barto, 2018).

The Bitter Lesson. In an essay dated 13 March 2019, Sutton argued that over 70 years of AI research the most effective approaches were general methods that leverage computation, and that building in human knowledge about how we think has tended not to work in the long run (Sutton, 2019). This is an interpretation of history, not an empirical theorem (Contested; see Section 7).

Barto, his doctoral supervisor, is his principal collaborator; the secondary source dates the start of their joint work to the early 1980s, with one news citation placing it in the late 1970s (Wikipedia contributors, 2026k). Sutton belongs to the RL tradition that draws on psychology (animal learning) and control theory. In 2025 he co-authored a position paper with David Silver, "Welcome to the Era of Experience", arguing for agents that learn from their own experience (Wikipedia contributors, 2026k). In a podcast interview of 25 September 2025 he argued that large language models cannot learn on the job and that new architectures are needed (as summarised by Wikipedia contributors, 2026k, which marks that source as non-primary) (neither the position paper nor the interview was read from a primary source).

The ACM A.M. Turing Award for 2024, announced in March 2025, was announced on 5 March 2025 and shared by Barto and Sutton, with the citation "for developing the conceptual and algorithmic foundations of reinforcement learning" (Wikipedia contributors, 2026k). Other reported honours include AAAI Fellow (2001), Fellow of the Royal Society of Canada (2016), Fellow of the Royal Society (2021), the IJCAI Research Excellence Award (2021) and a Canada CIFAR AI Chair (Wikipedia contributors, 2026k). The ACM page could not be opened, so these honours and the citation wording rest on that secondary source.

The Bitter Lesson is contested: critics argue that algorithmic and architectural innovations have contributed substantially alongside compute, and that human-designed structure remains important in data-scarce domains (Contested; see [ECON-COMP-001] for compute economics). RL itself faces well-known challenges of sample efficiency, reward specification and stability; these are open problems and are discussed in [TECH-RL-001].

Temporal-difference methods and policy-gradient methods are standard components of RL systems, and the 2018 textbook edition remains a standard reference. Claims about the future primacy of experience-driven agents are Preliminary and are positions, not established results.

No forecasts are made in this biography. Current developments in RL are recorded in the Living References (see [MODEL-001]).

• Foundational Concepts:
  • TECH-RL-001 Reinforcement Learning: His field.
• Related Methods & Technologies:
  • ECON-COMP-001 Compute Economics: Context for the Bitter Lesson.
  • AGI-PATH-001 Pathways to AGI: Scaling and learning-based paths.
• Advanced Implications & Future Trajectories:
  • BIO-HINTON-001 Geoffrey Hinton: Another Turing laureate in learning research.