--- id: AGI-CL-001 title: "Continual Learning and Persistent Memory" alternative_names: - "Lifelong Learning" - "Continual Learning" - "Catastrophic Forgetting and Memory in AI Systems" category: "Advanced Intelligence → General Intelligence" volume: V chapter: 27 entry_type: "Standard Article" level: 3 status: "Draft" last_reviewed: 2026-10-15 version: "0.1.0" prerequisites: - "AGI-PATH-001" - "TECH-ML-001" related_entries: - "AGI-REQ-001" - "AGI-WM-001" - "TECH-AGENT-001" - "TECH-RAG-001" --- # Continual Learning and Persistent Memory ## 1. Concise Definition **Continual learning** (also lifelong learning) is the capacity of a learning system to acquire new knowledge and skills sequentially over time while retaining what it has already learned. **Persistent memory** refers to mechanisms, inside or outside a model's parameters, that store information across interactions. Together they address a gap in standard practice, in which most deployed models are trained once and then used with fixed parameters. ## 2. Overview & Significance A person who works in a job for a year learns the organisation, the colleagues and the tasks, and carries that knowledge forward. A standard large language model deployment does not update its weights from its interactions; any apparent memory comes from text supplied in the context window, from retrieval over external stores, or from periodic retraining by the developer. Continual learning research asks how a system could learn on the job without destroying earlier competence. The topic matters for general intelligence for two reasons. First, long-horizon work, such as managing a project over weeks, seems to require accumulating information. Second, [AGI-REQ-001](../vol05_agi/AGI-REQ-001_requirements_for_general_intelligence.md) lists memory as a candidate requirement, and Hendrycks et al. (2025) report in their abstract that current AI systems have critical deficits in foundational cognitive machinery, particularly long-term memory storage. Whether this deficit is a fundamental barrier or an engineering gap is Contested. ## 3. Historical Development The core difficulty, **catastrophic forgetting**, is a long-recognised problem in connectionist research: when a network is trained on a new task, learning can overwrite what it learned before. Cognitive neuroscience offers a framework in which the brain uses complementary learning systems, with the hippocampus learning quickly and the neocortex consolidating knowledge slowly, as a solution to this problem (McClelland et al., 1995). Kirkpatrick et al. (2017) introduced elastic weight consolidation, which slows learning on parameters important to earlier tasks, and demonstrated sequential learning on supervised and reinforcement-learning tasks. Reviews organised the field into families of method (Parisi et al., 2019; De Lange et al., 2021; Wang et al., 2023). With the rise of language models, interest shifted to memory outside the weights: retrieval-augmented generation (Lewis et al., 2020), operating-system-inspired memory management for long contexts (Packer et al., 2023), and memory streams in simulated agents (Park et al., 2023). A separate line studies whether training itself degrades: Dohare et al. (2024) report loss of plasticity in standard deep-learning methods in continual settings. Titans (Behrouz et al., 2024) proposes a neural memory module that learns at test time. ## 4. Technical / Conceptual Explanation ### 4.1 The stability-plasticity trade-off Consider a sequence of tasks $1,\dots,T$ with losses $L_t(\theta)$. After training on task $t$, parameters $\theta_t$ are obtained by minimising $L_t$ starting from $\theta_{t-1}$. Without constraints, $L_s(\theta_t)$ for $s