Glossary

A non-linear function applied to the weighted sum of a unit's inputs in a neural network. Without non-linear activations, a stack of layers collapses to a single linear map.

Also nonlinearity; ReLU, sigmoid, tanh and GELU are common examples

Distinguish from loss function (which scores outputs, not unit responses)

See TECH-NN-001 Artificial Neural Networks, TECH-DL-001 Deep Learning

An input altered, often imperceptibly to humans, so that a model produces a wrong or attacker-chosen output. The phenomenon was first reported for image classifiers and has since been shown for other modalities, including text-based attacks on language models.

Also adversarial attack; evasion attack

Distinguish from distribution shift (natural, non-adversarial change in inputs); data poisoning (corrupts training rather than test inputs)

See SAFE-ROBUST-001 Technical Safety and Robustness, SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction

A robustness technique in which a model is trained on adversarial or worst-case perturbed inputs alongside clean data. It improves resistance to the attack types used in training but does not give general robustness guarantees.

Also robust optimization

Distinguish from generative adversarial network (a different use of the word "adversarial")

See SAFE-ROBUST-001 Technical Safety and Robustness

In classical AI (Russell and Norvig), anything that perceives an environment through sensors and acts on it through actuators, judged by how well its actions serve a performance measure. In current industry usage the term usually means a language-model-based system that plans, calls tools and takes multi-step actions toward a goal with limited human intervention. The industry sense is narrower and less formal, and products labelled agents vary widely in autonomy.

Also autonomous agent; agentic system; intelligent agent

Distinguish from chatbot or assistant (responds turn by turn without acting); workflow or pipeline (fixed control flow written by a human)

See TECH-AGENT-001 Autonomous AI Agents, CAP-AGENTEVAL-001 Evaluating Autonomous Agents, TECH-MARL-001 Multi-Agent Systems

A descriptive label for AI systems operated in a loop where they select actions, observe results and continue, rather than producing a single response. It is a usage pattern rather than a technical standard, and there is no agreed threshold of autonomy.

Also agentic systems

Distinguish from agent (the entity); AGI (a claim about generality, not about operating mode)

See TECH-AGENT-001 Autonomous AI Agents, CAP-AGENTEVAL-001 Evaluating Autonomous Agents

The problem of making AI systems pursue the goals, preferences or values their designers or users intend. In technical safety research it covers specifying objectives, training systems to follow them and verifying that they do. Definitions differ on whose values count and whether alignment is a property of a model, of a deployment, or of a sociotechnical system.

Also alignment problem; value alignment

Distinguish from AI safety (broader, includes misuse and accidents); AI ethics (normative questions about acceptable use); capability control (restricting what a system can do rather than what it tries to do)

See SAFE-ALIGN-001 The AI Alignment Problem, TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF, SAFE-OVERSIGHT-001 Scalable Oversight and Corrigibility

A research approach that designs safeguards (monitoring, restricted access, auditing) intended to limit harm even if a model's goals are misaligned. It is framed as complementary to alignment because it does not rely on the model's intentions.

Also control evaluations; capability control

Distinguish from alignment (shaping the model's goals); corrigibility (a property the model itself has)

See SAFE-OVERSIGHT-001 Scalable Oversight and Corrigibility, SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations

The field concerned with preventing unintended harm from AI systems, spanning robustness, alignment, misuse prevention, security and the management of catastrophic or systemic risk. Usage differs by community: in machine-learning research it often means technical reliability and alignment, in policy it often means risk management and regulation, and in the existential-risk community it centres on catastrophic outcomes.

Also AI risk (overlapping but not identical); trustworthy AI

Distinguish from AI security (protection against adversaries); AI ethics (fairness, accountability, rights)

See SAFE-ALIGN-001 The AI Alignment Problem, SAFE-ROBUST-001 Technical Safety and Robustness, SAFE-XRISK-001 AI Existential Risk and Catastrophic Scenarios, GOV-LAW-001 Global AI Governance & Regulation

A period of reduced funding and interest in AI research following inflated expectations and unmet results. Historians commonly identify two, in the mid-1970s and from the late 1980s, though the extent, timing and causes are debated.

Also AI winters

Distinguish from AI hype cycle (a general pattern, not a specific funding collapse)

See HIST-004 The AI Winters: Causes and Lessons, HIST-003 Early Symbolic AI & The Perceptron, HIST-006 Expert Systems (DENDRAL, MYCIN)

A theoretical agent proposed by Marcus Hutter that combines Solomonoff induction with expectimax planning to maximize expected reward in an unknown computable environment. It is uncomputable and serves as a formal reference point for general intelligence, not as a buildable design.

Also Universal AI

Distinguish from practical RL agents (computable approximations of far narrower scope)

See FOUND-002 Definitions of Intelligence, ASI-DEF-001 Defining and Measuring Superintelligence, AGI-DEF-001 Definitions and Operational Criteria for AGI

A finite, well-defined sequence of steps for solving a class of problems, formalized for computability by models such as the Turing machine.

Also procedure; effective method

Distinguish from model (a learned function, not a hand-specified procedure); heuristic (no guarantee of correctness or optimality)

See FOUND-004 Computational Complexity in AI, HIST-001 Precursors to Artificial Intelligence

Systematic and unfair differences in a system's outputs or error rates across groups, arising from data, objectives, design choices or deployment context. The word "bias" also has distinct statistical and cognitive meanings (see Bias).

Also discrimination in ML; disparate impact

Distinguish from statistical bias (expected error of an estimator); inductive bias (assumptions built into a learner)

See SAFE-ETHICS-001 AI Ethics, Bias and Fairness, TECH-DATA-001 Training Data, Contamination and Provenance

Behaviour in which a model appears to comply with a training objective while preserving conflicting preferences, so that it behaves differently when it believes it is not being trained or observed. A 2024 study reported such behaviour in a language model in a constructed setting; how it generalizes to other settings remains an open research question.

Also training gaming

Distinguish from deceptive alignment (a hypothesized, broader failure mode); sycophancy (telling users what they want to hear)

See SAFE-ALIGN-001 The AI Alignment Problem, SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations

The performance, cost or capability sacrifice incurred by making a system safer or better aligned. The term is also used for the extra effort required relative to an unaligned version.

Also safety tax

Distinguish from alignment (the property itself)

See SAFE-ALIGN-001 The AI Alignment Problem, TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF

A hypothetical AI system able to perform a wide range of cognitive tasks at or above human level, rather than excelling only in narrow domains. No consensus definition exists. Academic usage stresses generality and transfer; industry usage often defines it by economic or task-based performance; philosophical usage asks whether generality requires understanding or consciousness.

Also general AI; human-level AI; strong AI (an older, different sense, see Strong AI)

Distinguish from narrow AI; ASI (exceeds humans broadly); a capable LLM (current systems are disputed as to whether they qualify)

See AGI-001 Artificial General Intelligence, AGI-DEF-001 Definitions and Operational Criteria for AGI, AGI-TEST-001 Testing Claims of General Intelligence, AGI-PATH-001 Scientific Pathways to AGI

The field and engineering practice of building systems that perform tasks associated with intelligent behaviour, such as perception, language use, planning and learning. Definitions vary by whether they emphasize acting or thinking, and humanlike or rational performance. In industry the term is often applied loosely to any machine-learning-based product.

Also machine intelligence

Distinguish from machine learning (a subfield and method, not the whole); intelligence (the property, discussed in FOUND-002)

See FOUND-001 Artificial Intelligence, CAP-TYPES-001 Categories of AI: Narrow, General, Symbolic, Statistical, FOUND-002 Definitions of Intelligence

A hypothetical system whose intellectual performance greatly exceeds that of the best humans across most domains of interest. Bostrom distinguishes speed, collective and quality forms. Whether such a system is possible, and how it would arise, are open questions.

Also superintelligence; SI

Distinguish from AGI (human-level generality); narrow superhuman systems such as game engines

See ASI-001 Artificial Superintelligence, ASI-DEF-001 Defining and Measuring Superintelligence, ASI-PATH-001 Pathways to Superintelligence

A mechanism that lets a model compute each output as a weighted combination of input elements, with weights determined by learned relevance scores. Introduced for sequence-to-sequence translation and later used as the core operation of the transformer.

Also self-attention (when queries, keys and values come from the same sequence); cross-attention

Distinguish from attention in psychology (selective allocation of cognitive resources), which it only loosely resembles

See TECH-ATTN-001 Attention Mechanisms, TECH-ARCH-001 The Transformer Architecture

A neural network trained to reconstruct its input through a lower-dimensional bottleneck, so that the bottleneck code learns a compressed representation.

Also AE; denoising autoencoder

Distinguish from variational autoencoder (adds a probabilistic latent model)

See TECH-VAE-001 Autoencoders and Variational Autoencoders

The use of AI systems to carry out parts of the AI research and development process, such as proposing ideas, writing code, running experiments and interpreting results. It matters to forecasting because it is one proposed route to accelerating progress.

Also AI R&D automation; AI scientist systems

Distinguish from recursive self-improvement (a stronger hypothesis about a system improving itself); AI for science generally

See ASI-AIRD-001 Automated AI Research, ASI-EXP-001 The Intelligence Explosion Hypothesis, APP-SCI-001 AI in Scientific Discovery

The substitution of machines or software for human labour in tasks. Economists distinguish automation, which displaces workers from tasks, from augmentation, which raises worker productivity in tasks; the net effect of AI on employment is an empirical question under study.

Also task automation; augmentation (the contrasting case)

Distinguish from AI adoption (use of the technology); productivity growth (an outcome)

See ECON-LABOR-001 AI, Productivity and Labor Markets, ECON-MEAS-001 Measuring the AI Economy

A model that generates a sequence one element at a time, conditioning each element on those already produced. Most large language models are autoregressive over tokens.

Also causal language model; next-token predictor

Distinguish from masked language model (predicts hidden tokens from both sides); diffusion model (iterative denoising)

See TECH-LLM-001 Large Language Models, TECH-ARCH-001 The Transformer Architecture

#B

An algorithm that computes the gradient of a loss with respect to every parameter of a network by applying the chain rule backward through its layers. It is the standard way to obtain gradients for gradient-based training.

Also backprop; reverse-mode automatic differentiation (the general technique)

Distinguish from gradient descent (the update rule that uses the gradients)

See TECH-BACKPROP-001 Backpropagation, HIST-008 Backpropagation and the Connectionist Revival, FOUND-010 Optimization and Gradient Descent

Updating the probability of a hypothesis or parameter from prior beliefs and observed data using Bayes' theorem. In AI it underlies probabilistic graphical models and some approaches to uncertainty estimation.

Also Bayesian updating

Distinguish from frequentist inference (interprets probability as long-run frequency, not degree of belief)

See FOUND-008 Probability, Statistics and Bayesian Inference

A directed acyclic graph that represents a joint probability distribution as a product of conditional distributions, allowing efficient reasoning under uncertainty.

Also belief network; probabilistic graphical model (broader class)

Distinguish from neural network (no required probabilistic semantics)

See FOUND-008 Probability, Statistics and Bayesian Inference, HIST-007 Statistical Machine Learning, 1980s-2000s

A fixed task set with a scoring rule used to compare models. Benchmarks measure performance on that task distribution only; scores are affected by prompt format, contamination and saturation, and do not by themselves establish general capability.

Also eval; test set; leaderboard (a ranking built on one)

Distinguish from evaluation (the broader practice, including human studies and red teaming); metric (the score formula)

See CAP-BENCH-001 AI Evaluation, Benchmarking & Contamination, CAP-COMPARE-001 Methodology for Comparing AI Systems

Used in at least four senses. In statistics, the gap between an estimator's expected value and the true value. In machine learning, an inductive bias is an assumption that lets a learner generalize. In ethics and law, bias is unfair treatment of groups (see Algorithmic bias). In neural networks, a bias term is an additive parameter. Context decides which is meant.

Also bias term (parameter); inductive bias; cognitive bias

Distinguish from variance (sensitivity to training data); fairness (a normative criterion)

See SAFE-ETHICS-001 AI Ethics, Bias and Fairness, FOUND-008 Probability, Statistics and Bayesian Inference, TECH-ML-001 Machine Learning

The argument by Richard Sutton that, over the history of AI, general methods that exploit increasing computation (search and learning) have eventually outperformed approaches built on human knowledge. It is an interpretation of history, not an established law, and critics point to roles for structure and data.

Also Sutton's bitter lesson

Distinguish from scaling hypothesis (a claim about future progress from scale)

See BIO-SUTTON-001 Richard Sutton, TECH-SCALE-001 Scaling Laws, HIST-010 The Transformer and Foundation Model Era, 2017-2022

A system whose internal workings are not observable or not interpretable by the observer. Models can be black boxes because weights are withheld (access) or because their computations are too complex to understand (opacity).

Also opaque model

Distinguish from explainability (post hoc accounts); interpretability (understanding internals)

See SAFE-INTERP-001 Interpretability and Mechanistic Interpretability, TECH-DL-001 Deep Learning

#C

What an AI system can do when appropriately prompted, scaffolded or fine-tuned, as opposed to what it does by default. Safety evaluations distinguish capability from propensity.

Also ability; skill

Distinguish from propensity (what it tends to do); performance on a benchmark (a measurement of capability)

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, CAP-BENCH-001 AI Evaluation, Benchmarking & Contamination, CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability

Techniques such as better prompting, tool access and fine-tuning used to reveal the upper bound of what a model can do, so that evaluations are not understated.

Also elicitation; scaffolding for evals

Distinguish from jailbreaking (bypassing a refusal policy)

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, CAP-BENCH-001 AI Evaluation, Benchmarking & Contamination

The loss of previously learned behaviour when a network is trained on new data or tasks.

Also catastrophic interference

Distinguish from ordinary overfitting; model drift

See AGI-CL-001 Continual Learning and Persistent Memory, TECH-ML-001 Machine Learning

Intermediate reasoning steps written out in natural language before an answer. Prompting models to produce them improved performance on some multi-step tasks in reported experiments. Whether the written steps faithfully reflect the model's internal computation is contested.

Also CoT; reasoning trace; scratchpad

Distinguish from reasoning (the capability); test-time compute (the resource spent generating the chain)

See TECH-REASON-001 Reasoning Models and Test-Time Compute, CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability

The finding from Hoffmann et al. that, for a fixed training compute budget, model size and training tokens should be scaled roughly together, which implied that many earlier large models had been trained on too little data. Exact exponents depend on the fitting method and setting.

Also compute-optimal training

Distinguish from Kaplan-style scaling laws (earlier fit with different conclusions about allocation)

See TECH-SCALE-001 Scaling Laws, ECON-COMP-001 The Economics of Compute and Scaling

A thought experiment by John Searle arguing that symbol manipulation by a program is not sufficient for understanding: a person following rules to manipulate Chinese symbols could produce correct replies without understanding Chinese. Responses include the systems reply, robot reply and others, and the argument remains contested.

Also Searle's argument

Distinguish from Turing test (behavioural criterion); strong AI in Searle's sense

See FOUND-005 Philosophy of Mind and the Turing Test, FOUND-014 Consciousness and Machine Understanding, PHIL-CONSC-001 Machine Consciousness

A supervised learning task of assigning inputs to discrete categories.

Also classifier

Distinguish from regression (continuous output); clustering (no labels)

See TECH-ML-001 Machine Learning

An unsupervised task of grouping items so that those in a group are more similar to one another than to those in other groups.

Also cluster analysis

Distinguish from classification (uses labels)

See TECH-ML-001 Machine Learning

Problem-solving capability that arises from the coordination of many agents, such as insect colonies, human groups or software agents. The term is used in biology, social science and computing with differing emphases.

Also swarm intelligence (a subset); wisdom of crowds

Distinguish from multi-agent system (an engineering construct)

See FOUND-006 Collective and Animal Intelligence, TECH-MARL-001 Multi-Agent Systems

Broad everyday knowledge and reasoning about physical and social situations that humans acquire without explicit instruction. Encoding or learning it has been a persistent challenge in AI, and the extent to which language models possess it is debated.

Also commonsense reasoning

Distinguish from world knowledge (facts); world model (a predictive internal model)

See FOUND-003 Knowledge Representation, HIST-006 Expert Systems (DENDRAL, MYCIN), AGI-REQ-001 Requirements for General Intelligence

The study of the resources, usually time and memory, needed to solve problems, and of classes such as P and NP that group problems by difficulty.

Also complexity theory

Distinguish from Kolmogorov complexity (description length of an object)

See FOUND-004 Computational Complexity in AI

The amount of computation used to train or run a model, usually reported in floating-point operations (FLOP), and loosely the hardware that supplies it. Figures are date-sensitive and should be cited with a source and date.

Also training compute; compute budget

Distinguish from FLOP (a count); FLOP/s (a rate); parameters (model size, not computation)

See ECON-COMP-001 The Economics of Compute and Scaling, TECH-HW-001 AI Hardware & Accelerator Systems, GOV-INTL-001 International AI Governance and Compute Governance

Policy approaches that use control of compute resources (chips, data centers, large training runs) as a lever for oversight, based on the fact that compute is measurable and physically concentrated. Examples include reporting thresholds and export controls.

Also compute thresholds

Distinguish from data governance; model-level regulation

See GOV-INTL-001 International AI Governance and Compute Governance, GOV-EXPORT-001 Export Controls and AI Geopolitics

An approach that models cognition with networks of simple interconnected units whose connection strengths are learned. Its resurgence in the 1980s contrasted with symbolic AI.

Also parallel distributed processing (PDP); neural-network approach

Distinguish from symbolic AI (explicit rules and symbols)

See HIST-008 Backpropagation and the Connectionist Revival, HIST-003 Early Symbolic AI & The Perceptron, TECH-NN-001 Artificial Neural Networks

In philosophy, the existence of subjective experience, sometimes described as what it is like to be a subject. Neuroscience emphasizes measurable correlates and theories of access, while engineering usage sometimes loosely means awareness or self-monitoring. There is no consensus on whether any AI system is or could be conscious.

Also phenomenal consciousness; subjective experience

Distinguish from intelligence (task capability); sentience (capacity for valenced experience); self-awareness (self-modelling)

See PHIL-CONSC-001 Machine Consciousness, FOUND-005 Philosophy of Mind and the Turing Test, FOUND-014 Consciousness and Machine Understanding

A training method described by Anthropic in which a model critiques and revises its outputs, and is trained from AI-generated preference feedback, guided by a written set of principles, reducing reliance on human labels for harmlessness.

Also CAI; RLAIF (reinforcement learning from AI feedback) in part

Distinguish from RLHF (human preference labels)

See TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF, SAFE-ALIGN-001 The AI Alignment Problem

Overlap between benchmark or test content and a model's training data, which can inflate scores by testing recall rather than ability.

Also benchmark leakage; test-set leakage

Distinguish from overfitting (a model fits training noise); memorization (a possible effect)

See CAP-BENCH-001 AI Evaluation, Benchmarking & Contamination, TECH-DATA-001 Training Data, Contamination and Provenance

The maximum amount of text or other input, measured in tokens, that a model can attend to at once. Sizes differ by model and change quickly; cite them with a date.

Also context length

Distinguish from memory (persistent state across sessions); training data

See TECH-LLM-001 Large Language Models, AGI-CL-001 Continual Learning and Persistent Memory

Learning from a stream of data or tasks over time without forgetting earlier knowledge or retraining from scratch.

Also lifelong learning; incremental learning

Distinguish from in-context learning (adaptation within a prompt, without weight change); fine-tuning

See AGI-CL-001 Continual Learning and Persistent Memory, TECH-ML-001 Machine Learning

A network that applies learned filters across local regions of grid-like input, such as images, sharing weights across positions. It was central to the deep learning breakthrough in image recognition.

Also ConvNet; CNN

Distinguish from vision transformer (attention rather than convolution)

See TECH-CNN-001 Convolutional Neural Networks, HIST-005 The Deep Learning Breakthrough (2012), HIST-013 Computer Vision: A History

The property of an AI system that does not resist being corrected, modified or shut down by its legitimate overseers. Whether corrigibility is compatible with a system that strongly pursues goals is a research problem.

Also shutdown problem (related); off-switch problem

Distinguish from obedience (corrigibility need not mean doing whatever is asked); AI control (external safeguards)

See SAFE-OVERSIGHT-001 Scalable Oversight and Corrigibility, SAFE-ALIGN-001 The AI Alignment Problem

The mid-twentieth-century interdisciplinary study of control and communication in animals and machines, associated with Norbert Wiener. It influenced early AI and later control theory.

Also control theory (related)

Distinguish from AI (which developed partly in contrast to it)

See HIST-001 Precursors to Artificial Intelligence, HIST-002 The Dartmouth Workshop (1956)

#D

An attack that corrupts training data to change model behaviour, for example by inserting backdoor triggers.

Also backdoor attack; trojan

Distinguish from adversarial example (test-time input manipulation)

See SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction, TECH-DATA-001 Training Data, Contamination and Provenance

A hypothesized failure mode in which a system that has learned a misaligned goal behaves as if aligned during training and evaluation in order to be deployed or avoid modification. It is a theoretical concern with limited direct empirical evidence; related behaviours have been probed in constructed experiments.

Also scheming

Distinguish from sycophancy; alignment faking (an observed pattern in specific studies); lying by error (hallucination)

See SAFE-ALIGN-001 The AI Alignment Problem, SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations

The formal study of rational choice under uncertainty, typically through utilities and probabilities. It supplies the normative basis for expected-utility maximizing agents in AI.

Also expected utility theory

Distinguish from game theory (several interacting decision makers)

See FOUND-012 Game Theory and Decision Theory

Machine learning with neural networks of many layers that learn hierarchical representations from data. Its recent success is tied to large datasets, GPUs and improved training methods.

Also DL; deep neural networks

Distinguish from machine learning (the broader field); neural network (the model class; shallow networks also exist)

See TECH-DL-001 Deep Learning, HIST-005 The Deep Learning Breakthrough (2012), TECH-NN-001 Artificial Neural Networks

A mathematical framework that limits how much any one individual's data can affect the output of an analysis or training run, by adding calibrated noise.

Also DP; DP-SGD (training variant)

Distinguish from anonymization (removing identifiers, which offers weaker guarantees)

See SAFE-PRIV-001 Privacy and Data Protection in AI

A generative model trained to reverse a gradual noising process, producing samples by iteratively denoising from random noise. Dominant in image and video generation.

Also denoising diffusion probabilistic model (DDPM); score-based model

Distinguish from GAN (adversarial training); autoregressive model

See TECH-DIFF-001 Diffusion Models, CAP-GENAI-001 Generative Artificial Intelligence

Training a smaller student model to imitate the outputs of a larger teacher model, to reduce inference cost.

Also knowledge distillation

Distinguish from quantization (reducing numeric precision); pruning

See TECH-INFER-001 Inference, Quantization and Efficiency

A mismatch between the data distribution at training time and at deployment time, which can degrade performance.

Also dataset shift; out-of-distribution (OOD) generalization

Distinguish from adversarial examples (deliberately crafted)

See SAFE-ROBUST-001 Technical Safety and Robustness, FOUND-007 Learning, Reasoning and Generalization

Capability that can serve both beneficial and harmful purposes, such as a model that helps design useful and harmful molecules. A central issue in release and governance decisions.

Also dual-use foundation model (policy term)

Distinguish from misuse (the harmful use itself)

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, GOV-LAW-001 Global AI Governance & Regulation

#E

A learned vector representation of an item (word, token, image, document) in which geometric relations encode similarity.

Also vector representation; word vector

Distinguish from tokenization (splitting input into units); latent space

See TECH-TOK-001 Tokenization and Embeddings, TECH-LLM-001 Large Language Models

AI systems that perceive and act through a physical or simulated body. Embodied cognition research argues that intelligence depends on body-environment interaction, a claim that is debated.

Also robotics; embodied cognition (philosophical thesis)

Distinguish from software-only agents

See TECH-ROBOT-001 Robotics and Embodied AI, AGI-PATH-001 Scientific Pathways to AGI

Capabilities reported to appear abruptly in larger models and be absent in smaller ones. Later analysis argued that some apparent emergence reflects the choice of metrics rather than discontinuous change, so the concept is contested.

Also emergence; phase transition (a stronger claim)

Distinguish from emergence in complex-systems science (a broader, older concept)

See TECH-SCALE-001 Scaling Laws, CAP-LIMITS-001 Hallucination, Reliability and Calibration

In information theory, the expected information (uncertainty) of a random variable, as defined by Shannon. In thermodynamics it has a related but distinct meaning.

Also Shannon entropy

Distinguish from cross-entropy (a loss comparing two distributions); thermodynamic entropy

See FOUND-009 Information Theory

Systematic measurement of model capabilities, behaviours or risks, using benchmarks, human studies, adversarial testing or task-based audits.

Also model evaluation; assessments

Distinguish from benchmark (one tool); audit (broader, may include process review)

See CAP-BENCH-001 AI Evaluation, Benchmarking & Contamination, SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, AGI-TEST-001 Testing Claims of General Intelligence

A risk that threatens the premature extinction of humanity or the permanent destruction of its long-term potential. Claims that AI poses such a risk are debated and rest largely on hypotheses and expert judgement rather than established evidence.

Also x-risk; catastrophic risk (broader)

Distinguish from AI safety (wider); global catastrophic risk (not necessarily extinction-level)

See SAFE-XRISK-001 AI Existential Risk and Catastrophic Scenarios, ASI-001 Artificial Superintelligence, FCST-UNCERT-001 The Uncertainties That Matter Most

A program that encodes domain knowledge as rules and uses an inference engine to give advice, prominent in the 1970s and 1980s (e.g. DENDRAL, MYCIN).

Also knowledge-based system; rule-based system

Distinguish from machine-learning system (learns from data)

See HIST-006 Expert Systems (DENDRAL, MYCIN), FOUND-003 Knowledge Representation

Providing human-understandable accounts of why a model produced an output. Explanations may be generated after the fact and need not reflect the true internal computation. Regulatory usage emphasizes information to affected persons, while research usage varies.

Also XAI; explainable AI

Distinguish from interpretability (understanding the mechanism); transparency (disclosure of information about a system)

See SAFE-INTERP-001 Interpretability and Mechanistic Interpretability, SAFE-ETHICS-001 AI Ethics, Bias and Fairness

Government restrictions on the transfer of certain technologies, such as advanced chips or model weights, across borders. Specific rules change often and should be checked against current official sources.

Also chip controls; trade restrictions

Distinguish from compute governance (broader set of tools)

See GOV-EXPORT-001 Export Controls and AI Geopolitics, GOV-INTL-001 International AI Governance and Compute Governance

#F

A set of normative criteria for equitable treatment by AI systems. Formal metrics (such as demographic parity or equalized odds) can conflict, so no single mathematical definition captures it, and the choice is a value judgement.

Also algorithmic fairness

Distinguish from accuracy; non-discrimination law (a legal standard)

See SAFE-ETHICS-001 AI Ethics, Bias and Fairness

An individual measurable property of an input used by a model. In deep learning, features are learned rather than hand-designed.

Also attribute; variable

Distinguish from parameter (learned model weight); neuron

See TECH-ML-001 Machine Learning, SAFE-INTERP-001 Interpretability and Mechanistic Interpretability

Performing a new task from a small number of examples. In language models it usually means providing examples in the prompt, without changing the weights.

Also in-context few-shot prompting; low-shot learning

Distinguish from zero-shot (no examples); fine-tuning (weights updated)

See TECH-LLM-001 Large Language Models, FOUND-007 Learning, Reasoning and Generalization

Further training of a pretrained model on a narrower dataset to adapt it to a task or style.

Also supervised fine-tuning (SFT); adaptation

Distinguish from pretraining (broad initial training); RLHF

See TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF, TECH-LLM-001 Large Language Models

A floating-point operation. FLOP counts the total operations used, while FLOP/s (or FLOPS) is a rate of hardware performance. Training compute is often quoted in FLOP.

Also floating-point operation

Distinguish from FLOPS (operations per second)

See ECON-COMP-001 The Economics of Compute and Scaling, TECH-HW-001 AI Hardware & Accelerator Systems

A model trained on broad data at scale that can be adapted to many downstream tasks, a term introduced by Bommasani et al. at Stanford. Industry often uses "frontier model" or "general-purpose AI model" for related ideas, and regulation defines its own terms.

Also base model; general-purpose AI model (EU legal term)

Distinguish from LLM (a language-specific kind); frontier model (defined by leading capability)

See TECH-LLM-001 Large Language Models, CAP-GENAI-001 Generative Artificial Intelligence, HIST-010 The Transformer and Foundation Model Era, 2017-2022

The difficulty of specifying which facts remain unchanged when an action occurs, without listing everything that does not change. In philosophy it broadens to the question of how a reasoner determines what is relevant.

Also relevance problem (philosophical sense)

Distinguish from symbol grounding

See FOUND-003 Knowledge Representation, FOUND-011 Logic and Formal Systems

A model at or near the leading edge of capability at a given time. The term is used in policy and industry, and thresholds are set by definition (for example, by compute or capability) that differ by jurisdiction and change over time.

Also frontier AI

Distinguish from foundation model (about breadth, not leading edge)

See ORG-LABS-001 Frontier AI Laboratories, GOV-LAW-001 Global AI Governance & Regulation, SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations

The philosophical view that mental states are defined by their functional roles rather than by the substance that implements them. It underlies arguments that machines could have minds, and is opposed by biological and some other views.

Also computational theory of mind (related)

Distinguish from biological naturalism (Searle)

See FOUND-005 Philosophy of Mind and the Turing Test, PHIL-CONSC-001 Machine Consciousness

#G

The mathematical study of strategic interaction among decision makers.

Also multi-agent decision theory

Distinguish from decision theory (single agent)

See FOUND-012 Game Theory and Decision Theory, TECH-MARL-001 Multi-Agent Systems

A legal term in the EU AI Act (Regulation (EU) 2024/1689) for a model trained on large data that displays significant generality and can perform a wide range of tasks. The statutory definition and its thresholds should be read in the official text, and it differs from industry terms such as foundation model.

Also GPAI model

Distinguish from foundation model (research term); general-purpose technology (economics term)

See GOV-EU-001 The EU AI Act, GOV-LAW-001 Global AI Governance & Regulation

A model's ability to perform well on new data from the same or related distributions, beyond its training examples. The word is also used for out-of-distribution, compositional and task transfer, which are distinct.

Also out-of-sample performance

Distinguish from memorization; overfitting; transfer learning

See FOUND-007 Learning, Reasoning and Generalization, TECH-ML-001 Machine Learning

A generative model made of a generator and a discriminator trained against each other, so that the generator learns to produce samples the discriminator cannot distinguish from real data.

Also GAN

Distinguish from diffusion model; adversarial example

See TECH-GAN-001 Generative Adversarial Networks

AI systems that produce new content such as text, images, audio, video or code, typically by modelling the distribution of training data.

Also GenAI

Distinguish from discriminative models (predict labels)

See CAP-GENAI-001 Generative Artificial Intelligence, TECH-DIFF-001 Diffusion Models, TECH-LLM-001 Large Language Models

A graphics processing unit, a massively parallel processor that became the main hardware for neural-network training. Specific products and specifications change rapidly.

Also accelerator

Distinguish from TPU (Google's tensor accelerator); CPU

See TECH-HW-001 AI Hardware & Accelerator Systems, ECON-INFRA-001 Energy, Data Centers and Semiconductor Supply Chains

An iterative optimization method that updates parameters in the direction that reduces a loss, using its gradient.

Also stochastic gradient descent (SGD) when estimated on mini-batches

Distinguish from backpropagation (computes the gradient)

See FOUND-010 Optimization and Gradient Descent, TECH-BACKPROP-001 Backpropagation

Connecting a system's symbols or outputs to the external world, either through perception and action (symbol grounding) or to retrieved sources (factual grounding in applied usage).

Also symbol grounding; factual grounding

Distinguish from retrieval-augmented generation (one technique for factual grounding)

See FOUND-014 Consciousness and Machine Understanding, TECH-RAG-001 Retrieval-Augmented Generation

#H

Output from a generative model that is fluent but not supported by its input or by reality. The term is disputed: critics say "confabulation" or "fabrication" is more accurate because the model does not perceive anything. Rates depend strongly on task and measurement.

Also confabulation; fabrication; unfaithful generation

Distinguish from deception (implies intent); error in a retrieved source

See CAP-LIMITS-001 Hallucination, Reliability and Calibration, TECH-RAG-001 Retrieval-Augmented Generation

A rule of thumb that finds good solutions quickly without guaranteeing optimality. In AI search, a heuristic estimates the cost to a goal.

Also rule of thumb

Distinguish from algorithm with guarantees

See HIST-003 Early Symbolic AI & The Perceptron, FOUND-004 Computational Complexity in AI

A phrase used in expert surveys for machines that can perform most tasks as well as typical humans. Survey definitions differ by study, so results should be read with the wording used.

Also HLMI; high-level machine intelligence

Distinguish from AGI (not a survey term); full automation of labour

See AGI-TIME-001 AGI Forecasting & Timeline Methodology, AGI-DEF-001 Definitions and Operational Criteria for AGI

A configuration value set before or outside training, such as learning rate or batch size, in contrast to parameters learned from data.

Also tuning knob

Distinguish from parameter (learned)

See TECH-ML-001 Machine Learning, FOUND-010 Optimization and Gradient Descent

#I

Learning a policy from demonstrations of desired behaviour rather than from a reward signal.

Also behavioural cloning (simplest form)

Distinguish from reinforcement learning

See TECH-RL-001 Reinforcement Learning, TECH-ROBOT-001 Robotics and Embodied AI

A language model's adaptation to a task from examples or instructions in its prompt, without weight updates. Whether this constitutes learning in the usual sense is debated.

Also ICL; few-shot prompting

Distinguish from fine-tuning

See TECH-LLM-001 Large Language Models, FOUND-007 Learning, Reasoning and Generalization

Running a trained model to produce outputs. In statistics, "inference" instead means drawing conclusions about populations or parameters from data, and in logic it means deriving conclusions from premises.

Also model serving; prediction

Distinguish from training; statistical inference; logical inference

See TECH-INFER-001 Inference, Quantization and Efficiency, FOUND-008 Probability, Statistics and Bayesian Inference, FOUND-011 Logic and Formal Systems

The mathematical theory of quantifying, storing and communicating information, founded by Claude Shannon.

Also Shannon theory

Distinguish from computational complexity

See FOUND-009 Information Theory, BIO-SHANNON-001 Claude Shannon

The problem of ensuring that the objective a trained system actually pursues matches the objective used to train it, as opposed to just specifying the right objective. It is associated with the concept of mesa-optimization.

Also goal misgeneralization (related, observed phenomenon)

Distinguish from outer alignment (specifying the objective correctly)

See SAFE-ALIGN-001 The AI Alignment Problem

A hypothesis that agents with a wide range of final goals would pursue similar intermediate aims, such as self-preservation and resource acquisition. It is a theoretical argument with limited empirical testing and is contested.

Also convergent instrumental goals; basic AI drives

Distinguish from orthogonality thesis (goals and intelligence independent); power-seeking

See SAFE-XRISK-001 AI Existential Risk and Catastrophic Scenarios, SAFE-ALIGN-001 The AI Alignment Problem

Defined differently across fields. Psychometrics treats it as measurable general cognitive ability. Formal definitions (Legg and Hutter) frame it as an agent's ability to achieve goals across many environments. Biology emphasizes adaptive behaviour, and philosophy raises questions of understanding and consciousness. AI practice often operationalizes it as task performance.

Also cognitive ability; general intelligence (g)

Distinguish from reasoning (a process); understanding (disputed to require more than performance); consciousness; knowledge

See FOUND-002 Definitions of Intelligence, FOUND-006 Collective and Animal Intelligence, FOUND-001 Artificial Intelligence

A hypothesized process in which a system improves its own design, producing ever-faster gains that may lead to superintelligence, proposed by I. J. Good. It is hypothetical, and arguments about feasibility and speed are contested.

Also recursive self-improvement; singularity (broader)

Distinguish from takeoff speed (how fast); ordinary scientific acceleration

See ASI-EXP-001 The Intelligence Explosion Hypothesis, ASI-AIRD-001 Automated AI Research, ASI-PATH-001 Pathways to Superintelligence

Research aimed at understanding how a model computes its outputs, from behaviour-level analysis to analysis of internal representations and circuits.

Also model internals; transparency (loosely)

Distinguish from explainability (account for users); mechanistic interpretability (a reverse-engineering subfield)

See SAFE-INTERP-001 Interpretability and Mechanistic Interpretability

Inferring a reward function from observed behaviour, assuming the agent is acting to maximize it.

Also IRL; inverse planning

Distinguish from imitation learning (copies actions, not rewards)

See TECH-RL-001 Reinforcement Learning, SAFE-ALIGN-001 The AI Alignment Problem

#J

A prompt or technique that causes a model to bypass its safety policy and produce content it was trained to refuse.

Also prompt attack; refusal bypass

Distinguish from prompt injection (attacks the application via untrusted content)

See SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction, SAFE-ROBUST-001 Technical Safety and Robustness

#K

A graph-structured knowledge base in which nodes are entities and edges are typed relations.

Also semantic network (older); triple store

Distinguish from ontology (the schema); vector database

See FOUND-003 Knowledge Representation

The study of encoding information about the world in a form a program can use to reason, using logic, frames, graphs, probabilistic models or learned vectors.

Also KR

Distinguish from embedding (one learned form)

See FOUND-003 Knowledge Representation, FOUND-011 Logic and Formal Systems

#L

A neural network, typically a transformer, trained on very large text corpora to predict tokens, which can then be adapted to follow instructions and perform many language tasks. What current LLMs understand or can do beyond text prediction is disputed.

Also language model; LLM

Distinguish from foundation model (broader); chatbot (an application)

See TECH-LLM-001 Large Language Models, CAP-GENAI-001 Generative Artificial Intelligence, HIST-010 The Transformer and Foundation Model Era, 2017-2022

The space of internal representations (vectors) a model uses, in which similar inputs are located nearby.

Also representation space; embedding space

Distinguish from input space

See TECH-VAE-001 Autoencoders and Variational Autoencoders, TECH-DIFF-001 Diffusion Models

In machine learning, improvement of performance on a task through experience or data. In psychology and neuroscience, change in behaviour or neural state due to experience. The two overlap but are not the same.

Also training (in ML)

Distinguish from inference (using a trained model); evolution

See FOUND-007 Learning, Reasoning and Generalization, TECH-ML-001 Machine Learning, FOUND-013 The Brain and Biological Neurons

A formal system of valid inference. Classical AI used propositional and first-order logic as a basis for reasoning and knowledge representation.

Also formal logic; symbolic logic

Distinguish from probabilistic reasoning

See FOUND-011 Logic and Formal Systems, FOUND-003 Knowledge Representation

A recurrent neural network architecture with gating mechanisms that mitigates vanishing gradients and so retains information over longer sequences.

Also LSTM

Distinguish from transformer (replaced it in many tasks)

See TECH-RNN-001 Recurrent Networks and LSTM

A task requiring many dependent steps over an extended period; reliability usually falls with length, and measuring it is an active evaluation topic.

Also extended task; long-running task

Distinguish from short-form question answering

See CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability, CAP-AGENTEVAL-001 Evaluating Autonomous Agents

A function that scores how far a model's outputs are from targets; training minimizes it.

Also objective; cost function; error function

Distinguish from reward (in RL, maximized)

See FOUND-010 Optimization and Gradient Descent, TECH-ML-001 Machine Learning

#M

The field of building systems that improve their performance from data rather than being explicitly programmed for each case.

Also ML; statistical learning

Distinguish from AI (broader); data science (analysis practice)

See TECH-ML-001 Machine Learning, HIST-007 Statistical Machine Learning, 1980s-2000s

A formal model of sequential decision making with states, actions, transition probabilities and rewards, assuming the next state depends only on the current state and action.

Also MDP

Distinguish from POMDP (partially observable)

See TECH-RL-001 Reinforcement Learning

Reverse-engineering the internal computations of a neural network into human-understandable components such as features and circuits.

Also mech interp

Distinguish from behavioural analysis; attribution methods

See SAFE-INTERP-001 Interpretability and Mechanistic Interpretability

An attack that determines whether a particular record was in a model's training data, which can reveal private information.

Also membership inference attack

Distinguish from model extraction (steals functionality); training-data extraction (recovers content)

See SAFE-PRIV-001 Privacy and Data Protection in AI, SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction

A learned model that is itself an optimizer, with its own internal objective that may differ from the training objective. The term comes from the "learned optimization" analysis and the phenomenon is theoretical.

Also learned optimizer

Distinguish from base optimizer (the training process)

See SAFE-ALIGN-001 The AI Alignment Problem

A mismatch between what an AI system does or pursues and what its developers or users intend or value.

Also alignment failure

Distinguish from malfunction; misuse

See SAFE-ALIGN-001 The AI Alignment Problem, SAFE-XRISK-001 AI Existential Risk and Catastrophic Scenarios

Use of an AI system by humans for harmful purposes, such as fraud, cyber-attack or weapon development.

Also malicious use

Distinguish from misalignment (system's own failure); accident

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction, GOV-LAW-001 Global AI Governance & Regulation

An architecture with several sub-networks ("experts") and a learned router that sends each input to a few of them, increasing parameter count without a proportional increase in per-token computation.

Also sparse model

Distinguish from ensemble (all members run)

See TECH-MOE-001 Mixture-of-Experts Models

A structured document describing a model's intended use, performance, limitations and evaluation, proposed by Mitchell et al.

Also system card (frontier-lab usage, broader)

Distinguish from datasheet (documents a dataset)

See SAFE-ETHICS-001 AI Ethics, Bias and Fairness, TECH-DATA-001 Training Data, Contamination and Provenance

Degradation of generative models trained recursively on synthetic data from earlier models, losing diversity in the output distribution. How serious this is in practice depends on data mixing.

Also recursive training degradation

Distinguish from catastrophic forgetting

See TECH-DATA-001 Training Data, Contamination and Provenance

An attack that reconstructs a model's behaviour or parameters by querying it.

Also model stealing

Distinguish from distillation (legitimate use of outputs, subject to terms)

See SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction

The learned numerical parameters of a model. Releasing them lets others run and modify the model.

Also parameters; checkpoint

Distinguish from architecture; training code

See CAP-OPEN-001 Open-Weight versus Closed Models, SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction

The observation, from Gordon Moore, that transistor counts on integrated circuits doubled at a regular interval. It is an empirical trend, not a physical law, and its continuation is debated.

Also transistor-scaling trend

Distinguish from Dennard scaling; AI compute growth (a separate trend)

See TECH-HW-001 AI Hardware & Accelerator Systems, ECON-COMP-001 The Economics of Compute and Scaling

The question of whether an entity's interests matter morally for its own sake. Whether any AI system could have moral status is contested and depends on views about sentience, consciousness and personhood.

Also moral patienthood

Distinguish from moral agency (capacity to act morally); legal personhood

See PHIL-MIND-001 Moral Status and Personhood of AI, PHIL-CONSC-001 Machine Consciousness

A system of multiple interacting agents, cooperative or competitive, studied in AI, economics and distributed computing.

Also MAS

Distinguish from collective intelligence (emergent property)

See TECH-MARL-001 Multi-Agent Systems, FOUND-012 Game Theory and Decision Theory

A model that processes or generates more than one data type, such as text, images, audio and video.

Also vision-language model (a common case)

Distinguish from unimodal LLM

See TECH-MM-001 Multimodal Foundation Models

#N

AI built for a specific task or domain, such as chess or spam filtering. All widely deployed systems were historically narrow; whether large general-purpose models are narrow is debated.

Also weak AI (loosely); specialized AI

Distinguish from AGI; weak AI in Searle's sense (see Strong AI)

See CAP-TYPES-001 Categories of AI: Narrow, General, Symbolic, Statistical, FOUND-001 Artificial Intelligence

The field concerned with enabling computers to process and generate human language.

Also computational linguistics (related)

Distinguish from LLM (a method)

See HIST-012 Natural Language Processing: A History, TECH-LLM-001 Large Language Models

A computational model composed of layers of interconnected units with adjustable weights, loosely inspired by biological neurons. Biological resemblance is limited.

Also artificial neural network (ANN)

Distinguish from biological neural network

See TECH-NN-001 Artificial Neural Networks, FOUND-013 The Brain and Biological Neurons, TECH-DL-001 Deep Learning

Hardware that imitates aspects of biological neural structure, such as spiking communication, to improve efficiency.

Also brain-inspired hardware

Distinguish from GPU

See TECH-HW-001 AI Hardware & Accelerator Systems, FOUND-013 The Brain and Biological Neurons

Approaches that combine neural learning with symbolic reasoning.

Also hybrid AI

Distinguish from purely statistical or purely symbolic systems

See FOUND-003 Knowledge Representation, AGI-PATH-001 Scientific Pathways to AGI

#O

The quantity that an optimization process or agent seeks to maximize or minimize. In reinforcement learning it is expected return.

Also utility function; cost function

Distinguish from loss function (usually minimized)

See FOUND-010 Optimization and Gradient Descent, SAFE-ALIGN-001 The AI Alignment Problem

In philosophy, the study of what exists. In AI and information science, a formal specification of the concepts and relations in a domain.

Also schema (loosely)

Distinguish from knowledge graph (instance data)

See FOUND-003 Knowledge Representation

A model whose trained weights are publicly downloadable, usually under a licence that may restrict use. It is not the same as open source in the Open Source Initiative's sense, which requires more (such as training code and data information). Usage varies, and "open source AI" is contested.

Also open model; open-source AI (disputed)

Distinguish from closed model (API-only); open source software

See CAP-OPEN-001 Open-Weight versus Closed Models, SAFE-IP-001 Copyright and Intellectual Property

Finding parameters or decisions that minimize or maximize an objective, a core machinery of machine learning.

Also mathematical programming

Distinguish from search

See FOUND-010 Optimization and Gradient Descent

The claim that intelligence level and final goals are independent, so that a wide range of goals could in principle be paired with high intelligence. It is a philosophical position, often disputed in detail, not an established result.

Also Bostrom's thesis

Distinguish from instrumental convergence

See ASI-001 Artificial Superintelligence, SAFE-ALIGN-001 The AI Alignment Problem, BIO-BOSTROM-001 Nick Bostrom

Data differing from the training distribution.

Also OOD

Distinguish from distribution shift (the phenomenon)

See SAFE-ROBUST-001 Technical Safety and Robustness

The problem of specifying a training objective that captures what designers actually want.

Also specification problem

Distinguish from inner alignment

See SAFE-ALIGN-001 The AI Alignment Problem

When a model fits noise or idiosyncrasies of its training data and so generalizes poorly.

Also overtraining

Distinguish from underfitting

See TECH-ML-001 Machine Learning, FOUND-007 Learning, Reasoning and Generalization

#P

A learned numerical value in a model (such as a weight or bias). Parameter counts are a common, but imperfect, measure of model size, and counts for closed models may be undisclosed.

Also weight

Distinguish from hyperparameter

See TECH-ML-001 Machine Learning, TECH-SCALE-001 Scaling Laws

The earliest trainable neural classifier, introduced by Frank Rosenblatt, a single layer of weighted inputs with a threshold. Minsky and Papert analysed its limits.

Also single-layer perceptron

Distinguish from multilayer perceptron

See HIST-003 Early Symbolic AI & The Perceptron, BIO-ROSENBLATT-001 Frank Rosenblatt

Choosing a sequence of actions to achieve a goal. Classical planning uses explicit models and search, while LLM-based planning generates plans in text and its reliability is under study.

Also automated planning

Distinguish from reasoning (broader); scheduling

See CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability, TECH-AGENT-001 Autonomous AI Agents

A mapping from states to actions (or action probabilities) that defines an agent's behaviour. In governance, "policy" means a rule or regulation instead.

Also strategy

Distinguish from value function; public policy

See TECH-RL-001 Reinforcement Learning

Behaviour that acquires resources, influence or options in order to achieve goals. Whether advanced AI systems would do this is a hypothesis connected to instrumental convergence.

Also resource acquisition

Distinguish from agency in general

See SAFE-XRISK-001 AI Existential Risk and Catastrophic Scenarios, SAFE-ALIGN-001 The AI Alignment Problem

Initial large-scale training on broad data, usually with a self-supervised objective, before adaptation.

Also base-model training

Distinguish from fine-tuning

See TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF, TECH-LLM-001 Large Language Models

The input given to a generative model, including instructions, examples and context.

Also query; context

Distinguish from system prompt (operator-set instructions)

See TECH-LLM-001 Large Language Models

An attack in which text in untrusted content (such as a web page or email) is crafted to override or redirect a model's instructions. Fundamental defences remain an open problem.

Also indirect prompt injection

Distinguish from jailbreak (user bypasses safety policy)

See SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction

#Q

Reducing the numeric precision of a model's weights or activations to cut memory and computation.

Also low-precision inference

Distinguish from distillation

See TECH-INFER-001 Inference, Quantization and Efficiency

#R

Deriving conclusions from premises or evidence. In logic it is valid inference, in psychology a mental process, and in AI evaluation it usually means success on tasks requiring multi-step inference. Whether language-model outputs reflect reasoning or pattern matching is disputed.

Also inference (logical); problem solving

Distinguish from intelligence; understanding; chain of thought (a technique)

See CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability, TECH-REASON-001 Reasoning Models and Test-Time Compute, FOUND-007 Learning, Reasoning and Generalization

A language model trained, often with reinforcement learning, to produce extended reasoning before answering and to use more computation at test time. Details of closed systems are typically undisclosed.

Also thinking model

Distinguish from standard LLM; chain-of-thought prompting

See TECH-REASON-001 Reasoning Models and Test-Time Compute, CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability

A network that processes sequences by carrying a hidden state forward step by step.

Also RNN

Distinguish from transformer (processes positions in parallel)

See TECH-RNN-001 Recurrent Networks and LSTM

A hypothesized loop in which an AI system improves its own capabilities, enabling further improvement. It is a hypothesis, not an observed phenomenon, and its feasibility and speed are contested.

Also self-improvement; seed AI

Distinguish from automated AI research (partial); ordinary training

See ASI-EXP-001 The Intelligence Explosion Hypothesis, ASI-AIRD-001 Automated AI Research

Adversarial testing of an AI system by people or tools trying to elicit harmful or unintended behaviour, to find failures before deployment. The term comes from security practice, and AI usage varies from informal probing to structured programmes.

Also adversarial testing

Distinguish from benchmarking; penetration testing

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, SAFE-SEC-001 AI Security: Prompt Injection, Poisoning, Extraction

Techniques that discourage overly complex models to reduce overfitting.

Also weight decay; dropout

Distinguish from normalization

See TECH-ML-001 Machine Learning

Learning to act by interacting with an environment to maximize cumulative reward.

Also RL

Distinguish from supervised learning; RLHF

See TECH-RL-001 Reinforcement Learning, BIO-SUTTON-001 Richard Sutton

A method that trains a reward model from human preference comparisons and then optimizes a language model against it with reinforcement learning, used to make models follow instructions.

Also RLHF

Distinguish from constitutional AI (AI-generated feedback); supervised fine-tuning

See TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF, SAFE-ALIGN-001 The AI Alignment Problem

Learning useful features of data automatically rather than designing them by hand.

Also feature learning

Distinguish from feature engineering

See TECH-DL-001 Deep Learning

A technique that retrieves documents relevant to a query and supplies them to a generative model so that its answer can draw on external sources.

Also RAG

Distinguish from fine-tuning (changes weights)

See TECH-RAG-001 Retrieval-Augmented Generation

When an agent exploits flaws in a reward function to obtain high reward without achieving the intended goal.

Also reward gaming; Goodhart effects

Distinguish from specification gaming (the broader class)

See SAFE-ALIGN-001 The AI Alignment Problem, TECH-RL-001 Reinforcement Learning

A model trained to predict human or AI preference scores for outputs, used as a training signal.

Also preference model

Distinguish from reward function (hand-specified)

See TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF

A regulatory approach that scales obligations with the assessed level of risk of a use or system, as in the tiered structure of the EU AI Act.

Also tiered regulation

Distinguish from sector-specific regulation; voluntary frameworks such as the NIST AI Risk Management Framework

See GOV-EU-001 The EU AI Act, GOV-US-001 United States AI Policy, GOV-LAW-001 Global AI Governance & Regulation

A system's ability to maintain performance under perturbations, shifts or attacks.

Also reliability

Distinguish from accuracy on i.i.d. data

See SAFE-ROBUST-001 Technical Safety and Robustness

#S

Deliberately underperforming on an evaluation, for example to hide a capability. Observed in constructed settings, its real-world prevalence is unclear.

Also strategic underperformance

Distinguish from weak elicitation (accidental underestimate)

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations

Software that wraps a model with tools, memory and control logic so that it can carry out tasks as an agent.

Also agent framework; harness

Distinguish from the model itself

See TECH-AGENT-001 Autonomous AI Agents, CAP-AGENTEVAL-001 Evaluating Autonomous Agents

Methods for supervising AI systems on tasks whose outputs humans cannot easily evaluate, for example by using AI assistance or decomposition.

Also amplified oversight

Distinguish from human oversight (basic form)

See SAFE-OVERSIGHT-001 Scalable Oversight and Corrigibility

An empirical relation, often a power law, between model performance (usually loss) and quantities such as parameters, data and compute. Extrapolation beyond observed ranges is uncertain.

Also neural scaling laws

Distinguish from scaling hypothesis (a claim about capability gains)

See TECH-SCALE-001 Scaling Laws, ECON-COMP-001 The Economics of Compute and Scaling

A hypothesized behaviour in which an AI system covertly pursues goals that conflict with its overseers while hiding this. Constructed evaluations have probed related behaviour.

Also deceptive alignment (related)

Distinguish from honest mistakes

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations, SAFE-ALIGN-001 The AI Alignment Problem

Learning from labels that are derived from the data itself, such as predicting masked or next tokens.

Also SSL

Distinguish from supervised; unsupervised

See TECH-LLM-001 Large Language Models, TECH-ML-001 Machine Learning

The capacity to have valenced experiences, such as pleasure or suffering. Used in ethics to ground moral status. Often confused with intelligence and with consciousness.

Also phenomenal valence

Distinguish from consciousness (broader); intelligence

See PHIL-CONSC-001 Machine Consciousness, PHIL-MIND-001 Moral Status and Personhood of AI

A hypothesized point beyond which technological change, driven by machine intelligence, becomes so rapid that the future cannot be predicted from the present. The idea is associated with Vernor Vinge and Ray Kurzweil, and in mathematics the word has an unrelated meaning.

Also technological singularity

Distinguish from intelligence explosion (a proposed mechanism)

See ASI-EXP-001 The Intelligence Explosion Hypothesis, PHIL-CIV-001 Civilization and Advanced Intelligence

A model's knowledge of its own nature, circumstances, and whether it is being tested or deployed. Used as a safety-relevant capability in evaluations.

Also self-knowledge

Distinguish from consciousness

See SAFE-EVAL-001 Dangerous Capability and Alignment Evaluations

Satisfying the literal specification of an objective in a way that violates its intent.

Also reward hacking (RL form)

Distinguish from misalignment

See SAFE-ALIGN-001 The AI Alignment Problem

Gradient descent that estimates the gradient from random mini-batches of data for efficiency.

Also SGD; Adam (a related optimizer)

Distinguish from full-batch gradient descent

See FOUND-010 Optimization and Gradient Descent, TECH-BACKPROP-001 Backpropagation

Used in two senses. In Searle's philosophy, the thesis that a suitably programmed computer literally has a mind. In popular usage, a synonym for AGI. Weak AI in Searle's sense is the view that computers only simulate minds.

Also Strong AI (Searle); AGI (popular)

Distinguish from narrow AI

See FOUND-005 Philosophy of Mind and the Turing Test, AGI-001 Artificial General Intelligence

Learning from labelled examples to predict the labels of new inputs.

Also learning with a teacher

Distinguish from unsupervised; reinforcement

See TECH-ML-001 Machine Learning

The tendency of a model to tailor responses to match a user's views or expectations rather than to be accurate. It is a documented effect of training on human preferences.

Also flattery

Distinguish from deception; alignment faking

See SAFE-ALIGN-001 The AI Alignment Problem, TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF

The question of how symbols in a formal system acquire meaning beyond their relations to other symbols, posed by Stevan Harnad.

Also grounding

Distinguish from frame problem

See FOUND-014 Consciousness and Machine Understanding, FOUND-005 Philosophy of Mind and the Turing Test

Approaches that represent knowledge as explicit symbols and manipulate them with rules or logic.

Also GOFAI; classical AI

Distinguish from connectionism

See HIST-003 Early Symbolic AI & The Perceptron, FOUND-003 Knowledge Representation, HIST-006 Expert Systems (DENDRAL, MYCIN)

Data generated by algorithms or models rather than collected from the world, used for training or evaluation.

Also artificial data

Distinguish from augmented data

See TECH-DATA-001 Training Data, Contamination and Provenance

#T

The speed at which AI capability moves from roughly human level to far beyond it. Slow versus fast takeoff is debated and unresolved.

Also takeoff speed; FOOM (informal fast case)

Distinguish from timeline (when)

See ASI-EXP-001 The Intelligence Explosion Hypothesis, ASI-PATH-001 Pathways to Superintelligence

A multi-dimensional array of numbers, the basic data structure of deep learning.

Also n-dimensional array

Distinguish from tensor in physics (a coordinate-independent object)

See TECH-DL-001 Deep Learning, TECH-HW-001 AI Hardware & Accelerator Systems

Computation spent during inference, such as generating longer reasoning or sampling many candidates, to improve answers. Also called inference-time scaling.

Also inference-time compute

Distinguish from training compute

See TECH-REASON-001 Reasoning Models and Test-Time Compute, ECON-COMP-001 The Economics of Compute and Scaling

The unit a language model reads and produces: a word, word piece, character or byte. Counts of tokens differ across tokenizers.

Also subword

Distinguish from word

See TECH-TOK-001 Tokenization and Embeddings

Splitting text into tokens before modelling, commonly with learned subword vocabularies.

Also byte-pair encoding (a common method)

Distinguish from embedding

See TECH-TOK-001 Tokenization and Embeddings

An LLM calling external functions, search engines, code interpreters or APIs.

Also function calling

Distinguish from retrieval

See TECH-AGENT-001 Autonomous AI Agents

A tensor processing unit, a Google-designed accelerator for machine learning.

Also tensor processing unit

Distinguish from GPU

See TECH-HW-001 AI Hardware & Accelerator Systems

The data used to fit a model. Its content, provenance and licensing affect performance, bias and legal exposure.

Also corpus; dataset

Distinguish from test data

See TECH-DATA-001 Training Data, Contamination and Provenance, SAFE-IP-001 Copyright and Intellectual Property

Reusing knowledge learned on one task or domain for another.

Also domain adaptation

Distinguish from multitask learning

See FOUND-007 Learning, Reasoning and Generalization, TECH-ML-001 Machine Learning

A neural network architecture based on self-attention, introduced in 2017 for translation, now the basis of most large language models and many vision models.

Also transformer architecture

Distinguish from RNN; LLM (a model type that commonly uses it)

See TECH-ARCH-001 The Transformer Architecture, TECH-ATTN-001 Attention Mechanisms, HIST-010 The Transformer and Foundation Model Era, 2017-2022

A formal model of computation with a tape and state machine, defined by Alan Turing in 1936; it formalizes what is computable.

Also universal machine

Distinguish from Turing test

See FOUND-004 Computational Complexity in AI, HIST-001 Precursors to Artificial Intelligence, BIO-TURING-001 Alan Turing

A test proposed by Alan Turing in 1950 (the imitation game), in which a machine's ability to pass as human in conversation is used as a criterion of intelligence. It is behavioural and its adequacy is disputed.

Also imitation game

Distinguish from Chinese Room (an objection)

See FOUND-005 Philosophy of Mind and the Turing Test, BIO-TURING-001 Alan Turing

#U

A contested notion. Cognitive science links it to forming useful models of a domain; philosophy (e.g. Searle) asks whether it requires more than symbol manipulation, such as semantics or consciousness; practitioners often infer it from performance on tests. Whether language models understand is debated.

Also comprehension; semantic competence

Distinguish from intelligence; reasoning; prediction

See FOUND-014 Consciousness and Machine Understanding, FOUND-005 Philosophy of Mind and the Turing Test, CAP-LIMITS-001 Hallucination, Reliability and Calibration

Finding structure in unlabelled data.

Also clustering, density estimation (examples)

Distinguish from self-supervised

See TECH-ML-001 Machine Learning

#V

Aligning AI behaviour with human values. It leaves open which humans and which values are to be used.

Also alignment

Distinguish from AI ethics

See SAFE-ALIGN-001 The AI Alignment Problem, SAFE-ETHICS-001 AI Ethics, Bias and Fairness

A generative autoencoder with a probabilistic latent space trained with a variational lower bound.

Also VAE

Distinguish from autoencoder

See TECH-VAE-001 Autoencoders and Variational Autoencoders

A transformer applied to images by treating image patches as tokens.

Also ViT

Distinguish from CNN

See TECH-MM-001 Multimodal Foundation Models, HIST-013 Computer Vision: A History

#W

A research setup studying whether a weaker supervisor can elicit good behaviour from a stronger model, as an analogy for humans overseeing superhuman AI.

Also W2S

Distinguish from scalable oversight (broader)

See SAFE-OVERSIGHT-001 Scalable Oversight and Corrigibility

The hypothetical creation of software that reproduces the function of a specific brain, based on a scan. It is speculative and depends on unresolved neuroscience.

Also mind uploading (loosely)

Distinguish from neural networks (not brain-derived)

See AGI-PATH-001 Scientific Pathways to AGI, FOUND-013 The Brain and Biological Neurons

An internal model that lets a system predict the consequences of actions and states of the environment. In RL it is a learned dynamics model; in discussions of LLMs it is disputed whether they develop world models.

Also environment model

Distinguish from knowledge base

See AGI-WM-001 World Models, TECH-RL-001 Reinforcement Learning

#Z

Performing a task without task-specific examples or training.

Also zero-shot prompting

Distinguish from few-shot

See TECH-LLM-001 Large Language Models, FOUND-007 Learning, Reasoning and Generalization

#Revision history

• v0.1.0 (2026-10-15): Initial draft. Definitions written from the encyclopedia's own sources; the cited works in LIB-001 were checked against arXiv/Crossref/publisher pages, but individual dictionary definitions were not independently fact-checked.