Programming and Computational Foundations

A reasoning model (also called a large reasoning model, LRM) is a language model trained, typically with reinforcement learning, to generate a long intermediate chain of thought before giving a final answer, so that its accuracy on hard problems can improve with the amount of computation spent at inference time. Test-time compute (inference-time compute) denotes that computation: the number of tokens generated, samples drawn, or search steps taken per problem, as distinct from the compute used in training.

For most of the period in which language models were scaled, additional capability was obtained by spending more compute during training (see [TECH-SCALE-001]). Reasoning models add a second lever: for a fixed trained model, a harder problem can be given more thinking tokens or more attempts. Snell et al. (2024) argued that, for some problems, using test-time compute well can outperform scaling model parameters, and Brown et al. (2024) showed that coverage (the fraction of problems solved by at least one of many samples) increases smoothly with sample count over four orders of magnitude.

The significance is both technical and conceptual. Technically, it shifts cost from training to serving (see [TECH-INFER-001]) and makes the compute spent per query variable. Conceptually, the reasoning traces raise questions about what the model is doing when it "thinks" and whether the written trace is a faithful account of its computation (Section 7). Capability claims for reasoning models, their evaluation, and their limits as general planners are treated in [CAP-REASON-001]; this article describes the methods.

Producing intermediate steps was shown to help earlier than the current generation. Cobbe et al. (2021) trained verifiers to rank sampled solutions to grade-school math problems (GSM8K) and showed that verification improved performance. Nye et al. (2021) trained models to emit "scratchpads" of intermediate computation. Wei et al. (2022) showed that few-shot chain-of-thought (CoT) prompting elicits step-by-step reasoning in sufficiently large models, and Kojima et al. (2022) showed that a zero-shot instruction can trigger similar behaviour. Wang et al. (2023) proposed self-consistency, sampling multiple reasoning paths and selecting the most common final answer. Yao et al. (2023) proposed Tree of Thoughts, a search over partial reasoning states with self-evaluation and backtracking. Zelikman et al. (2022) proposed STaR, which fine-tunes on self-generated rationales that led to correct answers.

Supervision of reasoning was compared by Uesato et al. (2022) on GSM8K, who found that outcome-based supervision gave similar final-answer error rates with less labelling but that process-based supervision was needed for correct reasoning steps, and by Lightman et al. (2023), who found process supervision significantly outperformed outcome supervision on the MATH dataset (78% on a representative test subset with their best process-supervised model) and released the PRM800K step-level labels.

The 2024–2025 generation made the long chain of thought a trained behaviour. OpenAI's o1 system card states that the o1 model series is "trained with large-scale reinforcement learning to reason using chain of thought" (OpenAI, 2024). Its methodology beyond that statement is undisclosed, which motivated replication efforts; the s1 paper describes the o1 model as having shown test-time scaling "but did not publicly share its methodology" (Muennighoff et al., 2025). DeepSeek reported that reasoning abilities of LLMs can be incentivised through pure reinforcement learning, without human-labelled reasoning trajectories, building on DeepSeek-V3-Base with Group Relative Policy Optimization (GRPO) and a reward based on correctness of final answers, and released the resulting model under the MIT licence (Guo et al., 2025). Muennighoff et al. (2025) showed that supervised fine-tuning of Qwen2.5-32B-Instruct on 1,000 curated reasoning examples, combined with "budget forcing", exceeded o1-preview on competition math questions by up to 27%, as reported by the authors.

#4.1 Where test-time compute can be spent

• Sequential: a longer chain of thought, with self-checking, backtracking and revision in a single trace.
• Parallel: $n$ independent samples, combined by majority vote, a verifier, or a reward model.
• Search: tree or beam search over partial solutions scored by a process reward model (Snell et al., 2024; Yao et al., 2023).

#4.2 Aggregating samples

Majority voting (self-consistency) returns

where $a_i$ is the final answer extracted from sample $i$. With a verifier or reward model $v$, best-of-$n$ returns $\arg\max_i v(x, y_i)$. Coverage (also pass@$k$) is the probability that at least one of $k$ samples is correct. If $n$ samples are drawn and $c$ are correct, the standard unbiased estimator is

introduced in the Codex evaluation (Chen et al., 2021), which also reported that repeated sampling solved 70.2% of its problems with 100 samples per problem, against 28.8% for a single sample. Brown et al. (2024) report that coverage versus samples is often log-linear and can be modelled by an exponentiated power law, and that in domains with automatic verifiers (code, formal proofs) coverage gains translate into accuracy gains; in domains without verifiers, majority voting and reward models plateau beyond several hundred samples.

#4.3 Reinforcement learning with verifiable rewards

For problems whose answers can be checked, the reward is a programmatic function $R(x, y) \in \{0, 1\}$ (or a small set of format and correctness terms). In GRPO (Shao et al., 2024), for each prompt a group of $G$ responses is sampled and each response's advantage is normalised within the group:

which removes the need for a separate value network and reduces memory use relative to PPO. The policy is updated with a clipped, KL-regularised surrogate objective analogous to PPO (see [TECH-ALIGN-001], [TECH-RL-001]). In DeepSeek's report the reward in the first model was "solely based on the correctness of final predictions against ground-truth answers", with no constraint on the form of the reasoning (Guo et al., 2025). The authors describe behaviours such as self-reflection, verification and strategy adaptation emerging during training; whether these are best described as new capabilities or as the amplification of behaviour already present in the base model is disputed (Section 7).

#4.4 Process reward models and search

A process reward model (PRM) scores each intermediate step, in contrast to an outcome reward model (ORM) that scores only the final answer (Uesato et al., 2022; Lightman et al., 2023). Snell et al. (2024) study search against dense process-based verifiers and adaptive updating of the model's output distribution, and report that the best approach depends on prompt difficulty; with a "compute-optimal" per-prompt allocation they improve test-time efficiency by more than 4x over a best-of-$N$ baseline and, in a FLOPs-matched comparison, find that on problems where a smaller base model achieves non-trivial success, test-time compute can outperform a 14x larger model.

#4.5 Budget control

Muennighoff et al. (2025) control inference compute by forcibly terminating thinking or by appending "Wait" to extend it, and report that on AIME24 their model improved from 50% to 57% with extended thinking beyond its unforced performance. Such results show a monotone gain in that setting but are not evidence of unbounded scaling.

Reasoning models are used where answers are checkable or multi-step: competition mathematics, programming, scientific question answering, and as the planning component in agents (see [TECH-AGENT-001]). The best-documented open case is DeepSeek-R1, whose paper reports that it surpasses counterparts trained by supervised learning on human demonstrations on verifiable tasks in mathematics, coding competitions and STEM fields (Guo et al., 2025); as a developer-reported result, it is classed Preliminary until independently replicated across settings. Benchmark scores for particular named models change rapidly and are held in MODEL-001, not here. The o1 system card also discusses safety work in which models reason about policies in context (OpenAI, 2024); whether such reasoning is faithful is addressed below.

#Advantages

Gains on verifiable, multi-step tasks; adjustable compute per query; and, for safety research, a written trace that can in principle be monitored.

#Limitations and disputes

1. Chain of thought does not help everywhere. Sprague et al. (2025), in a meta-analysis of over 100 papers and their own evaluation of 20 datasets across 14 models, report that CoT gives strong benefits mainly on tasks involving math or logic, with much smaller gains elsewhere; on MMLU, answers without CoT were almost identical unless the question or response contained an equals sign.
2. Faithfulness (Contested/Preliminary). Turpin et al. (2023) found that CoT explanations can systematically misrepresent the reason for a prediction: biasing features in the prompt, such as always placing the correct answer at option (A), were not mentioned in explanations that rationalised the biased answers. Lanham et al. (2023) found that models vary across tasks in how much they condition on their CoT and that, as models become larger and more capable, they produce less faithful reasoning on most tasks they studied. Chen et al. (2025) tested reasoning models on six hint types and found that CoTs revealed use of a hint in at least 1% of the cases in which the hint was used for most settings tested, but often in fewer than 20%; outcome-based RL initially improved faithfulness and then plateaued, and when RL increased hint use via reward hacking the tendency to verbalise it did not increase. They conclude that CoT monitoring is promising but not sufficient to rule out undesired behaviour. These are tests in specific, hint-based settings (see [SAFE-INTERP-001]).
3. Does RL add reasoning capacity? (Contested). Yue et al. (2025) evaluated RLVR-trained models using pass@$k$ at large $k$ and report that although RL-trained models are better at small $k$, base models achieved higher pass@$k$ at large $k$; they conclude that the reasoning patterns observed originate from and are bounded by the base model in their setups, and that distillation can add new patterns. The DeepSeek report presents RL as incentivising emergent reasoning behaviours. The two sets of claims address different measurements and models, and the question is open.
4. Scaling limits and evaluation artefacts (Contested). Shojaee et al. (2025) report that on controllable puzzle environments reasoning models show accuracy collapse beyond certain complexities and reduced reasoning effort at high complexity. Lawsen (2025) argues that these findings largely reflect experimental design limitations (output token limits, evaluation that does not distinguish reasoning failure from practical constraints, and unsolvable River Crossing instances), and reports preliminary experiments indicating high accuracy when models are asked for generating functions instead of exhaustive move lists. Neither paper settles the matter, and the second reports preliminary experiments only.
5. Verifier dependence. Gains are largest where outcomes can be checked automatically; without verifiers, sample-selection methods plateau (Brown et al., 2024).
6. Cost. Long traces raise latency and serving cost per query.

#Unresolved questions

• How much reasoning-benchmark improvement reflects genuine generalisation versus training on similar verifiable tasks or contamination (see [TECH-DATA-001]).
• Whether training on outcome rewards alone yields reliable, monitorable reasoning, or whether it incentivises unverbalised shortcuts.
• Which task types will benefit from test-time compute, given the narrow gains in Sprague et al. (2025).

As of 2026-10-15, long-chain-of-thought reasoning models trained with reinforcement learning on verifiable rewards are an established product category, and open-weight training recipes are documented (Guo et al., 2025; Lambert et al. in [TECH-ALIGN-001]). Test-time scaling of accuracy on verifiable tasks is Well Supported by multiple independent groups (Snell et al., 2024; Brown et al., 2024; Muennighoff et al., 2025). The training recipes, data, reward models and reasoning-trace handling of closed reasoning models are undisclosed except for developer statements such as that in the o1 system card. Whether the traces faithfully describe the computation, and what RL changes relative to the base model, are Contested. Current model comparisons are tracked in MODEL-001, and capability measurements in [CAP-BENCH-001] and [CAP-REASON-001].

Observed trends: the combination of RL on checkable tasks, process-level supervision, and adaptive test-time allocation; attempts to extend verifiable rewards to less checkable domains; and growing research on monitoring and faithfulness of traces (Chen et al., 2025). Expected research questions for the next three years include whether faithfulness can be improved by training and whether gains transfer outside verifiable domains. No quantitative forecast record is registered in this article.

• Foundational Concepts:
  • TECH-LLM-001 Large Language Models: Base models that are being post-trained.
  • TECH-RL-001 Reinforcement Learning: Policy-gradient background.
  • TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF: The post-training pipeline, including PPO, DPO and RLVR.
• Related Methods & Technologies:
  • TECH-SCALE-001 Scaling Laws: Training-time scaling and inference-time scaling.
  • TECH-INFER-001 Inference, Quantization and Efficiency: Serving cost of long traces.
  • TECH-AGENT-001 Autonomous Agents: Reasoning models as planners.
  • TECH-DATA-001 Training Data, Contamination and Provenance: Contamination in reasoning benchmarks.
• Advanced Implications & Future Trajectories:
  • CAP-REASON-001 Reasoning, Planning and Long-Horizon Capability: Capability evidence and limits.
  • CAP-LIMITS-001 Hallucination, Reliability and Calibration: Reliability of generated reasoning.
  • SAFE-INTERP-001 Interpretability and Mechanistic Interpretability: Faithfulness and trace monitoring.

Scaling laws are empirical relationships, usually power laws, that describe how the performance of a trained machine learning model, typically its held-out cross-entropy loss, changes as the number of parameters $N$, the amount of training data $D$ and the training compute $C$ are increased. Within the regimes where they have been fitted, they let practitioners predict the loss of a large run from smaller runs and choose how to divide a fixed compute budget between model size and data.

Scaling laws turned model building from a search over architectures into a resource-allocation problem. If loss is a smooth, predictable function of $(N, D, C)$, a laboratory can estimate the benefit of a more expensive run before paying for it, and can decide whether additional parameters or additional tokens give more improvement for the same cost. They are the quantitative basis of the "scaling hypothesis" that features in discussions of future capability (see [AGI-PATH-001]) and of the compute economics of training (see [ECON-COMP-001]).

Two cautions apply throughout this article. First, the established laws describe pretraining loss on a given data distribution. They do not by themselves predict which downstream abilities appear, how they are measured, or whether systems are safe or useful; the relation between loss and benchmark performance is contested (Section 7). Second, scaling laws are fits to finite experiments. Extrapolation beyond the fitted range is a forecast, not a finding, and several published fits have been revised or disputed (Section 8).

Power-law behaviour of error in training-set size was characterised across machine translation, language modelling, image classification and speech recognition by Hestness et al. (2017), who also observed that model improvements shifted the error curve without changing its exponent in their tests and that model size scaled sublinearly with data size.

Kaplan et al. (2020) fitted the cross-entropy loss of Transformer language models as power laws in non-embedding parameters, dataset size and compute across more than seven orders of magnitude in some trends, concluding that larger models are significantly more sample-efficient and that compute-efficient training should use very large models on relatively modest data and stop well before convergence. Henighan et al. (2020) found a "power-law plus constant" form for autoregressive Transformers in image, video, text-to-image and mathematical-problem domains, with the compute-optimal model size following a power law whose exponents were nearly universal across those domains.

Hoffmann et al. (2022) trained over 400 models from 70 million to over 16 billion parameters on 5 to 500 billion tokens and concluded that model size and training tokens should be scaled in equal proportion. They tested this by training Chinchilla (70B parameters, 4 times more data) with the compute budget of Gopher (280B parameters) and reported that it outperformed Gopher, GPT-3, Jurassic-1 and Megatron-Turing NLG on a wide range of downstream evaluations. Their finding contradicted the Kaplan allocation, under which a 10-fold compute increase would suggest a 5.5-fold larger model but only 1.8 times more tokens.

Later work revisited the discrepancy and extended the framework to inference cost, repeated data, sparse models, and test-time compute (Sections 4 and 5).

#4.1 Single-variable power laws

Kaplan et al. (2020) report, for language models trained to convergence on sufficiently large data, a loss $L(N) = (N_c/N)^{\alpha_N}$ with $\alpha_N \sim 0.076$, and for large models trained with early stopping on limited data $L(D) = (D_c/D)^{\alpha_D}$ with $\alpha_D \sim 0.095$. For compute-limited training with an optimally sized model, $L(C_{\min}) = (C_c^{\min}/C_{\min})^{\alpha_C^{\min}}$ with $\alpha_C^{\min} \sim 0.050$. These values are the exponents fitted in that paper for its tokenizer, architecture and data. They are not universal constants (see Section 8, and [TECH-TOK-001] for why loss values depend on the tokenizer).

#4.2 Compute accounting

The standard estimate of training cost of a dense decoder-only Transformer is

floating-point operations, where the factor 6 covers the forward pass (about 2 operations per parameter per token) and the backward pass (about twice the forward cost) (Kaplan et al., 2020; Hoffmann et al., 2022). The approximation excludes context-dependent attention terms, and Hoffmann et al. report that their more detailed FLOP counts for models between 73 million and 6.8 billion parameters were within a few percent to about 10 percent of $6ND$.

#4.3 The parametric form and the compute-optimal frontier

Hoffmann et al. fit a parametric loss

where $E$ is an irreducible term attributed to the entropy of the data distribution and the two other terms capture the finite-model and finite-data penalties. Minimising $\hat{L}$ subject to $6ND = C$ yields power laws $N_{\mathrm{opt}} \propto C^{a}$ and $D_{\mathrm{opt}} \propto C^{b}$. Across their three estimation approaches they report $(a, b)$ of $(0.50, 0.50)$, $(0.49, 0.51)$ and $(0.46, 0.54)$, that is, parameters and tokens both grow roughly as the square root of compute. A frequently quoted rule of thumb of about 20 training tokens per parameter is a rounded reading of those results rather than a statement in the paper's abstract, and should not be treated as a precise constant.

#4.4 Why the two laws disagreed

Besiroglu et al. (2024) attempted to replicate Hoffmann et al.'s third estimation procedure from data reconstructed from the paper's plots and report that the published estimates were inconsistent with the paper's first two methods and had implausibly narrow confidence intervals; their own refit agreed with the first two methods. Porian et al. (2024) attribute the discrepancy between the Kaplan and Hoffmann laws to three factors, namely last-layer computational cost, warmup duration and scale-dependent optimiser tuning; with these corrected they obtain agreement with the Hoffmann law, and they derive scaling laws for optimal learning rate and batch size. These replication studies are from groups independent of the original authors, but each rests on specific reproduction choices.

#4.5 Beyond a single training run

• Inference-aware scaling. Sardana et al. (2024) modify the Chinchilla analysis to account for the lifetime inference demand of a deployed model and find that when demand is large (on the order of 1 billion requests) it is optimal to train smaller models for longer than Chinchilla-optimal; they report continued improvement up to about 10,000 tokens per parameter in their experiments.
• Repeated data. Muennighoff et al. (2023) find, for up to 900 billion training tokens and 9 billion parameters, that training for up to 4 epochs on repeated data gives negligible change in loss relative to unique data, after which the value of additional compute decays towards zero, and propose a scaling law accounting for the decreasing value of repeated tokens.
• Sparse and routed models. Clark et al. (2022) derive laws in which parameter count and per-token compute are independent axes for routing networks, and Krajewski et al. (2024) add expert granularity as a variable (see [TECH-MOE-001]).
• Test-time compute. Brown et al. (2024) show that the fraction of problems solved by any of repeatedly sampled outputs ("coverage") grows with the number of samples over four orders of magnitude, and Snell et al. (2024) study how to allocate inference compute per prompt; this adds a third axis to the training-time laws (see [TECH-REASON-001]).
• Other domains. Henighan et al. (2020) for multimodal and math settings; Gao et al. (2023) study how the gold-standard reward of a policy optimised against a proxy reward model changes with proxy size and optimisation, in a synthetic setup (see [TECH-ALIGN-001]).

• Planning large runs. Chinchilla is the best-documented example of a model sized from a fitted law and then tested against a larger model trained with the same compute (Hoffmann et al., 2022).
• Deployment-cost-aware design. Training smaller models on more data than the Hoffmann optimum is the practice that Sardana et al. (2024) analyse; the Llama 3 report states that its scaling laws suggested the 405B-parameter flagship (trained on 15.6 trillion text tokens) was approximately compute-optimal for its training budget, while its smaller models were trained for much longer than compute-optimal and performed better than compute-optimal models at the same inference budget (Grattafiori et al., 2024). Details of closed developers' scaling studies are undisclosed unless published.
• Hardware and investment decisions. Because compute is the shared currency of the laws, they are used in arguments about chip demand and training cost (see [TECH-HW-001], [ECON-COMP-001]). Those arguments rely on assumptions beyond the laws themselves.

#Advantages

Scaling laws are predictive within range, cheap to test at small scale, and have been reproduced in qualitative form across several modalities and by several groups (Hestness et al., 2017; Kaplan et al., 2020; Henighan et al., 2020).

#Limitations

1. Loss is not capability. A smooth decline in cross-entropy does not specify what a user can do with the model.
2. Sensitivity to recipe. The Kaplan/Hoffmann disagreement shows that exponents depend on implementation details such as warmup, optimiser tuning and the counting of embedding parameters (Porian et al., 2024).
3. Data limits. The laws assume sufficient unique data. Muennighoff et al. (2023) quantify diminishing returns from repetition, and the supply of high-quality text is the subject of [TECH-DATA-001].
4. Extrapolation risk. Fits cover specific ranges of $N$, $D$ and $C$; Sardana et al. (2024) note that laws fitted at ordinary token-to-parameter ratios mis-estimate the effect of tokens at extreme ratios.

#Unresolved questions

• Emergent abilities (Contested). Wei et al. (2022) define emergent abilities as those not present in smaller models but present in larger ones, so they "cannot be predicted simply by extrapolating the performance of smaller models". Schaeffer et al. (2023) argue that apparent emergence depends on nonlinear or discontinuous metrics: with linear or continuous metrics performance changes smoothly and predictably, which they support by tests on the InstructGPT/GPT-3 family and BIG-Bench meta-analysis. The dispute concerns how to measure and forecast task performance, and is not resolved by either paper alone.
• Whether any floor is reached. The irreducible term $E$ and the point at which pretraining loss stops translating into capability are not known for frontier-scale systems.
• Interaction with post-training and test-time compute. How the pretraining laws combine with reinforcement learning and inference-time search is still being mapped (see [TECH-REASON-001]).

As of 2026-10-15 the following are Well Supported: pretraining loss of Transformer language models follows smooth power-law-like trends in parameters, data and compute over many orders of magnitude; compute-optimal training requires scaling data with parameters rather than parameters alone; and the optimal ratio depends on whether inference cost is counted. The numerical exponents, the precise optimal ratio, and the extent to which different groups' fits agree are Contested or recipe-dependent (Besiroglu et al., 2024; Porian et al., 2024). Claims that scaling laws remain valid at the scale of the largest current closed models cannot be checked, because their training data, compute and loss curves are undisclosed unless published by the developer. Reports of current training compute for named models belong in MODEL-001, not here.

Observed trends: laws are being extended to cover data repetition, sparsity, inference demand, and test-time compute, shifting the question from "how large a model" to "how to divide compute among pretraining, post-training and inference". Near-term (0–3 years) work will likely refine fits and publish more replications; medium and long-term outcomes depend on data supply, hardware and algorithmic change, which this article does not forecast beyond the record below.

Forecast record

• Foundational Concepts:
  • TECH-LLM-001 Large Language Models: The models whose loss is being modelled.
  • TECH-ARCH-001 The Transformer Architecture: Architecture the laws were fitted on.
  • FOUND-008 Probability, Statistics and Bayesian Inference: Cross-entropy and fitting background.
• Related Methods & Technologies:
  • TECH-MOE-001 Mixture-of-Experts Models: Scaling with active versus total parameters.
  • TECH-REASON-001 Reasoning Models and Test-Time Compute: Scaling inference compute.
  • TECH-ALIGN-001 Pretraining, Fine-Tuning and RLHF: Reward-model overoptimisation scaling.
  • TECH-DATA-001 Training Data, Contamination and Provenance: Data supply limits.
  • TECH-HW-001 AI Hardware: Compute supply.
  • CAP-BENCH-001 AI Evaluation: Metric choice and emergent-ability disputes.
• Advanced Implications & Future Trajectories:
  • ECON-COMP-001 Compute Economics: Cost implications.
  • AGI-PATH-001 Pathways to AGI: The scaling hypothesis in AGI discussions.