thebestai.co encyclopedia

The Uncertainties That Matter Most

1 article9 min read10 topics

#The Uncertainties That Matter Most

#Concise Definition

The uncertainties that matter most for AI futures are those whose resolution would change many downstream forecasts and decisions, ranked by sensitivity (how much outputs move across the plausible range of the input) and by decision relevance, not by how salient or contentious they are. Such an uncertainty is often called a crux when disagreement about it explains disagreement about conclusions.

#Overview & Significance

AI forecasts disagree widely: the 50% aggregate date for unaided machines outperforming humans at every task was 2047 in a 2023 survey of 2,778 researchers, while the date for full automation of occupations was 2116 (Grace et al., 2024); in a 2022 tournament, domain experts' median probability of AI-caused extinction by 2100 was 3% against 0.38% for superforecasters (Karger et al., 2023, via the authors' announcement; see the verification note in FCST-METHOD-001). The question is which differences in underlying belief generate such gaps. Identifying them directs research and measurement effort, and it prevents disagreement over conclusions from being misread as disagreement over evidence.

#Historical Development

Sensitivity analysis and value-of-information reasoning are standard tools in decision analysis. In AI forecasting, model-based reports expose their parameters so that disagreement can be traced to inputs: the biological-anchor approach (Cotra, 2020) and compute-centric takeoff modelling (Davidson, 2023) both expose uncertain parameters and report outputs as distributions. The scenario work in FCST-SCEN-001 takes the next step by sorting uncertainties into scenario axes.

#Technical / Conceptual Explanation

#4.1 Sensitivity and value of information

Let a forecast be F=g(θ1,…,θn)F=g(\theta_1,\dots,\theta_n) with uncertain inputs θi\theta_i. A simple sensitivity index for input ii is the share of output variance attributable to it,

Si=Var⁡(E[F∣θi])Var⁡(F),S_i=\frac{\operatorname{Var}\left(\mathbb{E}[F\mid\theta_i]\right)}{\operatorname{Var}(F)},

the first-order variance-based index. Inputs with large SiS_i are candidates for measurement. The expected value of perfect information about θi\theta_i is the expected improvement in the decision if θi\theta_i were revealed before the decision is made; it is high only if the decision would change for some values of θi\theta_i. An input that moves a forecast a lot may still be unimportant for decisions.

#4.2 Types of uncertainty

  • Parametric: the value of a quantity in a given model (for example, the compute needed for a capability).
  • Structural: whether the model is the right one (for example, whether capability is a smooth function of compute).
  • Definitional: what counts as the event (AGI, automation of an occupation).
  • Strategic: what actors will choose, including responses to the forecast itself.

Structural and definitional uncertainty are usually under-reported, because model outputs show parametric ranges only.

#Types, Components & Classifications

The following list is the encyclopedia's editorial selection of cruxes. It is organised by type and is not a ranking; no sensitivity analysis was performed for this draft.

  1. Definition of the target (definitional). Milestones such as "every task better than humans" and "full automation of occupations" received 50% dates 69 years apart in the same survey (Grace et al., 2024). Disagreement about AGI timelines is partly disagreement about meaning (AGI-TIME-001).
  2. Whether measured trends continue (structural). Reported doublings of training compute (about 6 months in the deep-learning era; Sevilla et al., 2022) and of task length (about 7 months since 2019; Kwa et al., 2025) are descriptive; whether they continue to the capabilities at issue is unknown.
  3. Binding resource constraints (parametric/structural). Epoch AI concluded that runs of about 2×10292\times10^{29} FLOP were likely to be feasible by 2030, with power the first constraint to bind, and gave wide ranges for data (400 trillion to 20 quadrillion effective tokens) and chips (a median of about 100 million H100-equivalents, range 20 million to 400 million), with synthetic data excluded from the central data estimate (Sevilla et al., 2024). Feasibility is not the same as willingness to spend.
  4. Strength of AI-assisted research feedback (structural). Whether AI systems speeding AI research produce accelerating growth or run into diminishing returns is a central parameter in compute-centric models (Davidson, 2023). It is a hypothesis in the sense of ASI-EXP-001, and no standard measurement exists (see FCST-SIGNALS-001).
  5. Generalisation of measured capability (structural). Task-length results come from software tasks (Kwa et al., 2025); transfer to other domains and to deployed settings is not established by that work.
  6. Compute-to-capability mapping (parametric). Biological-anchor estimates depend on how brain-equivalent compute and training procedure are mapped to required training compute, and a widely cited summary says the report may be too aggressive in the next few years and too conservative beyond about fifty years (Karnofsky, n.d., summarising Cotra, 2020). [VERIFICATION NEEDED: secondary source.]
  7. Safety and control outcomes (strategic/structural). Whether alignment and oversight keep pace with capability is the largest source of divergence between the tournament's expert and superforecaster groups, though the tournament does not isolate the reason (Karger et al., 2023). See SAFE-ALIGN-001 and SAFE-XRISK-001.
  8. Governance and geopolitics (strategic). Regulatory timetables change; for example, secondary sources reported in mid-2026 a provisional agreement to postpone EU high-risk obligations (see FCST-NEAR-001; status unverified).
  9. Diffusion and economic translation (structural). Whether capability becomes measured productivity quickly, slowly, or unevenly; the timescale of institutional adaptation (see ECON-COMP-001).

#Applications & Concrete Examples

  • Decomposing a disagreement. Two forecasters with the same data but different answers to cruxes 1 and 4 will produce different timelines without either being irrational; stating their answers to the cruxes makes the disagreement tractable.
  • Allocating measurement. A funder choosing between building a better capability benchmark and tracking power procurement can use the cruxes above to see which uncertainty each reduces.
  • Stress-testing a forecast. Re-run a model forecast with each crux set to its extreme values to see which one moves the output most.

#Advantages, Limitations & Unresolved Questions

  • The list is a judgment. A different set of cruxes would be defensible; the ordering above is not a ranking.
  • Interactions. Cruxes are not independent (resource constraints affect the strength of research feedback, for example); one-at-a-time analysis can miss interactions.
  • Unknown unknowns. Uncertainties not on any list may dominate.
  • Unresolved: which cruxes are resolvable by observation in the next three years, and which only by the outcome itself.

#Current State of Knowledge

As of 2026-10-15 none of the nine cruxes is settled by evidence. Several have measured proxies (2, 3, 5), one has a hypothesised but unstandardised indicator (4), and some (7, 8, 9) are dominated by actor choices. No quantitative sensitivity ranking across them is cited here.

#Future Trajectory & Strategic Outlook

Cruxes with measured proxies (2, 3) are expected to be partly resolved by data over the next three years; the rest are expected to remain open. The forecast below concerns crux 3, the resource-constraint question, using a physical-capacity test.

#Forecast record

Field Content
Forecast ID FCST-POWER-001
Forecast Statement By 2029-10-15, at least one AI data-centre campus with a publicly stated power capacity of 1 GW or more will be reported as operational for AI workloads.
Target Event An operational AI campus at or above 1 GW
Operational Definition Resolves true if, by 2029-10-15, the operator or a utility publicly states that a single AI-focused campus with at least 1 GW of delivered (not merely planned or contracted) power is operating. Planned or under-construction capacity does not count. Baseline: whether this already holds on 2026-10-15 is to be checked at first review; if already true, the record resolves immediately.
Time Horizon 2029-10-15
Assumptions Continued investment in AI infrastructure; the power constraint identified by Epoch AI (Sevilla et al., 2024) is surmountable for at least one campus of that size, since Epoch considered 1-5 GW campuses as supporting runs of 102810^{28} to 3×10293\times10^{29} FLOP.
Confidence Level Medium-high (about 65-80%; editorial judgment)
Alternative Scenarios Scenario A: grid interconnection, equipment lead times, or financing delay delivery; or operators do not publicly state capacity, leaving the record unresolved.
Review Schedule 2027-01-15 (baseline), then each October; final 2029-10-15
Status Active
Sources, evidence and revision historyReferences · Evidential Classification Matrix · Revision History

#References & Foundational Literature

  1. Grace, K., Stewart, H., Sandkühler, J. F., Thomas, S., Weinstein-Raun, B., Brauner, J., & Korzekwa, R. C. (2024). Thousands of AI Authors on the Future of AI. arXiv:2401.02843; JAIR 84:9 (2025). https://arxiv.org/abs/2401.02843. [2,778 respondents; 2047 and 2116 aggregate 50% dates for the two milestones.]
  2. Karger, E., Rosenberg, J., Jacobs, Z., Hickman, M., Hadshar, R., Gamin, K., Smith, T., Williams, B., McCaslin, T., Thomas, S., & Tetlock, P. E. (2023). Forecasting Existential Risks: Evidence from a Long-Run Forecasting Tournament. Forecasting Research Institute, 2023-07-10 (rev. 2023-08-08). https://forecastingresearch.org/xpt. [80 experts and 89 superforecasters; largest disagreement on AI risk; medians via the authors' announcement.]
  3. Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022). Compute Trends Across Three Eras of Machine Learning. arXiv:2202.05924. https://arxiv.org/abs/2202.05924. [Compute doubling times.]
  4. Sevilla, J., Besiroglu, T., Cottier, B., You, J., Roldán, E., Villalobos, P., & Erdil, E. (2024). Can AI scaling continue through 2030? Epoch AI, 2024-08-20. https://epoch.ai/blog/can-ai-scaling-continue-through-2030. [Power, chip, data, latency constraint ranges; 2030 feasibility of about 2×10292\times10^{29} FLOP.]
  5. Kwa, T., West, B., Becker, J., et al. (2025). Measuring AI Ability to Complete Long Software Tasks. arXiv:2503.14499. https://arxiv.org/abs/2503.14499. [Seven-month doubling of the 50% task time horizon, software tasks.]
  6. Davidson, T. (2023). What a compute-centric framework says about takeoff speeds. Open Philanthropy. https://coefficientgiving.org/research/what-a-compute-centric-framework-says-about-takeoff-speeds/. [Takeoff model with explicit uncertain parameters; not directly retrieved: VERIFICATION NEEDED for final-report details.]
  7. Cotra, A. (2020). Forecasting Transformative AI with Biological Anchors (draft report). Open Philanthropy. [Original not retrieved: VERIFICATION NEEDED.]
  8. Karnofsky, H. (n.d.). Forecasting transformative AI: the "biological anchors" method in a nutshell. Cold Takes. https://www.cold-takes.com/forecasting-transformative-ai-the-biological-anchors-method-in-a-nutshell/. [Secondary summary, including the report's own caveat on near- and far-term bias; date not recorded: VERIFICATION NEEDED.]

#Evidential Classification Matrix

Section / Claim Evidence Category Supporting Source Confidence / Status
Definitions of milestones drive date differences Well Supported (within one survey) Grace et al. (2024) High
Variance-based sensitivity index Editorial method (no specific citation) -- Moderate
Resource-constraint ranges Forecast Sevilla et al. (2024) Report-stated
Compute and time-horizon doublings Well Supported / Preliminary Sevilla et al. (2022); Kwa et al. (2025) Moderate-high
Research-feedback strength Hypothetical Davidson (2023); ASI-EXP-001 Low
Expert/superforecaster gap on AI risk Preliminary Karger et al. (2023) Moderate
Selection of cruxes Editorial judgment -- Not a ranking
FCST-POWER-001 Forecast Sevilla et al. (2024) About 65-80%, editorial

#Revision History

  • v0.1.0 (2026-10-15): Initial draft. Citations checked against arXiv/Crossref/publisher pages where marked; not independently fact-checked.
10 topics this chapter covers
  • 68.1The relationship between scale and capability
  • 68.2The limits of current architectures
  • 68.3Data constraints
  • 68.4Reasoning and generalization
  • 68.5Autonomous learning
  • 68.6Recursive improvement
  • 68.7Economic incentives
  • 68.8Human control
  • 68.9Governance and coordination
  • 68.10Uncertainty in long-term forecasting