---
id: ASI-COEX-001
title: "Human and Machine Coexistence"
alternative_names:
  - "Human-AI Coexistence"
  - "Human-AI Collaboration and Oversight"
category: "Advanced Intelligence → Superintelligence → Society and Coexistence"
volume: VI
chapter: 36
entry_type: "Standard Article"
level: 3
status: "Draft"
last_reviewed: 2026-10-18
version: "0.1.0"
prerequisites:
  - "ASI-001"
  - "SAFE-ALIGN-001"
related_entries:
  - "PHIL-CIV-001"
  - "PHIL-MIND-001"
  - "SAFE-OVERSIGHT-001"
  - "SAFE-ETHICS-001"
  - "ECON-LABOR-001"
  - "GOV-INTL-001"
---

# Human and Machine Coexistence

## 1. Concise Definition
**Human and machine coexistence** denotes the study of how people, institutions and increasingly capable AI systems can work alongside one another while humans retain meaningful agency, oversight and a fair share of the benefits. It spans empirically studied present-day questions (human-AI team performance, over-reliance, effective oversight) and speculative questions about arrangements that would follow systems far exceeding human performance.

## 2. Overview & Significance
Chapter 36 of the encyclopedia blueprint treats coexistence as the point where technical capability, economics, governance and philosophy meet. The subject matters because most proposed benefits of advanced AI depend on humans being able to use, check and direct it, and because most proposed harms involve humans losing the ability or the standing to do so.

This article separates three kinds of claim. **Empirical findings** come from experiments, field studies and reviews on existing systems and are rated by the evidence framework. **Philosophical arguments** concern what ought to be preserved and why; they are labelled as arguments. **Speculation** about systems that do not exist (superintelligence) is labelled Hypothetical or Contested. No forecasts are made here; any future-facing statement is a scenario, not a prediction.

## 3. Historical Development
Concern about humans working with automated systems predates modern AI. Bainbridge (1983) argued that automating most of a task leaves human operators with the residual, hardest duties (monitoring and rare-event intervention) while eroding the skill and practice needed to perform them. Human-factors research on automation use through the 1990s and 2000s led to the review by Parasuraman and Manzey (2010), which characterised complacency and automation bias as general features of human attention interacting with automation, observed in both novices and experts.

A parallel philosophical line asked whether tools become part of cognition. Clark and Chalmers (1998) proposed the "extended mind" thesis, in which external resources can under some conditions count as parts of a cognitive process. Bostrom (2014) shifted the debate toward the long-term question of how humans could coexist with, or control, machine agents that outperform them. Russell (2019) proposed a design stance in which machines are uncertain about human preferences and defer to humans accordingly. From 2022 onward, widely deployed generative systems turned the empirical side into a field-experiment literature (Noy & Zhang, 2023; Brynjolfsson et al., 2025).

## 4. Technical / Conceptual Explanation

### 4.1 When do human-AI teams outperform either alone?
Let $P_H$, $P_A$ and $P_{HA}$ denote task performance of the human alone, the AI alone and the combined team. *Complementarity* holds when $P_{HA} > \max(P_H, P_A)$; *augmentation* holds when $P_{HA} > P_H$ only. These are distinct, and the literature shows that augmentation is common while complementarity is not.

### 4.2 Over-reliance and under-reliance
Reliance is calibrated when a person accepts correct AI outputs and rejects incorrect ones. **Automation bias** is the tendency to accept automated advice, including when it is wrong, and to omit checks that would catch errors (Parasuraman & Manzey, 2010; Goddard et al., 2012). Oversight is therefore a human performance question, not only a procedural one.

### 4.3 Oversight as a design problem
Meaningful oversight requires that the overseer has the information, time, authority and skill to detect errors and act on them. Technical approaches to supervising systems whose outputs are hard for humans to evaluate are covered in [[SAFE-OVERSIGHT-001]]; this article concerns the human and institutional side.

## 5. Types, Components & Classifications

| Dimension | Variants |
| :--- | :--- |
| Collaboration mode | Human-in-the-loop (human approves each action); human-on-the-loop (human supervises and can intervene); human-out-of-the-loop (no real-time human role) |
| Task type | Decision tasks (choose a label or action); creation tasks (produce content) |
| Cognitive relationship | Substitution of a human function; augmentation of it; offloading to an external store |
| Locus of power | Distributed among many users and institutions; concentrated in few developers, states or compute owners |

The scope of this chapter comprises seven topics treated in the sections below: collaboration, cognitive enhancement, AI-mediated decisions, autonomy and oversight, the distribution of technological power, economic and institutional adaptation, and human values in advanced systems.

## 6. Applications & Concrete Examples

### 6.1 Human-AI collaboration (empirical)
- **Writing tasks.** In a preregistered experiment with 453 college-educated professionals, access to ChatGPT cut task time by 40% and raised rated quality by 18%, and reduced inequality between workers (Noy & Zhang, 2023).
- **Customer support.** In a staggered-rollout study of 5,172 agents, AI assistance raised issues resolved per hour by 15% on average; gains were concentrated among less experienced workers, while the most experienced saw small gains in speed and small declines in quality (Brynjolfsson et al., 2025).
- **Pooled evidence.** A preregistered meta-analysis of 106 experiments (370 effect sizes) found human-AI combinations performed, on average, significantly worse than the best of humans or AI alone (Hedges' g = -0.23; 95% CI -0.39 to -0.07). Losses appeared in decision tasks and gains in content-creation tasks; combinations gained when humans outperformed the AI alone and lost when the AI outperformed humans (Vaccaro et al., 2024). The review covered studies published 1 January 2020 to 30 June 2023, so it predates much recent deployment.
- **Explanations.** In user studies on three datasets, AI explanations did not produce complementary performance; they increased the chance that people accepted the AI's recommendation regardless of correctness (Bansal et al., 2021). A later experiment (N=199) found that "cognitive forcing" interfaces reduced over-reliance relative to simple explanations, but users rated those designs least favourably (Buçinca et al., 2021).

### 6.2 Human cognitive enhancement
- **Skill and learning.** In a field experiment with nearly a thousand high school mathematics students, GPT-4 access during practice improved performance (48% for a standard interface, 127% for a tutor-style interface), but students who then lost access scored 17% lower than controls in the standard-interface condition; the tutor-style safeguards largely mitigated the loss (Bastani et al., 2025). This is evidence that performance with a tool and learning can diverge.
- **Offloading.** Sparrow et al. (2011) reported experiments in which people expecting future access to information showed lower recall of it. This is a single-paradigm finding and its generality to AI assistants is **Preliminary**.
- **Neural or biological enhancement and brain-machine interfaces** are outside the scope of the evidence reviewed here. [VERIFICATION NEEDED: no peer-reviewed sources were checked for cognitive enhancement via neurotechnology; no claims are made.]

### 6.3 AI-mediated decision-making
Human oversight is a central regulatory mechanism. Article 14 of Regulation (EU) 2024/1689 (the EU AI Act) requires that high-risk AI systems be designed so that natural persons can effectively oversee them during use, with the aim of preventing or minimising risks to health, safety or fundamental rights. (Article 14(1) and (2), Official Journal text read 2026-10-18; application dates for high-risk systems are in [GOV-EU-001].) In a survey of 41 policies prescribing human oversight of government algorithms, Green (2022) argued that evidence suggests people cannot reliably perform the desired oversight functions, so such policies may legitimise flawed systems. This is a policy-analytic argument built on cited experiments, rated **Well Supported** for the first claim only to the extent of the cited human-factors literature, and **Contested** as a policy conclusion.

## 7. Advantages, Limitations & Unresolved Questions

**Advantages (empirical).** Measured gains exist for specific tasks and populations, particularly for less experienced workers in the studies above (Noy & Zhang, 2023; Brynjolfsson et al., 2025).

**Limitations (empirical).**
1. Combined teams do not reliably beat the better party alone (Vaccaro et al., 2024).
2. Automation bias persists across expertise levels and is not removed by explanations (Parasuraman & Manzey, 2010; Bansal et al., 2021).
3. Task-level gains may coexist with learning losses (Bastani et al., 2025).
4. The studies reviewed use short horizons and specific tasks; none addresses AI systems substantially more capable than their human users.
5. Skill erosion under automation is an old observation (Bainbridge, 1983), but the encyclopedia has not located controlled long-run evidence for modern AI assistants. [VERIFICATION NEEDED: long-run skill-atrophy evidence for generative AI.]

**Unresolved questions (philosophical and speculative).**
- *Hypothetical.* **Preference deference.** Russell (2019) argues that AI should be built to be uncertain about human objectives. This is a design proposal, not a demonstrated solution; see [[SAFE-ALIGN-001]].
- *Contested.* **Whether oversight can scale.** If a system exceeds its overseers in the relevant domain, the empirical pattern that humans add value mainly when they outperform the AI (Vaccaro et al., 2024) suggests that oversight must rely on structure (audits, tooling, institutional checks) rather than case-by-case human judgement. Whether such structure suffices is disputed ([[SAFE-OVERSIGHT-001]]).
- *Contested.* **Distribution of power.** Bostrom (2014) discusses scenarios in which control over a decisive technology concentrates in a few actors. Whether advanced AI concentrates or diffuses power depends on compute access, open-weight availability and governance, and the question is unsettled ([[GOV-INTL-001]]).
- *Contested.* **Economic adaptation.** Whether labour markets and institutions adapt through reallocation, redistribution or neither is the subject of [[ECON-LABOR-001]]; field evidence on current tools does not settle the answer for much more capable systems.
- *Hypothetical.* **Human values in advanced systems.** Which values to encode, whose, and how to handle disagreement are open; fairness aspects of present systems are in [[SAFE-ETHICS-001]], and questions about the moral standing of AI systems themselves are in [[PHIL-MIND-001]].
- *Contested.* **Extended-mind framing.** Clark and Chalmers (1998) offer a philosophical framework under which AI tools could count as part of a person's cognition; philosophers dispute its scope and conditions.

## 8. Current State of Knowledge
As of 2026-10-15, and from the sources above:
- **Well Supported:** Automation bias exists and affects experts (Parasuraman & Manzey, 2010; Goddard et al., 2012). Generative AI assistance raised measured productivity in the specific professional tasks studied, with larger gains for less experienced workers (Noy & Zhang, 2023; Brynjolfsson et al., 2025).
- **Well Supported with bounds:** Human-AI combinations are not generally superior to the best single party; the pooled result depends on task type and relative performance (Vaccaro et al., 2024).
- **Preliminary:** Learning and offloading effects of AI assistants (Bastani et al., 2025; Sparrow et al., 2011), and interface interventions against over-reliance (Buçinca et al., 2021).
- **Contested:** Whether formal human-oversight requirements are effective safeguards in practice (Green, 2022).
- **Hypothetical:** All claims about coexistence with superintelligent systems ([[ASI-001]]), including the effectiveness of deference-based designs and the long-run distribution of power ([[PHIL-CIV-001]]).

Fast-moving figures (adoption rates, productivity estimates, model capabilities) are not tabulated here; see the Living Reference collections `MODEL-001`, `FACT-001`, `TIME-001` and `FCST-REG-001`.

## 9. Future Trajectory & Strategic Outlook
This article registers no forecasts. The following are observed trends and labelled scenarios, not predictions.

- **Observed trend (0-3y):** Deployment of AI assistants in knowledge work, with research shifting from "does it help" to "for whom, and at what cost to learning and judgement" (Bastani et al., 2025).
- **Hypothetical scenario (3-10y):** Regulatory and organisational requirements for human oversight meet systems that outperform their overseers on more tasks; the meta-analytic pattern (Vaccaro et al., 2024) suggests that oversight design would need to change from per-decision approval to audit and structural control. No evidence yet shows this works at scale.
- **Hypothetical scenario (10y+):** Arrangements in which humans and systems far beyond human performance share the world are discussed by Bostrom (2014) and in [[PHIL-CIV-001]]. These are philosophical models; their likelihood is not assessed here.

Any quantified claim about these horizons would require a full forecast record under the evidence framework (`FCST-<topic>-NNN`) and is deliberately absent.

## 10. Related Entries & Knowledge Graph
- **Foundational Concepts**:
  - [[ASI-001] Artificial Superintelligence](ASI-001_artificial_superintelligence.md): the hypothetical systems that raise the long-run coexistence questions.
  - [[SAFE-ALIGN-001] AI Alignment](../vol08_safety_governance/SAFE-ALIGN-001_ai_alignment.md): the problem of making systems pursue intended goals, including human values.
- **Related Methods & Technologies**:
  - [[SAFE-OVERSIGHT-001] Scalable Oversight and Corrigibility](../vol08_safety_governance/SAFE-OVERSIGHT-001_scalable_oversight_and_corrigibility.md): technical approaches to supervising hard-to-evaluate systems.
  - [[SAFE-ETHICS-001] AI Ethics, Bias and Fairness](../vol08_safety_governance/SAFE-ETHICS-001_ai_ethics_bias_and_fairness.md): fairness and values in present-day systems.
  - [[ECON-LABOR-001] AI, Productivity and Labor Markets](../vol09_economy/ECON-LABOR-001_ai_productivity_and_labor_markets.md): economic adaptation evidence and analysis.
  - [[GOV-INTL-001] International AI Governance and Compute Governance](../vol08_safety_governance/GOV-INTL-001_international_ai_governance_and_compute_governance.md): institutions bearing on the distribution of power.
- **Advanced Implications & Future Trajectories**:
  - [[PHIL-CIV-001] Civilization and Advanced Intelligence](../vol11_philosophy/PHIL-CIV-001_civilization_and_advanced_intelligence.md): long-run philosophical treatment of civilisation and advanced intelligence.
  - [[PHIL-MIND-001] Moral Status and Personhood of AI](../vol11_philosophy/PHIL-MIND-001_moral_status_and_personhood_of_ai.md): whether AI systems themselves have moral standing.

## 11. References & Foundational Literature
1. Bainbridge, L. (1983). Ironies of automation. *Automatica*, 19(6), 775-779. DOI: 10.1016/0005-1098(83)90046-8. [Supports the argument that automation leaves operators with hard residual tasks while eroding skill.]
2. Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. *Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems*, 1-16. DOI: 10.1145/3411764.3445717; arXiv:2006.14779. [Supports the finding that explanations increased acceptance of AI recommendations regardless of correctness.]
3. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakci, O., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. *Proceedings of the National Academy of Sciences*, 122(26). DOI: 10.1073/pnas.2422633122. [Supports the performance-versus-learning divergence in a field experiment.]
4. Bostrom, N. (2014). *Superintelligence: Paths, Dangers, Strategies*. Oxford University Press. ISBN 978-0-19-967811-2. [Supports the philosophical discussion of control and concentration of power under superintelligence scenarios.]
5. Brynjolfsson, E., Li, D., & Raymond, L. (2025). Generative AI at work. *The Quarterly Journal of Economics*, 140(2), 889-942. DOI: 10.1093/qje/qjae044. [Supports the customer-support field study results and heterogeneity by worker experience.]
6. Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. *Proceedings of the ACM on Human-Computer Interaction*, 5(CSCW1), 1-21. DOI: 10.1145/3449287. [Supports the effect of cognitive forcing on over-reliance and the user-preference trade-off.]
7. Clark, A., & Chalmers, D. (1998). The extended mind. *Analysis*, 58(1), 7-19. DOI: 10.1093/analys/58.1.7. [Supports the philosophical extended-mind thesis; title and venue confirmed, argument summarised from general knowledge.]
8. Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: a systematic review of frequency, effect mediators, and mitigators. *Journal of the American Medical Informatics Association*, 19(1), 121-127. DOI: 10.1136/amiajnl-2011-000089. [Supports the existence of automation bias in clinical decision support as a systematically reviewed phenomenon.]
9. Green, B. (2022). The flaws of policies requiring human oversight of government algorithms. *Computer Law & Security Review*, 45, 105681. DOI: 10.1016/j.clsr.2022.105681; arXiv:2109.05067. [Supports the survey of 41 oversight policies and the argument that people cannot perform the desired oversight functions.]
10. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. *Science*, 381(6654), 187-192. DOI: 10.1126/science.adh2586. [Supports the writing-task experiment results (40% time, 18% quality).]
11. Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. *Human Factors*, 52(3), 381-410. DOI: 10.1177/0018720810376055. [Supports the characterisation of automation complacency and bias.]
12. Russell, S. (2019). *Human Compatible: Artificial Intelligence and the Problem of Control*. Viking. ISBN 978-0-525-55861-3. [Supports the proposal that AI be uncertain about human preferences.]
13. Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: Cognitive consequences of having information at our fingertips. *Science*, 333(6043), 776-778. DOI: 10.1126/science.1207745. [Supports the finding on reduced recall when future access is expected.]
14. Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. *Nature Human Behaviour*, 8(12), 2293-2303. DOI: 10.1038/s41562-024-02024-1. [Supports the pooled finding that human-AI combinations underperform the best single party on average.]
15. European Parliament and Council. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 14 (Human oversight). https://eur-lex.europa.eu/eli/reg/2024/1689/oj. [Article 14(1) and (2), Official Journal text read 2026-10-18; supports the human-oversight requirement for high-risk systems.]

## 12. Evidential Classification Matrix
| Section / Claim | Evidence Category | Supporting Source | Confidence / Status |
| :--- | :--- | :--- | :--- |
| Automation bias exists, including among experts | Well Supported | Parasuraman & Manzey (2010); Goddard et al. (2012) | High |
| Generative AI raised productivity in studied tasks | Well Supported (task-bounded) | Noy & Zhang (2023); Brynjolfsson et al. (2025) | Moderate-High |
| Human-AI teams underperform best single party on average | Well Supported (one meta-analysis, 2020-2023 studies) | Vaccaro et al. (2024) | Moderate |
| Explanations raise acceptance regardless of correctness | Well Supported (user studies) | Bansal et al. (2021); Buçinca et al. (2021) | Moderate |
| AI use can impair later unaided performance | Preliminary | Bastani et al. (2025) | Moderate |
| Expected access reduces recall; relevance to AI assistants | Preliminary | Sparrow et al. (2011) | Low-Moderate |
| Human-oversight policies are ineffective safeguards | Contested | Green (2022) | Moderate |
| Oversight can scale beyond human ability to evaluate | Contested | Bostrom (2014); Vaccaro et al. (2024) (inference) | Low |
| AI uncertainty about preferences yields coexistence safety | Hypothetical | Russell (2019) | Low |
| Extended-mind view of AI tools | Contested (philosophical) | Clark & Chalmers (1998) | N/A |
| Concentration of power under superintelligence | Hypothetical | Bostrom (2014) | Low |
| EU AI Act oversight requirement | Established (legal text; mirror source) | Regulation (EU) 2024/1689, Art. 14 | Moderate pending check |

## 13. Revision History
- **v0.1.0** (2026-10-15): Initial draft. Citations checked against arXiv/Crossref/publisher pages where marked; not independently fact-checked.