Why Do AI Models Hallucinate?

The encyclopedia's full article is Hallucination, reliability and calibration. Terms such as calibration and retrieval-augmented generation are in the glossary.

#Quick reference

#What is an AI hallucination?

In the research literature, a hallucination is generated content that is nonsensical or unfaithful to its source. Ji and colleagues' survey in ACM Computing Surveys (2023) organizes earlier work into two types: intrinsic hallucinations, where the output contradicts the source (a summary that changes a 2019 approval date to 2021), and extrinsic hallucinations, where the output cannot be verified from the source at all.

For chat assistants, which answer open questions without a fixed source document, Huang and colleagues (2023) proposed a different split. Factuality hallucination is a conflict with verifiable real-world facts. Faithfulness hallucination is a departure from the user's instructions, from supplied context, or from the model's own earlier statements. Fabricated citations, invented software functions and made-up quotations are all factuality hallucinations.

Some authors prefer "confabulation", since "hallucination" borrows a term from perception. Farquhar and colleagues (2024) use "confabulation" for a narrower subset: answers that are both arbitrary and incorrect.

#Why do language models make things up?

A large language model is trained to predict the next piece of text (see how transformers work and large language models). Nothing in that objective checks whether a sentence is true. Three published explanations go further.

Some facts cannot be learned from the data. Kalai and Vempala (2023) prove that a model satisfying a statistical calibration condition must hallucinate on "arbitrary" facts, such as an individual's birthday, at a rate close to the fraction of such facts that appear exactly once in the training data. This holds even if the training data contain no errors. They also note that their analysis gives no statistical reason for hallucination on facts that appear many times.

Errors behave like classification mistakes. In "Why Language Models Hallucinate" (September 2025), Kalai, Nachum, Vempala and Zhang, three of them at OpenAI, relate generating valid text to a binary classification problem: deciding whether a given output is valid. If incorrect statements cannot be told apart from facts, then pretraining will produce errors through ordinary statistical pressure. They write that hallucinations "need not be mysterious".

Tests reward guessing. The same paper argues that hallucinations survive post-training because most evaluations are graded right or wrong. An answer of "I don't know" scores zero, so a model that guesses whenever it is unsure scores higher. The authors reviewed the grading of widely used benchmarks and found that most give no credit for expressing uncertainty. This argument is Preliminary: it is a preprint with a theoretical analysis, and how much each cause contributes in a given product has not been measured. Benchmark design more broadly is covered in the Turing test and AI benchmarks and AI evaluation.

#Are hallucinations inevitable?

Not in every sense. Kalai and colleagues (2025) dispute the claim that hallucinations are inevitable: a system that answers only from a fixed question-and-answer database and a calculator, and otherwise says "I don't know", would never hallucinate. Their lower bound applies to calibrated base models, and they show that a model which never errs on such facts cannot be calibrated in their sense. The practical question is the trade-off: abstaining more often reduces hallucination but also makes the system less useful. Whether general-purpose assistants can get hallucination close to zero without becoming unhelpful is Contested.

#What reduces hallucination?

Retrieval. Retrieval-augmented generation (RAG), introduced by Lewis and colleagues in 2020, has the model consult a document index (Wikipedia, in the original paper) before it writes. Retrieval helps only if the retrieved text is correct and the model uses it faithfully. Kalai and colleagues add that binary grading still rewards a guess whenever search fails to find a confident answer. See retrieval-augmented generation.

Calibration and uncertainty signals. A model is calibrated when its stated confidence matches how often it is right. Kadavath and colleagues at Anthropic (2022) reported that larger models were well calibrated on multiple-choice and true/false questions in a suitable format, and could be trained to predict whether they knew an answer. Farquhar and colleagues (Nature, June 2024) proposed semantic entropy: sample several answers, group them by meaning, and treat high disagreement as a warning sign of confabulation.

Abstention and scoring. Kalai and colleagues (2025) propose that benchmarks state a confidence target in the instructions: answer only if more than t confident, with wrong answers losing points, correct answers gaining one point and "I don't know" scoring zero. A threshold of 0.75, for example, means a wrong answer costs two points. The aim is to make admitting uncertainty the best strategy when a model is unsure.

Human checking. Where an error is costly, review against primary sources remains the reliable safeguard. This encyclopedia uses the same rule for its own text, which is why unverified claims carry visible markers.

#Have hallucinations caused real harm?

Yes. In Mata v. Avianca, a personal-injury case against the Colombian airline Avianca, US District Judge P. Kevin Castel in Manhattan fined two lawyers and their firm 5,000 US dollars in total on 22 June 2023, Reuters reported. Their brief had cited six fictitious cases generated by ChatGPT. Reuters quoted the judge as finding "acts of conscious avoidance and false and misleading statements to the court", and quoted the firm as saying it had made "a good faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth." Fabricated citations in court filings are the best-documented kind of harm. Rates for current named models vary by task and test, and are not given here.

#Frequently asked questions

#What is an AI hallucination in simple terms?

It is when an AI system states something false or unsupported in the same fluent, confident style it uses for true statements. Examples include invented citations, wrong dates and functions that do not exist in a software library.

#Do newer models still hallucinate?

Yes. Kalai and colleagues wrote in September 2025 that hallucinations persist even in state-of-the-art systems, though rates vary widely by model, task and grading method.

#Does retrieval-augmented generation stop hallucination?

It reduces it on knowledge-intensive tasks but does not stop it. The model can still misread or ignore retrieved text, and the retrieved text itself can be wrong.

#Why doesn't the model just say "I don't know"?

Training and evaluation often give no credit for abstaining, so guessing scores better on most benchmarks (Kalai et al., 2025). Changing how benchmarks score uncertain answers is one proposed fix.

#Is hallucination an alignment problem?

Partly. A model that states falsehoods it has no basis for is not doing what users intend, which links hallucination to the AI alignment problem.

#Sources

• Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., Fung, P.: Survey of Hallucination in Natural Language Generation (ACM Computing Surveys 55(12), 2023)
• Huang, L., Yu, W., Ma, W., et al.: A Survey on Hallucination in Large Language Models
• Kalai, A. T., Nachum, O., Vempala, S. S., Zhang, E.: Why Language Models Hallucinate
• Kalai, A. T., Vempala, S. S.: Calibrated Language Models Must Hallucinate
• Lewis, P., Perez, E., Piktus, A., et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Kadavath, S., Conerly, T., Askell, A., et al.: Language Models (Mostly) Know What They Know
• Farquhar, S., Kossen, J., Kuhn, L., Gal, Y.: Detecting hallucinations in large language models using semantic entropy (Nature 630, 2024)
• Merken, S. (Reuters): New York lawyers sanctioned for using fake ChatGPT cases in legal brief (22 June 2023, syndicated copy)

Status: draft, reviewed 26 October 2026 against the sources above. The Mata v. Avianca facts and quotations come from Reuters' report of the sanctions order, not from the court opinion itself. Hallucination rates and mitigation results change with each model generation, and the 2025 incentive argument has not yet been tested at scale.