What is Hallucination?
Also called Confabulation, AI Hallucination.
A hallucination is output from a language model that is fluent and confident but factually wrong or unsupported by any source. It arises because such models predict likely continuations of text rather than retrieving verified facts, so plausibility and truth can diverge. Common forms include invented citations, fabricated figures, and imagined product features or API methods.
The mechanism is statistical. A model learns which token sequences are probable given the preceding context, and a fabricated citation in the correct format is often more probable than an admission of uncertainty. Nothing in the generation process distinguishes a remembered fact from a well-formed guess, which is why hallucinated content usually reads as confidently as accurate content.
Some researchers argue that hallucination is inherent to the approach and cannot be fully eliminated, only reduced. Others argue that better grounding, calibration, and training on when to abstain can push the rate very low for bounded tasks. Both camps agree the rate is task-dependent, rising sharply for obscure facts, precise numbers, recent events, and long chains of reasoning.
Mitigations work by reducing the model's need to guess. Retrieval augmentation supplies source documents and asks for answers grounded in them. Tool use replaces recalled arithmetic or lookups with actual computation and queries. Citation requirements make claims checkable. Abstention training encourages the model to say it does not know rather than fill the gap.
Detection is a separate discipline. Approaches include checking every claim against retrieved sources, sampling several answers and measuring their agreement, using a second model as a verifier, and constraining output to a schema whose values can be validated. None is perfect, so high-stakes outputs still warrant a human reader who can check the underlying source.
Key points
- Fluent, confident output that is factually wrong or unsupported
- Caused by next-token prediction, not by a lookup failure
- Rate rises for obscure facts, exact numbers, and recent events
- Reduced by retrieval grounding, tool use, and abstention
- Whether it can be eliminated entirely is contested
In practice
A research assistant is asked for peer-reviewed studies on a niche topic and returns five references with authors, journals, years, and page numbers. Three are real. Two do not exist, though their author names are real researchers in the field and the journal titles are correct. The formatting is identical across all five, so nothing in the output signals which entries were fabricated.