Saying Untrue Things Plausibly
A hallucination is when an LLM states something untrue in plausible sentences. Typical examples are titles of books that do not exist, wrong numbers, and quotes of things nobody said. Because the sentences read naturally, readers find it hard to notice.
Anthropic's guide also says that even capable models sometimes write text that is factually wrong or inconsistent with the documents they were given. Switching models does not bring it to zero.
Why It Happens
An LLM writes by choosing words that are likely to come next in the text. It does not check whether they are correct as it writes. The less a topic appears in the training data, the more easily the likely words drift from the facts.
A 2025 paper argues that hallucinations persist because training and evaluation make writing a guess more rewarding than answering that it does not know. Most evaluations are graded so that a wild guess earns more points than leaving the answer blank.
When It Happens and What to Do
It tends to happen with proper names, numbers, dates, quotes, events after the knowledge cutoff, and specialized topics with few sources. The countermeasures are as follows.
- Pass in documents (use RAG to have the model read your documents and answer from them)
- Tell the model it may say so when it does not know
- Have it quote the basis for its answer directly from the documents and show the sources
- Have a person check the important parts against the original documents
Anthropic's guide says that just allowing the model to say it does not know can greatly reduce errors. On topics where errors matter a lot, such as law, medicine, and tax, check the answer with an expert.