Why AI makes things up, and how to check its answers
Updated
Sometimes an AI assistant answers confidently with something false: a wrong date, a court ruling that doesn't exist, an invented source. The usual term is "hallucination".
It isn't an occasional glitch: it follows from how these systems work. That is exactly why you can learn to protect yourself.
Why it happens
A language model produces the most plausible text after what has already been written. It doesn't consult an archive of verified facts and it doesn't check what it says.
When it knows a topic well, the plausible text usually matches the correct one. When the topic is rare, recent or very specific, the model can generate something that sounds right without being right, and it says it in the same tone.
Where it goes wrong most often
- Sources and quotations: article titles, court rulings, legal references and links can be invented.
- Numbers and dates: precise percentages, amounts and years.
- Little-known people and places: less training data, more invention.
- Recent events: the model only knows what was in its data up to a certain date, unless it has a search tool.
- Long calculations: use a calculator.
Warning signs
Be wary when the answer is more precise than your question allowed for, when it cites sources you can't find, when it changes its story if you repeat the question, or when it confidently confirms your own mistaken assumption.
A four-step method
- Ask for sources and actually open them: if the link doesn't exist or doesn't say what the AI claims, discard the information.
- Cross-check important facts against a second reliable source.
- Have the AI work on a text you provide, such as a document or an article, and ask it to point to the passages behind each claim.
- Repeat the question differently or in a new conversation: if the answers diverge, you've found a weak spot.
When AI isn't enough
For health, legal, tax or financial matters, use AI to understand and to prepare your questions, but confirm the decision with a professional or an official source. How much to check depends on what a mistake would cost, not on how convenient the answer is.
Fewer errors, not zero errors
Web-connected tools, documents you supply and precise requests reduce invention but don't eliminate it. The right attitude is to treat the answer as a draft to verify, not as a verdict.
Go deeper in the book
"LLMs Unpacked" explains how an answer is built, one token at a time, and why a model can write with confidence about things it does not know. The final chapter helps you read claims about new models without being dazzled.
See the book