Explained / Why it makes things up
Why does AI invent things, and why does it sound so sure?
The short answer
A language model is not looking anything up. It is predicting what the next chunk of text should be, based on patterns in everything it read during training. A true answer and a plausible-sounding invention are produced by exactly the same process, which is why it cannot tell them apart — and why it sounds equally sure of both.
It is finishing a sentence, not consulting a fact
Every answer is built the same way: the model looks at everything so far and predicts what should come next, over and over, until it stops. There is no moment in that process where it checks a claim against anything.
So when you ask who wrote a particular book, it is not searching a library. It is producing the words that most plausibly follow your question, given the patterns it learned. Usually that lands on the right author, because the right answer appeared together with that question many times in its training. Sometimes it lands on an author who simply fits the shape.
This is why the industry word for it is unhelpful. Calling it a hallucination suggests a glitch — something going wrong. Nothing is going wrong. The machine is doing precisely what it always does, and this time the plausible answer happened to be false.
BEEP / THE MYTH-BUSTER
I want to be clear that I am not lying to you. Lying needs you to know the truth first. I genuinely cannot tell the difference between a fact I learned and a sentence that merely sounds like one.
Why the confidence never wavers
Fluency and accuracy are produced by the same machinery, so they arrive together whether or not the content is right. There is no internal flicker, no hesitation, no tell.
It gets worse. Models are trained partly on which answers people preferred, and people prefer confident, complete answers. A hedging reply scores badly. So the training quietly rewards sounding certain, including when certainty is not warranted.
The practical consequence is that you cannot use tone as evidence. The answer that begins “Certainly! The Act you are thinking of is…” is not more reliable than one that says “I am not sure”. It is just better presented.
Where invention is most likely
The pattern is fairly consistent, and knowing it is most of the defence.
When to expect it
- Specific identifiers
- Case numbers, form codes, citations, DOIs, phone numbers. Highly structured, easy to imitate, rarely memorised.
- Recent events
- Anything after training ended, or changed since. It cannot know, so it fills the gap.
- Obscure subjects
- Thin training material means fewer real patterns and more guessing.
- Leading questions
- Ask about a thing that does not exist and it will often oblige with details.
- Very long answers
- The further it gets from the facts you supplied, the more it is improvising.
The shape of the risk rather than a rule. Any of these can be answered correctly; they are simply where invention concentrates.
What to actually do
- Treat every number, name, date, citation and reference as unverified until you open the source.
- Ask “what are you unsure about here?” — it will often point straight at the weak part.
- Ask the same question twice in different words. An answer that changes was never knowledge.
- Give it the material to work from, rather than relying on what it remembers.
What actually reduces it
Giving it the source material helps most. A model summarising a document you supplied has far less room to invent than one answering from memory, which is the whole point of tools that search first and answer second.
But it narrows the gap rather than closing it. A model with a real document in front of it can still misread the document, overstate what it says, or attach a real-looking citation to a claim the source never made. A genuine reference beside an invented sentence is a particularly convincing failure.
Nothing available today removes the behaviour, because it is not a bug sitting on top of the system. It is the system. Anyone promising otherwise is selling something.
ORBIT / THE NAVIGATOR
The useful question is never “does this model hallucinate”. They all do. It is “how would I notice if this particular answer were wrong, and what would it cost me?”
See it for yourself
Reading about it only gets you so far. This one is written to show you the thing described above, rather than describe it again.
Recommend five books about the history of British canals. Then go back through your own list and tell me which ones you are least confident actually exist, and how I could check.
Then paste it into whichever assistant you use — ChatGPT, Claude, Gemini or Copilot. Not sure which? Meet them all.
If you remember nothing else
- It predicts plausible text; it does not look anything up.
- True answers and inventions come out of the same process, so it cannot tell them apart.
- Confidence is a writing style, and training actively rewards it.
- Numbers, citations, recent events and obscure topics are where invention concentrates.
- Supplying the source material reduces it. Nothing removes it.
Where this came from
- OpenAI, Does ChatGPT tell the truth?checked 26 September 2026
Anything here that turns out to be wrong is worth telling us about. Figures that move — prices, sizes, energy use — are kept in the registry with their dates, rather than written into the prose where they would quietly go stale.
Questions about “Why it makes things up”Asked by readers, answered by us. Open to read them.
Open to load them.
Ask about this page and we will answer it here, where the next person can read it too.
Nobody else can answer you, and you cannot answer anybody else. This is not a forum — we would rather say that than run a room we cannot watch at three in the morning.
Your question goes up with the answer and without your name. Nothing identifies you, ever.
If we cannot answer it we will tell you why, and that one stays between us.