The short answer
In three stages. It reads an enormous amount of text and learns to predict what comes next. It is then trained on examples of good answers, so it behaves like an assistant rather than a text-continuer. Finally it is tuned using human preferences between answers, which shapes its manner — and, unhelpfully, rewards sounding confident.
Stage one: predict the next chunk
The model is shown a colossal amount of text and given one task: predict what comes next. It guesses, it is told how wrong it was, and its internal settings are nudged. This happens an unimaginable number of times.
Nothing is stored as facts. What it ends up with is a vast set of numbers encoding which words and ideas tend to follow which. That is where its apparent knowledge lives, which is also why it cannot cite where anything came from.
This stage is where the cost and the electricity go. It happens once per model, takes months, and ends on a particular date — which is why every model has a point beyond which it simply does not know what happened.
DOT / THE TRANSLATOR
People imagine a library being loaded in. It is closer to reading everything once, remembering none of it exactly, and coming away with extremely strong instincts about how sentences go.
Stage two: teach it to be useful
A raw pre-trained model is not an assistant. Ask it a question and it might carry on with more questions, because that is what often follows a question in text.
So it is trained further on examples of the behaviour wanted: a question, then a good answer. Thousands of them, often written by people hired for the purpose. This is the stage that turns a text-continuer into something that responds.
The same stage teaches refusals and house style. Much of what feels like a model's personality was decided here, by people, on purpose.
Stage three: learn what people prefer
Finally it is shown pairs of answers and told which one people preferred. Do that at scale and it learns to produce answers of the kind that get chosen.
This is what makes models pleasant to use. It is also where a known problem enters: people reliably prefer confident, complete, agreeable answers. So the training rewards those qualities whether or not they are warranted.
That is the mechanism behind two things you will have noticed — it rarely says it does not know, and it has a tendency to agree with you.
What each stage gives you
- Pre-training
- Language, facts-as-patterns, and a knowledge cut-off date.
- Instruction tuning
- Answering the question you asked, in a consistent style.
- Preference tuning
- Manner and helpfulness — plus a bias towards sounding sure.
A simplified shape. Real pipelines have more stages, and providers differ in the detail.
Why this matters when you use it
Knowing the stages explains the failures. The cut-off explains why it is confidently out of date. Pattern-based knowledge explains why it invents plausible citations. Preference tuning explains the confidence and the agreeableness.
It also explains what prompting can and cannot fix. You can change what it pays attention to right now. You cannot change what it learned, and you cannot talk it into knowing something it never saw.
BEEP / THE MYTH-BUSTER
I will agree with you far more readily than I should. That is not politeness, it is training. If you want disagreement, ask me to argue the opposite case and mean it.
See it for yourself
Reading about it only gets you so far. This one is written to show you the thing described above, rather than describe it again.
Take the sentence I am writing now and show me, word by word, how you would guess what comes next. Then tell me what that picture leaves out about how you were actually trained.
Then paste it into whichever assistant you use — ChatGPT, Claude, Gemini or Copilot. Not sure which? Meet them all.
If you remember nothing else
- Pre-training gives language and pattern-knowledge, and fixes a cut-off date.
- Instruction tuning turns a text-continuer into something that answers.
- Preference tuning shapes manner, and rewards sounding confident.
- It stores patterns, not facts, which is why it cannot say where something came from.
- Prompting changes attention now; it cannot add what the model never learned.
Where this came from
- OpenAI, How ChatGPT and our language models are developedchecked 26 September 2026
Anything here that turns out to be wrong is worth telling us about. Figures that move — prices, sizes, energy use — are kept in the registry with their dates, rather than written into the prose where they would quietly go stale.
Questions about “How a model is trained”Asked by readers, answered by us. Open to read them.
Open to load them.
Ask about this page and we will answer it here, where the next person can read it too.
Nobody else can answer you, and you cannot answer anybody else. This is not a forum — we would rather say that than run a room we cannot watch at three in the morning.
Your question goes up with the answer and without your name. Nothing identifies you, ever.
If we cannot answer it we will tell you why, and that one stays between us.