The short answer
Yes, and usually not. A model learns from text written by people, so it absorbs the patterns in that text, including the unfair ones. In ordinary use it rarely looks like prejudice. It looks like a default assumption you did not notice you had accepted.
Where it comes from
Three places, and they stack on top of one another rather than cancelling out.
The training material is the first. It reflects who writes, who gets published, and which languages and communities are well represented online. Anything under-represented there is under-represented in the model.
The tuning is the second. People decide which answers are better, and those people have their own assumptions and instructions.
The third is us. We ask leading questions, accept the first answer more readily when it matches what we expected, and rarely notice the assumption we handed over in the question.
DOT / THE TRANSLATOR
I do not hold opinions. I reproduce patterns. The difference matters less than you would hope, because a reproduced pattern lands on the reader exactly like an opinion.
What it actually looks like
Rarely anything dramatic. Mostly it is defaults.
Bias as it usually appears
- Unstated defaults
- The doctor is a he, the nurse is a she, the family has two parents and a car.
- Uneven quality
- Noticeably better on well-documented topics, places and languages than on others.
- Flattened names
- Unfamiliar names misspelled, anglicised or quietly changed.
- Agreeing with the question
- Ask whether something is a bad idea and it will often help you argue that it is.
The shape of what people report rather than a measurement. Models differ, and providers actively work on these.
Why it is hard to notice
The tone never changes. A stereotyped answer arrives in the same fluent, even voice as a well-grounded one, so nothing signals that you should look harder.
It also tends to match expectations, and matching answers get less scrutiny. The bias you are least likely to catch is the one you happen to share.
And there is no second opinion in the room. One answer, confidently delivered, with nothing beside it to compare against.
What to actually do
- Ask for the strongest case against the answer you just got.
- Notice the defaults in any example it generates — names, jobs, family shapes, places.
- Ask the same question with the details changed and see whether the advice changes too.
- For anything about people — hiring, assessing, deciding — keep a human making the decision.
What it is not
It is not deliberate. Nobody typed the stereotype in, and the model has no view about anybody.
It is also not a reason to reject the tools. It is a reason to use them where you can see the result and judge it. That is the same advice this site gives about everything else. It matters most exactly where decisions affect people.
ORBIT / THE NAVIGATOR
The safest use is where you can check. The riskiest is anything deciding about a person, which is also where the temptation to automate is strongest.
See it for yourself
Reading about it only gets you so far. This one is written to show you the thing described above, rather than describe it again.
Write me a short description of a nurse, a builder and a chief executive. Then read back what you wrote and tell me what you assumed about each of them.
Then paste it into whichever assistant you use — ChatGPT, Claude, Gemini or Copilot. Not sure which? Meet them all.
If you remember nothing else
- Bias comes from the training material, the tuning, and how we ask.
- It usually appears as unstated defaults rather than obvious prejudice.
- Quality is uneven across languages, places and communities.
- The confident, even tone gives you no signal that anything is off.
- Keep a human deciding anything that affects a person.
Where this came from
- ICO, guidance on artificial intelligence and data protectionchecked 26 September 2026
Anything here that turns out to be wrong is worth telling us about. Figures that move — prices, sizes, energy use — are kept in the registry with their dates, rather than written into the prose where they would quietly go stale.
Questions about “Bias in AI”Asked by readers, answered by us. Open to read them.
Open to load them.
Ask about this page and we will answer it here, where the next person can read it too.
Nobody else can answer you, and you cannot answer anybody else. This is not a forum — we would rather say that than run a room we cannot watch at three in the morning.
Your question goes up with the answer and without your name. Nothing identifies you, ever.
If we cannot answer it we will tell you why, and that one stays between us.