What is hallucination in an LLM?
PICTURE THIS: A SENTENCE BECOMES TOKENS
The model does not read letters like humans. It reads these pieces, then predicts the next one.
Simple meaning
Hallucination is fluent text that is false, unsupported, or fabricated.
WHY — Tokens instead of words?
LLMs use tokens (not full words) because it helps them:
who only read docs from people who shipped.
and tied to GenAI / LLM work.
Each piece maps to a number the network can learn.
Fixed pieces are what transformers expect as input.
STEPS — What happens with tokens?
Before the model can read a sentence, it goes through these steps:
- 1Hallucination is fluent text
that is false, unsupported, or fabricated.
- 2The model is optimizing
next-token likelihood, not a truth database.
- 3Users still read it
as confident fact, which is the product risk.
- 4Context mix
Attention looks at nearby tokens together.
- 5Next token
The model scores what should come next.
- 6Decode
IDs turn back into readable text.
EXAMPLE — See it in action
Let's see how a real sentence is tokenized (tokens may vary by model):
Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).
Key takeaway
Hallucination is fluent text that is false, unsupported, or fabricated. The model is optimizing next-token likelihood, not a truth database.