What is the difference between input tokens and output tokens?
PICTURE THIS: A SENTENCE BECOMES TOKENS
The model does not read letters like humans. It reads these pieces, then predicts the next one.
Simple meaning
Input tokens are the prompt, system message, history, and any retrieved context you send in.
WHY — Tokens instead of words?
LLMs use tokens (not full words) because it helps them:
contrast on Tokens, not two memorised paragraphs.
the developer, then one case where picking wrong hurts.
Each piece maps to a number the network can learn.
Fixed pieces are what transformers expect as input.
STEPS — What happens with tokens?
Before the model can read a sentence, it goes through these steps:
- 1Input tokens are the
prompt, system message, history, and any retrieved context you send in.
- 2Output tokens are the
completion the model generates.
- 3APIs often price them
differently, and output tokens usually dominate latency because they are produced one step at a time.
- 4Context mix
Attention looks at nearby tokens together.
- 5Next token
The model scores what should come next.
- 6Decode
IDs turn back into readable text.
EXAMPLE — See it in action
Let's see how a real sentence is tokenized (tokens may vary by model):
Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).
Key takeaway
Input tokens are the prompt, system message, history, and any retrieved context you send in. Output tokens are the completion the model generates.