What is temperature in an LLM API?
PICTURE THIS: AN LLM TURN
Text inTokens
TransformerAttention
Text outNext token
Simple meaning
Temperature controls randomness.
WHY — Tokens instead of words?
LLMs use tokens (not full words) because it helps them:
LLM questions separate people
who only read docs from people who shipped.
Keep it short, concrete,
and tied to GenAI / LLM work.
Stable token IDs
Each piece maps to a number the network can learn.
Fits the model
Fixed pieces are what transformers expect as input.
STEPS — What happens with tokens?
Before the model can read a sentence, it goes through these steps:
- 1Tokenization
Temperature controls randomness.
- 2Low values make output
more deterministic
- 3Embeddings
high values more creative.
- 4For code and facts
I keep temperature low.
- 5Next token
The model scores what should come next.
- 6Decode
IDs turn back into readable text.
EXAMPLE — See it in action
Let's see how a real sentence is tokenized (tokens may vary by model):
Input text
“Low values make output more deterministic”
Tokenized output
Lowvaluesmakeoutputmoredeterministic
Token IDs (example)
2987408337471632900
Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).
Key takeaway
Temperature controls randomness. Low values make output more deterministic