Easy Temperature Question 41 of 223

What is temperature in LLM sampling?

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

PICTURE THIS: A SENTENCE BECOMES TOKENS

The model does not read letters like humans. It reads these pieces, then predicts the next one.

Simple meaning

Temperature rescales the logits before the token distribution is sampled.

1

WHY — Tokens instead of words?

LLMs use tokens (not full words) because it helps them:

Temperature questions separate people

who only read docs from people who shipped.

Keep it short, concrete,

and tied to GenAI / LLM work.

Stable token IDs

Each piece maps to a number the network can learn.

Fits the model

Fixed pieces are what transformers expect as input.

2

STEPS — What happens with tokens?

Before the model can read a sentence, it goes through these steps:

  1. 1
    Temperature rescales the logits

    before the token distribution is sampled.

  2. 2
    Higher temperature flattens the

    distribution and increases variety.

  3. 3
    Lower temperature sharpens it

    toward the most likely tokens.

  4. 4
    Context mix

    Attention looks at nearby tokens together.

  5. 5
    Next token

    The model scores what should come next.

  6. 6
    Decode

    IDs turn back into readable text.

3

EXAMPLE — See it in action

Let's see how a real sentence is tokenized (tokens may vary by model):

Input text
“Higher temperature flattens the distribution and increases variety.”
Tokenized output
Highertemperatureflattensthedistributionand
Token IDs (example)
2987408337471632900

Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).

Key takeaway

Temperature rescales the logits before the token distribution is sampled. Higher temperature flattens the distribution and increases variety.

Previous question Next question
Chat with us