Easy Attention Question 15 of 223

What is attention in a neural network?

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

The model does not read letters like humans. It reads these pieces, then predicts the next one.

Attention lets the model weight how much each token should influence another token.

LLMs use tokens (not full words) because it helps them:

Attention questions separate people

who only read docs from people who shipped.

Keep it short, concrete,

and tied to GenAI / LLM work.

Stable token IDs

Each piece maps to a number the network can learn.

Fits the model

Fixed pieces are what transformers expect as input.

Before the model can read a sentence, it goes through these steps:

1
Attention lets the model
weight how much each token should influence another token.
2
Token IDs
Important context gets high weight
3
less useful tokens get
low weight.
4
It is the mechanism
that lets transformers mix information across the sequence.
5
Next token
The model scores what should come next.
6
Decode
IDs turn back into readable text.

Let's see how a real sentence is tokenized (tokens may vary by model):

Input text

“Important context gets high weight”

Tokenized output

Importantcontextgetshighweight

Token IDs (example)

298740833747163290

Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).

Attention lets the model weight how much each token should influence another token. Important context gets high weight