Easy Transformers Question 13 of 223

What is the difference between an encoder and a decoder in transformers?

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

PICTURE THIS: A SENTENCE BECOMES TOKENS

The model does not read letters like humans. It reads these pieces, then predicts the next one.

Simple meaning

An encoder builds bidirectional representations of the full input.

WHY — Transformers instead of guessing?

Why interviewers care about Transformers:

They want a clean

contrast on Transformers, not two memorised paragraphs.

Say what changes for

the developer, then one case where picking wrong hurts.

Stay structured

Name the idea, why it exists, then one short example.

Close cleanly

End with when you use it and one common pitfall.

STEPS — What happens with tokens?

Before the model can read a sentence, it goes through these steps:

1
An encoder builds bidirectional
representations of the full input.
2
A decoder generates tokens
left to right and can only see past tokens plus optional encoder output.
3
Embeddings
BERT is encoder-style
4
Context mix
GPT is decoder-style
5
Next token
T5 uses both.
6
Decode
IDs turn back into readable text.

EXAMPLE — See it in action

Let's see how a real sentence is tokenized (tokens may vary by model):

Input text

“A decoder generates tokens left to right and can only see past tokens plus optio”

Tokenized output

Adecodergeneratestokensleftto

Token IDs (example)

2987408337471632900

Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).

Key takeaway

An encoder builds bidirectional representations of the full input. A decoder generates tokens left to right and can only see past tokens plus optional encoder output.