Moderate Transformers Question 78 of 223

What role do residual connections and layer normalization play in transformers?

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

Train 70%Val 15%Test 15%

Fit on train, tune on val, report on test once.

Residuals add the block input to its output so gradients flow through deep stacks.

LLMs use tokens (not full words) because it helps them:

Transformers questions separate people

who only read docs from people who shipped.

Keep it short, concrete,

and tied to GenAI / LLM work.

Stable token IDs

Each piece maps to a number the network can learn.

Fits the model

Fixed pieces are what transformers expect as input.

Before you speak the answer, walk the interviewer through these steps:

1
Residuals add the block
input to its output so gradients flow through deep stacks.
2
Layer norm keeps activations
well scaled.
3
Together they make 20-plus
layer transformers trainable instead of collapsing.
4
Give an example
One tiny concrete case you can say aloud.
5
Common mistake
What juniors usually get wrong.
6
Close
When you pick this over the alternative.

Here's a short line you can speak, broken into clear beats:

Say this line

“Layer norm keeps activations well scaled.”

Break into beats

Layernormkeepsactivationswellscaled

Speaking order

2987408337471632900

Note: Adapt this scaffold to your own project — keep it under 60–90 seconds.

Residuals add the block input to its output so gradients flow through deep stacks. Layer norm keeps activations well scaled.