Moderate RAG Question 90 of 223

Walk through a typical RAG pipeline from query to answer.

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

PICTURE THIS: AN LLM TURN

Text inTokens

TransformerAttention

Text outNext token

Simple meaning

The query may be rewritten, then hybrid search returns top chunks, optionally reranked.

WHY — Tokens instead of words?

LLMs use tokens (not full words) because it helps them:

RAG questions separate people

who only read docs from people who shipped.

Keep it short, concrete,

and tied to GenAI / LLM work.

Stable token IDs

Each piece maps to a number the network can learn.

Fits the model

Fixed pieces are what transformers expect as input.

STEPS — What happens with tokens?

Before the model can read a sentence, it goes through these steps:

1
The query may be
rewritten, then hybrid search returns top chunks, optionally reranked.
2
Selected text is packed
into a prompt with instructions to cite and not invent.
3
The LLM generates an
answer, and you log retrieval IDs for evals.
4
Context mix
Attention looks at nearby tokens together.
5
Next token
The model scores what should come next.
6
Decode
IDs turn back into readable text.

EXAMPLE — See it in action

Let's see how a real sentence is tokenized (tokens may vary by model):

Input text

“Selected text is packed into a prompt with instructions to cite and not invent.”

Tokenized output

Selectedtextispackedintoa

Token IDs (example)

2987408337471632900

Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).

Key takeaway

The query may be rewritten, then hybrid search returns top chunks, optionally reranked. Selected text is packed into a prompt with instructions to cite and not invent.