High Chunking Question 162 of 223

How do hierarchical or summary indexes work in RAG?

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

PICTURE THIS: DATABASE INDEX

Without indexScan every row

With indexJump to keys

CostWrites slower

Simple meaning

You store summaries of sections or documents, retrieve at the coarse level, then drill into child chunks.

WHY — Tokens instead of words?

LLMs use tokens (not full words) because it helps them:

This is a process

question about Chunking.

Panels listen for order,

trade-offs, and what you would actually do on a GenAI / LLM project - not buzzwords.

Stable token IDs

Each piece maps to a number the network can learn.

Fits the model

Fixed pieces are what transformers expect as input.

STEPS — What happens step by step?

Before you speak the answer, walk the interviewer through these steps:

1
You store summaries of
sections or documents, retrieve at the coarse level, then drill into child chunks.
2
That supports both 'overview
this repo' and 'find this function.' Stale summaries after edits are a silent failure mode.
3
Close with when you
would choose this approach on a real task.
4
Give an example
One tiny concrete case you can say aloud.
5
Common mistake
What juniors usually get wrong.
6
Close
When you pick this over the alternative.

EXAMPLE — See it in action

Here's a short line you can speak, broken into clear beats:

Say this line

“That supports both 'overview this repo' and 'find this function.' Stale summarie”

Break into beats

Thatsupportsboth'overviewthisrepo'

Speaking order

2987408337471632900

Note: Adapt this scaffold to your own project — keep it under 60–90 seconds.

Key takeaway

You store summaries of sections or documents, retrieve at the coarse level, then drill into child chunks. That supports both 'overview this repo' and 'find this function.' Stale summaries after edits are a silent failure mode.