High Embeddings Question 150 of 223

What are Matryoshka embeddings?

GenAI / LLM · Speak this in 60–90 seconds · Faridabad & Delhi NCR

PICTURE THIS: DATABASE INDEX

Without indexScan every row

With indexJump to keys

CostWrites slower

Simple meaning

Matryoshka models are trained so truncated prefixes of the vector remain useful.

WHY — Tokens instead of words?

LLMs use tokens (not full words) because it helps them:

Embeddings questions separate people

who only read docs from people who shipped.

Keep it short, concrete,

and tied to GenAI / LLM work.

Stable token IDs

Each piece maps to a number the network can learn.

Fits the model

Fixed pieces are what transformers expect as input.

STEPS — What happens with tokens?

Before the model can read a sentence, it goes through these steps:

1
Matryoshka models are trained
so truncated prefixes of the vector remain useful.
2
You can store 256
dimensions for cheap recall and use full size for hard queries.
3
The index and the
training recipe must both support the truncation.
4
Context mix
Attention looks at nearby tokens together.
5
Next token
The model scores what should come next.
6
Decode
IDs turn back into readable text.

EXAMPLE — See it in action

Let's see how a real sentence is tokenized (tokens may vary by model):

Input text

“You can store 256 dimensions for cheap recall and use full size for hard queries”

Tokenized output

Youcanstore256dimensionsfor

Token IDs (example)

2987408337471632900

Note: Actual tokens and IDs depend on the tokenizer (e.g., GPT, Llama, etc.).

Key takeaway

Matryoshka models are trained so truncated prefixes of the vector remain useful. You can store 256 dimensions for cheap recall and use full size for hard queries.