GenAI and LLM interview questions

FlashAttention computes attention in GPU SRAM tiles so it avoids materializing the full n-by-n matrix in HBM.

They share key and value heads across many query heads, shrinking the KV cache.

Full attention has a strong empirical quality record and simple associative recall over the whole context.

Vision or audio tokens are projected into the same residual stream and attend with text tokens.

MoE routes each token to a few expert FFNs instead of one dense MLP.

A large teacher generates targets or preferences that a smaller student learns.

Weights use fewer bits, so more of the model fits in GPU memory and memory bandwidth drops.

Retrieval still works because embedders have their own tokenizer, but chunk token budgets for the prompt must use the generator's tokenizer.

Some languages spend far more tokens per word, so the same UX hits the context wall sooner and costs more.

Matryoshka models are trained so truncated prefixes of the vector remain useful.

Fine-tune on query-passage pairs from your search logs or synthetic pairs from a teacher LLM.

ColBERT keeps token-level embeddings and scores a query against a document with MaxSim over tokens.

Rotary position embeddings encode relative position in the attention product.

Models often park large attention mass on the first few tokens.

PagedAttention, as in vLLM, stores the KV cache in non-contiguous pages like virtual memory.

Score retrieval (recall, MRR), then generation (faithfulness, context precision, answer relevancy).

Agentic RAG lets the model rewrite queries, retrieve again, compare sources, or call tools before answering.

When answers depend on entities and relations across many documents, a graph can traverse links that nearest-neighbor search never joins.

Decompose the question, retrieve per sub-question, and condition later queries on intermediate facts.

Require the model to quote spans, then check that each quote is a substring of a retrieved chunk.