GenAI and LLM interview questions

Prompt caching reuses computation for a repeated prefix such as a long system prompt or static RAG header.

The context window is the maximum number of tokens the model can consider at once, including prompt and usually the completion.

The call fails, or your client must drop history, summarize, or retrieve fewer chunks.

Embedding a whole book as one vector blurs topics and may exceed model limits.

Overlap copies a few tokens or sentences from the end of one chunk into the start of the next.

Hybrid search combines lexical matching such as BM25 with dense vector search.

BM25 is a ranking function for keyword search that scores term frequency with document-length normalization.

BOS marks the start of a sequence, EOS marks the end of generation, and PAD fills batches to equal length.

Often yes with a symmetric embedder, but some models are asymmetric: one tower for queries and one for passages.

A jailbreak is a prompt that tries to bypass safety policy, for example by role-play or hypothetical framing.

An LLM is a neural net trained on huge text to predict the next token.

It is writing clear instructions, context, and examples so the model does the task you want.

Temperature controls randomness.

A piece of text the model reads — often a word piece.

The instruction and context you send the model.

Randomness in token sampling.

A numeric fingerprint of meaning so similar text sits nearby.