GenAI and LLM interview questions

Prompt caching reuses computation for a repeated prefix such as a long system prompt or static RAG header.

The context window is the maximum number of tokens the model can consider at once, including prompt and usually the completion.

The call fails, or your client must drop history, summarize, or retrieve fewer chunks.

Embedding a whole book as one vector blurs topics and may exceed model limits.

Overlap copies a few tokens or sentences from the end of one chunk into the start of the next.

Hybrid search combines lexical matching such as BM25 with dense vector search.

BM25 is a ranking function for keyword search that scores term frequency with document-length normalization.

BOS marks the start of a sequence, EOS marks the end of generation, and PAD fills batches to equal length.

Often yes with a symmetric embedder, but some models are asymmetric: one tower for queries and one for passages.

A jailbreak is a prompt that tries to bypass safety policy, for example by role-play or hypothetical framing.

A fixed word vocabulary would map rares to unknown and explode in size.

Many BPE vocabularies treat capitalization and leading spaces as part of the token.

Use the same tokenizer the model uses, such as tiktoken for many OpenAI models.

Higher dimensions can hold more nuance but cost more storage, RAM, and distance compute.

Dense vectors are short learned embeddings where every dimension is used.

After unit normalization, inner product equals cosine similarity, which simplifies indexes and scores.

A bi-encoder embeds query and document separately so documents can be precomputed.

Residuals add the block input to its output so gradients flow through deep stacks.

After attention mixes information across tokens, a position-wise MLP transforms each token independently.

Encoder-decoder models such as T5 shine at transduction: translate, summarize, or map a full input to a structured output.