GenAI and LLM interview questions

They scan the user input and the model output for hate, self-harm, sexual content, or PII.

The model may regurgitate copyrighted or private strings it saw in pretraining or fine-tuning.

They are a separate model or rules engine that scores text independently of the chat model.

Tighten prompts, cap history, retrieve fewer better chunks, and cache embeddings and repeated prefixes.

Larger models and longer chain-of-thought raise quality and delay the first token.

Offline scoring, embeddings for a corpus, and nightly reports can use batch APIs at lower price.

Use exponential backoff with jitter, queue work, and fall back to a smaller model or cached answer.

Chunk and retrieve, map-reduce summarize, or use a long-context model for the last mile only.

Models often use the start and end of a long prompt more than the middle.

They reduce aggressive chunking and can attend over a whole report.

Semantic chunking splits on headings, paragraphs, or embedding breakpoints so a chunk is a coherent idea.

You embed small child chunks for precise search, then return a larger parent section to the LLM.

Tiny chunks match precisely but lose surrounding meaning.

Keep a table with its header and nearby caption

Common methods are reciprocal rank fusion, weighted score mixes after normalization, or a learned reranker on the union.

Exact SKUs, error codes, names, and legal citations often fail in embedding space.

RRF combines ranked lists by summing 1 over k plus rank for each document across systems.

MMR reranks hits to balance relevance with diversity so near-duplicate chunks do not fill the prompt.

The context window is the hard cap on prompt plus completion tokens the model can see.

Streaming sends tokens as they are generated instead of waiting for the full completion.