GenAI and LLM interview questions

If test items leaked into pretraining or fine-tuning, scores look like generalization when they are memorization.

Humans set the rubric and calibrate judges on a gold subset.

Randomize at the session, log prompt version, and measure a north-star such as task success plus cost and latency.

An attacker poisons a wiki, email, or web page that your retriever will fetch.

System and developer rules outrank user text, which outranks retrieved content.

Systematically try jailbreaks, injection in docs, PII exfil via tools, and long-context smuggling.

The model critiques and revises its own answers against a written constitution of principles, reducing some human preference labeling.

Define allowed domains, disallowed assistance, and escalation paths in writing.

Statistical watermarks and C2PA-style metadata try to mark machine text, but they are brittle to paraphrase and not universally deployed.

Factuality is correctness versus the real world.

Floating-point math, batching, and mixture-of-experts routing can change logits across runs.

Token probabilities are not well-calibrated probabilities of truth.

You sample multiple chain-of-thought paths and majority-vote the final answer.

A cheap draft model proposes several tokens

Put the large static system prompt, tool schemas, and stable few-shot block first.

A classifier or small model handles easy intents

Cache embeddings for documents and repeated queries

New requests join an in-flight batch at token boundaries instead of waiting for a full batch to drain.

If the task is strictly transduction with a bounded output, such as normalizing records, a T5-class model can be cheaper and more controllable.

Open-weight chat models expect a specific mix of special tokens and role markers from training.