MLOps interview questions and answers

Keep MLflow as the catalog and export the same version to Triton, ONNX Runtime, or a compiled custom server.

Short horizon: ops metrics, input drift, score distribution, feature freshness, and shadow diffs.

Log null rates and default-imputation counts per feature, alert on spikes, and compare live vectors to training quantiles.

Availability and p99 latency, plus data freshness, prediction coverage, and a quality SLO on a delayed metric with an error budget.

That is a closed-loop experiment: assignment must be sticky, and you analyze user-level outcomes, not request-level scores.

I would check training-serving skew, candidate generation differences, latency hurting UX, and whether the A/B splitter is broken.

Interleaving mixes results from two rankers in one list so users see both, which can need less traffic than a pure A/B.

A batch job would score known entities into a KV store

You cannot easily get exactly-once compute, so you make sinks idempotent with a prediction_id and model version as a primary key.

Use event-time watermarks, allowed lateness, and versioned re-aggregation windows rather than processing-time counts.

Prefer a shared feature platform over copy-pasted SQL, delete unused paths, and put ownership on each feature.

Code SHA, config, dataset snapshot, dependency lock, Docker digest, seed, and hardware notes from the tracking run.

Notebooks stay for EDA

Daily data stages update .dvc pointers on a data branch or via pipeline outputs in object storage tagged by date.

CI uses a sampled dataset with its own DVC file, while full data stays for cluster jobs.

Use ops and proxy metrics on the 1 percent, shadow the rest for prediction diffs, and run a longer parallel experiment on a statistically powered user holdout.

You monitored the service, not the decision quality or a leading proxy.

I would compile or vectorize inference, skip Python per-row overhead, cache hot entities, and shard by key across many CPU replicas close to the feature store.

Increase batch size or concurrent streams, use multi-model Triton instances, or bin-pack more replicas per GPU if memory allows.

End-to-end p99 is not the sum of p99s