How can mix shifts and Goodhart's law make a KPI look healthy while the business is not?
PICTURE THIS: HOW TO EXPLAIN IT
Simple meaning
If you reward a metric, teams will optimize the number rather than the underlying value, which is Goodhart's law.
WHY — Business Metrics instead of guessing?
Why interviewers care about Business Metrics:
question about Business Metrics.
trade-offs, and what you would actually do on a Data Science project - not buzzwords.
Name the idea, why it exists, then one short example.
End with when you use it and one common pitfall.
STEPS — What happens step by step?
Before you speak the answer, walk the interviewer through these steps:
- 1If you reward a
metric, teams will optimize the number rather than the underlying value, which is Goodhart's law.
- 2Mix shifts can raise
overall conversion while every true segment is flat, which is Simpson's paradox in KPI clothing.
- 3Pair the headline metric
with guardrails, cohort views, and a causal experiment when the decision is large.
- 4Give an example
One tiny concrete case you can say aloud.
- 5Common mistake
What juniors usually get wrong.
- 6Close
When you pick this over the alternative.
EXAMPLE — See it in action
Here's a short line you can speak, broken into clear beats:
Note: Adapt this scaffold to your own project — keep it under 60–90 seconds.
Key takeaway
If you reward a metric, teams will optimize the number rather than the underlying value, which is Goodhart's law. Mix shifts can raise overall conversion while every true segment is flat, which is Simpson's paradox in KPI clothing.