In this article
Before you start
This page is one question inside the LLM evaluation guide for AI and machine learning (ML) interviews; read it for the wider map of tests, benchmarks and judges.
Why interviewers ask this
A team that ships on the wrong metric ships the wrong model, and the mistake often stays hidden until users complain. Interviewers ask which metrics matter and which mislead because the answer shows whether you treat a number as evidence or as a score to maximize. The question is reported in interviews at AI labs and large tech companies.
A weak answer lists metric names: accuracy, F-score, perplexity, win rate. A strong answer says what each number measures, the data condition under which it lies, and what you would use instead. A Staff-level answer starts from the decision the number drives and describes the check that would reveal a metric gone bad. That check is the part most candidates skip, and the rest of this page builds it.
Evaluation metrics from first principles
- Balanced accuracy
- Balanced accuracy is the average of the recall of each class, so every class counts equally however rare it is. On balanced data it equals plain accuracy, and for a classifier that always names the majority class it falls to one divided by the number of classes.
- Goodhart's law
- Goodhart's law is the observation that a measure used as a target stops being a reliable measure, because people and optimizers push on the number instead of on the goal it stood for.
In plain words
Think of a step counter on your wrist. Steps stand for exercise. If you shake your wrist, the count rises and the exercise does not. The analogy breaks in one place: nobody has to game the model's number on purpose, because an optimizer pushes it up automatically.
In one precise sentence, every evaluation metric is a proxy metric, and a good answer names when the proxy and the goal come apart.
Step 1: Ask what decision the number will drive
A metric matters only if a named decision changes when its value changes. Before you pick any number, answer three questions. Which decision will it change: ship, roll back, or choose between two models? Which mistake costs more, a missed positive or a false alarm? Has the number been checked against human judgment on a sample?
The third question is the one that exposes a bad metric. Label a small sample by hand, keep it fixed, and see whether the metric ranks examples the way people do; that sample is a golden set, covered in golden sets and regression testing for LLM apps. If the metric and the people disagree, the metric is not ready to drive a decision. The wider design of the whole test is in how would you design an eval for X. You now have the filter; the next four steps are the common ways a number fails it.
Step 2: Plain accuracy misleads on imbalanced data
Plain accuracy misleads whenever one class is much rarer than the other, because a model that always names the common class still scores high. Take 1,000 items with 950 negatives and 50 positives (illustrative). A model that always answers negative is right on 950 of them, so its accuracy is 0.95, yet it finds no positive: its recall is 1.0 on negatives and 0.0 on positives.1
Balanced accuracy is built to avoid this inflation, and the scikit-learn reference defines it as the average of recall on each class. Here it is 0.5, which is one divided by two classes.1 The reference's own six-item example returns 0.625; its plain accuracy is 0.6667 and its recalls are 0.75 and 0.5 (our arithmetic).1
| Quantity | Always-negative model (illustrative) | scikit-learn example |
|---|---|---|
| Accuracy | 0.95 | 0.6667 |
| Recall, positive class | 0.0 | 0.5 |
| Recall, negative class | 1.0 | 0.75 |
| Balanced accuracy | 0.5 | 0.625 |
Balanced accuracy is not a universal fix. If a missed positive costs far more than a false alarm, report recall and precision, the share of the model's positive calls that are right, separately.
Text description of the diagram
- A model that always answers negative scores 0.95 accuracy but only 0.5 balanced accuracy.
- The scikit-learn example returns a balanced accuracy of 0.625 and has accuracy 0.6667.
Step 3: Perplexity measures fit to text, not quality of answers
Perplexity is the exponential of a language model's average negative log-probability per token, so it measures how well the model fits a text. The Hugging Face documentation defines it as "the exponentiated average negative log-likelihood of a sequence". A lower value means the model finds the text less surprising. That it says nothing about whether answers are correct, safe or useful is our inference from the definition, not a claim of the documentation.
A toy case shows the scale: if a model gives every token probability 0.25, perplexity is 4, like choosing among four equally likely tokens.1
The same documentation says tokenization affects perplexity and must be taken into consideration when comparing models, so compare perplexity only under one tokenizer. Suppose one text has a total negative log-likelihood of 120 nats (illustrative). A tokenizer that splits it into 40 tokens gives an average of 3.0 and a perplexity of 20.0855; one that splits it into 60 tokens gives 2.0 and 7.3891, for the same text and the same total fit.1 The Hugging Face documentation also limits the metric to causal models, not masked ones. More on reading it is in LLM benchmarks and perplexity.
Two other scores mislead in the same way. Word-overlap scores such as Bilingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) count shared words with a reference answer; Liu et al. (2016) report that such metrics correlate very weakly, or not at all, with human judgments of dialogue responses. A benchmark score is also inflated when test items leaked into the training data, because the model then remembers answers instead of showing skill. Check both against a hand-labelled sample and a fresh, held-back test set.
Step 4: A judge model carries its own biases
A model used as a judge is itself a metric, and it has measurable biases. Zheng et al. (2023) examine position, verbosity and self-enhancement bias, plus limited reasoning ability. Position bias means a judge favors an answer because of where it appears; self-enhancement means a judge favors answers from its own model.
With the default prompt and the answer order swapped, Generative Pre-trained Transformer 4 (generative pre-trained transformer (GPT)-4) gave a consistent verdict 65.0% of the time and Claude-v1 only 23.8% of the time, as reported in Zheng et al. (2023), Table 2 and Section 3.3. In the same paper GPT-4 gave itself a win rate 10% higher than humans gave it and Claude-v1 25% higher, while GPT-3.5 showed no self-preference; the authors caution that the data are limited.
Their mitigations are swapping answer positions, few-shot examples and reference-guided grading. Our advice: trust judge scores only after comparing them with human labels on a sample, as in Step 1. The practice is covered in LLM-as-a-judge.
Step 5: Once a metric becomes a target it stops measuring the goal
Optimizing against a metric makes it less trustworthy, and the four mechanisms differ, so the fixes differ. Manheim and Garrabrant (2018) name four: regressional, extremal, causal and adversarial. The popular wording, "When a measure becomes a target, it ceases to be a good measure", is Marilyn Strathern's 1997 paraphrase; El-Mhamdi and Hoang (2024) report that Goodhart's own 1975 statement concerned statistical regularities collapsing under pressure for control. They also show that the harm depends on the tail of the gap between goal and measure, with long-tailed gaps favoring a Goodhart effect.
The regressional case is easy to simulate. Let true quality and measurement noise each be a standard normal number (illustrative), and let the proxy be their sum. Among 10,000 candidates, the top 100 by proxy have a mean proxy of 3.688 but a mean true quality of 1.769, about half.1 Selecting on the number selects its noise too. Keep a held-out check that nobody optimizes against, and refresh it.
Step-by-step description
- Name the decision the number drives.
- Skewed classes inflate plain accuracy.
- Tokenizers change the number.
- Judges show position and self bias.
- A target drifts from the goal.
- A metric matters only if a named decision changes when its value changes.
- Plain accuracy misleads whenever one class is much rarer than the other.
- Perplexity is not comparable across different tokenizers.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
What metrics matter, and which ones mislead you?
I judge a metric by the decision it drives. First I name the decision and which error costs more, then I check the metric against human labels on a sample. On imbalanced data I skip plain accuracy: with an illustrative five percent positive class, a model that always says negative scores ninety-five percent. I use per-class recall or balanced accuracy. I never compare perplexity across tokenizers. With a model as judge, I swap answer order and spot-check against people. Once a metric is a target, I keep a held-out check nobody optimizes.
What makes it Staff-level is that it leads with a decision and a validation step, names the data condition under which each metric lies, and ends with the defense against optimization pressure.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Lists metric names without saying what each measures. | States what accuracy, perplexity and judge scores measure. | Adds the exact condition under which each one lies. |
| Depth | Says accuracy is bad on imbalance. | Shows a majority guesser and uses per-class recall. | Also covers tokenizer effects, word-overlap scores, contamination and judge bias with sources. |
| Trade-offs | Names one best metric. | Picks the metric by which error costs more. | Reports several metrics and explains why one never decides alone. |
| Communication | Rambles through a list. | Gives the failure, then the fix, for each metric. | Opens with the decision, then the validation check. |
| Follow-up readiness | Cannot say how to test a metric. | Checks the metric against human labels. | Keeps a held-out check and refreshes it. |
Three variants of this question
Which metrics would you track for a classifier with rare positives?
Track recall and precision for the rare class, plus balanced accuracy as a single summary, and keep plain accuracy only as context. Choose between recall and precision by which error is costlier.
Which metrics mislead you when you evaluate a chatbot?
Perplexity misleads because it measures fit to text, not answer quality, and a judge score misleads until position and verbosity bias are checked. A sample of human-labeled answers is the anchor.
What if the metric improves but users complain?
Suspect that the metric has become a target: compare it with fresh human labels and a held-out check, and look for the regressional effect of Step 5. Fix the metric before you tune the model further.
Common mistakes
- "Our accuracy is 0.95, so the model is good." (wrong) On skewed data that score can come from always answering the common class. (right) "I check per-class recall and balanced accuracy."
- "Model A has lower perplexity than model B, so A is better." (wrong) Different tokenizers change the number, and it measures fit, not helpfulness. (right) "I compare perplexity only with one tokenizer, and judge quality separately."
- "A high BLEU or benchmark score proves the answers are good." (wrong) Word overlap tracks human judgment weakly, and leaked test items inflate a benchmark. (right) "I check the score against human labels and a fresh test set."
- "The judge model is objective." (wrong) Judges show position, verbosity and self-enhancement bias. (right) "I swap answer order and check against human labels."
- "More tuning against the benchmark always helps." (wrong) Once the metric is the target, the gap to the goal can widen. (right) "I keep a held-out check that is not optimized."
Follow-up questions
How do you check that a metric is valid?
Label a small fixed sample by hand and see whether the metric orders examples the way people do. If it does not, the metric is not ready to drive a decision.
Is perplexity a good metric for a chatbot?
Not on its own. Perplexity measures how well a causal language model fits text, so it can fall while answers do not improve; use it to compare models only under one tokenizer.
Can you trust an LLM judge?
Only after checking it. The judge can favor an answer by position, length or its own origin, so swap order, add reference answers and compare with human labels on a sample.
What is Goodhart's law in machine learning?
It is the pattern that optimizing a proxy metric makes it a worse stand-in for the goal. Step 5 shows a simulation where selecting on a noisy proxy keeps only about half of the apparent gain.
How do you measure a retrieval-augmented generation (RAG) system?
Evaluate retrieval and generation separately, for example context quality and faithfulness of the answer to the retrieved text; what Ragas is and how it evaluates retrieval-augmented generation (RAG) covers one toolkit. Faulty answers also tie to why LLMs hallucinate.
- A metric matters when a named decision changes with its value, so start from the decision.
- Plain accuracy lies on imbalanced data; use per-class recall or balanced accuracy.
- Perplexity measures fit to text and is not comparable across tokenizers.
- A judge model has position, verbosity and self-enhancement biases; check it against human labels.
- A metric that becomes a target drifts from the goal, so keep a held-out check.
Sources
- Metrics and scoring: quantifying the quality of predictions (scikit-learn 1.9.1) — scikit-learn documentation (accessed 2026-10-26), 26 Oct 2026
- balanced_accuracy_score (scikit-learn 1.9.1) — scikit-learn documentation (accessed 2026-10-26), 26 Oct 2026
- Perplexity of fixed-length models — Hugging Face Transformers documentation (accessed 2026-10-26), 26 Oct 2026
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv 2306.05685) — arXiv (Zheng et al.), NeurIPS 2023 Datasets and Benchmarks, 9 Jun 2023
- Categorizing Variants of Goodhart's Law (arXiv 1803.04585) — arXiv (Manheim and Garrabrant), 13 Mar 2018
- On Goodhart's law, with an application to value alignment (arXiv 2410.09638) — arXiv (El-Mhamdi and Hoang), 12 Oct 2024
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, full paper (arXiv 2306.05685v4) — arXiv (Zheng et al.), Table 2 and Section 3.3, 9 Jun 2023
- How NOT To Evaluate Your Dialogue System (arXiv 1603.08023) — arXiv (Liu et al.), 25 Mar 2016
