LLM evaluation: the complete guide for AI and ML interviews

In this article

1. The words this guide uses
2. Start here: the learning path
3. What does an evaluation measure?
4. How many examples does an evaluation need?
5. How do you score a retrieval-augmented system?
6. Can one model grade another?
7. When must people grade the outputs?
8. How does online evaluation differ from a pre-deployment test set?
9. How do you test safety and agent security?
10. Where do public benchmarks fit?
11. The map of the topic
12. Questions interviewers ask on LLM evaluation
13. All 30 guides in LLM evaluation
14. Follow-up questions
15. Sources

The words this guide uses

LLM evaluation

LLM evaluation is the practice of measuring a large language model or an application built on one against a stated definition of quality, using test inputs (a fixed evaluation set before release, real traffic after it), a scoring method and a decision rule. It answers whether a change made the system better, not only whether the system works.

Evaluation set

An evaluation set is a fixed collection of inputs, each with a reference answer or a scoring rubric, that is held out from training and prompt tuning and reused to compare versions of a system.

Benchmark

A benchmark is a published evaluation set with a defined scoring procedure that many models are run on, so their results can be compared with each other.

Start here: the learning path

Read the pages in this order. First, how a transformer generates text, because evaluation scores what that process outputs. Second, classification metrics, which the confusion matrix page explains, because many language-model checks reduce to right or wrong decisions. Third, this guide, which ties the metrics to a test set, a scoring rule and a decision.

1. AI agent security: how prompt injection attacks work and how to stop them
2. What is LLM-as-a-judge, and how do you use it correctly?
3. Active learning explained: how a model picks which data to label
4. Offline vs online evaluation: why do you need both?
5. How would you design an eval for X?
6. What is bias and fairness in machine learning, and how do you measure it?
7. Golden sets and regression testing for LLM apps: gate each change case by case
8. What are LLM benchmarks and perplexity, and how do you read them?
9. Why do LLMs hallucinate, and how do you reduce it?
10. What are LLM guardrails, and how do they work?
11. What is privacy in ML systems, and how do you protect it?
12. Design an internal eval platform that gates every change on a versioned test set
13. How do you design a frontier-model benchmark harness?
14. How do you evaluate the current state of LLM applications?
15. How would you measure whether a new UI feature is improving engagement?
16. YouTube watch time dropped 5% last Tuesday: how do you investigate?
17. How would you approach building a churn prediction model from scratch?
18. What metrics would you track to evaluate the performance of your ML pipeline?
19. What is generative AI, and how does it work?
20. How do you measure factuality and hallucinations?
21. Pre-deployment metrics looked great, the A/B test did not: why?
22. What metrics matter and which ones mislead you, from accuracy to perplexity
23. How do tool-result and memory poisoning attacks compromise agents?
24. Why are offline metrics higher than online, and how do you fix it?
25. What's the trade-off in offline metrics?
26. What is the project deep dive, and how do you answer it?
27. How do you design CI-friendly eval suites?
28. Why is annotation hard?
29. How do you minimize and fix hallucinations over time?
30. How do you do red-teaming on LLM features?
31. How do you detect alignment over-refusal?
32. How to build GPT from scratch, step by step in PyTorch
33. How do you deploy LLMs for a regulated enterprise?

What does an evaluation measure?

An evaluation measures whatever the metric rewards, so the metric must be chosen from the product's risks. Reference-based checks compare an output with a known answer, such as exact match or accuracy. Model-graded checks ask another model to score the output against a rubric. Human ratings ask people. Most real systems mix the three: cheap automatic checks on every change, a model grader for open-ended quality, and humans on a sample.

Breadth matters too. The Holistic Evaluation of Language Models paper (Liang et al., November 2022) measures 7 metrics: accuracy, calibration, robustness, fairness, bias, toxicity and efficiency. It runs them over 16 core scenarios and 30 models, and reports raising the share of core scenarios on which models were evaluated from an average of 17.9 percent to 96.0 percent. The lesson for an interviewer's question is that accuracy alone leaves most of the risk unmeasured.

How many examples does an evaluation need?

An evaluation needs enough examples that the difference you care about is larger than the noise in the score. A score measured on n examples has a margin of error that shrinks with the square root of n, so quadrupling the set only halves the margin. The margin of error is the distance either side of the measured score within which the true score probably lies; the 95 percent interval is the range that would contain the true score in 95 percent of repeated measurements.

A difference between two systems is significant when it is too large to be explained by that noise alone, and the significance level (here 5 percent) is the chance you accept of calling a difference real when it is not. Power (here 80 percent) is the chance of detecting a difference that really exists.

The margins use the normal approximation, 1.96 times the square root of p(1-p)/n, where p is the measured score (0.8 for 80 percent) and n is the number of examples. The Wilson interval is a refinement that behaves better on small sets. The independent-sample sizes use the standard two-proportion formula at a 5 percent significance level and 80 percent power¹. Comparing two versions on the same examples is cheaper: a paired comparison with McNemar's test looks only at the items where the two systems disagree, so it needs far fewer examples when they mostly agree. The 906 figure holds only if the two systems' errors are unrelated, which is rare for two versions of one product.

A hundred hand-checked examples are a good smoke test and a poor way to rank two systems that differ by a few points. If the test set must stay small, report the interval, not the single number.

Text description of the diagram

• Margin of error at 80 percent agreement is 11.1 points at 50 examples and 7.8 points at 100 examples.
• It is 3.9 points at 400 examples and 2.5 points at 1,000 examples.

How do you score a retrieval-augmented system?

A retrieval-augmented system is scored in two places: did retrieval fetch the right passages, and did the answer stay faithful to them. The Ragas paper (Es et al., September 2023) describes reference-free evaluation along three dimensions: retrieval quality, faithful use of the context and generation quality, without needing ground-truth human annotations.

The Ragas documentation (read 2026-10-24) defines faithfulness as the claims in the response supported by the retrieved context divided by the total claims in the response. Its context precision page defines context precision as the mean of precision at each rank where a relevant chunk appears, so order matters. Precision is the share of retrieved items that are relevant; the confusion matrix page explains it, and precision at rank k applies it to only the top k chunks.

With four retrieved chunks and relevant chunks at ranks 1 and 3, the score is 0.8333; with the same two relevant chunks at ranks 3 and 4 it falls to 0.4167¹. A response with 4 supported claims out of 5 has a faithfulness of 0.8 (illustrative). The Ragas page walks through all four metrics.

Can one model grade another?

A strong model can grade another model's answers, with known biases. Zheng et al. (2023) report that strong judges such as the generative pre-trained transformer (GPT) model GPT-4 reach over 80 percent agreement with human preferences, the same level of agreement as between humans. They also name three biases to control: position bias, verbosity bias and self-enhancement bias, along with limited reasoning ability.

Treat a judge as an instrument that needs calibration. Score a sample of outputs with people and with the judge, then report the agreement with its interval, using the sample-size table above: 80 agreements in an illustrative 100 judgments only supports a claim of 71.1 to 86.7 percent. Swap answer positions to check for position bias, and do not use a model to judge its own outputs when you can avoid it.

When must people grade the outputs?

People must grade whenever the metric cannot be written as a rule and no judge has been checked against them yet. Human labels are the reference that automatic scores are tested against: a rubric, a few worked examples and two or more graders per item. Where graders disagree, the rubric is unclear, and a score built on it is noisy before the model is even involved. Measure the agreement between graders on a shared sample before trusting any label.

Label a small, carefully graded sample first, use it to calibrate the model judge above, then let the judge score the bulk and have people re-check a fresh sample every so often. The sample-size table above applies to labels too: 100 graded items (illustrative) give a wide interval.

How does online evaluation differ from a pre-deployment test set?

A pre-deployment test set is fixed and known in advance. Online evaluation scores what real users send after release. The test set tells you whether a change is safe to ship, because every version runs on the same inputs. Live traffic tells you whether the product works for the inputs you did not think of, so it catches drift and new kinds of requests that no fixed set contains.

Neither replaces the other: a test set is comparable across versions but ages, and live checks are current but noisy, since inputs differ every day. Connect the two by adding failures found in production to the fixed set, so the next release is tested on the inputs that broke the last one.

How do you test safety and agent security?

Safety is tested with adversarial inputs, scored on whether the system refuses, stays on task or leaks data. A model that only answers questions fails by saying something wrong. A model that acts, by calling tools, reading email or browsing, can fail by doing something harmful because text it read told it to. That second case is called prompt injection, and it needs its own tests: put instructions inside the documents, web pages and tool results the agent reads, and count how often the agent obeys them.

Report the attack success rate next to the task success rate, because a defense that stops every attack and also stops the useful work is not a good result. For the attacks and the defenses in detail, see AI agent security: how prompt injection attacks work and how to stop them.

Where do public benchmarks fit?

Public benchmarks compare models, and your own evaluation set compares versions of your product. A benchmark is useful for choosing a starting model because every candidate has been run the same way, which is the point of the holistic benchmark above. It cannot tell you whether your prompts, retrieval index and users behave the same way.

Use both. Pick candidate models from public results, then build an evaluation set from your real inputs, hold it out from prompt tuning, and re-run it on every change. For the instrumentation side, tracing what happens in production, see the LLM observability page, and for structured JavaScript Object Notation (JSON) outputs that are easy to score see getting reliable JSON from an LLM.

The map of the topic

The topic is one definition and six questions, drawn below. Each question has its own section above.

Text description of the diagram

• The guide covers six questions around one definition of evaluation.
• They range from test-set size to safety and agent security.

Questions interviewers ask on LLM evaluation

Interviewers return to a few recurring probes. Expect: How would you evaluate a chatbot before launch? How do you know your judge is reliable? How many test examples do you need? Which metrics would you track for a retrieval-augmented system? Why can a benchmark score and user satisfaction disagree? In each case, define quality for the product, say how you will measure it, and give the sample size and the failure mode.

Follow-up questions

What are LLM evaluation metrics?

They are rules that turn outputs into numbers: accuracy against a reference, claim-level faithfulness against a source, rubric scores from a model grader or from people. For classification-style checks the confusion matrix page gives precision and recall, and the Ragas page covers retrieval metrics.

What are LLM evals?

Evals are repeatable tests of a language-model system, usually an evaluation set plus a scoring method that is run on every change. They play the role that unit tests play for ordinary code, with scores instead of pass or fail.

How to evaluate LLM output quality?

Write down what good means for the task, turn it into a rubric or a reference answer, score a fixed sample with a model grader or people, and report the score with its margin. Check the grader against human labels first, as in the judge section above.

How to evaluate LLM?

Start with a defined use, build an evaluation set from real inputs, pick metrics tied to the risks, and compare versions on the same set. Public benchmarks help choose a starting model; they do not replace your own set. For models that act as agents, see human-in-the-loop AI agents.

How do I keep an evaluation from drifting as the product changes?

Version the evaluation set, add real failures to it as they appear, and keep a frozen core that never changes so scores stay comparable. Re-check the judge against fresh human labels every so often.

Sources

1. Holistic Evaluation of Language Models (arXiv 2211.09110) — arXiv (Liang, Bommasani et al., Stanford CRFM), 16 Nov 2022
2. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv 2306.05685) — arXiv (Zheng et al.), 9 Jun 2023
3. Ragas: Automated Evaluation of Retrieval Augmented Generation (arXiv 2309.15217) — arXiv (Es, James, Espinosa-Anke, Schockaert), 26 Sep 2023
4. Faithfulness, Ragas documentation — Ragas (page shows no date; read 2026-10-24), 24 Oct 2026
5. Context precision, Ragas documentation — Ragas (page shows no date; read 2026-10-24), 24 Oct 2026