In this article
Before you start
This page is one step of the large language model (LLM) evaluation learning path, because labels from people are how evaluation sets and training sets get made.
Why interviewers ask this
Labels cost money and time, so an interviewer who asks about active learning is really asking how you spend a labeling budget. Candidates report variants of this theme in machine-learning system design and coding rounds: a harmful-content detector built from weak labels and active learning, a coding task that finds a low-quality annotator in label data, and the design of a human preference dataset. These are reports from candidates, not confirmed by a public source.
A weak answer says "the model asks for the hard examples" and stops. A strong answer names the loop, an uncertainty measure, the stopping rule, and the risk that noisy labels land exactly where the model asks. A Staff-level answer adds how you would measure label quality first, because a selection rule built on bad labels optimizes the wrong thing.
Active learning from first principles
- Active learning
- Active learning is a training method in which the learner chooses which unlabeled examples to send to a human for labeling, instead of receiving labels for randomly chosen ones, aiming for higher accuracy from a smaller number of labels.
- Oracle
- An oracle is the source of correct labels that an active learner queries, usually a human annotator, sometimes an experiment or a trusted expert, and it is treated as the label authority even though real annotators make mistakes.
- Uncertainty sampling
- Uncertainty sampling is a query strategy that asks the oracle to label the unlabeled examples on which the current model is least confident, such as those with a top class probability close to a coin flip.
In plain words
Picture a student with a few hours of tutoring. Asking about problems already solved wastes the hours. Asking about the problems that confuse the student most teaches more. Active learning does the same: the model asks a human about the cases it finds hardest.
The analogy breaks in one place. A student knows what confuses them. A model only has a confidence score, and that score can be wrong. A model that is sure and wrong never asks.
A query strategy is a rule that scores every unlabeled example so the highest-scoring ones can go to the oracle. Pool-based sampling is the setting in which those examples are drawn from a fixed pool.
Step 1: Train a first model on a small labeled seed
The loop starts from a small labeled set, the seed, plus a large unlabeled collection. Settles calls this layout pool-based sampling: a small labeled set and a large pool of unlabeled data are available, and queries are drawn from the pool (Settles, §2.3). The survey names two other scenarios, membership query synthesis and stream-based selective sampling; this page uses the pool setting because it is the common one.
Input: the seed labels. Output: a first model that can give a probability for every class. The seed can be tiny, but the model must be good enough that its confidence means something. You now know the starting point: a weak model, a pool and an empty budget.
Step 2: Score every unlabeled example with a query strategy
The query strategy turns the model's probabilities into one number per pool example. The most common family is uncertainty sampling. Settles gives three measures (Settles, §3.1): least confidence, which is 1 minus the top class probability; margin sampling, the gap between the top two class probabilities, where a small gap means a query; and entropy, which measures how spread out the probabilities are.
Take three classes with probabilities 0.5, 0.3 and 0.2 (illustrative). Least confidence is 0.5, the margin is 0.2 and the entropy is 1.0297 nats. A confident case at 0.9, 0.05 and 0.05 has least confidence 0.1, margin 0.85 and entropy 0.3944 nats.1 The first example is queried before the second. For two classes all three rankings agree.
In a driving setting, the pool is logged camera frames and the same scoring picks frames the model finds hardest; that is the general pattern, not a claim about any company.
Step 3: The oracle labels the picks, and noise hits here
The top-scoring batch goes to the oracle, and this is where the common error lives. Active learning assumes the labels are right. Settles flags that assumption: human experts may not always be reliable, because some instances are hard for people as well as machines, and people get distracted or tired (Settles, §6.2).
Uncertainty sampling picks the examples near the model's decision boundary, which are the borderline cases. Borderline cases are plausibly where two annotators disagree most, so the batch you pick is the batch most likely to carry label noise. In the worked run below, noise concentrated at the boundary shrinks the gain.1
Practical response: send each picked example to more than one annotator, measure agreement (see the annotator section below), and keep a random sample of the pool labeled for honest evaluation.
Step 4: Add the new labels, retrain and repeat
The new labels join the labeled set, the model retrains, and the pool is scored again. Each pass is a round. The loop stops when the label budget is spent or when accuracy on a held-out set stops improving.
Selecting one example at a time is slow if retraining is expensive or many annotators work in parallel. Batch-mode active learning lets the learner query groups of examples; the hard part is that the top scores may all be near-copies of each other (Settles, §6.1). Settles calls the plain take-the-top-scores version "Q-best" and describes methods built for batches as generally better.
Input: the new labels. Output: a retrained model and a fresh set of scores. You now have the whole loop, which the diagram shows.
Step-by-step description
- A first model learns from a small labeled seed.
- Each unlabeled example gets an uncertainty score.
- Annotators label the top-scoring batch.
- New labels join the set and the loop restarts.
- A first model trains on a small labeled seed.
- A query strategy scores every unlabeled example.
- The oracle, usually a human annotator, labels the highest-scoring picks.
- The new labels join the labeled set and the model retrains.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
The run below uses a synthetic problem, so every number is illustrative of the method, not a benchmark. The pool holds 10,000 unlabeled examples with 20 features each. Logistic regression is a classifier that turns a weighted sum of features into a class probability, and it is the model here. Each run starts from 30 labels and adds 20 labels per round for 10 rounds, ending at 230 labels. Results are the average of 20 runs with different random seeds, scored on 2,000 clean test examples.1
Uncertainty sampling picks the 20 pool examples whose predicted probability is closest to 0.5. Random sampling picks 20 at random. The "noisy" condition flips 35% of the labels that fall in a band around the true boundary (illustrative), which imitates annotators disagreeing on borderline cases.1
| Condition | Labels | Random accuracy | Uncertainty accuracy | Gap (points) |
|---|---|---|---|---|
| Clean labels | 50 | 81.92% | 83.99% | 2.07 |
| Clean labels | 110 | 87.69% | 90.59% | 2.90 |
| Clean labels | 230 | 90.36% | 92.43% | 2.07 |
| Noisy boundary labels | 110 | 87.60% | 89.26% | 1.66 |
| Noisy boundary labels | 230 | 90.48% | 91.61% | 1.13 |
Uncertainty sampling reaches 90.59% accuracy with 110 labels, while random sampling with 230 labels reaches 90.36%, so it needed about half the labels here.1 Noisy random accuracy at 230 labels (90.48%) is slightly above the clean figure (90.36%); that is run-to-run variance across 20 runs, not a benefit of noise.1 The gap is largest at 110 labels, 2.90 points, and shrinks to 2.07 points at 230 as the random sample catches up.1 With noisy boundary labels the lead at 230 labels falls to 1.13 points.1
Text description of the diagram
- Uncertainty sampling reaches 90.59% accuracy with 110 labels.
- Random sampling with 230 labels reaches 90.36%.
Finding a low-quality annotator
Active learning only works if the labels are good, so measure the annotators first. In a second illustrative simulation, eight annotators each label the same 400 binary items. Seven have an accuracy between 89% and 93% and one has 60% (illustrative settings).1
For each annotator, build the majority label of the other seven, then compute the agreement rate and Cohen's kappa (Cohen, 1960) against that majority. Seven annotators agree with the others 87.7% to 91.7% of the time, with kappa between 0.755 and 0.835. Annotator 8 agrees 58.8% of the time, with a kappa of 0.176.1 The outlier is obvious.
Text description of the diagram
- Seven annotators agree with the others 87.7% to 91.7% of the time.
- Annotator 8 agrees 58.8% of the time.
Before removing anyone, check that the cause is the person and not the task. Low agreement can also come from ambiguous guidelines or hard items, so compare each annotator on the same items and look at which classes they disagree on.
How do you train on noisy multi-annotator labels?
Majority vote turns each item's labels into one hard label. It throws away the disagreement and treats a careless annotator like a careful one. A better route models reliability: Dawid and Skene (1979) estimate a confusion matrix for each annotator, the chance of each reported label given each true label, and infer the true labels and those matrices together with expectation-maximization.
A column such as annotator_id is what makes this possible. Then train on soft labels, the estimated probability of each class per item, using cross-entropy loss instead of hard labels, so the model sees that borderline items are uncertain. Down-weight or drop annotators whose confusion matrices look close to random. Send low-agreement items to an expert adjudicator, and keep the resolved labels as a clean evaluation slice.
Trade-offs and edge cases
Active learning trades simplicity and unbiased data for label efficiency. The main failure modes follow.
Biased labeled set. Because the model chose the examples, the labeled instances are a biased distribution, not a random draw, and a training set built this way may be less useful if you later switch model class (Settles, the survey's practical-considerations discussion). Schein and Ungar, cited in the same survey, found that active learning can sometimes need more labels than random sampling even with the same model class. The advice: keep a random labeled slice for evaluation and never score the model on the examples it chose.
Noisy oracles. Step 3 and the worked table show the cost: the lead drops to 1.13 points at 230 labels in the noisy condition.1
Weak seed model. If the first model is poor, its confidence is a poor guide, so the first queries may be no better than random. A larger or more varied seed is the usual fix.
Diminishing returns. The gap peaked at 2.90 points and fell to 2.07 points as labels accumulated.1
| Strategy | What it scores | Cost |
|---|---|---|
| Random sampling | Nothing; picks uniformly | Lowest |
| Uncertainty sampling | Model confidence on each example | One scoring pass |
| Query-by-committee | Disagreement among several models | Several models |
| Expected error reduction | Estimated drop in future error per label | Highest |
In current LLM work the same ideas appear in how teams spend human review. A practical design is to let an automated judge pre-score outputs and send the low-confidence or disagreeing ones to people; see LLM-as-a-judge for the judge side and Ragas for evaluating retrieval systems.
How to answer it in an interview
What is active learning and when would you use it?
Active learning is a loop where the model picks which unlabeled examples a human should label next, so I spend a fixed labeling budget on the examples that teach the most. I train on a small seed, score the pool with an uncertainty measure like margin or entropy, send the top batch to annotators, add the labels, and retrain. It helps when labels are expensive and the pool is large. It breaks when labels are noisy, because the most ambiguous examples are also where annotators disagree, and the labeled set becomes biased toward one model. I'd keep a random slice for evaluation.
The spoken structure: say the loop, name the uncertainty measure, say when it pays off, then name the noise and bias risks and the random evaluation slice.
Common mistakes
- "Active learning always beats random sampling." (wrong) The gain shrinks with noise and with more labels, and the survey cites a case where it needed more labels. (right) "It usually helps, so I compare it to a random baseline on the same budget."
- "It removes the need for humans." (wrong) The oracle is still a human annotator. (right) "It reduces how many labels I need, not who provides them."
- "It fixes noisy labels." (wrong) Selection favors ambiguous examples, where annotators disagree most. (right) "I measure annotator agreement first and use several annotators on the picked batch."
- "I evaluate on the examples the model asked for." (wrong) That set is biased toward the model's weak spots. (right) "I evaluate on a random labeled slice."
- "It is the same as active retrieval." (wrong) They share a word, not a goal. (right) "One selects data to label; the other decides when to retrieve while generating."
Follow-up questions
When is active learning worth it?
It is worth it when unlabeled data is abundant and labels are difficult, slow or expensive, which is the motivation Settles gives (Settles, 2009). It also needs a seed model whose confidence means something and labels you trust. Run a random baseline on the same budget before committing.
Is active learning the same as active retrieval augmented generation?
No. Active learning chooses which examples get a human label. Active retrieval augmented generation, as in Forward-Looking Active REtrieval augmented generation (FLARE), is a way for a language model to retrieve documents several times while writing: it retrieves again when the upcoming sentence contains low-confidence tokens, unlike a single retrieve-then-generate setup (Jiang et al., 2023). Both use model uncertainty to trigger an action, but one acts on training data and the other on a live answer.
How do you find a low-quality annotator from label data?
Compare each annotator with the majority label of the others, using agreement rate and Cohen's kappa, and look for the outlier, as in the worked example. Then check whether the cause is the annotator or ambiguous items and guidelines.
How do you design high-quality human preference datasets?
Write clear guidelines with examples, have several annotators compare the same pairs, and measure agreement before scaling. Remove or retrain annotators who sit far from the others, review the pairs with the lowest agreement for unclear guidelines, and keep a small gold set of items with known answers to audit quality over time.
Can you assume you can use user reports and human annotation to generate labels?
Not without checking. User reports are self-selected, uneven across topics, and open to abuse, so treat them as weak labels and audit a random sample with trained annotators before training on them. If an LLM tool reads report text, injected instructions are a risk; see AI agent security. For LLM outputs more broadly, combine automated checks and a calibrated model judge for scale with human review where the judge is uncertain; the LLM evaluation guide covers the full toolkit.
Your preference data has low annotator agreement. How do you measure and fix it?
Measure agreement per annotator and per pair, with agreement rate and Cohen's kappa against the other annotators' majority. Then find the cause: unclear guidelines, genuinely ambiguous pairs, or a few unreliable annotators. Fix guidelines first, retrain or remove outliers, and adjudicate the lowest-agreement pairs. For the rest, aggregate with a per-annotator reliability model or keep the vote split as a soft label rather than forcing a hard one.
- Active learning lets the model choose which unlabeled examples a human labels, to reach higher accuracy with fewer labels.
- The loop is seed, score the pool, label the picks, retrain, repeat.
- Uncertainty sampling scores examples by least confidence, margin or entropy.
- Noisy labels concentrate on the ambiguous examples the strategy picks, so measure annotator agreement first.
- Keep a random labeled slice for evaluation and compare against random sampling on the same budget.
Sources
- Active Learning Literature Survey (Computer Sciences Technical Report 1648, 2009; revised January 2010) — University of Wisconsin-Madison (Settles), 26 Jan 2010
- A Coefficient of Agreement for Nominal Scales — Educational and Psychological Measurement 20(1):37-46 (Cohen), 1 Apr 1960
- Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm — Journal of the Royal Statistical Society, Series C (Dawid and Skene), 1 Jan 1979
- Active Retrieval Augmented Generation (arXiv 2305.06983) — arXiv (Jiang et al.), 11 May 2023
