In this article
Before you start
This page is one step of the retrieval-augmented generation (RAG) learning path.
Why interviewers ask this
Candidates report being asked how to verify that an answer's citations support its claims, in generative-AI design rounds for search and for legal-AI products. The question matters because a citation looks like proof while proving nothing by itself. In an audit of commercial generative search engines, Liu et al. (2023) found that on average only 51.5% of generated sentences were fully supported by their citations.
A weak answer says "we add links to the sources." A strong answer separates the pointer from the proof, names the two citation scores, and says how the check runs without a human reading every answer. A Staff-level answer adds where the check fails: a faithful answer built from a wrong passage is still wrong, and the cost of the check is one or more model calls per answer.
Grounding and citations from first principles
- Grounding
- Grounding is the property that every claim in a generated answer is supported by the source passages the system retrieved for it, rather than by the model's memory alone, so that a reader can trace each claim back to its evidence.
- Citation
- A citation is a pointer, such as a passage number in square brackets, that links one statement in a generated answer to the source passage that is meant to support it, so that a reader or a program can check the statement.
- Citation recall
- Citation recall is the share of generated statements that are fully supported by the passages they cite, so it measures whether the answer is comprehensive in its evidence.
- Citation precision
- Citation precision is the share of generated citations that actually support the statement they are attached to, so it measures whether each pointer is correct.
In plain words
Think of a student essay with footnotes. Grounding is the essay being true to its books. A footnote is only a number at the bottom of the page. A teacher who wants to know whether the essay is honest opens each book at the cited page and reads. The analogy breaks in one place: a model writes the footnote number and the sentence at the same time, so it can attach a number to a sentence the book never says.
In one precise sentence, a grounded answer is one whose claims can each be matched to a retrieved passage, and a citation is the label that makes the match checkable. The rest of this page is the check, in four steps.
Step 1: Retrieve passages and give each a stable id
The system retrieves passages for the question and gives each one an id that names its document and paragraph. The id is what a citation will later point to, so the passages must be addressable down to the size of a claim. If a passage is a whole page, a citation to it says almost nothing; the way text is split decides how precise a citation can be, as covered in chunking strategies for RAG.
Input: the question and a search index of embeddings. Output: a short numbered list of passages. You now know what a citation can point to: one of these ids, nothing else.
Step 2: Generate each statement with citation markers
The model receives the question and the numbered passages and writes an answer in which each statement carries markers such as [1][2]. In the Automatic Large Language Model Citation Evaluation (ALCE) benchmark (Gao et al., 2023), the system generates text with citing passages from a large retrieval corpus, and a statement may hold several citations.
Input: question plus numbered passages. Output: sentences with markers. The markers are produced by the same model that wrote the sentences, so they are claims about support, not evidence of it. The model can cite a passage that is on the topic but silent on the exact fact. That gap is what the next step measures.
Step 3: Check each claim and score citation recall and precision
A checker takes each statement together with the passages it cites and asks whether they support it. The checker can be a person, a language model used as a judge, or a natural language inference (NLI) model, which is a classifier that labels a pair of texts as supporting, contradicting, or neither. Two scores follow. Liu et al. (2023) define citation recall as the share of statements fully supported by their citations, and citation precision as the share of citations that support their statement.
Definitions of "supports" differ between papers, so this page states its own convention: it counts a citation as supporting if it backs at least one claim of its sentence, and a sentence as fully supported only if its citations together back every claim. A system that cites many passages per sentence tends to raise recall and lower precision; a system that cites one cautious passage does the reverse.
Step 4: Score faithfulness and locate the failing stage
Faithfulness is the share of claims in a response that the retrieved context supports, a number from 0 to 1 in the Ragas documentation. It checks the answer against what the generator was given, not against the world. Alone it cannot say whether a bad answer came from retrieval or generation, so you read it beside a retrieval score such as context recall, and the table in the worked example shows the four patterns. The RAG evaluation guide covers the retrieval metrics in full.
Input: claims, passages and the retrieval score. Recall, precision and faithfulness are then scored, which gives a number per answer and a decision about which stage to fix.
Step-by-step description
- Passages get stable ids.
- The model adds citation markers.
- Each claim is tested against its passages.
- Recall, precision and faithfulness.
- Retrieved passages get stable ids.
- The model writes statements with citation markers.
- A checker tests each claim against its cited passages.
- Recall, precision and faithfulness are then scored.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
A six-sentence answer with eight citations shows the arithmetic. Every value is illustrative, and the scoring rules are the ones from Step 3.1 Sentence 2 cites two passages and only one supports it; sentence 5 cites one passage that supports only half of its claims.
| Sentence | Citations | Citations that support it | Fully supported |
|---|---|---|---|
| 1 | 1 | 1 | yes |
| 2 | 2 | 1 | yes |
| 3 | 1 | 0 | no |
| 4 | 2 | 2 | yes |
| 5 | 1 | 1 | no |
| 6 | 1 | 0 | no |
Citation recall is 3 of 6 sentences, so 50%. Citation precision is 5 of 8 citations, so 62.5%.1 Faithfulness works the same way on claims: if the answer holds four claims and the context supports three, the score is 3 divided by 4, which is 0.75 (illustrative).1
| Retrieval score | Faithfulness | Likely failing stage |
|---|---|---|
| 0.2 | 0.95 | retrieval |
| 0.9 | 0.4 | generation |
| 0.2 | 0.3 | both, fix retrieval first |
| 0.9 | 0.95 | stages pass; check the sources |
The first row is the trap: the model answered faithfully from passages that did not contain the answer. The last row is the other trap: a faithful answer built on a wrong source passes every score here. A faithful answer built from a wrong passage passes the scores but is still wrong, while an unfaithful answer built on a right passage fails the faithfulness score.
Text description of the diagram
- Audited engines averaged 51.5% citation recall and 74.5% citation precision.
- The illustrative toy answer scores 50% recall and 62.5% precision.
The audited engines averaged 51.5% recall and 74.5% precision (Liu et al., 2023), and the toy answer's illustrative 50% and 62.5% are in the chart only to show how the scores read. On a long-form question dataset, Gao et al. (2023) report that even the best models lacked complete citation support 50% of the time.
Text description of the diagram
- A faithful answer built from a wrong passage passes the scores but is still wrong.
- An unfaithful answer built on a right passage fails the faithfulness score.
| Fact | Value | Source |
|---|---|---|
| Average share of sentences fully supported, 4 generative search engines | 51.5% | Liu et al., 2023 |
| Average share of citations that support their sentence | 74.5% | Liu et al., 2023 |
| Best models lacking complete citation support on a long-form question dataset | 50% of the time | Gao et al., 2023 |
| Hallucination rate of audited legal research tools | 17% to 33% | Magesh et al., 2024 |
| Ragas faithfulness range | 0 to 1 | Ragas documentation, read October 2026 |
| Toy answer citation recall and precision (illustrative) | 50% and 62.5% | computed in the data script1 |
Trade-offs and edge cases
Citation precision and usefulness can pull apart. Liu et al. (2023) hypothesize that precision is inversely correlated with perceived utility because the engines often copy or closely paraphrase their cited pages. An answer that copies its source is easy to verify and less helpful to read, so a team picks its balance on purpose.
Faithful does not mean correct. Faithfulness compares claims with the retrieved context, so an answer repeating a wrong or outdated passage scores well. Check source quality and freshness separately, and apply access control to what a citation can reveal, as in securing a RAG system.
Retrieval does not remove hallucination. Vendors of legal research tools promoted retrieval as eliminating or avoiding it, and an independent preregistered study (Magesh et al., 2024) found that the tools of two of those vendors each hallucinated between 17% and 33% of the time, less than general chatbots but far from none.
Span-level citations give a cheap first filter. In a span-level citation the model must quote the exact text of the passage that supports each statement, not just give a passage id. A program can then check, with plain string matching after whitespace and case are normalised, that the quote really is a substring of the cited passage.
This costs almost nothing and runs before any NLI or judge call, so it removes invented or misquoted citations at once. It is only a first filter: a real quote can still fail to support the claim it is attached to, so the model-based check of the statement against the quoted span still follows, and a short span is easier to judge than a whole passage. A common design is therefore deterministic checks first and model-based checks second.
Checkers are imperfect. A judge model or an NLI model can be wrong, so calibrate it against a small human-labeled sample before trusting its scores, and keep human review for high-stakes domains. Each model-based check also adds one or more model calls per answer; the cost is laid out in where RAG latency and cost go.
How to answer it in an interview
How do you verify that an answer's citations actually support its claims?
I treat a citation as a pointer, not as proof. Every retrieved passage gets a stable id, the model cites those ids, and then a separate check splits the answer into claims and tests each one against the passages it cites, using an NLI model or a judge model. I have the model quote its supporting span and first check, with plain string matching, that the quote appears in the cited passage. I report citation recall and citation precision, plus faithfulness against the retrieved context. I calibrate the checker on a human-labeled sample, and when a check fails I look at retrieval scores to see which stage broke.
The spoken structure: say a citation is a pointer, name the check, name the scores, then say how you trust the checker and where the failure sits.
Common mistakes
- "The answer has citations, so it is grounded." (wrong) A citation is written by the model and can point at a passage that is silent on the claim. (right) "Citations make an answer checkable; the check decides whether it is grounded."
- "Recall and precision of citations are the same score." (wrong) One counts supported sentences, the other counts correct citations. (right) "I report both, because a system can be high on one and low on the other."
- "High faithfulness means the answer is right." (wrong) It only compares the answer with the retrieved context. (right) "Faithfulness says the answer follows its sources; I check the sources separately."
- "Retrieval-augmented generation removes hallucination." (wrong) Independent testing of legal tools still found 17% to 33%. (right) "It reduces it; I measure what remains."
Follow-up questions
How do you evaluate hallucinations automatically?
Split each answer into atomic claims, check each claim against the retrieved text with an NLI model or a carefully prompted judge model, and aggregate into a faithfulness score. Add citation matching for the pointers, and calibrate the judge on human labels.
How do you tell if a wrong answer is a retrieval failure or a generation failure?
Check whether the needed fact was in the retrieved passages. If it was missing, retrieval failed, even when the answer is faithful; if it was present and the answer ignored or contradicted it, generation failed. The retrieval metrics are in the RAG evaluation guide.
How would you build evaluation infrastructure for legal-AI outputs without in-house lawyers?
Start with automatic checks that need no domain expertise: claim-to-passage support, citation matching to paragraph ids, and consistency across runs. Then send a small, sampled set to outside expert reviewers to calibrate the automatic scores, since human review stays necessary for high-stakes domains.
How do you make every answer point back to the source paragraphs?
Index passages with paragraph-level ids, require the model to cite those ids, and reject or flag any sentence whose marker does not resolve to a retrieved passage or fails the support check. Paragraph-sized chunks keep each pointer precise.
Should the system say when it cannot find an answer?
Yes. If retrieval returns nothing that supports the question, a cautious "the sources do not say" is faithful, while an answer from model memory is not grounded. More questions of this kind are in the RAG interview questions.
- Grounding means claims are supported by retrieved passages; a citation is only the pointer to that support.
- Citation recall counts fully supported sentences; citation precision counts correct citations; report both.
- Audited commercial engines averaged 51.5% recall and 74.5% precision in 2023.
- Faithfulness compares claims with the retrieved context, so pair it with a retrieval score to locate the failing stage.
- Retrieval reduces hallucination without removing it; calibrate any automatic checker on human labels.
Sources
- Evaluating Verifiability in Generative Search Engines (arXiv 2304.09848) — arXiv (Liu, Zhang, Liang), 19 Apr 2023
- Enabling Large Language Models to Generate Text with Citations (arXiv 2305.14627) — arXiv (Gao, Yen, Yu, Chen), EMNLP 2023, 24 May 2023
- Ragas documentation: Faithfulness metric — Ragas, 26 Oct 2026
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (arXiv 2405.20362) — arXiv (Magesh et al.), J. Empirical Legal Studies 2025, 30 May 2024
