In this article
Before you start
This question builds on AI agent architecture and prompt engineering.
Why interviewers ask this
A PDF workflow looks easy in a demo and breaks in production, so this question separates candidates who have shipped document pipelines from those who have only called a model once. Candidates report it in agent and applied large language model (LLM) interviews, for example as "you have a simple workflow that sends PDFs to a model with a long context window; make it reliable and fast."
A weak answer says "use a bigger context window and retry on errors." A strong answer names what actually fails: wasted tokens, facts the model overlooks in the middle of long input, silent extraction errors and repeated cost. A Staff-level answer adds how you would measure it, where a person reviews, and when each fix stops paying off. The same instinct, find the failure before choosing the tool, appears in why agents fail on long tasks.
A reliable PDF workflow from first principles
- Schema validation
- Schema validation is a programmatic check that an extracted result has exactly the expected fields, types and allowed values, so a missing, mistyped or impossible value is caught before anything downstream uses it.
- Provenance
- Provenance is the record of where an extracted value came from, here the file and page number, so a person or a second check can open that page and confirm the value against the source.
In plain words
Think of a clerk who must answer questions about a long contract. A poor clerk rereads the whole contract for every question. This clerk trusts memory and never double-checks. A good clerk indexes the contract once and opens only the relevant pages. Each answer goes on a form with fixed boxes. The figure is then checked against the page. The analogy breaks in one place. A model does not tire or forget on purpose. It can still misread a table with total confidence, so the double-check has to be built in.
In one precise sentence, a reliable PDF workflow converts each file into checked, page-addressable pieces once, and every later question reads only the pieces it needs.
Step 1: Extract each page once and keep its page number
Extraction turns a PDF into data you control. The Claude application programming interface (API) documentation on PDF support (read October 2026) says each page is converted into an image and its text is extracted and supplied alongside that image, so charts and tables can be read as well as text. The same page lists limits: a 32 MB request, and 600 pages per request, or 100 when the context window is under 1M tokens.
Dense pages may fill the context window before the page limit (an inference), so split big documents into sections. Input: a file. Output: one record per page, holding its number, its text and a flag for whether the page has a usable text layer. Each page's text is then split into chunks of about 500 tokens (illustrative), and every chunk keeps its page number. For a scan, the page image carries the content; a separate optical character recognition (OCR) stage, which turns picture-of-text into characters, is a design choice, not a requirement. You now have pieces to address one by one.
Step 2: Retrieve the relevant chunks instead of pasting the whole PDF
Retrieval is the step that selects a few chunks that match a question and sends only those to the model. It matters because a long input is both costly and read unevenly: Liu et al. (2023) report that performance is often highest when the relevant information is at the beginning or end of the input and degrades significantly when it is in the middle, even for models built for long contexts.
The cost side is arithmetic. The documentation puts a page at typically 1,500–3,000 text tokens, so a 100-page PDF pasted whole is 150,000–300,000 text tokens per question, before image tokens, and at the high end the file may not even fit in a 200,000-token window.1 Retrieving five chunks of about 500 tokens each, which are passages smaller than a page, plus 300 tokens of instructions (illustrative) is 2,800 tokens per question.1 Retrieval can miss the right page, so it needs a recall check, covered in Step 5.
Text description of the diagram
- A whole 100-page PDF is 150,000 to 300,000 text tokens per question.
- Retrieval of five chunks is 2,800 tokens per question in the illustrative example.
Step 3: Retry transient failures, but not misreadings
Retries fix failures that are temporary, and only those. Retries with exponential backoff, and jobs written so that a repeat is harmless (idempotent), are the right fix for transient failures such as rate limits, overload and timeouts; work that still fails goes to a dead-letter queue for review.
You tell the two kinds apart by what the error is: a Hypertext Transfer Protocol (HTTP) status or error type such as a timeout or an overloaded response is transient, while a response that arrives successfully but holds a wrong value is deterministic. A retry often returns the same wrong value, so a misreading needs a check, not a retry. Log the file, page, error type and attempt number for every failure, so you can see which kind dominates.
Step 4: Validate every field against a schema and its source page
Where the platform allows it, enforce the output shape at generation time with structured output or tool use, which the documentation lists for extraction. Then run schema validation on every extraction: required fields present, numbers parse, dates are real, totals add up. Then check the value against its provenance by asking a narrow question of the cited page, or by recomputing it, for example a total from its rows. Route any disagreement to a person; the human in the loop page covers when that is worth it.
The numbers explain the priority. Suppose a document has 20 fields and each is right 98% of the time (illustrative). All 20 are right in only 67% of documents.1 If the checks catch 80% of the wrong fields (illustrative), the residual field error falls to 0.4%, and 92% of documents come out fully correct, with 28% flagged for review.1 These figures assume each field errs independently, the checks raise no false alarms, and a reviewer corrects every flagged field (illustrative).
Step 5: Cache, batch and measure the repeated work
Prompt caching is the reuse of an unchanged beginning of a request, such as a long document, across several calls so it is not processed from scratch each time. The PDF documentation lists caching for repeated analysis and document batches for high volume as its scaling options, and its best-practice list adds placing the PDF before the question, using upright pages and logical page numbers, and splitting large files.
Use caching when many questions hit the same document, and batching when no one is waiting for the answer. Then measure. Keep a labelled sample of documents, score field accuracy and retrieval recall after every change, and watch the flagged-for-review rate; evaluating agents covers the method. A change with no measured gain is removed. For latency, process pages or sections concurrently, and send less input, since shorter input shortens the time to the first token.
Step-by-step description
- Each page is read once and keeps its number.
- Only the relevant chunks reach the model.
- Each field is checked against a schema and its page.
- Repeated work is cached and batched.
- Each page is extracted once and keeps its page number.
- Retrieval sends only the chunks that match a question to the model.
- Every field is validated against a schema and its source page.
- Use caching when many questions hit the same document, and batching when no one is waiting for the answer.
Worked numbers for one 100-page PDF
The table shows what the steps change for one 100-page PDF queried 1,000 times, using the documented per-page token range and illustrative accuracy figures. Read it top to bottom: the first rows are the token bill with and without retrieval, the next two are the accuracy gain from validation, and the last row shows why a long file needs splitting into several requests. A 250-page file needs three requests of at most 100 pages each, so the parts must be merged and validated.1 The token rows come from the documented 1,500–3,000 tokens per page, the accuracy rows are illustrative, and the whole table is computed in the script named in the footnote.1
| Quantity | Value (computed or illustrative) |
|---|---|
| Text tokens per question, whole 100-page PDF | 150,000–300,000 |
| Text tokens per question with retrieval (illustrative) | 2,800 |
| Total over 1,000 questions, whole PDF | 150,000,000–300,000,000 |
| Total with retrieval plus one extraction pass | 2,950,000–3,100,000 |
| Reduction factor | about 50× to 97× |
| Documents fully correct, no checks (20 fields at 98%, illustrative) | 67% |
| Documents fully correct, checks catch 80% of errors (illustrative) | 92% |
| Requests needed for a 250-page file at 100 pages each | 3 |
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
How would you improve reliability and performance in a PDF-based workflow?
I'd build four stages. First, extract each page once and keep its page number with every piece. Second, retrieve only the chunks a question needs, each tagged with its page, instead of pasting the whole file, because long input costs tokens and models use the middle of it worse. Third, validate every extracted field against a schema and its source page, and send disagreements to a person. Fourth, cache the repeated document, batch bulk jobs, and measure accuracy and recall on a labelled sample. Retries with backoff handle transient errors, but not a misreading: the same wrong value often comes back, so I check it instead.
What makes it Staff-level is the order and the evidence: it fixes the failure that is silent (wrong values) before the one that is visible (slowness), and it ends with a measurement, not a technique.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | "Use a bigger model." | Extracts per page and validates against a schema. | Also checks each value against its source page. |
| Performance | "Paste the whole PDF." | Retrieves relevant chunks only. | Adds caching and batching, with a stated reason for each. |
| Trade-offs | None named. | Notes retrieval can miss the right page. | Measures recall and sets a fallback. |
| Communication | Lists tools. | Orders steps by failure removed. | Gives the first check to build and why. |
| Follow-up readiness | Stuck on scans. | Handles scans and tables separately. | Quotes limits and checks them in the documentation. |
Three variants of this question
How would you extract tables from scanned PDFs reliably?
Treat scans and text PDFs as different inputs: flag pages with no text layer, read their images with a vision-capable model or an OCR stage, and validate numeric tables with arithmetic checks such as row sums. A page-level review queue handles what fails.
How would you cut the cost of asking the same PDF many questions?
Extract and index once, retrieve only the relevant chunks per question, and cache the repeated document prefix when a whole document must be sent. Batch any work no person is waiting on.
What if the PDF is far larger than the model's limit?
Split it by section within the documented page and size limits, extract each part, and retrieve across the parts. A question that spans parts, such as a total across chapters, needs a step that merges the parts' results and then validates the merge.
Common mistakes
- "A long context window means I can paste everything." (wrong) Cost grows with every page, and the Lost in the Middle results show facts in the middle are used less well. (right) "I retrieve the relevant chunks and keep the prompt short." The context rot page covers the same effect.
- "If extraction is wrong, I just retry." (wrong) Retries suit transient errors, but a retry often repeats the same misreading. (right) "I retry transient failures with backoff, and validate against a schema and the source page, then escalate to a person."
- "Scans and text PDFs behave the same." (wrong) A page with no text layer gives the model only an image. (right) "I classify each page first and route scans to a stage I can check."
- "I tested it on three files, so it works." (wrong) Three files cannot reveal a per-field error rate. (right) "I score a labelled sample after every change."
Follow-up questions
Given a simple PDF-based workflow with long context windows, how would you improve reliability and performance?
Stop sending the whole document each time. Extract once with page numbers, retrieve only what a question needs, validate each result against a schema and its source page, and cache or batch repeated work.
Why not paste the whole PDF into a long context window?
It is costly, since a page typically uses 1,500–3,000 text tokens according to the PDF documentation, and long inputs are used unevenly. See why agents fail on long tasks.
How do you know the extraction is correct?
Score field-level accuracy on a labelled sample, check every value against its page, and track how many documents are flagged for review. The agentic AI interview questions page shows how related evaluation questions are asked.
What changes if the workflow runs on thousands of PDFs?
Per-document cost and review load dominate. Batch the work, cache shared prefixes, and watch the flagged rate: with the illustrative figures above, about 28% of documents need a look.1
- Extract each page once and keep its page number as provenance.
- Retrieve relevant chunks; pasting a whole PDF costs tokens and loses mid-document facts.
- Validate every field against a schema and its source page; retries fix transient errors, not misreadings.
- Cache repeated documents, batch bulk work, and measure on a labelled sample.
Sources
- PDF support (Claude API documentation) — Anthropic (read 2026-10-26), 26 Oct 2026
- Lost in the Middle: How Language Models Use Long Contexts (arXiv 2307.03172) — arXiv (Liu et al.), 6 Jul 2023
