In this article
Before you start
This page belongs to the interview loops guide, where take-home rounds are one stage of the process.
Why interviewers ask this
A take-home that extracts data from documents shows a reviewer something a live round cannot: whether you treat model output as a claim to check or as a fact. Candidates report versions of it for insurance-claim agents, a full-stack extraction app that combines optical character recognition (OCR) and an LLM with a React front end, academic marksheets and contract redlines; this page cites no public report for any company, so treat each as reported by candidates.
The task is open-ended on purpose, so your choices are the evidence. A weak submission is a demo on one clean file. A strong submission has a small labeled set, a score per field and a path for fields the system doubts. A Staff-level submission also says what the scores do not show, such as new document layouts. Other take-home types include the refactor take-home, the agent take-home, which fits the insurance-claims variant, and the retrieval-augmented generation (RAG) take-home. After any of them, a project deep dive can follow, where you defend your choices out loud.
Document extraction from first principles
- Document extraction
- Document extraction is a process that turns the text of unstructured files, such as scans, forms and contracts, into named fields with typed values, so that software can use them without a person retyping. It combines reading the file, mapping content to a schema, checking the values and handling doubt, and it is judged field by field, not file by file.
- Field-level F1 score
- The field-level F1 score is a measure of extraction accuracy that counts each field value as correct, wrong or missed, then takes the harmonic mean of precision and recall over those counts. It is more informative than a per-document score, because it shows which fields fail instead of marking a whole document right or wrong.
- Evidence span
- An evidence span is the quoted source text, with its page number and position, that shows where an extracted value came from. It lets a reviewer or an automatic check confirm the value against the document in seconds, and it makes an invented value easy to spot because no matching span exists.
In plain words
Picture a mail-room clerk with a paper form. The form has fixed boxes: vehicle number, date, amount. The clerk reads each letter and copies values into the boxes. When a box has no answer, the clerk leaves it blank. When a handwritten number is smudged, the clerk walks it to a supervisor.
The analogy breaks in one place. A model does not leave a box blank or walk to a supervisor on its own. Unless you build that behavior, it fills the box with a plausible value.
In one precise sentence, the take-home asks you to build the clerk, the form and the supervisor path, and to prove with numbers how often each one fails.
Step-by-step description
- The file becomes text and page images.
- A schema fixes field names, types and nulls.
- Formats, totals and evidence are checked.
- Doubtful fields go to a reviewer.
- The file becomes text and page images.
- A schema fixes field names, types and nulls.
- Validation checks each value, not only the shape.
- Doubtful fields go to a reviewer.
Step 1: Turn the file into text and page images
A system cannot extract from a file it cannot read, so the first step turns each file into text and page images. A portable document format (PDF) file may carry a text layer, which you read directly, or be a scan, which is only pixels. Optical character recognition (OCR) is software that reads the pixels of an image and outputs text, usually with the position of each word on the page. Many OCR engines also report a confidence for each word or line. A vision-capable LLM can read the page image itself, which helps with layout and handwriting.
Input: one file. Output: text with page numbers and positions, plus one image per page. Keep the page and position, because the review step needs them to show where a value came from. You now have raw material; nothing has been decided yet.
Step 2: Ask for a schema, with room for nothing
A schema fixes field names, types and nulls, so the model returns the same shape for every document. The schema is usually written in JSON Schema, a vocabulary for describing and validating JSON documents. List every field you want, such as a vehicle identification number (VIN), a date of loss and a payout amount. Allow null for "not in this document", and add an evidence span for each value.
A structured output is a model response constrained to follow a schema you supply. As of October 2026, OpenAI's Structured Outputs guide says the feature ensures responses adhere to your JSON Schema, so the model does not omit a required key or invent an invalid enum value, and it says refusals can be detected in code.
That promise is about shape. The guide does not say a number was read correctly, so this step ends with a well-formed answer that may still be wrong. The next step checks it.
Step 3: Validate the values, not only the shape
Validation is the step that checks each value is right, not just well formed. Run three checks: normalize formats so "14 March 2026" and "2026-03-14" compare equal; apply cross-field rules such as line items that must add up to a total; and confirm the evidence span appears in the source. Then compare with a hand-labeled set. Precision is the share of returned values that are correct. Recall is the share of values in the documents that were returned correctly.
In an illustrative set of six documents with four fields each, 22 of the 24 slots hold a value.1 The field-level F1 score is 0.6818 on exact string match and 0.9545 after normalizing formats.1 The gap is measurement, not model quality: six of the seven exact-match errors were only formatting.1 Two real errors remain. A document with no loan balance got 4500.00, a well-formed value with confidence 0.81 that passes the schema, and a balance of 0.00 was left empty.1
Text description of the diagram
- The field-level F1 score is 0.6818 on exact string match.
- The field-level F1 score is 0.9545 after normalizing formats.
Step 4: Route doubtful fields to a reviewer by confidence
A review threshold is a confidence cutoff below which a field, or any empty field, goes to a person instead of being accepted. A model's own stated confidence is a claim to test on your labeled set, not a measured probability. OCR confidence is a different signal: it measures how clearly characters were read, not whether the right field was picked.
Choose the cutoff from the labeled set, and show the reviewer the evidence span next to each value, for example on a React reviewer screen. The high, medium and low labels some assignments ask for are bands of the same cutoffs. Deciding what to do next, as the insurance-claims agent variant asks, is the same routing extended: accept the claim, ask for a missing document, or send it to a person, chosen by rules over the validated fields.
| Review threshold | Fields to review (of 24) | Share to review | Wrong values accepted |
|---|---|---|---|
| 0.6 | 3 | 12.5% | 1 |
| 0.8 | 6 | 25% | 1 |
| 0.9 | 9 | 37.5% | 0 |
The invented value carried confidence 0.81, so only the highest cutoff catches it.1 The cost is reviewer time, and the right trade depends on what a wrong field costs.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
How would you build the extraction system, and how do you know it works?
I split it into four stages. First I turn each file into text and page images, keeping page positions. Second I ask the model for a schema with nulls and a quoted evidence span, so the shape is fixed. Third I validate values: normalize dates and money, check cross-field totals, and confirm each value appears in the source. I score that against a small hand-labeled set, per field, so a format mismatch does not look like a wrong read. Fourth I route low-confidence and empty fields to a reviewer, and I pick the threshold from the labeled set, because I cannot assume the model's own confidence is calibrated. What I would report is which errors still slip through.
What makes it Staff-level is the last sentence: it names the remaining failure instead of presenting a single accuracy number.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Demo works on one clean file | Works on a varied labeled set | Reports failures by document type and field type |
| Validation | Trusts valid JSON | Normalizes formats and checks totals | Checks evidence spans and flags invented values |
| Measurement | One overall accuracy figure | Precision, recall and F1 per field | Separates formatting errors from real misreads |
| Failure handling | Silent wrong values | Low-confidence fields go to review | Threshold chosen from data, with the review cost stated |
| Communication | Describes the code | Explains the pipeline and trade-offs | Names known gaps and what to build next |
Three variants of this question
What if the documents are marksheets with handwriting and complex tables?
A marksheet-extraction application programming interface (API) for complex tables and handwriting is reported by candidates in one take-home. Treat table structure as its own step (find rows, columns and merged cells before reading values), send every handwritten field to review until your labeled set shows otherwise, and report scores per field type. The API should return the evidence span with each value.
What if the task is to generate contract redlines?
A contract-redline prototype is reported by candidates for a contract-software company. Extraction becomes review against a standard: split the incoming contract into clauses, match each clause to your playbook's preferred position, and ask the model to propose a tracked-change edit per clause that deviates, quoting the clause and the playbook rule it applies. A person accepts or rejects each proposal, and you measure how often the proposals are accepted.
What if the system must improve from reviewer corrections?
Store every correction as a new labeled example, keep a held-out set that never receives corrections, and track the field-level score over time. Use corrections as examples in the prompt or as rule updates, and re-run the whole set after each change so a fix for one firm does not break another.
Common mistakes
- "The JSON validates, so the extraction is right." (wrong) A schema checks shape, not truth. (right) "The shape is fixed by the schema; correctness comes from validation and a labeled set."
- "Accuracy is high." (wrong) One figure hides which fields fail. (right) "Here is precision and recall per field, and here are the two real errors."
- "The model says it is 95% sure, so I accept it." (wrong) Stated confidence is not measured. (right) "I test confidence against the labeled set and pick the threshold from it."
- "Empty means zero." (wrong) A missing balance and a balance of 0.00 are different facts. (right) "I keep
nullfor absent and send it to review when the document might hold the value."
Follow-up questions
How do you build a document-extraction system that combines OCR and an LLM?
OCR turns the pixels into text with positions, the LLM maps that text to a schema, a validator checks the values, and a reviewer screen shows each value beside its source span. Say which stage owns which error.
How do you show high, medium and low confidence for each extracted field?
Pick two cutoffs from the labeled set and label fields below the lower cutoff as low, between the cutoffs as medium and above the upper one as high. Report how many wrong values land in each band.
What should you hand in with the take-home?
Hand in a short readme with how to run it, the labeled set, the per-field scores, the known failures and what you would build next. A reviewer can then judge your choices without guessing.
What if the document mix or the threshold changes?
Re-run the labeled set on the new mix before trusting any number, because scores come from the documents you tested. A lower threshold sends fewer fields to review and lets more wrong values through, as the routing table shows.
Does this come up in a live round too?
Yes: a project deep dive or a design round can ask you to walk through the same pipeline out loud. Prepare by practicing the four-stage answer above. See what forward deployed and applied AI engineer rounds ask and the AI engineer interview questions. If you use an AI coding assistant on the take-home, read how an AI-assisted coding interview is graded.
- The extraction take-home tests whether you check model output instead of trusting it.
- A schema fixes the shape of the answer, not the correctness of the values.
- Validate formats, cross-field totals and evidence spans, then score per field on a labeled set.
- Separate formatting errors from real misreads before you quote any accuracy figure.
- Choose the review threshold from your own data and state what a wrong field costs.
Sources
- Structured model outputs (OpenAI API guide) — OpenAI (API documentation, read 2026-10-26), 26 Oct 2026
- JSON Schema: a vocabulary for annotating and validating JSON documents — JSON Schema (specification, read 2026-10-26), 26 Oct 2026
