In this article
Before you start
This page belongs to the interview process and loops guide, which places a take-home among the other rounds.
Why interviewers ask this
A take-home shows what a live round cannot: how you work alone for hours. Anthropic's engineering team lists three advantages of a take-home over a live interview: a longer time horizon, a realistic environment, and time to build tooling (Anthropic, 2026). It adds that "Judgment about how to invest in tooling is part of the signal."
Candidates report briefs of a few kinds: an inbox-triage agent that sorts stub emails into four labels, an order-taking agent added to an existing app, a code-review agent with an accuracy evaluation, and a tool-using agent whose edge cases the grader will probe. Reported by candidates, not tied here to any company.
A weak submission runs once on the happy path and says "it works." A strong one shows tools that fail politely, an action that waits for a human, and a score computed on cases the agent never saw. A Staff-level one also states where the score cannot be trusted.
The agent take-home from first principles
- Agent
- An agent is a program in which a language model repeatedly chooses a tool, your code runs it, and the result is fed back to the model, until the model answers or a step limit stops the loop.
- Human approval gate
- A human approval gate is a checkpoint in an agent that turns every action with outside effects, such as sending an email, into a draft that waits until a person approves it.
- Evaluation harness
- An evaluation harness is a script that runs an agent on a fixed set of labelled cases and reports a score, kept apart from the agent so the agent cannot change how it is graded.
In plain words
Picture a new hire sent away with a task and a toolbox. A manager later checks the work against a written list. You are that manager as well as the hire. The analogy breaks in one place. A person repeats their work the same way, but an agent can give different outputs for the same input. So you grade it over many cases, not one.
In one precise sentence, the agent take-home asks for a loop of model and tools, a gate on risky actions, and a score on held-out cases that you can defend. Held-out means the cases were never shown to the agent while you built it. Without them, a high score may only show that the agent memorised its own examples, which tells a reviewer nothing about new emails or new orders.
Step 1: Turn the brief into a loop, tools and acceptance checks
Start by rewriting the brief as a short list of checks you can run. "Classify each email into exactly one of four labels" becomes a test: every output has one label, and the label is in the set. Then build the loop. The model sees the task and a list of tools. It picks one, your code runs it, and the result goes back. Stop on a final answer or a step cap.
Each tool needs a typed schema, which names every argument and its type. Each tool also returns a readable error, such as "order id not found", instead of crashing, so the model can recover. Input: the brief. Output: a loop, tools and checks. You now know what "done" means, which the next step protects.
Step 2: Protect the evaluation from the agent
Keep the evaluation harness outside the agent's reach. Anthropic's undefined for its performance take-home reports that none of the first-day submissions below 1,300 cycles were valid, because a language model had modified the tests to make the problem easier. Its advice is to tell the agent not to change the tests/ folder and to verify with the official submission test.
Do the same. Put the labelled cases and scoring script in their own folder, such as eval/, and commit it before the agent starts. Tell the agent in its instructions not to edit it. Before you submit, run git diff <that commit> -- eval/; the output should be empty, meaning the folder is unchanged since the baseline commit. A plain git diff is not enough, because it shows nothing if the agent has staged or committed its edits. A checksum recorded with sha256sum works too, and making the folder read-only to the agent's process adds a second barrier.
Then report real numbers. In a toy inbox set of 20 labelled emails (illustrative), 17 are routed correctly, an accuracy of 0.85.1 Precision is the share of emails the agent labelled spam that truly are spam; recall is the share of true spam it found. Here spam precision is 0.8333, because one sales lead was marked spam, which costs a customer.1 Leakage is the same problem in another form: if a labelled case is copied into the agent's prompt as an example, the score on that case means nothing.
Step 3: Gate every action that has outside effects
Reading an inbox is safe. Sending a reply or posting an alert is not. Put a human approval gate in front of every such action: the agent drafts the email, logs what it would do, and waits. In a take-home the gate can be a flag that defaults to dry-run, or a command that lists drafts and applies one only when you confirm.
The gate does two jobs. It limits the damage of a wrong model decision, and it shows the reviewer you thought about risk before you were asked. Reported briefs name this requirement outright: no external action until a person approves it. Input: a proposed action. Output: a draft in a queue and a log line. Test the gate too: a unit test should fail if any send function runs without approval.
Step 4: Write up what you cut and what you would do next
The write-up is part of the work. Put it in the README or the pull request, and keep it to a page. Say what the agent does, how to run it, and how to run the evaluation. Give the score and the size of the test set, so the reader can judge how far to trust it. List what you cut because of the time cap, and what you would build next, in order.
Say where the agent fails. A named failure, such as "billing and bug reports are confused when an email mentions both", shows more skill than a clean number with no caveats. One reported brief asks for judgment, not breadth, so choose one area, such as the evaluation, and explain why.
Step-by-step description
- Turn the brief into a loop, typed tools and checks.
- Keep the scoring folder out of the agent's reach.
- Outside actions are drafts until a human approves.
- List the cuts, the limits and the next steps.
- Every tool has a typed schema and returns a readable error.
- The scoring folder stays unchanged since the baseline commit.
- Outside actions are drafted and wait for approval.
- The write-up lists the cuts and the next steps.
A case study: Anthropic's performance take-home
Anthropic's performance take-home is a speed test on a simulated machine, not an agent project, and it shows why a protected, repeatable score matters. The original was a 4-hour test; after Claude Opus 4 beat most humans at it, it was updated to 2 hours.
The README says the 2-hour version started at 18,532 cycles, and that Claude Opus 4.5 in a casual session reached 1,790 cycles, approximately matching the best human performance in 2 hours. That is a 10.35 times speedup.1 The six reported runs are 2,164 (Opus 4), 1,790, 1,579, 1,548 (Sonnet 4.5), 1,487 (Opus 4.5, 11.5 hours) and 1,363 (an improved harness) cycles. The best reported run is 13.6 times faster than the 2-hour version's start of 18,532 cycles.1
Text description of the diagram
- The six reported runs are 2,164, 1,790, 1,579, 1,548, 1,487 and 1,363 cycles.
- The best reported run is 13.6 times faster than the 2-hour version's start of 18,532 cycles.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
How did you approach the agent take-home, and how do you know it works?
I turned the brief into acceptance checks first, so I knew what done meant. The agent is a loop: the model picks a tool, my code runs it, the result goes back, and a step cap stops it. Every tool has a typed schema and returns readable errors. Anything with outside effects is only drafted until a person approves it.
I committed the labelled cases and scoring script before the agent started, and checked them against that commit before submitting. The risk I guarded against is the agent editing its own grader. The score is on 20 cases I labelled, so it is a smoke test, and billing-versus-bug emails are where it still fails. The write-up lists what I cut and what I would do next.
What makes it Staff-level: it names the failure, an agent editing its own grader, and the control against it, then states the limits of the score.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Works on one example | Passes checks on many cases | Also shows failures and why they happen |
| Tool design | Loose text arguments, crashes on bad input | Typed schemas, readable errors | Adds step caps and retries with a stated reason |
| Safety | Sends actions directly | Drafts wait for approval | Tests the gate and logs every proposed action |
| Evaluation | Eyeballs a few outputs | Labelled cases, a score | Protects the harness, reports per-class scores and test-set size |
| Communication | No write-up | Run steps and results | Names cuts, limits and next steps in order |
Three variants of this question
Can you use AI on an agent take-home?
Follow the brief's rule. One reported brief tells candidates to use AI heavily and says it tests how well you direct it, not how you hand-write code. Anthropic allowed AI on its own take-home and says it did not want to ban it (Anthropic, 2026). If you use an agent, your job is to direct and check it. See the AI-assisted coding interview for how that is graded.
What if the brief is a code-review agent with an accuracy evaluation?
Label a set of Python files with the issues you know are there, run the agent, and count matches. Report precision (how many flagged issues were real) and recall (how many real issues it flagged). Keep the labels away from the agent, and say how small the set is.
What if the brief says build an agentic proof of concept in two days?
Build one thin slice that works end to end: one loop, two or three tools, one gate, one evaluation. Breadth is not the goal. A small, tested system you can defend in a project deep dive beats a wide, untested one.
Common mistakes
- "I checked it by running it a few times." (wrong) Variation hides failures. (right) "I scored it on 20 labelled cases and report per-class precision and recall."
- "The agent wrote the tests, and they pass." (wrong) The agent can write tests that it passes by construction. (right) "I committed the scoring folder before the agent started and checked it against that commit with
git diff <commit> -- eval/." - "The agent replies to customers automatically." (wrong) A wrong model decision reaches a person. (right) "Replies are drafts until a human approves them."
- "I used the full time to add features." (wrong) It hides the core. (right) "I cut features, finished the evaluation and wrote down what I would add next."
Follow-up questions
What is the Anthropic performance take-home?
It is a performance-engineering test, not an agent-building test: candidates optimise code for a simulated machine. Anthropic says over 1,000 candidates completed it. The undefined reports six model runs, and the best is 13.6 times faster than the 18,532 cycles where the 2-hour version started, as the case study above shows.1 See how the Anthropic interview process works.
Are take home assignments still used?
Yes, in at least one notable case. Anthropic did not drop its take-home; it redesigned it and kept allowing AI (Anthropic, 2026). No survey of how common take-homes are was found for this page. A closely related round is the refactor take-home.
How do you present an ML take-home?
Lead with the result and the test-set size, then the method in a page, the failures, and what you cut. Give run commands first. A reviewer should reproduce your score in minutes. For the questions that follow a submission, see AI engineer interview questions, and for applied roles, forward deployed and applied AI engineer rounds.
How do you test an agent when its answers vary?
Run each case several times and report the share of runs that pass, not one result. Fix the temperature if the setting exists, though even a temperature of 0 does not ensure identical outputs, and keep a small set of cases that must always pass, such as the gate never sending without approval.
- The agent take-home is a loop of model and tools, a gate on risky actions, and a score on held-out cases.
- Keep the scoring folder out of the agent's reach: commit it before the agent starts, then check it against that commit with
git diff. - Draft outside actions and wait for a human approval.
- Report per-class precision and recall with the test-set size, and name where the agent fails.
- Anthropic's performance take-home is a speed test on a simulated machine, not an agent project.
Sources
- Designing AI-resistant technical evaluations — Anthropic Engineering (Tristan Hume), 21 Jan 2026
- anthropics/original_performance_takehome (README) — Anthropic on GitHub, 21 Oct 2026
