In this article
Before you start
This page belongs to the AI agent architecture learning path. If the words "prompt" and "model reply" are new, read prompt engineering basics first.
Why interviewers ask this
An agent that handles only the happy path breaks the first time a tool call fails, and tool calls fail often because they cross networks, hit rate limits and depend on other teams' services. Candidates report this question in agent design rounds. It also fits beside the agentic AI interview questions people prepare, because it tests whether you have run an agent outside a demo.
A weak answer is "wrap it in try and except and retry three times." A strong answer sends the error back to the model, retries only what is safe and returns what succeeded. A Staff-level answer also deals with the timeout that did the work anyway, with retries that multiply across layers, and with what the user is told when only part of the job finished.
Tool calling failures from first principles
- Tool result
- A tool result is the message your code sends back to the model after running a tool call, tied to that call by its identifier, that carries either the output or an error flagged with
is_error. The model never runs client tools itself; it reads this result to decide what to say or do next. - Partial result
- A partial result is the usable subset of what a request produced when some of its parts failed, returned together with a label that says which parts are missing and why, so the reader does not mistake it for the complete answer.
In plain words
Picture ordering three dishes through a waiter. The kitchen runs out of one. A good waiter does not leave the restaurant. He tells you which dish is gone and what is on offer instead, and he brings the other two. He also does not ask the kitchen to cook a dish again without checking whether it is already on its way.
The analogy breaks in one place. A waiter can walk to the kitchen and look. An agent often cannot check whether a timed-out action happened.
In one precise sentence, handling tool calling failures means turning each failed call into data the model can read, repeating only calls that are safe to repeat, and reporting honestly what finished and what did not.
Step 1: Return every failure to the model as a tool result
A failed tool call is returned to the model as a tool result flagged as an error, never thrown as an exception that ends the run. In the tool-use loop of Anthropic's documentation on handling tool calls, the model asks for a call, your code runs it, and you send back a tool_result.
A failure still needs that reply. Set is_error to true and put the error text in content. The docs require the result to follow its tool_use block immediately, with a matching tool_use_id.
Write the message so the model can act on it. The docs advise saying what went wrong and what to try next, and give "Rate limit exceeded. Retry after 60 seconds." as the kind of text to send instead of a bare "failed" (Anthropic, 2026). The model then folds the error into its reply or tries again. You now know where a failure goes; the next question is whether to send the same call again.
Step 2: Retry only what is safe, because a timeout proves nothing
A tool call should be retried only when it is safe, which means the error is transient and repeating the action cannot do harm twice. A transient failure is a request that fails for a short period of time, in the words of the Amazon Builders' Library (Brooker, 2019).
Client-style errors such as bad arguments are different: the article says a client error will not succeed later if the same request is repeated, while a server error may succeed on a later try. Fix the arguments or tell the model; do not repeat them. A rate-limit response (Hypertext Transfer Protocol (HTTP) 429) is client-class but is the exception: it is worth retrying after the stated wait, as in Step 1's example.
The trap is the timeout. The same article says "a timeout or failure doesn't necessarily mean that side effects haven't happened," and that an application programming interface (API) with side effects isn't safe to retry unless it provides idempotency. So a payment, email or ticket tool carries an idempotency key, and a retry reuses the same key. A side-effecting tool with no key is not retried at all; the agent reports the uncertainty instead.
Step 3: Bound every retry with one layer, backoff and a timeout
Retries are bounded by three limits: one retrying layer, backoff with jitter, and a timeout on every call. Layers multiply: the Amazon article shows that if each layer of a five-deep stack retries three times independently, the load on the database increases 243 times. Its advice is to retry at a single point in the stack. Each extra layer that retries multiplies the load by three, so retry at one layer only.
Backoff with jitter is called that because each wait grows, is capped, and is randomized. The delays double and stop growing: 1, 2, 4, 8 and 8 seconds (illustrative).1 Jitter "adds some amount of randomness to the backoff to spread the retries around in time." With full jitter each wait is a random value between zero and the delay, so the five waits average 11.5 seconds in total instead of the 23-second maximum.1
For the timeout, the Amazon article accepts a 0.1% false-timeout rate (an example it gives, not a rule) and sets the timeout to the downstream p99.9 latency, the time that 99.9 percent of calls beat.
Text description of the diagram
- Three retries at each of five layers multiply the load on the database by 243.
- Each extra layer that retries multiplies the load by three.
Step 4: Return a labeled partial result when some calls succeed
When an agent calls several tools at once and only some fail, it returns every result, flags only the failed one as an error, and lets the model answer with caveats. A design example (illustrative): the agent calls three tools, A succeeds, B fails with a server error and C succeeds. Three results go back, one per call, with is_error true on B only. The model then writes an answer that uses A and C and says that B's part is missing.
The Amazon article defines a partial failure as one where a percentage of requests succeed. This matters because failures compound across tools. At 99 percent success per call (illustrative), a run of three tools fully succeeds 97.03 percent of the time, so 2.97 percent of runs contain a failure; with ten tools the failure share is 9.56 percent.1
A partial result needs a label: which parts finished, which did not, and why. Without that label the next reader treats an incomplete answer as complete.
Sequential steps that change state are the harder case. If step 1 already committed, such as a booked flight, and step 2, the hotel, fails, either run step 1's compensating action (the undo) or record it as done and resume from a checkpoint instead of redoing it. This is the idea of sagas (Garcia-Molina and Salem, 1987): a long job split into steps that each have a compensating action. Report which state changes stand. If a dependency keeps failing, stop calling it and use a fallback tool.
Step 5: Stop, report and escalate when retries stop helping
An agent stops retrying when more attempts no longer help, then tells the user plainly what happened. The Amazon article puts it this way: "We stop retrying when the retries are not helping to improve availability."
A retry budget is a cap on how many retries a client may spend, for example a local token bucket that earns a little credit for each success and spends one credit per retry. If each success earns one tenth of a credit (illustrative), retries stay at most about 10 percent of successful calls, plus the bucket's starting credit; the budget is a cap, not a target.1
When the budget is spent, report three things: what finished, what failed, and whether any side effect may have happened. An irreversible action with an unknown outcome goes to a person; see human in the loop AI agents.
In short, a failed tool call returns an error result to the model as data. Only safe transient errors are retried, at one layer, with backoff and jitter. Successful results come back with a label that says what is missing. When the retry budget is spent the agent stops and reports honestly.
Step-by-step description
- The failure goes back to the model.
- Only transient errors qualify.
- One layer, backoff and jitter.
- Successes come back labeled.
- The budget is spent, so say so.
- A failed tool call returns an error result to the model as data.
- Only safe transient errors are retried, at one layer, with backoff and jitter.
- Successful results come back with a label that says what is missing.
- When the retry budget is spent the agent stops and reports honestly.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
How would you handle tool calling failures and partial results in an agent?
I treat a failure as data, not an exception. Every failed call gets a paired result flagged as an error, with a message that says what went wrong and what to try next, so the model can recover or explain. Before retrying I classify: bad arguments are fixed, not repeated; transient errors get capped exponential backoff with jitter, at one layer only, under a retry budget. A timeout does not prove nothing happened, so side-effecting tools carry an idempotency key. When some calls fail, I return the successes labeled as partial, name what is missing, and let the model answer with caveats. If the budget runs out, I stop and report, and irreversible actions go to a person.
This is Staff-level because it separates retryable from non-retryable errors, treats a timeout as an unknown outcome, and says what the user is told. The spoken structure: failure as data, classify, bound the retries, label the partial result, then stop and escalate.
The rubric
This rubric is our own framing of what a strong answer shows, not a company's scoring sheet.
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Wraps the call in try and except, then retries. | Returns the error as a tool result and retries transient errors only. | Adds idempotency keys for side effects and treats a timeout as an unknown outcome. |
| Depth | Names retry as the only tool. | Adds backoff with jitter and a timeout on every call. | Retries at one layer and caps the total with a retry budget. |
| Trade-offs | Retries until it works. | Says retries add load and delay. | Weighs a longer wait against load amplification and against telling the user early. |
| Communication | Says handle the error, with no sequence. | Walks through classify, retry, return, report in order. | Says what the user is told, including which parts finished and what may have happened. |
| Follow-up readiness | Has no answer for a timeout that did the work. | Knows idempotency makes a retry safe. | Also compensates or resumes committed side effects, and covers partial results, strict mode limits and untrusted tool output. |
Three variants of this question
What if the failed tool is the one the whole task depends on?
Then there is no useful partial result, and the agent must not invent one. Retry within the budget, and if the tool still fails, stop and tell the user which step could not run and what they can do next.
How do you retry a tool call that charges a card or sends an email?
Only with an idempotency key, reused on every retry, so a repeat returns the first outcome instead of acting twice. If the tool offers no key and the outcome is unknown, do not retry; report the uncertainty or ask a person.
What do you do when the model keeps calling a tool with bad arguments?
Return an instructive error that names the bad field, and cap the number of attempts. Anthropic's docs say Claude retries invalid calls two to three times with corrections before apologizing (Anthropic, 2026), and strict tool use removes schema-invalid calls.
Common mistakes
- "I catch the exception and end the run." (wrong) The loop dies and the user learns nothing. (right) "I return the failure to the model as a tool result with
is_errorset, so it can recover or explain." - "I retry every failure three times at every layer." (wrong) Retries multiply: three retries across five layers is 243 times the load (Amazon Builders' Library, 2019). (right) "I retry at one layer, with backoff, jitter and a budget."
- "A timeout means it failed, so I retry." (wrong) The side effect may have happened. (right) "A timeout is an unknown outcome, so side-effecting calls carry an idempotency key."
- "With strict mode on, tool calls cannot fail." (wrong) OpenAI's documentation says strict mode makes generated arguments follow the schema, on supported schemas (OpenAI function calling guide); the tool itself can still error or time out, and the Help Center adds that the schema match holds only when the schema uses the supported subset. (right) "Using the strict setting fixes argument shape; I still handle runtime failures."
- "I return the text failed." (wrong) The model cannot act on it. (right) "I say what went wrong and what to try next."
Follow-up questions
How do you stop a tool result from hijacking the agent?
Treat tool output as untrusted input. Anthropic's docs note that results often carry content from outside your control, such as webpages, inbound email, uploads and third-party APIs (Anthropic, 2026), so the agent should not follow instructions found inside them.
Does strict mode remove the need for error handling?
No. With strict mode on, arguments match the schema, but the tool can still return an error, time out or give empty data. OpenAI recommends always enabling strict mode (OpenAI function calling guide), and it requires additionalProperties set to false and every property marked required.
Should you turn off parallel tool calls to avoid partial results?
You can: setting parallel_tool_calls to false ensures zero or one tool is called per turn (OpenAI function calling guide). The trade-off is simpler failure handling against more round trips and a slower run.
How do you test failure handling in an agent?
Inject faults on purpose: a rate limit, a timeout after the side effect, invalid arguments and an empty result. Then check that the final message names what is missing. Evaluating agents covers trajectory and tool-accuracy measures.
When should a person step in?
When the action is irreversible and its outcome is unknown, or when the retry budget is spent on a task that matters. The human in the loop pattern gives the person the facts the agent collected.
How does this fit into a full agent design?
Failure handling is the reliability layer around the tool loop, next to memory and safety. Designing an AI agent system shows where it sits in the whole design.
- Return every failure to the model as a tool result with
is_errorset, never as a crash. - Retry only transient errors, at one layer, with capped backoff and jitter.
- A timeout does not prove the action did not happen, so side effects need an idempotency key.
- Return successes as a labeled partial result that names what is missing.
- When the retry budget is spent, stop, report honestly and escalate irreversible cases.
Sources
- Handle tool calls (Claude Platform Docs) — Anthropic, 26 Oct 2026
- Timeouts, retries, and backoff with jitter (Amazon Builders' Library, Marc Brooker) — Amazon Web Services, 5 Dec 2019
- Function calling (OpenAI developer docs) — OpenAI, 26 Oct 2026
- Function Calling in the OpenAI API (Help Center) — OpenAI, 26 Oct 2026
- Sagas (Garcia-Molina and Salem, ACM SIGMOD 1987) — ACM, 1 Dec 1987
