In this article
Before you start
This page belongs to the AI agent architecture path and assumes the basics of prompt engineering.
Why interviewers ask this
The question tests judgment, not enthusiasm. A candidate who has only built agents sounds like someone who reaches for the newest pattern; a candidate who can say when not to is someone a team can trust with budget and risk. It is reported by candidates in AI-engineering interviews, and it matches the stance of Anthropic's guide to building effective agents, which says the simplest solution "might mean not building agentic systems at all."
A weak answer is vague ("agents are risky") or absolute ("I never use them"). A strong answer names a concrete task, the criterion that ruled an agent out, and what was built instead. A Staff-level answer also says what evidence would change the decision, and how the choice was measured rather than assumed.
Agent or no agent from first principles
- Compounding error
- A compounding error is a mistake in an early step of a multi-step run that is carried into every later step. The chance that a whole run is correct therefore falls as steps are added, even when each single step is usually right, unless a later step catches and fixes the error.
In plain words
Think of a bus and a taxi. A bus follows a fixed route, so you know the stops and the cost before you board. A taxi is given an address and chooses its own route. That helps when roads are unpredictable, but it costs more. The analogy breaks in one place: a model's route can differ from run to run, and every choice is paid for in tokens.
An agent is a system in which a language model dynamically directs its own process and tool use, keeping control over how it accomplishes the task, as Anthropic describes it. A workflow is the bus. OpenAI's guide adds that a simple chatbot, a single-turn model call or a sentiment classifier is not an agent. The rest of the page runs four checks; the first one that fails points to a simpler design.
Step-by-step description
- Predictable steps call for a workflow.
- Test one call with retrieval first.
- High stakes or cost argue against it.
- Only then build one agent.
- Predictable steps call for a workflow rather than an agent.
- One model call with retrieval is the next simpler option to test.
- High stakes or a failing cost case argue against an agent.
- An agent needs success criteria you can measure.
Step 1: Check whether the steps are predictable
Predictable steps call for a workflow rather than an agent. Anthropic says agents suit open-ended problems where the number of steps is hard or impossible to predict and a fixed path cannot be hardcoded. If you can write the steps on a whiteboard, write them in code.
Input: the task description. What changes: you list the steps a careful person would follow. Output: either a fixed list, which means build a workflow, or a list with branches nobody can enumerate, which keeps an agent possible.
OpenAI gives three signals that favor an agent: complex decision-making, rules that are hard to maintain, and heavy reliance on unstructured data. If the use case does not clearly meet them, the guide says "a deterministic solution may suffice." You now know the first filter; the next one asks whether a single call can already do the work.
Step 2: Check whether one call is enough
One model call with retrieval is the next simpler option to test. Anthropic writes that for many applications, optimizing single calls with retrieval and in-context examples is usually enough. Retrieval means looking up relevant documents and adding them to the prompt; in-context examples are sample inputs and outputs written into the prompt, the subject of prompt engineering.
Input: a real set of cases. What changes: you write the prompt, add retrieval and examples, and run it on those cases. Output: a pass rate. If the single call passes, that is your answer, and "we measured the single call and it was enough" is a stronger sentence than "an agent seemed too complex."
Anthropic's advice is to start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short. If the single call fails, the third check weighs what a loop would cost.
Step 3: Check the stakes and the cost of a loop
A loop costs more tokens, adds delay and puts your risk in the model's hands, so check the stakes and the cost before building. Anthropic says agentic systems often trade latency and cost for better task performance, and that autonomy means higher costs and the potential for compounding errors. In its own data, agents use about 4 times the tokens of chat interactions and multi-agent systems about 15 times, so the task must be worth that.
| Design | Tokens per task (illustrative baseline) | Tokens for 10,000 tasks |
|---|---|---|
| One chat call | 2,000 | 20,000,000 |
| Single agent (about 4 times) | 8,000 | 80,000,000 |
| Multi-agent system (about 15 times) | 30,000 | 300,000,000 |
The baseline and task count are illustrative; the multipliers are Anthropic's, and the products are arithmetic.1
Errors compound too. A per-step success rate of 0.95 over ten steps gives 0.5987 for a fully correct run (0.8574 over three steps and 0.3585 over twenty), while 0.99 gives 0.9044 (0.9703 and 0.8179) (illustrative rates).1 This assumes each step succeeds independently and no error is caught and fixed; checks and retries raise the real figure, which is why validation steps matter. OpenAI adds that sensitive, irreversible or high-stakes actions should trigger human oversight until confidence in the agent's reliability grows; see human in the loop for agents. High stakes or a failing cost case argue against an agent.
Text description of the diagram
- At 0.95 per step, ten steps all succeed 0.5987 of the time.
- At 0.99 per step, ten steps all succeed 0.9044 of the time.
Step 4: Check that you can measure success, then decide
An agent needs success criteria you can measure. Anthropic's appendix says agents add the most value where tasks have clear success criteria, enable feedback loops and integrate meaningful human oversight. Without a test set or a check such as passing unit tests, you cannot tell whether the loop helps, and the earlier advice to add complexity only when it demonstrably improves outcomes cannot be followed. The guide to evaluating agents covers trajectories and tool accuracy.
Two illustrative scenarios show the decision. An invoice-field extractor has the same steps every time and a checkable output, so one call with a schema, or a short workflow, wins. A refund-approval flow with nuanced exceptions matches OpenAI's own example of complex decision-making, so an agent is plausible, with a human-approval gate for large refunds.
Industry context points the same way. Gartner forecasts that over 40% of agentic AI projects will be canceled by the end of 2027. It is a forecast, not a measured outcome.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
When have you decided not to build an agent?
I looked at a task that extracted fixed fields from invoices. The steps were identical every time, so I used one model call with a schema and a validation step in plain code. An agent would have cost several times the tokens, added delay and left more places to fail, and nothing in the task needed the path to change. I measured the single call on real cases first, and it passed. I would revisit the decision if evaluation showed failures that only a loop with tools could fix. My rule is to start simple, measure, and add autonomy only when it demonstrably improves outcomes.
Use a project of your own; the structure matters, not this example. What makes it Staff-level is the last two sentences: a measured baseline, and the named trigger that would reverse the decision.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Treats any model application as an agent | Separates workflows, single calls and agents | Uses the sources' definitions and says where they differ |
| Depth | "Agents are risky" | Names the failed criterion, such as predictable steps | Quantifies cost and compounding error with stated assumptions |
| Trade-offs | None given | Cost, delay and risk against benefit | Adds when the benefit would justify the loop |
| Communication | Abstract and absolute | Concrete task, decision, outcome | Spoken in under a minute, with the reversal trigger last |
| Follow-up readiness | Stuck on "what would change your mind?" | Names one trigger | Names the metric, the threshold and the first design to add |
Three variants of this question
Tell me about a time you chose a workflow over an agent.
Pick a task with fixed steps, say what you measured about the single call or workflow, and say why a loop would have added cost without benefit. The structure is the model answer above.
How do you decide between a single agent and several agents?
Default to one: OpenAI recommends maximizing a single agent's capabilities first, noting that some implementations manage more than 15 well-defined tools while others struggle with fewer than 10 overlapping ones. Cognition's June 2025 post warns that "Actions carry implicit decisions, and conflicting decisions carry bad results" (Cognition). Anthropic reports that multi-agent systems fit valuable, heavily parallel work, as in a deep-research agent.
What if the task is high-stakes but an agent would help?
Build it with guardrails: a human approval step for sensitive or irreversible actions until reliability is shown, and limits on what the agent may do. Name the action that triggers review.
Common mistakes
- "I never build agents." (wrong) It signals a rule, not judgment. (right) "I start with the simplest design and add an agent when measurements show it helps."
- "Any app that calls a model is an agent." (wrong) OpenAI says single-turn calls, chatbots and classifiers are not agents. (right) "This was a workflow, because my code set the order of the steps."
- "I would use several agents to be faster." (wrong) More agents cost more tokens and can make conflicting choices. (right) "One agent first; more only if parallel, read-heavy work justifies the cost."
- "It felt too risky." (wrong) No criterion is named. (right) "The action was irreversible, so I would need a human approval gate."
- "I would see how it goes." (wrong) There is no way to judge the result. (right) "I would define success criteria and a test set before choosing a design."
Follow-up questions
What is the difference between a workflow and an agent?
In a workflow, predefined code paths decide the order of steps; in an agent, the model directs its own process and tool use. Anthropic treats both as agentic systems.
When is an agent the right choice?
When the problem is open-ended, the steps cannot be predicted, success can be measured and the cost is worth it. The agent system design guide covers the loop, tools, memory and safety.
What evidence would make you build the agent later?
Measured failures of the single call or workflow that depend on intermediate results, task paths that vary too much to hardcode, and a task value high enough to pay for the extra tokens.
Why do agentic AI projects get canceled?
Gartner names escalating costs, unclear business value and inadequate risk controls, and says many use cases positioned as agentic do not require agentic implementations.
Is multi-agent design a bad idea?
No; the advice is to default to one agent and add more only for valuable, heavily parallel work. More questions like this are in agentic AI interview questions.
- Decide against an agent when the steps are predictable, one call with retrieval is enough, stakes or cost are high, or success is unmeasurable.
- Anthropic says to add complexity only when it demonstrably improves outcomes, and OpenAI says to maximize a single agent first.
- Agents use about 4 times the tokens of chat, and multi-agent systems about 15 times.
- Errors compound: ten steps at 0.95 per step finish correctly 0.5987 of the time.
- A Staff answer names the measurement and the evidence that would reverse the decision.
Sources
- Building effective agents — Anthropic Engineering (page modified 2026-10-09), 19 Dec 2024
- A practical guide to building agents — OpenAI (undated; released about April 2025), 1 Apr 2025
- Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 — Gartner newsroom, 25 Jun 2025
- How we built our multi-agent research system — Anthropic Engineering, 13 Jun 2025
- Don't Build Multi-Agents — Cognition (Walden Yan), 12 Jun 2025
