Should you build agent orchestration yourself or use a framework like LangGraph?

In this article

1. Before you start
2. Why interviewers ask this
3. Agent orchestration from first principles
4. The Staff-level answer
5. The rubric
6. Three variants of this question
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page sits on the AI agent architecture path and assumes you have met prompting basics.

Why interviewers ask this

The wrong answer to this question costs a design round its credibility, because it shows a candidate who picks tools by fashion. A design round on an agent system tends to reach the build-or-buy choice early, and the agent system design guide shows where it sits in a full answer.

A weak answer says "frameworks save time" or "frameworks are bloated" and stops. A strong answer names what a framework adds, what it hides, and the condition that flips the choice. A Staff-level answer ties the choice to failure: what happens to a half-finished run when a process dies, a person takes two days to approve, or a deploy restarts the server. Interviewers want a rule you could defend, not a brand preference.

Agent orchestration from first principles

Agent orchestration

Agent orchestration is the control flow around model calls and tool calls: which step runs next, what state carries between steps, when a run stops, and what happens when a step fails or must wait for a person.

Agent runtime

An agent runtime is the production machinery that runs agents: it stores state, resumes runs after failures or pauses, and streams progress. LangChain's documentation places frameworks above runtimes and names LangGraph as a runtime.

In plain words

Think of a cook working from a paper checklist in a small kitchen. If the cook is interrupted, the checklist in their head is gone. A restaurant with an order-ticket system keeps every ticket on a rail, so any cook can pick up a half-finished order. Writing your own loop is the paper checklist. Adopting a runtime is the ticket rail. The analogy breaks in one place: a ticket rail also makes you learn its rules, and a runtime's rules can hide what your prompts do.

In one precise sentence, the choice is between control flow you write and read yourself, and control flow plus saved state that someone else maintains.

Step 1: The loop you can write yourself

The smallest orchestration is a loop of model calls and tool calls, and that is enough for a short run. Anthropic (December 2024) puts it plainly: agents are "typically just large language models (LLMs) using tools based on environmental feedback in a loop" (LLM stands for large language model). The post notes that much of its tooling survey has changed since December 2024; its start-simple advice is the part used here.

The same post separates two kinds of system. A workflow is a system in which models and tools are orchestrated through predefined code paths. An agent is a system in which the model directs its own process and tool use. Both can start as a few functions; an agent adds a while loop.

Your loop needs a stopping rule, such as a maximum number of iterations, and a place for each tool result. Anthropic's advice is to start with direct calls to the large language model's application programming interface (API), because many patterns take only a few lines. You now know the baseline: a loop with no memory beyond the process it runs in.

Step 2: A crash during a human pause shows what a runtime saves

Take a refund agent with six steps (illustrative): read the order, check the policy, draft a decision, wait for human approval, issue the refund, send the email. The wait may last hours or days. Suppose 200 runs are waiting (illustrative) when a deploy restarts the process. A loop that keeps state in the process memory loses all 200 of them.¹

The second danger is a repeat. If a hand-rolled loop restarts a crashed run from the start, then without saved state a crash repeats every finished step plus the one that was running. If the crash hits during a random one of the six steps (illustrative), the run repeats 3.5 steps on average, and in 2 of the 6 crash points (illustrative) the refund is issued twice.¹

A checkpointed run resumes after the last saved step, so it repeats at most the step that was running when the crash hit. That is why the refund step needs an idempotency key, so a repeated call cannot pay twice. An idempotency key is a unique id sent with a request so the receiver ignores a repeat of it.

A checkpoint is written after a step finishes. If the crash lands after the refund call returns but before the checkpoint is written, that step runs again, so 1 of the 6 crash points (illustrative) still pays twice without the key.¹ LangGraph's durable execution page advises designing workflows to be idempotent and wrapping side effects in tasks for this reason.

Text description of the diagram

• Without saved state a crash repeats every finished step plus the one that was running.
• A checkpointed run repeats only the step in flight.

You can fix this by hand with a database and a resume routine, and that routine is the work a runtime sells.

Step 3: What a runtime like LangGraph adds

LangChain's documentation describes LangGraph as a low-level orchestration framework and runtime for long-running, stateful agents. Its stated benefits include persistence, so agents resume where they left off after failures, and human-in-the-loop (HITL) oversight, where you inspect and change state at any point. It also lets you mix hand-coded logic and model-driven decisions in one graph.

Durable execution is saving a run's progress at key points so it can pause and later resume where it stopped, as the durable execution page describes it. A checkpoint saves progress so a paused run resumes after a crash. In LangGraph, checkpointers persist a thread's graph state as checkpoints, and a thread_id names the run. The checkpoint is saved when the run pauses, before the wait, so a restart during the wait is survivable.

1 · The loop2 · Pause and crash3 · The resumeReplay

Step-by-step description

1. Model and tool calls make a short run.
2. State is saved before the wait, so a restart is survivable.
3. The run continues from the saved checkpoint once approved.

• A loop of model calls and tool calls is enough for a short run.
• A checkpoint saves progress so a paused run resumes after a crash.
• The checkpoint is saved when the run pauses, before the wait, so a restart during the wait is survivable.

LangGraph "does not abstract prompts or architecture," says its overview, which answers part of Anthropic's worry about hidden prompts. The overview says LangGraph is built by LangChain Inc but "can be used without LangChain", so choosing it does not mean adopting that whole stack. The human-in-the-loop agents page covers approval design in more depth.

Step 4: The decision rule, and its traps

The rule below is our synthesis of Anthropic's advice and LangChain's own criteria, not a quote from either. Build it yourself when a run is short, stateless and lives in one process. Use a runtime when you need pause and resume, long runs, human approval or crash recovery. LangChain's documentation lists the same triggers for LangGraph: fine-grained control, durable execution for long-running stateful agents. It also says straightforward agent applications without complex orchestration needs suit a higher-level framework. That is vendor documentation, so weigh it accordingly.

Traps remain even with a runtime. The in-memory checkpointers lose every checkpoint when the process restarts, and checkpoints accumulate over long conversations, raising latency and storage costs, per the persistence documentation.

The Staff-level answer

What makes it Staff-level: it gives a condition that flips the choice, names a failure mode of the framework itself, and ends on a production detail.

The rubric

Three variants of this question

Do you need a framework to build an AI agent at all?

No. Anthropic describes agents as typically just a model using tools in a loop, and suggests starting with direct API calls. A framework helps once the run needs saved state or approvals.

When is a hand-written agent loop enough?

A hand-written loop is enough when a run is short, finishes inside one process, and can be safely started over if it fails. Add a stopping rule and log every prompt.

What if the agent must wait days for a human approval?

Then in-process state cannot survive, so you need checkpoints in persistent storage, either your own or a runtime's. Design each step to be repeatable so a resume never doubles a side effect such as a refund.

Common mistakes

• "An agent needs a framework." (wrong) Agents are typically a model using tools in a loop. (right) "A framework is optional; I choose one for saved state and resume."
• "Frameworks are always bad." (wrong) Anthropic warns about hidden prompts; it does not say never use one. (right) "If I use one, I understand its code and can reduce abstraction later."
• "My checkpointer makes state durable." (wrong) In-memory savers lose state on restart. (right) "Production uses a database-backed checkpointer."
• "LangGraph and LangChain are the same thing." (wrong) LangGraph can be used without LangChain. (right) "LangGraph is the runtime; higher-level frameworks sit above it."

Follow-up questions

Is LangGraph the same thing as LangChain?

No. LangChain's documentation says frameworks are generally higher level and run on agent runtimes, and that LangGraph can be used without LangChain.

What breaks if you use an in-memory checkpointer?

Every checkpoint is lost when the process restarts, so paused runs vanish. Use a persistent store in production and prune old checkpoints.

When would you pick a durable execution engine instead of a graph framework?

LangChain's documentation lists Temporal and Inngest as examples of agent runtimes, alongside other durable execution engines. Choose one when your team already runs it for other workflows; the LangGraph, CrewAI and AutoGen comparison covers the framework side.

How do you stop a framework from hiding your prompts?

Keep prompts in your own files, log every model request, and read the framework's source. Anthropic notes that incorrect assumptions about what is under the hood are a common source of error.

Does a framework make an agent more accurate?

No source here says so. A runtime adds reliability, not answer quality, so measure quality with agent evaluation. More practice questions are in the agentic AI interview questions, and the goal-based agents page explains planning.

Sources

1. Building effective agents — Anthropic Engineering, 19 Dec 2024
2. LangGraph overview — LangChain Docs, 26 Oct 2026
3. Runtimes, frameworks, and harnesses — LangChain Docs, 26 Oct 2026
4. Persistence (LangGraph) — LangChain Docs, 26 Oct 2026
5. Durable execution (LangGraph) — LangChain Docs, 26 Oct 2026