Scaling long-horizon LLM agents via context-folding to keep the context small

In this article

1. Before you start
2. Why interviewers ask this
3. Context folding from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one step on the AI agents learning path.

Why interviewers ask this

Long tasks break agents long before they break the model's context limit. An agent that browses fifty pages or edits a large code base fills its window with tool output, and its accuracy drops as the window fills. Interviewers use context folding to separate candidates who treat context as unlimited from candidates who treat it as a budget to be spent and managed.

A weak answer says "use a bigger context window" or "summarise the history." A strong answer says what is kept, what is dropped, who decides, and what is lost when a summary is wrong. The follow-up is nearly always the same: what happens when the dropped detail turns out to matter later?

Context folding from first principles

Context folding

Context folding is a technique that lets an LLM agent split a long task into sub-tasks, run each in a branch that is removed from the context on return, and fold the finished branch into a short message so that only the outcome stays in the main context.

Long-horizon task

A long-horizon task is a task that needs many sequential tool calls, such as deep research across many web pages or fixing a bug across many files, so the record of the work outgrows a single context window.

Return message

A return message is the short text an agent writes at the end of a branch to report the outcome of the sub-task to the main thread, and it is the only part of the branch that survives the fold.

In plain words

Think of a manager who asks an analyst one question. The analyst reads for two days. Then the analyst sends back a half-page note. The manager never reads the source material. The note is what enters the file. The analogy breaks in one place. A human analyst can be phoned for details later. A folded branch is gone, so the note must carry everything the manager will need.

In one precise sentence: the agent gets two extra tools, one to start a branch and one to end it, and an ended branch leaves the context that later steps read. The idea targets ReAct. ReAct is a prompting pattern in which a model alternates reasoning and tool calls (Yao et al., October 2022), and it appends every step to the context. The active context is the set of tokens the model reads at its current step.

Step 1: A flat agent appends everything

A flat agent adds each reasoning step and each tool result to one growing context. The input is the task; the output after n steps is a context that holds all n steps. The paper models this directly: the agent's next action depends on the question and the full history of earlier actions and observations.¹

Two things go wrong as the history grows. The history can exceed the window, and before that happens the model gets worse at using it. Liu et al. (July 2023) found that models do best when the relevant fact sits at the start or end of a long input and worse in the middle, and Chroma's July 2025 report tested 18 models and found performance becomes less reliable as input length grows. What you now know: more context is not free, even before it is full.

Step 2: Branch to open a separate context

The agent calls a tool named branch with two fields: a short description of the sub-task and a detailed prompt for it. The input is a self-contained sub-task, such as "find the founding year of the company that made this product"; the output is the same context with the branch call appended: the branch reads the main thread above it, and everything it adds from here on is removed when it returns.¹

The main thread itself does not move on while the branch is open. Its job in this design is planning: the paper's plan-execution scheme discourages token-heavy tool use in the main thread and disables new branches while a branch is running, so branches do not nest. A good sub-task is independent, has a checkable answer and is expected to need many tool calls; a one-call lookup does not earn a branch. What you now know: a branch is a temporary stretch of context with a clear question.

Step 3: Work inside the branch

Inside the branch the agent runs the ordinary loop: search, read, run code, read the output. Every tool result lands in the branch part of the context, which is removed on return. This is where the tokens go: in the illustrative run below, each step adds about 2,300 tokens and one branch of eight steps adds about 18,400.²

The agent stops the branch when it can answer the sub-task or decides it cannot. The input to this step is the branch prompt and the tools; the output is a finished piece of work, still inside the branch. The branch can be long, but the main thread and the open branch share one window (32,768 tokens in the paper¹), so a branch still has to fit beside what the main thread already holds. The one rule is that the agent must finish by calling return. What you now know: the cost of exploring is paid in the branch.

Step 4: Return and fold, and keep only the message

The agent calls return with a message that describes the outcome, and the context switches back to the main thread with only that message appended. Everything the branch did is folded away.¹ The paper also notes an inference benefit: when return is called, the system rolls its key-value (KV) cache back to the position where the branch began, so the prefix is reused; the KV cache page explains why that matters.

The return message is the missing term in most candidate answers. It must state the result, the evidence the main thread needs to trust it, and anything unresolved. The paper names two failure modes it had to train against: agents leaving token-heavy work in the main thread, and agents failing to return and carrying on inside a branch.¹ What you now know: the quality of the fold is the quality of the message.

1 · Plan in main2 · Branch3 · Work4 · Return and foldReplay

Step-by-step description

1. The main thread holds the plan and results so far.
2. A branch call opens a separate working context.
3. Tool calls fill the branch, not the main thread.
4. Only a short message returns to the main thread.

• The agent calls a tool named branch with two fields: a short description of the sub-task and a detailed prompt for it.
• Every tool result lands in the branch context, not the main thread.
• Everything the branch did is folded away.

Worked example

The numbers below are illustrative assumptions, not measurements, and the script that produced them is saved with the page.² Assume a 1,500-token prompt, 300 tokens the model writes per step, 2,000 tokens of tool output per step, and a task of 64 tool calls. A flat agent ends with 1,500 + 64 × 2,300 = 148,700 tokens in one context. A folding agent that opens a branch every 8 calls, spends 200 tokens opening each and keeps a 400-token return message, never holds more than 24,300 tokens, and its main thread ends at 6,300.²

The last row uses the per-token cache size of Llama 3 70B computed on the KV cache page; it shows scale, not a real deployment, because that model's own window is shorter than 148,700 tokens.²

Text description of the diagram

• A flat agent ends with 1,500 + 64 × 2,300 = 148,700 tokens in one context.
• The folding agent never holds more than 24,300 tokens.

What the paper measured is different from this toy. It used one open 36-billion-parameter base model, a 32K-token active budget and at most 10 branches.

On BrowseComp-Plus, a deep-research benchmark, the untrained model scored 0.286 pass at one as a ReAct agent limited to 32K, 0.386 as a summarisation-based agent, 0.478 as a ReAct agent with 327K tokens and 0.420 as a folding agent. After reinforcement learning, the 327K ReAct agent scored 0.540, the summary agent 0.527, the folding agent with plain GRPO (Group Relative Policy Optimization) 0.567 and the folding agent with FoldGRPO 0.620. On SWE-Bench Verified, a software-engineering benchmark, the like-for-like margin is thin: 0.580 for FoldGRPO folding against 0.574 for the trained 327K ReAct agent.¹ The training method, FoldGRPO (Folded-context Group Relative Policy Optimization), is reinforcement learning that rewards good branching, not only the final answer.

Text description of the diagram

• Before reinforcement learning the 327K-token ReAct agent scored 0.478 and the folding agent 0.420 on BrowseComp-Plus.
• After reinforcement learning the folding agent (FoldGRPO) scored 0.620 against 0.540 for the 327K-token ReAct agent.

Trade-offs and edge cases

Context folding trades fidelity for headroom: whatever the return message omits is gone from the main thread. The paper's untrained folding agent, before reinforcement learning, scored 0.420 on BrowseComp-Plus, above the 32K ReAct agent at 0.286 but below the 327K ReAct agent at 0.478.¹ The method works because the model is trained to branch and return well; prompting alone gets partway.

Three edge cases matter in practice. First, dependent sub-tasks: if branch two needs a detail branch one saw but did not report, the agent must re-do the work or fail. Second, branches that do not return: an agent that never calls return keeps adding to one context, which is the flat agent again. Third, evidence: a folded branch cannot be inspected later unless the system stores it outside the context, so production systems log the branch for debugging; see LLM observability.

Other methods spend context differently. Anthropic's September 2025 engineering post describes compaction (summarising a nearly full conversation), structured notes kept outside the window, and sub-agents that return condensed summaries, often 1,000 to 2,000 tokens.³ Folding sits between compaction and sub-agents. The paper (§2.4) lists three differences from sub-agent systems: branches are created on the fly rather than predefined, they share the same context prefix with the main agent (which is friendly to the KV cache), and they interleave with the main agent instead of running in parallel.¹

How to answer it in an interview

The spoken structure: say the budget idea, describe branch and return, name the cost of a lossy message, then say how you would test it.

Common mistakes

• "Just use a longer context window." (wrong) Longer windows still degrade with length, and every step re-reads more tokens. (right) "Even with a long window I would keep the active context small, because accuracy and cost both worsen as it grows."
• "Folding is the same as summarising the chat." (wrong) A summary runs when the window is full and compresses everything; a fold is chosen by sub-task and drops only a finished branch. (right) "Folding removes completed sub-task work and keeps its result."
• "The return message can be one line." (wrong) A bare "done" loses the evidence the main thread needs. (right) "The message carries the result, the proof and open questions."
• "More branches are always better." (wrong) A one-call lookup costs more to open and return than it saves. (right) "I branch only for work that needs many tool calls."

Follow-up questions

What is context folding in an LLM agent?

Context folding is a technique in which an agent branches into a sub-task, completes it in a temporary branch of the context and returns a short message, so only the outcome stays in the main context. The agent controls this through two tools, branch and return (Sun et al., October 2025).

How is context folding different from summarising the conversation?

A summary compresses the whole history when the window fills; a fold removes one finished sub-task and keeps its result. In the paper's tests, both trained with reinforcement learning, a summarisation-based agent scored 0.527 and the folding agent 0.620 on BrowseComp-Plus; untrained, they scored 0.386 and 0.420.¹ The agent decides when to fold instead of waiting for the limit.

Why not just use a longer context window?

Because long inputs are used unevenly: models recall middle content worse than the start or end (Liu et al., 2023), and each step still reads everything. In the paper a 32K-budget folding agent trained with FoldGRPO scored 0.620 against 0.540 for a similarly trained 327K ReAct agent.¹

What if the return message leaves out something the agent needs later?

The agent must re-run the sub-task or fail, which is the main cost of folding. Mitigations are richer return messages, storing full branches outside the context so they can be searched, and a human check for high-stakes steps; see human-in-the-loop agents.

How is context folding different from using sub-agents?

Folding branches differ from sub-agents in three ways (paper §2.4). They are created on the fly, not predefined. They share the main agent's context prefix, which is KV-cache friendly. They interleave with the main agent instead of running in parallel.¹ For tool access and agent plumbing, see the Model Context Protocol overview and the agentic AI interview questions.

Sources

1. Scaling Long-Horizon LLM Agent via Context-Folding (arXiv 2510.11967v1) — arXiv (Sun, Lu, Ling, Liu, Yao, Yang, Chen), 13 Oct 2025
2. ReAct: Synergizing Reasoning and Acting in Language Models (arXiv 2210.03629) — arXiv (Yao et al.), 6 Oct 2022
3. Effective context engineering for AI agents — Anthropic Engineering, 29 Sep 2025
4. Lost in the Middle: How Language Models Use Long Contexts (arXiv 2307.03172) — arXiv (Liu et al.), 6 Jul 2023
5. Context Rot: How Increasing Input Tokens Impacts LLM Performance — Chroma Research, 14 Jul 2025