How do you code a minimal agent loop and a minimal RAG pipeline?

In this article

1. Before you start
2. Why interviewers ask this
3. A minimal agent loop and RAG pipeline from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page belongs to the machine learning (ML) coding round guide, which covers the other from-scratch problems of that round.

Why interviewers ask this

The task tests whether a candidate can build something small that works and then reason about how it fails. Candidates report live coding rounds where they are handed access to a model backend and asked to build a small agent or a retrieval tool, and report a round advertised as a data structures and algorithms exercise that turned into redesigning agent code that calls tools. These are candidate reports, not a company policy.

A weak answer pastes a framework call and cannot say what happens between two model calls. A strong answer writes the loop by hand, names the stopping rule, handles a tool error, and says what the retriever gets wrong. The code is short; the judgment about its failure modes is what separates answers.

A minimal agent loop and RAG pipeline from first principles

Step limit

A step limit is a fixed maximum number of model calls an agent loop may make, so a model that never answers cannot run forever.

Dense retriever

A dense retriever is a search component that embeds the query and each passage into vectors and ranks passages by closeness in that vector space, so it can find paraphrases that share no words, at the cost of an embedding model and an index.

In plain words

Picture a research assistant at a desk with a phone. You ask a question; the assistant either answers or says "look up X". You do the lookup, hand back the page, and the assistant decides again. You stop after a set number of lookups so the assistant cannot send you around forever. The analogy breaks in one place: the model has no memory between calls, so you resend the whole conversation each time.

In one precise sentence, the model is a function from messages to a next message, and the agent is the ordinary loop you wrap around it. Anthropic (December 2024) describes agents as typically "just large language models (LLMs) using tools based on environmental feedback in a loop", where a large language model (LLM) is the model being called. Yao et al. (2022) study the interleaving of reasoning and actions that this loop carries out.

Step 1: The loop calls the model and checks for a tool request

The loop starts with a message list holding the user's text and calls the model with it. The model's reply is one of two kinds: a final answer, or a tool call, which is a structured request naming a tool and its argument. Input: the message list. Output: a reply you can branch on. If the reply is final, return it and stop.

The branch is the heart of the loop, and it is plain if code. Real application programming interface (API) clients differ in field names, but each returns something equivalent to "answer" or "tool request", so write the branch against your own small dictionary and keep vendor details at the edge. You now know where the loop decides to continue.

Step 2: The host runs the tool and appends the result

The program, not the model, executes the tool. It looks the requested name up in a dictionary of Python functions, calls the function with the argument, and appends two messages: the model's request and the tool's result. Input: a tool call. Output: a longer message list. The next model call treats the result as feedback from the environment, and Anthropic's agents post notes that agents need this feedback at each step. An unknown tool name raises a KeyError, which the except below returns to the model as text. On the last step a requested tool is still run but its result is never shown to the model, a wasted call that a stricter loop would skip.

def run_agent(call_model, tools, user_text, max_steps=5):
    messages = [{"role": "user", "content": user_text}]
    for step in range(1, max_steps + 1):
        reply = call_model(messages)           # the model sees the whole history
        if reply["type"] == "final":           # no tool requested: done
            return reply["text"], step
        try:
            result = tools[reply["name"]](reply["arg"])   # the host runs the tool
        except Exception as err:               # report errors back to the model
            result = f"error: {err}"
        messages.append({"role": "assistant", "content": reply})
        messages.append({"role": "tool", "content": result})
    return None, max_steps                     # step limit reached

Step 3: The step limit stops a loop that never finishes

A model can keep requesting tools forever, for example by repeating the same search. The for loop above bounds the damage: after max_steps model calls it returns a clear "no answer" result instead of hanging and spending money. Anthropic's agents post notes that agent tasks usually end on completion but that it is also common to include stopping conditions, such as a maximum number of iterations, to maintain control.

The limit is a safety floor, not a fix. This loop returns None with the step count, and None tells the caller the limit was hit; a larger system would return an explicit stop reason, so the caller can retry with a hint or show an honest failure. The worked example below runs a stuck model through this guard. You now have a complete, safe agent loop; the next two steps give it something useful to call.

Step 4: Retrieval is the tool: rank passages by similarity

A retriever turns every passage and the query into word-weight vectors and ranks passages by cosine similarity, the cosine of the angle between two vectors, which is larger when they share more weighted words. Weights come from term frequency (TF), which counts a word in a passage, and inverse document frequency (IDF), which gives rare words more weight than common ones; multiplying them gives the TF-IDF weight. Input: a query and a list of passages. Output: the top k passages, where top-k means the k highest-scoring items.

Lewis et al. (2020) combine pre-trained parametric memory with a non-parametric memory, a dense vector index of Wikipedia, and fine-tune the retriever's query encoder and the generator together; the minimal version here only pastes retrieved passages into the prompt, so it needs just a function that returns relevant text. A chunk is one stored passage, usually a few sentences cut from a longer document.

import math, re
from collections import Counter

STOP = {"the","a","of","is","to","are","on","in","and","with","how","long",
        "it","for","an","can","be","from","do","i","my"}

def tokens(text):
    return [w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in STOP]

def make_retriever(docs):
    toks = [tokens(d) for d in docs]
    df = Counter(w for t in toks for w in set(t))
    idf = lambda w: math.log((1 + len(docs)) / (1 + df.get(w, 0))) + 1
    vec = lambda ts: {w: c * idf(w) for w, c in Counter(ts).items()}
    dvs = [vec(t) for t in toks]

    def cosine(a, b):
        dot = sum(x * b.get(w, 0) for w, x in a.items())
        na = math.sqrt(sum(x * x for x in a.values()))
        nb = math.sqrt(sum(x * x for x in b.values()))
        return dot / (na * nb) if na and nb else 0.0

    def retrieve(query, k=2):
        qv = vec(tokens(query))
        ranked = sorted(range(len(docs)), key=lambda i: -cosine(qv, dvs[i]))
        return [docs[i] for i in ranked[:k]]
    return retrieve

Step 5: The retrieved text goes into the prompt, and the answer follows

The pipeline's last move is to paste the retrieved passages into the prompt and tell the model to answer from them. Input: a question and k passages. Output: one prompt string. A good instruction also says what to do when the passages lack the answer, because the model otherwise fills the gap from memory.

def rag_prompt(question, passages):
    context = "\n".join(f"[{i + 1}] {p}" for i, p in enumerate(passages))
    return ("Answer using only the passages. If they do not contain the answer, "
            f"say so.\nPassages:\n{context}\n\nQuestion: {question}")

To join the two halves, register retrieve as a tool named search in the tools dictionary of Step 2. The model then decides when to retrieve, which is the augmented LLM that Anthropic calls the basic building block of agentic systems: an LLM extended with retrieval, tools and memory. A fixed pipeline that always retrieves once is the simpler cousin.

1 · Call the model2 · Run the tool3 · Step limit4 · Retrieve5 · Build the promptReplay

Step-by-step description

1. The loop sends the history and checks the reply.
2. Your code runs the tool and appends the result.
3. A step limit ends a loop that never finishes.
4. Retrieval is the search tool and returns the best passages.
5. The passages go into the prompt.

• The loop calls the model and runs requested tools.
• A step limit ends a loop that never finishes.
• Retrieval returns the best passages for the prompt.

Worked example

A six-passage store and a scripted fake model make the run deterministic with no API key. The passages are illustrative support-policy sentences, and the question is "Can I get a refund after 30 days?".¹ After stop-word removal the query has five tokens, and the store has a 37-word vocabulary.¹

Only two passages score above zero. The refund-policy passage scores 0.2482 and the shipping passage scores 0.0998; the other four score 0.¹ The match comes from the tokens "30" and "days": the query word "refund" does not match "Refunds" because this retriever does no stemming, and so the gift-card passage, which mentions "refunded", scores 0. The shipping passage is second only because it contains "days".¹

The last row shows what top-k costs: each extra passage adds words to every later model call, so a larger k raises cost and can bury the useful passage.¹

Text description of the diagram

• A model that answers uses two steps and one tool call.
• A stuck model uses all five steps and five tool calls.

Text description of the diagram

• The refund-policy passage scores 0.2482 and the shipping passage scores 0.0998.
• The other four passages score zero.

Trade-offs and edge cases

Word matching is simple and explainable but misses synonyms and word forms. A dense retriever embeds text into vectors and finds paraphrases, at the cost of an embedding model and an index; a word-matching retriever is enough to start, and a candidate should say which one the time allows. Chunk size is a trade-off too: small chunks match precisely but lose context, and large chunks carry context but dilute the score.

On the loop side, the cases that matter are a tool that raises an error (return the error text to the model so it can adapt), a tool that returns a huge result (truncate it, since every later call resends it), and repeated identical tool calls (detect them and stop early rather than wait for the limit). Reported practice differs on whether to let a model call tools in a free loop or to use a fixed workflow; Anthropic's post (as of December 2024) advises adding agent complexity only when simpler designs fall short.

Expect the interviewer to read your code the way a reviewer would; the code review interview questions page lists what reviewers look for. The same from-scratch habit applies to models: see building generative pre-trained transformer (GPT) from scratch and scaled dot-product attention from scratch.

How to answer it in an interview

The spoken structure: loop first, then the stopping rule, then the error path, then retrieval as one tool, then how you would test it.

Common mistakes

• "I trust the model to stop by itself." (wrong) A model can request tools forever. (right) "The loop has a step limit and returns a clear no-answer result."
• "I let the model run the tool." (wrong) The model only emits a request. (right) "My code looks up the tool by name, runs it and appends the result."
• "RAG needs a vector database." (wrong) It needs a retriever, and a word-matching one works for a small store. (right) "I would start with TF-IDF and move to embeddings if paraphrases are missed."
• "I paste every retrieved passage." (wrong) Extra text costs tokens and can bury the answer. (right) "I pick a small k and tell the model to say so when the passages do not answer."

Follow-up questions

Do you need a vector database for RAG?

No. A vector database earns its place when the store is too large to scan on every query or when you need approximate search over embeddings; for a small store, scoring every passage as this page does is simpler and easier to explain.

How would you implement a response safety filter with scoring, caching and explainability?

Candidates report this as a coding prompt, and no public source gives an official answer, so treat it as a design exercise. Keep a list of rules, each returning whether it matched, a weight and a reason. Combine the weights into a score between zero and one, compare it with a threshold that a strictness setting moves, cache results by a hash of the text, and return the matched reasons as the explanation.

How do you choose top-k and chunk size?

Treat both as tunable settings and measure them on a held-out set of questions with known answers. A small k keeps the prompt short; a larger k raises the chance the right passage is included, at higher cost.

Does RAG stop a model from hallucinating?

No. Retrieval can miss the right passage, as the shipping passage ranked second in the worked example, and the model can ignore or misread what it is given. Lewis et al. (2020) report that RAG models generate more factual language than a parametric-only baseline: an improvement, not a full fix.

How do you keep tool output within a token budget?

Count the size of every message before each model call and cap each tool result, for example by truncating it, because every later call resends it. If the total still exceeds the budget, drop the lowest-scoring passages first and lower k, as the words-by-k row of the worked example shows.

How would you extend the loop to several agents?

Give each agent its own loop and tool set and let a coordinating agent call the others as tools; the step limit then applies at every level, a pattern the AI engineer coding interview questions page also covers. The ML coding round guide covers the neighbouring from-scratch questions, and the beam search page covers another decoding loop.

Sources

1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401) — arXiv (Lewis et al.), 22 May 2020
2. Building effective agents — Anthropic, 19 Dec 2024
3. ReAct: Synergizing Reasoning and Acting in Language Models (arXiv 2210.03629) — arXiv (Yao et al.), 6 Oct 2022