AI Agents

What different strategies could you use for long-term memory?

By Interview Clozo · Published 27 October 2026 · Last verified 26 October 2026 · 12 min read

What different strategies could you use for long-term memory? Four: a searchable store, memory files, self-managed tiers and a scored memory stream. Long-term memory is storage outside the context window that persists across sessions. The follow-up is which memories load when few fit: a score adds recency, importance and relevance, and the most recent memory can rank last.

In this article

Before you start
Why interviewers ask this
Long-term memory from first principles
The Staff-level answer
The rubric
Three variants of this question
Common mistakes
Follow-up questions
Sources

Before you start

This page is part of the AI agent architecture learning path and builds on prompt engineering.

Why interviewers ask this

A candidate who answers "use a vector database" has named one tool, not a design. The question separates people who can list storage options from people who can say what is stored, when it is written, how it is recalled and what happens when the window is full.

A weak answer equates memory with a larger context window, or with retrieval-augmented generation alone. A strong answer names the memory types, gives at least two storage strategies, and says that recall must be ranked because only a few memories fit in the prompt. A Staff-level answer adds write timing, staleness and deletion, and how you would test the memory.

The question comes up in agent design rounds; see the agentic AI interview questions for its neighbours and how to design an AI agent system for the full loop.

Long-term memory from first principles

Long-term memory: Long-term memory is a store outside the model's context window that lets an agent keep information across conversations or sessions. The model reads it back only when something loads it into the prompt, so the memory lives in your system, not inside the model.
Namespace: A namespace is a label or path, such as one per user, under which memories are saved so that searches stay inside one scope and one user's memories never mix with another's. LangChain's documentation saves long-term memories in custom namespaces, kept apart from thread-scoped short-term memory.
Reflection: A reflection is a higher-level memory an agent writes by synthesizing many lower-level memories, then stores and retrieves like any other memory. It turns a pile of events into one insight that later retrieval can rank. The term comes from Park et al. (2023).

In plain words

Think of a desk and a filing cabinet. The desk is the context window: everything on it is visible at once, but it is small. The filing cabinet is long-term memory: large and durable, but you must walk over, pick a folder and put it on the desk. The analogy breaks in one place. A person remembers on their own, while a model is stateless: it sees only what your code places in the prompt.

In one precise sentence, long-term memory is a store plus read and write operations that decide what moves between the store and the context window. The LangChain documentation contrasts it with short-term memory, which is scoped to one thread, a single conversation.

Step 1: Decide what kind of memory you are storing

Memory types decide the storage form, so the first step is to classify what the agent must remember. LangChain's documentation borrows three types from human memory. Semantic memory is facts, such as facts about a user. Episodic memory is experiences, such as past agent actions. Procedural memory is rules and instructions, such as the agent's system prompt.

Each type has a common form. The DeepLearning.AI course on long-term agentic memory uses episodic memory as few-shot examples, procedural memory as system prompts improved with feedback, and semantic memory as facts saved in a store and searched later. Retrieval-augmented generation (RAG) is a pattern that retrieves relevant text from a store and adds it to the prompt; it is one common way to implement semantic memory. So RAG is one implementation of one memory type, not the whole answer. You now know what to store; next is how to get the right piece back.

Step 2: Retrieval ranks memories instead of dumping them all

Recall must be ranked, because only the top-ranked memories that fit the context window go into the prompt (Park et al., 2023). Their agents score every memory as a weighted sum of three parts: recency, importance and relevance, with every weight set to 1. Recency decays exponentially with a factor of 0.995 per game hour since the memory was last retrieved. Importance is a rating the language model gives. Relevance is cosine similarity, a measure of how closely two embedding lists point the same way, between the memory and the current query. The paper min-max scales each part to between 0 and 1; the example below uses simple scaling for readability, and the ranking is the same either way.

Worked scoring of three memories

Here is an illustrative example of three memories and the query "recommend dinner plans for my Berlin trip".¹ Importance ratings on the paper's 1 to 10 scale are 9, 8 and 2, divided by ten, and the cosine values are made up.

In this illustrative example the flight-date memory ranks first with 2.5267, even though the train-times memory is the most recent.
Memory	Hours since access	Recency	Importance	Relevance	Total
Flight to Berlin on a set date	24	0.8867	0.9	0.74	2.5267
User is vegetarian	100	0.6058	0.8	0.82	2.2258
Asked about train times	2	0.9900	0.2	0.31	1.5

Here the flight-date memory ranks first at 2.5267, even though the train-times memory is the most recent one. Recency alone would rank the train-times memory first, since 0.995 to the power 2 is 0.9900. The total puts it last. With room for two memories, the flight and the diet go into the prompt and the train times stay in the store.¹

Importance and relevance lift older memories above a recent but trivial one.CC BY 4.0 — source: What different strategies could you use for long-term memory?

Text description of the diagram

In this illustrative example the flight-date memory ranks first with 2.5267, even though the train-times memory is the most recent.
With room for two memories, the flight and the diet go into the prompt and the train times stay in the store.

Stuffing everything into the window is not free either; the context rot page explains why more text can make answers worse. You now know that recall is a ranking problem.

Step 3: Choose where memories live and when they are written

Two storage strategies are common. The first is a store with search. LangGraph stores long-term memories as JavaScript Object Notation (JSON) documents, where JavaScript Object Notation (JSON) is a plain text format for structured data, under a namespace such as one per user, and the agent searches them later.

The second is file-based memory. In the Anthropic memory tool, Claude creates, reads, updates and deletes files in a memory directory. The tool is client-side: Claude requests a file operation and your application runs it. The /memories path is a prefix your handler maps onto real storage, such as a per-user directory or database keys, and the documentation says to restrict operations to that directory for security.

Semantic memory itself can take two shapes, as LangChain's documentation describes. A profile is one continuously updated document per user: easy to load, but each update needs a careful merge so new facts do not overwrite good ones. A collection is a growing set of small memories: easy to append to, but it needs search, deduplication and conflict resolution.

Writing is a separate choice. LangChain's memory documentation describes two ways to write memories: writing during the conversation, called the hot path, and writing in the background, after it. A reasonable design rule, offered as advice rather than a sourced finding: write urgent facts on the hot path and heavier summarizing in the background, so replies are not slowed by memory work.

Step 4: Page memory in and out, and reflect on it

Two further strategies manage scale. The first is self-managed tiers. In MemGPT, the context window acts as fast main memory and external storage as slow memory. The model uses function calls to read and write external data, retrieve relevant historical data and evict less relevant data from context. This gives the appearance of large memory; it does not enlarge the window.

The second is a memory stream with reflection. Park et al. (2023) store a complete record of the agent's experiences in natural language, synthesize them over time into reflections, and retrieve them dynamically to plan behavior. Reflection compresses many events into one lasting insight, so retrieval has fewer, richer items to rank.

The four strategies differ in who manages memory and in what each one risks.
Strategy	Who decides what is saved	Storage	Main risk
Store plus search	Your code or the model	Documents under a namespace	Weak ranking returns wrong memories; a profile needs careful merges, a collection needs deduplication and conflict resolution
Memory files	The model, through file tools	A directory you control	Messy files and unsafe paths
Memory tiers (MemGPT)	The model, through function calls	Context plus external storage	Poor eviction choices
Memory stream with reflection	Scoring rules and reflection	Full experience log	Cost of scoring and summarizing

A memory design is four choices stacked: what to remember, how to recall it, where it lives and how it scales.CC BY 4.0 — source: What different strategies could you use for long-term memory?

Step-by-step description

Facts, experiences or rules.
Rank, then load the top few.
A store or memory files.
Paging and reflection.

Four layers: what to remember, how to recall it, where it lives, how it scales.
Only the top-ranked memories that fit the context window go into the prompt.

Four layers: what to remember, how to recall it, where it lives, how it scales. The context-folding page covers a related way to keep long tasks inside the window.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.