In this article
Before you start
This page is one step of the generative AI system design learning path.
Why interviewers ask this
If a retrieval system returns a document the user may not read, the model will happily quote it. The leak happens before any answer is written, so no polite wording in the answer can undo it. Questions of this kind are reported by candidates in interviews on enterprise search, legal-technology and healthcare-style systems.
A weak answer says "add a line to the system prompt telling the model not to reveal restricted files." A strong answer puts the check in the retrieval layer, explains how permissions get onto every chunk, and says what happens when a permission changes. A Staff-level answer also separates two problems that sound alike: per-user access inside one organization, and isolation between tenants, which are different customers sharing one system.
Permission-aware RAG from first principles
- Access control list
- An access control list (ACL) is the list of users or groups allowed to read one document, stored with the document at its source and copied to the search index so that a query can be checked against it.
- Security trimming
- Security trimming is the practice of removing from search results every document the calling user may not read, by comparing the caller's identity with the access list stored on each indexed document.
In plain words
Think of a company archive with a clerk. You ask for papers on a topic. A careless clerk fetches the five best matches, then checks your badge and hands over only what you may see. A careful clerk checks your badge first and searches only the shelves you may open. The analogy breaks in one place. A search index has no badge reader of its own, so your system must store the permissions and apply them on every query.
In one precise sentence, permission-aware retrieval stores each chunk's allowed readers beside its vector and filters on the caller's identity inside every search.
Microsoft's documentation describes the same goal: Azure AI Search enforces permissions at the document level from ingestion through query execution, and names RAG applications as a use case.
Step 1: Capture the access list when you ingest
Ingestion is the job that reads source documents and loads them into the index. In this step it also reads each document's access control list from the source system, such as a file store or a wiki. The input is a document with its permissions. The output is a record that carries the text and a list of allowed groups.
Filter on groups rather than single users, because groups change less often than people move between teams. Store the list in a field of the index, as a collection of strings. In the Azure pattern, you create a field for the principal identifiers and load documents that include them. You now know where the permission data comes from: the source system, copied at load time.
Step 2: Copy the permissions onto every chunk
A chunk is a short passage cut from a document so it can be embedded and retrieved on its own. Search returns chunks, not documents, so the permission list must sit on each chunk. A chunk with no list must be treated as readable by no one (fail closed), and the missing list is an ingestion error to alert on; a chunk that defaults to readable-by-all is a leak.
The input is one document with its list. The output is many chunks, each carrying the same list. Microsoft notes that when a skillset splits documents into chunks, the permission metadata moves from indexer field mappings to index projections, which is the same idea in Azure's terms. The same rule applies to tables, charts and images extracted from a file, a case the page on designing RAG over real documents covers. You now have chunks that know who may read them. Next the query has to use that knowledge.
Step 3: Filter inside the search, not after it
At query time the application resolves the caller's identity from the authenticated session, expands it to the caller's groups, and sends them as a filter with the vector query. Pre-filtering applies that filter while the index is searched. Post-filtering is the opposite: it runs the search first, takes the top few results, and then removes the unreadable ones.
Post-filtering leaks nothing, but it starves the answer. If the best five chunks are mostly unreadable, little is left. With a user who can read only 10% of the corpus, and unreadable chunks spread independently through the ranking, five retrieved chunks leave 0.5 on average (illustrative).1 In the Azure pattern the filter has the form group_ids/any(g:search.in(g, 'group_id1, group_id2')), per the Azure security-filter article. The result list is trimmed to documents that match the caller's strings. You now have a retrieval step that returns only readable chunks, and a full list of them.
Step 4: Log every retrieval and keep permissions fresh
Two jobs remain. First, record which user retrieved which chunks, because the Open Worldwide Application Security Project (OWASP) recommends detailed, immutable logs of retrieval activity to detect suspicious behavior. Second, keep the copied lists current. A permission copied at ingestion goes stale the moment someone changes it at the source.
Suppose the index syncs once a day at midnight and a user is removed at 01:00. That user keeps access for 23 hours (illustrative).1 Fixes are a shorter sync, a deletion event that updates the index at once, or a live check against the source for sensitive collections. Freshness is a design concern in managed products too, because the index holds a copy of the permissions. The federated alternative skips the copy: the system asks the source system's own application programming interface (API) at query time which items the user may read, which trades extra latency per query for freshness. Each retrieval is logged, with the user and the chunks returned. You now have the whole loop.
Step-by-step description
- Ingestion reads each document's access list.
- Every chunk carries the list.
- The search keeps only readable chunks.
- Each retrieval is recorded.
- Ingestion captures the access list of each source document.
- Every chunk carries a copy of the access list.
- The filter runs inside the search, so the model sees only readable chunks.
- Each retrieval is logged, with the user and the chunks returned.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
Take a toy corpus of 10 chunks. Each has a similarity score to one question and one allowed group: legal, finance, people or public. All values are illustrative. Alice belongs to public and legal. Bob belongs to public and people. Alice may read 5 of the 10 chunks and Bob may read 4.1
Without any filter, the top five chunks for both users are the same, and they come from legal, finance and people. Three of those five are unreadable to Alice and four are unreadable to Bob, so a plain search leaks.1 Filtering after the search leaves Alice 2 chunks and Bob 1. Filtering inside the search returns Alice 5 readable chunks and Bob all 4 that he can read.1
| Quantity (illustrative toy corpus) | Alice | Bob |
|---|---|---|
| Chunks the user may read, of 10 | 5 | 4 |
| Unreadable chunks in the unfiltered top 5 | 3 | 4 |
| Chunks returned when filtering after the search | 2 | 1 |
| Chunks returned when filtering inside the search | 5 | 4 |
Text description of the diagram
- Filtering after the search returns 2 chunks for Alice and 1 for Bob.
- Filtering inside the search returns 5 chunks for Alice and 4 for Bob.
The staleness arithmetic works the same way. With a 24-hour sync at midnight and a removal at 01:00, the exposure is 23 hours; the average exposure for a removal at a random moment is 12 hours (illustrative).1 A live check at query time brings it to zero for the sources you check, at the cost of an extra call per query.
Trade-offs and edge cases
Permission-aware RAG trades simplicity for safety, and the right level of isolation depends on who shares the system. The OWASP entry on vector and embedding weaknesses names context leakage between users in a shared vector database as a risk, and recommends permission-aware vector stores with strict logical partitioning between groups.
Five edge cases decide real designs:
- Embeddings are not anonymous. OWASP notes that attackers can invert embeddings to recover source information, so the vector store itself needs access control, not only the text store.
- Filtered vector search changes recall and latency. With an approximate-nearest-neighbour index, a highly selective filter (a user who sees 1% of the corpus, illustrative) can return fewer than k results or push the engine into a slow exhaustive scan. A common mitigation for post-filtering is to over-fetch, searching for more than k and then trimming; check how your vector store orders filter and search.
- Caches and derived artifacts leak too. A semantic or answer cache, conversation memory or a precomputed summary shared across users bypasses the retrieval filter unless it is keyed by permission scope.
- Not every product is native. As of October 2026 the Azure overview marks its built-in ACL, role-based access control (RBAC) and SharePoint patterns as preview; the string-filter pattern is Microsoft's documented route for solutions that can't use that built-in support. Verify the status of any managed feature before you rely on it.
- Tools widen the surface. An agent that calls retrieval as a tool must pass the user's identity into every call, not the agent's own service identity; the agentic RAG page shows where those calls sit. The large language model (LLM) gateway questions cover how identity travels through a gateway layer, and the Perplexity design walkthrough shows retrieval in a search product.
Text description of the diagram
- A shared index with a filter is the cheapest and rests entirely on the filter being right.
- A separate index per tenant is the strongest and costs the most.
For different customers, you choose where to draw the wall. A shared index with a tenant filter is cheapest and rests entirely on the filter being right. A partition per tenant adds a boundary the query must name. A separate index per tenant is the strongest and costs the most. For very sensitive data, such as law-firm matters, many designs combine a boundary with attribute-based access control (ABAC), where rules compare attributes of the user and the document.
How to answer it in an interview
Your RAG system needs per-user access control on internal documents. How do you implement it?
I enforce permissions in the retrieval layer, never in the prompt. At ingestion I copy each document's access list, as group names, onto every chunk. At query time I resolve the caller from the authenticated session, expand it to groups, and apply them as a filter inside the vector search, so unreadable chunks are never candidates. The model only sees authorized text. I sync permission changes on a short schedule, or check live for sensitive sources, and I log every retrieval. For separate customers I add a tenant boundary on top.
The spoken structure: say where the check lives, say how permissions reach chunks, say the filter runs inside the search, then name freshness and logging.
Common mistakes
- "I tell the model in the system prompt not to reveal restricted files." (wrong) The text is already in the context. (right) "Authorization happens before the model sees the context."
- "I retrieve the top results, then drop the ones the user can't read." (wrong) The user may be left with almost nothing. (right) "I pass the filter into the search so the limit is filled with readable chunks."
- "Permissions are set once at ingestion." (wrong) They go stale. (right) "I sync changes, and check live where the cost of a leak is high."
- "Embeddings are just numbers, so any tenant can share them." (wrong) They can be inverted to recover source information. (right) "The vector store gets the same access control as the documents."
- "I filter on the user's name." (wrong) Single-user lists are hard to manage and break when people move teams. (right) "I filter on groups and resolve the user's groups at query time."
Follow-up questions
How will you handle restricted vs unrestricted questions before retrieval?
Treat a classifier or router as an optimization, not as the security check. It may send obviously public questions down a cheaper path, but every retrieval still carries the caller's permission filter, so a misrouted question cannot leak. This is a design judgment, not a sourced standard.
How do you do matter-level isolation in a multi-tenant vector store?
Give each matter its own group or attribute, store it on every chunk, and filter on it inside the search; add a partition or separate index when the risk justifies the cost. OWASP recommends strict logical and access partitioning between groups, plus tagging and classifying data to control access levels. Keep encryption and logging as separate layers.
How would you design a private, VPC-deployed RAG system for a healthcare customer?
A virtual private cloud (VPC) keeps traffic inside a private network, which limits where data travels. It does not decide who may read which document, so the per-chunk permission filter and the retrieval log stay in place. Check the customer's Health Insurance Portability and Accountability Act (HIPAA) requirements with their compliance team before you state specifics.
How would you design human-in-the-loop review for a RAG system?
Route low-confidence or high-stakes answers to a reviewer, and show the reviewer only the sources that reviewer is authorized to read. The retrieval log lets a reviewer reconstruct what the model saw.
What happens when someone's permissions change after indexing?
Until the index syncs, the user keeps the old access. Shorten the sync, push deletion events, or check live against the source; the worked example shows a 23-hour window for a daily sync.
- Enforce access in the retrieval layer; a prompt instruction cannot unsend text the model already has.
- Copy the access list onto every chunk at ingestion.
- Apply the user's groups as a filter inside the search, not after it.
- Keep permissions fresh and log every retrieval.
- Separate per-user access from tenant isolation, and protect the vector store itself.
Sources
- Document-level access control in Azure AI Search — Microsoft Learn (Azure AI Search documentation), 17 Sep 2026
- Security filters for trimming results in Azure AI Search — Microsoft Learn (Azure AI Search documentation), 26 Oct 2026
- LLM08:2025 Vector and Embedding Weaknesses — OWASP Gen AI Security Project, 28 Apr 2025
