What is metadata filtering in vector search, and how does it work?

In this article

1. Before you start
2. Why interviewers ask this
3. Metadata filtering from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one step of the embeddings and vector search learning path. The encoders behind most embeddings are covered in the transformer overview.

Why interviewers ask this

Real systems almost never search a whole collection. A shop search must stay inside one category, a support assistant inside one customer's documents, a news feed inside the last week. A retrieval-augmented generation (RAG) assistant, for example, may only quote documents the user is allowed to read and that are newer than a given date. So a design round on retrieval soon reaches the question of how a filter and a similarity search combine, and the wrong plan quietly returns too few or the wrong results.

A weak answer says "we add a filter condition to the query." A strong answer names three plans (filter after the search, filter before it, filter during it), says what each does to the number of results and to the cost, and knows that a very selective filter is the hard case. A Staff-level answer adds how to measure it: recall with the filter on, the count of results returned, and an index on the filtered fields. The page hybrid search shows where the same question returns when keyword and vector lists are merged.

Metadata filtering from first principles

Metadata filtering

Metadata filtering is a retrieval step that restricts a similarity search to records whose stored attributes, such as category, date, owner or tenant, satisfy a condition, so that only matching records can appear in the results, ranked by closeness to the query.

Selectivity

Selectivity is the share of all records that pass a filter, such as 1% for a filter that keeps one record in a hundred. A selective filter has a small share, a loose filter a large one. Documentation differs on the word: Qdrant, for instance, calls weak filters "high-selectivity", the opposite sense. This page always means the share kept.

Post-filtering

Post-filtering is a plan that runs the similarity search first, takes the top k results, and then removes those that fail the filter, so the answer can contain fewer than k records.

In plain words

Picture a huge clothes shop. A customer says "something like this blue jacket, but only in the women's section, size medium." You can walk the whole shop looking for the most similar jackets and then throw away the wrong sections. Or you can go to the women's section, size medium rail first and compare only those. Both are metadata filtering; they differ in when the filter happens. The analogy breaks in one place. A shop has few jackets, but a vector index holds millions, and its fast search structure was built without your filter in mind.

In one precise sentence, a metadata filter is a yes-or-no test on stored attributes that the search must respect, and the plan decides whether the test runs before, after or during the nearest-neighbour search.

Step 1: Each vector is stored with its metadata

A vector database stores, for every item, its vector and a small record of attributes next to it. Qdrant calls this record the payload, other systems call it metadata or properties. An item might be a vector plus category: hats, year: 2026 and tenant: acme. A tenant is one customer of a shared system, so a tenant filter keeps one customer's data apart from another's.

Input: items with vectors and attributes. Output: a collection that can answer two kinds of question, "which items are near this vector" and "which items satisfy this test." Qdrant's documentation describes payload indexes for accelerating similarity search with filters on structured fields: an index on the attribute, separate from the vector index, that finds matching records quickly. You now know what is stored; the next steps are about how the two questions are combined.

Step 2: Post-filtering searches first and filters second

Post-filtering asks the vector index for the top k nearest items and then discards those that fail the filter. It is the simplest plan and works with any index. Its flaw is arithmetic: the top k are chosen without knowing the filter, so if the filter keeps a small share of items, few of the k survive.

In the illustrative case of 1,000,000 items, a filter with 1% selectivity and k = 10, the expected number of matches among the top 10 is 0.1, and the chance of at least one is 0.0956.¹ The pgvector README states the same effect: with approximate indexes, queries with filtering can return fewer results, because filtering is applied after the index is scanned. The README gives its own figure: with a filter that matches 10% of rows and the default hnsw.ef_search of 40, only 4 rows will match on average. Input: top k. Output: at most k, often far fewer. The usual repair is to fetch more candidates than k, which the next steps and the worked example price.

Step 3: Pre-filtering finds the matches first and searches only them

Pre-filtering builds the set of matching records first, using the attribute index, and then finds the nearest items inside that set. In its simplest form it skips the graph index and compares the query with every matching vector. Weaviate's blog describes this: brute force over a small number of objects is fast, and it slows down as more objects pass the filter.

Input: a condition and a query vector. Output: exactly the k nearest matching items, because the comparison is exhaustive. The cost is the number of matches times the vector length. In the illustrative case of 10,000 matches and 768 numbers per vector, that is 7,680,000 multiply-adds, one hundredth of a full scan of the collection.¹ A loose filter, with 500,000 matches, costs 384,000,000, so brute force over the matches stops paying off.

Step 4: Filtered graph search tests the filter while walking the index

Many vector indexes are graphs. A Hierarchical Navigable Small World (HNSW) index links each vector to near neighbours, and a search walks from an entry point towards the query. Filtered graph search is a plan that checks the filter during that walk, so only matching items can enter the result list while the walk still uses the graph's speed. Qdrant says HNSW is "well-compatible with the modification that allows Qdrant to use filters during a search". Its documentation also says Qdrant extends the HNSW graph with additional edges based on indexed payload values, which is why the payload index has to exist for in-graph filtering to work well.

Weaviate's 1.34 release note introduces Automatic Constraint Optimization for Retrieval Networks (ACORN) as the default for new collections; Patel et al. (2024) describe the method. Weaviate's concepts page says its filtered search is built on pre-filtering without a brute-force scan, and that from version 1.34 it uses ACORN by default, which helps most when the filter has low correlation with the query: the matching items are not the ones near the query. Pgvector takes a different route: iterative index scans, added in 0.8.0, keep scanning the index until enough results pass the filter or a scan limit is reached.

1 · Record2 · Post-filter3 · Pre-filter4 · In-graphReplay

Step-by-step description

1. Each item keeps a vector and metadata.
2. Search first, filter second.
3. Filter first, search the matches.
4. Test the filter during the walk.

• Each item is stored with its vector and its metadata.
• Post-filtering searches first and filters second, so it can return fewer than k results.
• Pre-filtering finds the matching items first and searches only them.
• Filtered graph search tests the filter while walking the index.

Worked example

Take a collection of 1,000,000 items with 768 numbers per vector, and a search for the top 10 results (k = 10). All of these values are illustrative, and the matches are assumed to be spread at random, unrelated to the query. The table runs four filters from loose to very selective.¹

A full unfiltered scan costs 768,000,000 multiply-adds.¹ Read the 1% row: a post-filter plan must fetch about 1,000 candidates to expect ten matches, which is 100 times the k asked for, while brute force over 10,000 matches costs one per cent of a full scan. A tenant filter in a shared system with 1,000 equal tenants is the last row.

Text description of the diagram

• A filter keeping 1% of items needs about 1,000 candidates to expect ten matches.
• A filter keeping 0.1% needs 10,000 candidates, in this illustrative case.

A ten-item toy shows the failure by hand (illustrative). Items are listed from nearest to farthest: bag, shoe, shoe, hat, bag, hat, shoe, bag, hat, shoe. The filter is category = hat and k = 3. Post-filtering takes the three nearest, which hold zero hats, so it returns nothing. Pre-filtering finds the three hats at positions 4, 6 and 9 and returns all three. A search-then-filter plan would need to fetch 9 candidates to get three hats.¹

Trade-offs and edge cases

No plan wins everywhere; the filter's selectivity and its relation to the query decide. Post-filtering is simplest but can return too few results. Pre-filtering with brute force is exact but costs more as the matching set grows. Filtered graph search keeps the speed of the index but is engine-specific and can lose recall when the filter removes most of the graph's useful paths.

Text description of the diagram

• Post-filtering searches first and filters second, so it can return fewer than k results.
• Pre-filtering with brute force is exact but its cost grows with the number of matches.
• Filtered graph search tests the filter during the walk of the index.

Edge cases to name:

• Correlation. If the matching items sit far from the query, as in a "women's section" filter on a query about men's coats, a graph walk that tests the filter has to travel through non-matching regions. Weaviate's concepts page names this low-correlation case as the one its default strategy targets.
• Iterative scans. In pgvector, hnsw.iterative_scan accepts strict_order, which keeps results in exact distance order, and relaxed_order, which allows slight disorder for better recall.
• Unindexed fields. A filter on a field with no attribute index forces a scan of the records' metadata. Qdrant's documentation recommends payload indexes for fields you filter on.
• The engine picks the plan. Engines estimate how many records match and switch between brute force and the graph. Weaviate's blog says it brute-forces small filters, with the size set by the flatSearchCutOff parameter.
• Filters that are rules, not preferences. Tenant and permission filters must always apply; a plan that returns fewer results is acceptable, one that returns another customer's item is not. A mandatory filter on a shared index is one option; stronger isolation is a partition or table per tenant, and the pgvector README says: "For tenant isolation, use list partitioning or separate tables."
• Reranking. A cross-encoder reranker applied after a filtered search only sees the allowed items. The two-tower model page explains why the first stage is built to be cheap.

How to answer it in an interview

The spoken structure: define the filter, name the three timings, say what breaks for a selective filter, then say how you would measure it.

Common mistakes

• "Filtering is free." (wrong) A filter changes the plan and the cost. (right) "Filtering changes how many candidates I must examine, so I test latency and recall with the filter on."
• "Post-filtering always returns k results." (wrong) It can return fewer, even none. (right) "Post-filtering returns at most k, and I over-fetch or use iterative scans to fill the list."
• "Pre-filtering is always exact and fast." (wrong) It is exact only if it compares every match, and it slows as the matching set grows. (right) "Pre-filtering with brute force suits small matching sets; for large ones I use the index."
• "The filter field needs no index." (wrong) An unindexed field makes the match lookup slow. (right) "I create an attribute index on the fields I filter on often or that are selective, because indexes cost memory."

Follow-up questions

What is the difference between pre-filtering and post-filtering?

Pre-filtering finds the matching records first and searches only among them; post-filtering searches first and then removes records that fail the filter. The first can return a full list of k; the second can return fewer.

Why does a filtered vector search return fewer than k results?

The search probably chose its top k before the filter was applied, so some failed it. In pgvector this is documented for approximate indexes, and iterative index scans, available from version 0.8.0, keep scanning until enough results pass or a limit is reached.

Do you need an index on metadata fields?

Yes, for fields you filter on often. Qdrant's documentation covers payload indexes for exactly this, to speed up similarity search with filters on structured fields.

How would you keep one customer's data out of another's vector search results?

One option is a tenant attribute on every item, indexed and applied as a mandatory filter on every query of a shared index; because the filter is a rule, accept fewer results rather than any leak. The stronger design is a partition, collection or table per tenant: the pgvector README advises list partitioning or separate tables for tenant isolation, and it also removes the very selective filter case. Either way, test the smallest tenant.

How does metadata filtering work with hybrid search?

Apply the same filter to both the keyword and the vector search, so the merged list comes from one allowed set. See hybrid search for the merging step.

How is metadata filtering used in RAG?

A RAG pipeline filters the stored chunks by attributes such as source document, date, language or the user's permissions at retrieval time. The language model then never sees a disallowed or outdated chunk, so it cannot quote one.

Sources

1. Indexing - Qdrant documentation — Qdrant (undated page, read 2026-10-26), 26 Oct 2026
2. pgvector: open-source vector similarity search for Postgres (README, filtering and iterative index scans) — pgvector project on GitHub (read 2026-10-26), 26 Oct 2026
3. Filtering - Weaviate documentation (concepts) — Weaviate (undated page, read 2026-10-26), 26 Oct 2026
4. How we speed up filtered vector search with ACORN — Weaviate blog (read 2026-10-26), 26 Oct 2026
5. Weaviate 1.34 Release — Weaviate blog (read 2026-10-26), 26 Oct 2026
6. ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data (arXiv 2403.04871) — arXiv (Patel et al.), 7 Mar 2024