What is a vector database? How it works

In this article

1. The vocabulary of vector search
2. Start here: the learning path
3. What each step of the path teaches
4. Questions interviewers ask on vector search
5. All 20 guides in Embeddings and vector search
6. Follow-up questions
7. Sources

The vocabulary of vector search

Vector database

A vector database is a data store that keeps embeddings next to their metadata and returns the stored vectors nearest to a query vector, usually through an approximate index, so that similar items can be found by meaning rather than by exact keywords.

Embedding

An embedding is a fixed-length list of numbers produced by a model for a piece of content, such as a sentence. The model is trained so that content with similar meaning lands at nearby points in the space those numbers describe, which lets a program measure how alike two pieces of content are by measuring the distance between their lists.

Approximate nearest-neighbour search

Approximate nearest-neighbour (ANN) search is a way of finding the vectors closest to a query without comparing the query against every stored vector. It accepts that it may miss a few of the true neighbours in exchange for a much lower cost per query, and the share it finds is measured as recall.

Start here: the learning path

The path starts with the whole loop of embedding, indexing, searching and measuring. It then adds hybrid search, which merges a keyword list into the vector results, and the cross-encoder reranker, which reorders the shortlist. Along the way it covers what an embedding is, how the two-tower model trains query and item vectors, and which similarity metric to use.

Underneath all of it sits the transformer, the network that produces most text embeddings today.

Text description of the diagram

• An embedding model turns text into a vector and the index stores the vectors.
• The index returns candidates, which a reranker reorders.
• Recall at k measures whether the index found the true nearest neighbours.

1. What is hybrid search, and how do you merge BM25 and vector results?
2. What is a cross encoder reranker? How it works
3. What are embeddings? Embeddings explained from scratch
4. What is a two-tower model, and how do you train one?
5. Can you describe your past projects? How to answer it
6. What is the difference between an embedding and encoding?
7. How does a vectorDB actually work?
8. What are the similarity metrics you would choose in a RAG system?
9. How do bi-encoders and cross-encoders differ in architecture and use cases?
10. How do you pick negative examples and positive examples in this case?
11. GenAI system design interview walkthrough: design a Perplexity answer engine

What each step of the path teaches

How does a vector database actually work?

A vector database embeds every document once, stores the vectors in an index, and answers a query by embedding the query with the same model and asking the index for the nearest vectors. The expensive work, embedding the corpus and building the index, happens before any query arrives. At query time the cost is one embedding of the query plus one index lookup, then the stored identifiers are mapped back to documents and metadata. A vector database adds to a bare index the parts an application needs around it: durable storage, updates and deletes, filters on metadata, and access control. pgvector, for example, adds vector search to an ordinary Postgres database (pgvector documentation, v0.8.7, as of October 2026).

1 · Embed the documents2 · Build the index3 · Embed the query4 · Look up neighbours5 · Return documentsReplay

Step-by-step description

1. The embedding model turns every document into a fixed-length vector.
2. Vectors are added to a Flat, IVF or HNSW index, with metadata beside them.
3. The query goes through the same model and becomes one vector.
4. The index returns the k vectors closest to the query vector.
5. Ids map back to documents, optionally reranked before use.

• Documents are embedded and added to the index before any query arrives.
• The query must use the same embedding model as the documents.
• The index returns the k nearest vectors, exactly or approximately.

Should you search exactly or approximately?

Exact search compares the query with every stored vector and always returns the true nearest neighbours; approximate search checks only part of the data and may miss some. By default pgvector performs exact search, which it says "provides perfect recall", and adding an approximate index "trades some recall for speed" (pgvector documentation, v0.8.7, as of October 2026).

The three index families you must name are flat (exact, scans everything), IVF (inverted file index: the vectors are split into cells and only a few cells are scanned) and HNSW (Hierarchical Navigable Small World: a layered graph of links between vectors that a search walks from coarse to fine). The HNSW authors claim logarithmic complexity scaling with data size (Malkov and Yashunin, 2016). Independent comparison matters: the ANN-Benchmarks authors found that very different approaches give comparable quality-performance trade-offs (Aumüller et al., 2018). The Faiss library paper describes the same inverted-file idea for billion-scale search (Johnson, Douze and Jégou, 2017).

Three settings control how hard each index searches, and the measurement below uses all of them. In HNSW, M is the number of links each vector keeps to its neighbours, efConstruction is how many candidates are considered while the graph is built, and efSearch is how many candidates the search keeps while it walks the graph; a larger efSearch means more work and a better chance of finding the true neighbours. In IVF, nprobe is how many of the cells are scanned for each query.

We measured this rather than quoting it.¹ We embedded 10,000 posts from the 20 Newsgroups training set with a 384-dimension model, took 300 other posts as queries, and used exact inner-product search as the truth. With HNSW (M = 16), recall at 10 was 0.854 at efSearch 8, 0.975 at 32 and 0.999 at 128. With an IVF index of 100 lists, recall at 10 was 0.695 at nprobe 2, 0.936 at 10 and 0.977 at 20.

We timed one query at a time, as an online service would see them, taking the best of three passes over the 300 queries on one thread: exact search took 0.44 milliseconds per query, so it is under one millisecond at 10,000 vectors and is perfectly fine for a small collection. The approximate indexes were already faster per query: HNSW at efSearch 32 took 0.033 milliseconds (about 13 times faster, recall 0.975) and at efSearch 128 took 0.115 (about 4 times faster, recall 0.999), and IVF at nprobe 20 took 0.080 (about 5 times faster, recall 0.977). Exact cost grows in proportion to the number of vectors, so the gap widens as the corpus grows toward millions.¹

Text description of the diagram

• HNSW recall at 10 was 0.854 at efSearch 8, 0.975 at 32 and 0.999 at 128.
• IVF with 100 lists had recall at 10 of 0.695 at nprobe 2, 0.936 at 10 and 0.977 at 20.
• Truth was exact inner-product search over the same vectors.

Which similarity metric should you use?

Use the metric the embedding model was trained with. On normalised vectors, meaning vectors scaled to length one, cosine similarity, inner product and L2 distance (the straight-line distance between two points) all rank neighbours the same way; on unnormalised vectors they do not.

Vectors of length one have a dot product equal to their cosine, which is why many pipelines normalise once and use the cheaper inner product, as we did in the experiment above. pgvector exposes L2 distance, negative inner product, cosine distance and L1 distance (the sum of the differences along each dimension, sometimes called city-block distance) as separate operators, and an index serves one distance function (pgvector documentation, v0.8.7, as of October 2026). A query written with a different operator does not use that index and scans every vector, which is slower but still correct. The real mistake is applying a metric the model was not trained for to vectors that were never normalised.

How much memory do vectors need?

A flat index of float32 vectors needs 4 × d bytes per vector, where d is the dimension (Faiss index documentation, as of October 2026). One million vectors of 768 dimensions is therefore about 3.07 gigabytes (10^9 bytes) before any index overhead, and graph indexes such as HNSW add more per vector, growing with the connectivity setting M. Storing half-precision or 8-bit values cuts the raw size by two or four, and product quantisation compresses further at the cost of precision.

Product quantisation stores each vector as a short code, so memory and each comparison get cheaper, but the Faiss documentation lists a plain product-quantisation index as exhaustive, meaning it still visits every vector. These sizes are why memory often decides which index you can afford.²

What is hybrid search?

Hybrid search runs a keyword search and a vector search over the same collection and merges the two ranked lists. Dense vector retrieval blurs exact identifiers and rare words, such as a product code, a surname or an error number, because the embedding stores meaning rather than spelling. Keyword retrieval with BM25 (Best Matching 25, a standard ranking formula that scores a document higher when the query's words appear in it often, gives more weight to words that are rare across the collection, and corrects for document length) keeps those exact tokens but will not recognise synonyms, acronyms or spelling variations (sentence-transformers documentation, as of October 2026).

A common way to merge the lists is reciprocal rank fusion: each document scores the sum, over the lists, of 1 divided by (60 plus its rank in that list), and the documents are sorted by that score. Its authors report that it consistently beats any single system it combines (Cormack, Clarke and Büttcher, 2009).

What does a reranker add?

A reranker reorders the vector search's shortlist with a model that reads the query and each candidate together, so the ordering improves at the price of one model pass per candidate. The index compares two vectors that were computed separately, which is fast but coarse; the reranker looks at the actual words of the query beside the actual words of the passage, which is slower but sharper.

It cannot find what the index missed, which is why recall at k is measured first. Recall at k measures whether the index found the true nearest neighbours; if the true answer is not in the shortlist, no ordering can surface it. The full explanation, with a measured example and a cost table, is in what a cross-encoder reranker is and how it works.

What is an embedding?

An embedding is a list of numbers that a model assigns to a piece of content so that similar content gets nearby lists. A sentence embedding model of the kind described by Reimers and Gurevych maps a sentence to one fixed-length vector, and two sentences are compared with a cosine similarity between their vectors (Reimers and Gurevych, 2019). The number of entries, called the dimension, is fixed by the model: the model we use above produces 384.

The reason this works is that the model was trained on pairs of related texts to place them close together, so distance in the space stands in for similarity of meaning, not for shared words. That is why a search for "car" can find a passage about "automobile".

What is a two-tower model?

A two-tower model uses one encoder for queries and another for items, trained together so that matching pairs end up close in the same vector space. The towers may share weights or not. Because the two towers never read a pair jointly, item vectors can be precomputed and indexed, which is what makes the design usable for retrieval at scale. The dense passage retrieval paper trains a dual encoder of this kind from a small set of questions and passages and reports that it outperforms a strong BM25 system by 9 to 19 percentage points in top-20 accuracy on several question-answering datasets (Karpukhin et al., 2020).

Questions interviewers ask on vector search

Interviewers ask this topic in three forms. The definition form asks what a vector database is and how it differs from a vector index (the index is the search structure; the database adds storage, updates, filters and access control). The trade-off form asks which index to use and what recall you would accept, which the measurements above answer. The design form places the index inside a RAG system and asks about chunking, reranking and evaluation. For a broader set of practice questions across the AI engineering loop, see the AI engineer question list; for how retrieval fits into a full design round, see machine learning system design and AI agents.

Follow-up questions

What is a vector database in AI?

It is the retrieval component that lets an application find content by meaning: embeddings go in, nearest vectors come out. It is used behind semantic search and behind RAG systems; the reranker page shows the next step after retrieval.

How do embeddings actually work?

A model trained on related pairs of texts maps each text to a vector so that related texts are close; a search is then a nearest-neighbour lookup in that space. Most text embedding models are transformers.

What is vector indexing and how is it used in vector databases?

Vector indexing organises the stored vectors so that a query does not have to compare against all of them. IVF scans only a few cells and HNSW walks a graph, so each query skips most vectors. Product quantisation instead compresses the vectors so memory and each comparison are cheaper, and it is often combined with IVF (Faiss index documentation, as of October 2026). See the measured trade-offs above and machine learning system design.

What is vector database and RAG?

In RAG the vector database holds embeddings of document chunks; a question is embedded, the nearest chunks are retrieved, and a language model answers using them. See AI agents for how retrieval becomes a tool, and inference for language models for what the answering step costs.

Which search would you choose if accuracy matters and speed does not?

Exact (flat) search, because it always returns the true nearest neighbours. For a small collection such as 10,000 documents it is also fast enough: it took 0.44 milliseconds per query in the measurement above. Approximate indexes were about 4 to 13 times faster per query, and that gap grows with corpus size. Practise this on the AI engineer question list.

Why does a query need the same embedding model as the documents?

Different models place texts in different spaces with different dimensions, so distances between vectors from two models mean nothing. When you change the model you must re-embed the corpus, which is why the transformers hub matters when choosing one.

Sources

1. Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs — arXiv (Malkov and Yashunin), v4 revised 2018-08-14, 30 Mar 2016
2. Billion-scale similarity search with GPUs — arXiv (Johnson, Douze, Jegou), 28 Feb 2017
3. ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms — arXiv (Aumuller, Bernhardsson, Faithfull), v2 revised 2018-07-17, 15 Jul 2018
4. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — arXiv (Reimers and Gurevych), 27 Aug 2019
5. Faiss indexes (wiki) — Meta Research, Faiss repository (read 2026-10-24), 24 Oct 2026
6. pgvector: open-source vector similarity search for Postgres — pgvector README, v0.8.7 as shown (read 2026-10-24), 24 Oct 2026
7. Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods — Cormack, Clarke and Buttcher, University of Waterloo and Google; ACM SIGIR 2009 (read 2026-10-24 from the authors' PDF copy), 19 Jul 2009
8. Dense Passage Retrieval for Open-Domain Question Answering — arXiv (Karpukhin et al.), v3 revised 2020-09-30; EMNLP 2020, 10 Apr 2020
9. Retrieve and Re-Rank — Sentence Transformers documentation (read 2026-10-24), 24 Oct 2026