What is GraphRAG and when do you use it over vector RAG?

In this article

1. Before you start
2. Why interviewers ask this
3. GraphRAG versus vector RAG from first principles
4. The Staff-level answer
5. The rubric
6. Three variants of this question
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one answer in the retrieval-augmented generation guide, and it builds on how embeddings and vector search work. For the mechanism of GraphRAG in depth, see how GraphRAG works.

Why interviewers ask this

Choosing between vector RAG and GraphRAG costs money and engineering time if you get it wrong, so interviewers use the question to see whether you match a retrieval method to a type of question instead of following a trend. Candidates report it in rounds about retrieval systems at AI labs and large tech companies.

A weak answer says "GraphRAG is RAG with a graph database" or "it is more accurate." A strong answer defines both methods, names the question types where vector similarity fails, and mentions the indexing cost. A Staff-level answer also says which question set it would build to decide, and keeps vector search for lookups.

The rest of this page goes in that order: the mechanism, a counted toy example, a model answer, a rubric and the mistakes that cost the point.

GraphRAG versus vector RAG from first principles

Community report

A community report is a summary that a language model writes for one cluster of closely connected entities in a knowledge graph. It states what the cluster contains and how its parts relate, so a question about a whole collection can be answered by reading summaries instead of raw text chunks.

Local search

Local search is a GraphRAG query mode that answers a question about specific entities by combining relevant data from the extracted knowledge graph with text chunks of the raw documents. The graph supplies the entities and relations around the ones the question names, and the chunks supply the original wording.

Global search

Global search is a GraphRAG query mode that answers a question about a collection as a whole by searching over all of the community reports in a map-reduce fashion. Each report is read separately for a partial answer, and the partial answers are then combined into one response.

In plain words

Vector RAG is like asking a librarian for the pages that sound most like your question. GraphRAG first builds a corkboard of people, places and links. It adds a one-page note for each cluster, then answers from the board. The analogy breaks in one place. A model reads every page to build the board, so the board costs money and can contain mistakes.

In one precise sentence: vector RAG retrieves text chunks by similarity to the question. GraphRAG has a model extract entities and relations into a graph. It groups them into communities and summarizes each. It answers from those structures.

Step 1: Vector RAG finds the chunks closest to the question

Vector RAG is a retrieval-augmented generation setup that splits documents into chunks, stores an embedding for each, and at question time returns the chunks whose embeddings are nearest to the question's embedding. Microsoft writes that "the majority of RAG approaches use vector similarity as the search technique, which we call Baseline RAG" (Microsoft GraphRAG documentation, accessed 2026-10-26).

Input: a question. Output: a few chunks the model reads before it answers. This works when the answer sits in a chunk that resembles the question.

It struggles in two cases that Microsoft names in its Microsoft Research post. First, "Baseline RAG struggles to connect the dots": answering needs a path through disparate pieces of information that share attributes. Second, it performs poorly on holistic questions over large data collections. You now know what the graph is meant to fix.

Step 2: GraphRAG pays at indexing time to build a graph and community reports

GraphRAG extracts a knowledge graph from the chunks, with entities such as companies and products joined by relations such as "acquired", and summarizes communities in community reports. Microsoft describes the process as extracting a knowledge graph out of raw text, building a community hierarchy and generating summaries for these communities (GraphRAG documentation). Input: the chunks. Output: a graph and a set of community reports.

The price is model calls. In the toy corpus below (illustrative: one extraction call per chunk and one summary per community), indexing takes 15 generation calls, twelve extraction calls plus three community summaries, where vector indexing makes none.¹ Both methods also make embedding calls (12 for vector RAG here).¹ The 15 is a lower bound, because real pipelines can add further passes over each chunk and write summaries at several levels of the hierarchy. The cost grows with the amount of text, and it is paid before any question arrives. That is the price the lead mentioned.

Text description of the diagram

• In the toy corpus GraphRAG indexing makes 15 generation calls and vector indexing makes none.
• The count is twelve extraction calls plus three community summaries.
• Both methods also make embedding calls, 12 for vector RAG here.

Step 3: A toy corpus shows where each method wins

A toy corpus makes the difference countable. Take twelve short chunks (illustrative), such as "Nimbus Labs makes the Falcon sensor" and "Orbitech acquired Nimbus Labs", and ask which company owns the maker of the Falcon sensor. The answer sits in the second chunk, which shares no content word with the question, while two other chunks that mention the sensor share two words each.¹ A real embedding model sees more than shared words, so this is a crude stand-in; the point is that the answer chunk never mentions the sensor.

In the graph, the path from the Falcon sensor to Orbitech is two edges, so a traversal finds the owner in two hops, while a question about what the sensor measures needs one edge.¹ For a themes question, a vector search that returns three chunks sees a quarter of the corpus, while global search reads three community reports that together cover all twelve entities.¹

Step 4: Local and global search read the graph at question time

Microsoft's query engine has several modes (Query Engine overview). Local search "generates answers by combining relevant data from the AI-extracted knowledge-graph with text chunks of the raw documents", and suits questions about specific entities, such as the healing properties of chamomile in Microsoft's own example. Global search works over all community reports in a map-reduce fashion. Drift search is a variant of local search that adds community information to the search.

Local search reads the knowledge graph and the text chunks, and global search reads the community reports. The same engine also lists a basic search mode, plain vector RAG kept for comparison, so GraphRAG sits next to vector search instead of replacing it. For more on the mechanism, see how GraphRAG works.

1 · Vector search2 · Graph indexing3 · Question type4 · Graph searchReplay

Step-by-step description

1. Vector RAG retrieves the chunks nearest the question.
2. A model extracts a graph and writes community reports.
3. Lookups suit vector search; links and themes need the graph.
4. Local search reads the graph and chunks; global search reads the reports.

• Vector RAG returns the chunks whose embeddings are nearest to the question's embedding.
• GraphRAG extracts a knowledge graph from the chunks and summarizes communities in community reports.
• Local search reads the knowledge graph and the text chunks, and global search reads the community reports.

The Staff-level answer

What makes it Staff-level: it names the question type that decides the choice, states the indexing cost, and ends with a measurement plan instead of a verdict.

The rubric

Three variants of this question

When should I pick GraphRAG over vector RAG for a document set?

Decide by the question type your users ask. Choose GraphRAG for connect-the-dots or whole-collection questions, and vector RAG for single-fact lookups. If you are unsure, start with vector RAG and add the graph only for the question types that fail.

How do you answer "what are the main themes in this collection?" with RAG?

A vector search returns a handful of chunks, which is a small slice of a large collection. Global search reads community reports that summarize the whole collection, which is the case it was built for.

What if the documents rarely mention the same entities?

The graph links documents through shared entities, so by that design few shared entities mean few cross-document edges. The graph then adds cost with little gain, and vector RAG is the safer start.

Common mistakes

• "GraphRAG replaces vector RAG." (wrong) Microsoft's query engine keeps a basic search mode, and local search combines graph data with raw text chunks. (right) "GraphRAG adds graph structures to retrieval; vector search stays."
• "GraphRAG is always more accurate." (wrong) VentureBeat reports a comparison where plain RAG edged ahead on single-hop lookup (F1, the balance of precision and recall, was 64.8 versus 63.0 for the best graph method on Natural Questions) and graph-guided retrieval led on multi-hop reasoning (70.3 versus 67.0 overall accuracy on MultiHop-RAG). The report describes the systematic evaluation of Han et al. (2025). (right) "It helps on multi-hop and whole-collection questions."
• "GraphRAG just means a graph database." (wrong) In Microsoft's design the graph is extracted from raw text by a model, then clustered and summarized, and the word is also used loosely for any graph-based retrieval. (right) "I say which GraphRAG I mean."
• "A bigger context window makes the choice irrelevant." (wrong) The Microsoft Research post says its news dataset was far too large to fit into a large language model (LLM) context window. (right) "Fitting the collection is not the only test: a long context still costs per query, and answer quality can degrade as it grows, so I measure both approaches."

Follow-up questions

How much does GraphRAG indexing cost compared with vector indexing?

It costs more, because a model reads every chunk to extract entities and then writes a summary per community. In the toy corpus that is 15 generation calls against none. The GraphRAG repository warns that indexing can be an expensive operation, and Microsoft Research's LazyGraphRAG post (November 2024) reports a variant whose indexing cost equals vector RAG's and is 0.1% of full GraphRAG's. Measure on your own data; see where RAG latency and cost go.

How would you test which one fits your questions?

Write a question set split into lookups, multi-hop questions and whole-collection questions, run both systems, and compare answer quality and cost per group. The RAG evaluation guide covers the metrics.

What goes wrong when GraphRAG extracts a wrong relation?

A wrong edge can send multi-hop retrieval to the wrong chunks, and community reports can repeat the error. Check answers against their cited source text; see grounding and citations.

Where does GraphRAG fit in a retrieval chatbot design?

Treat it as one retrieval path next to vector search, chosen by question type. The retrieval chatbot design walkthrough shows the surrounding system, and the RAG interview questions list the neighboring prompts.

Sources

1. Welcome to GraphRAG (official documentation; page shows no date, read date given) — Microsoft (GraphRAG project), 26 Oct 2026
2. Query Engine overview (official documentation; page shows no date, read date given) — Microsoft (GraphRAG project), 26 Oct 2026
3. GraphRAG (official repository README) — Microsoft (GraphRAG project), 26 Oct 2026
4. GraphRAG: Unlocking LLM discovery on narrative private data — Microsoft Research blog, 13 Feb 2024
5. LazyGraphRAG: Setting a new standard for quality and cost — Microsoft Research blog, 25 Nov 2024
6. RAG vs. GraphRAG: A Systematic Evaluation and Key Insights (arXiv 2502.11371) — arXiv (Han et al.), 17 Feb 2025
7. Stop graphing everything: When GraphRAG actually beats vector RAG (article shows no date to the reader of this page; read date given) — VentureBeat, 26 Oct 2026