What is GraphRAG, and how does it work?

In this article

1. Before you start
2. Why interviewers ask this
3. GraphRAG from first principles
4. Worked example
5. What does GraphRAG cost to build?
6. Trade-offs and edge cases
7. How to answer it in an interview
8. Common mistakes
9. Follow-up questions
10. Sources

Before you start

Why interviewers ask this

The consequence of getting this wrong is a retrieval system that sounds fluent and misses the answer. Plain RAG (Lewis et al., 2021) finds the text chunks most similar to a question; nothing in it follows a chain of facts that live in different chunks (a later link is found only if its chunk happens to rank high), and it cannot summarise a collection it never read in full. Interviewers ask about GraphRAG to see whether you know where plain RAG fails and whether you can name the price of the fix.

Microsoft Research's February 2024 post says baseline RAG "struggles to connect the dots" and struggles with queries that need information aggregated across a dataset, such as "What are the top 5 themes in the data?". A weak answer says "GraphRAG is RAG with a graph database". A strong answer names the two failure modes, the pipeline that builds the graph, the two query styles and the indexing cost. The topic comes up in AI-engineer and machine learning (ML) system-design loops that include a retrieval design question.

GraphRAG from first principles

GraphRAG

GraphRAG is a retrieval-augmented generation method that has a language model build a knowledge graph and summaries of its communities from a document collection, and that answers questions from those structures rather than from the nearest text chunks alone.

Knowledge graph

A knowledge graph is a graph whose nodes are entities, such as people, companies and places, and whose edges are the relationships between them, each tied to the text that stated it.

Community

A community is a group of nodes in a graph that are connected to each other more densely than to the rest of the graph, so that it usually covers one topic.

Query-focused summarization

Query-focused summarization is the task of writing a summary of a document collection that answers a specific question, as opposed to retrieving a few passages that match it.

In plain words

GraphRAG is a library whose librarian has first read everything and drawn a map. Plain RAG is a librarian who pulls the few books whose titles sound closest to your question. The mapped librarian can say that these people and companies belong together. The librarian can also say what that cluster is about. The analogy breaks in two places. The model draws the map, so it can contain mistakes. Drawing it for a large library takes a lot of reading before the first question. In one sentence: GraphRAG pre-computes structure and summaries, so answering means reading the structure instead of searching raw text.

Step 1: Split the documents into chunks

A chunk is a piece of a document cut to a fixed size so that each piece fits in a model's input. GraphRAG's default dataflow (read Oct 2026) calls its chunks TextUnits, with a default size of 1,200 tokens, and keeps a link from every document to its TextUnits so that any answer can point back to its source. The paper's own experiments used 600-token chunks.¹ This is the same first step as plain RAG, so the extra work comes after it. What you now know: the corpus is a list of chunks with source links, ready to be read by a model.

Step 2: Extract entities and relationships into a graph

A language model reads each chunk and lists the entities it mentions and the relationships between them, then the descriptions of repeated entities are merged. The documentation says this produces the graph tables, and that extracting claims, short factual statements about entities, is optional and off by default. Each node and edge keeps the identifier of the chunk it came from. This is the expensive step, because the model reads the whole corpus once, so the bill grows with the size of the collection.

Microsoft Research says the indexing cost of its LazyGraphRAG variant is 0.1% of the cost of full GraphRAG (Microsoft Research, November 2024), so, by that post's figures, full GraphRAG indexing costs about 1,000 times vector RAG's (1 ÷ 0.001). What you now know: the corpus has become a knowledge graph in which "Maria Lopez led engineering at Brightwave" is an edge, not just a sentence.

Step 3: Find communities

The graph is clustered into communities, and the clusters are organised in levels. The documentation says GraphRAG uses hierarchical Leiden clustering. The Leiden algorithm is a method that groups graph nodes into densely connected communities by repeatedly moving nodes between groups, and "hierarchical" means each community is then split again into sub-communities, recursively, until a community cannot be split further, so broad communities contain narrower ones (Edge et al. 2025, Section 3.1.4). The paper names the levels C0 for the root level with the fewest communities, then C1, C2 and C3, the lowest level with the most. What you now know: every entity belongs to a community at each level, from broad themes down to small topics.

Step 4: Summarise each community

The graph is grouped into communities and each community is summarised: for each community the model writes a report, built bottom-up; when a higher-level community is too large to fit, its sub-communities' reports stand in for their raw details (Edge et al. 2025, Section 3.1.5). The February 2024 post describes these reports as pre-generated "for each semantic cluster".

The documentation then adds a final stage that creates embeddings, the number lists used for similarity search, for the chunks, entity descriptions and report text, so that a query can later be matched against any of them. The reports are the part that lets GraphRAG answer a question about the whole collection, because the model never has to read the raw text again to describe a theme. What you now know: the collection now has a table of contents written by a model, and it was written before anyone asked a question.

Step 5: Answer a question, locally or globally

Global search answers broad questions about the whole collection using the community summaries. In the paper the summaries are split into token-sized batches, each batch writes a partial answer with a 0 to 100 helpfulness score, zero scores are dropped, and the highest-scoring partial answers are combined into the final one (Edge et al. 2025, Section 3.1.6). Local search answers questions about specific entities by fanning out to their neighbours and related concepts.

A third mode, DRIFT (Dynamic Reasoning and Inference with Flexible Traversal), works like local search but adds community information, and basic search is ordinary top-k vector search for questions that plain RAG already handles well. Top-k retrieval is the method of returning the k chunks whose embeddings are most similar to the question's. What you now know: the question decides the route, and a good system picks the cheapest mode that works.

1 · Chunk2 · Graph3 · Communities4 · Summaries5 · AnswerReplay

Step-by-step description

1. Split documents into chunks.
2. Extract entities and relationships.
3. Group the graph into levels.
4. Write a report per community.
5. Answer locally or globally.

• A language model extracts the graph from the chunks.
• The graph is grouped into communities and each community is summarised.
• A question is answered from the graph and the summaries.

Worked example

Take eight short text chunks about two companies and ask: "In which city is the company headquartered where the chief executive of Orbit Labs previously led engineering?" The chunks and the extracted facts are illustrative, written by hand for this page; in a real system a language model would extract them. The answer, Rotterdam, needs three links: Orbit Labs has the chief executive Maria Lopez, she previously led engineering at Brightwave, and Brightwave is headquartered in Rotterdam. No single chunk states all three.²

Rank the chunks by cosine similarity of word counts, a simple stand-in for embedding search. The chunk about Maria Lopez leading engineering scores 0.527, the founding chunk scores 0.447, and the chunk that names Rotterdam scores 0.258, third of eight. With top-2 retrieval the answer chunk is missed, so a model reading only those two chunks can name Brightwave but not its city.² Retrieving the top 3 would have caught it here, so read this as an illustration of the risk, not a proof: whether the last link of a chain ranks high enough is luck.

Text description of the diagram

• The chunk about Maria Lopez leading engineering scores 0.527.
• The founding chunk scores 0.447.
• The chunk that names Rotterdam scores 0.258, third of eight.

Now follow the graph. This walk is a simplified stand-in: GraphRAG's real local search picks entry entities by embedding similarity to the question, then gathers their connected entities, relationships, covariates, community reports and linked text units (local search docs, read Oct 2026). In the stand-in, start at the entity named in the question, Orbit Labs, and expand along edges for three hops.

The first hop reaches the founding chunk and the product chunk; the second reaches the chunk about Brightwave; the third reaches the chunk naming Rotterdam, plus one about Brightwave's acquirer. Within three hops the walk collects 5 of the 8 chunks and includes the answer chunk.² It also pulled in two chunks the question did not need, which is the cost of walking a graph: more context, some of it irrelevant.

Text description of the diagram

• Orbit Labs has the chief executive Maria Lopez.
• She previously led engineering at Brightwave.
• Brightwave is headquartered in Rotterdam.

The second use is global questions, where a chain is not the issue and coverage is. Edge et al. (2025) tested on podcast transcripts of about one million tokens and a news set of about 1.7 million tokens, with 125 generated questions per set.

The ratios in the fourth and fifth rows are computed from the paper's token counts.² The win rates are the ranges the paper reports for its global approaches, the four GraphRAG levels plus source-text summarisation, against vector RAG. Source-text summarisation means running the same batch-and-combine answering over every raw chunk, with no graph; it is a separate baseline from vector RAG, which is why the 2% to 3% token figure compares root summaries with it and not with plain RAG. Also, the paper's abstract frames the gain as comprehensiveness and diversity of answers, not factual accuracy.

Text description of the diagram

• Summarising the podcast source text takes 1,014,611 tokens; root summaries take 26,657.
• Summarising the news source text takes 1,707,694 tokens; root summaries take 39,770.

What does GraphRAG cost to build?

GraphRAG costs one language-model pass over the whole corpus at indexing time, plus the model calls to write every community report, before the first query. The documentation home page lists the indexing stages and does not state a price, so any figure is something you measure on your own data and model. Microsoft Research's November 2024 post on a variant it calls LazyGraphRAG says its indexing cost is "identical to vector RAG and 0.1% of the costs of full GraphRAG", which confirms the full pipeline is the expensive end of the range. That makes the interview answer a cost-benefit one: pay the indexing cost when many questions need connected or corpus-wide answers over a stable collection.

Trade-offs and edge cases

GraphRAG trades indexing cost for better answers on connected and corpus-wide questions, and it is not the better choice for single-passage facts.

When plain RAG wins. The documentation keeps a basic search mode for queries that are "best answered by baseline RAG (standard top k vector search)", so GraphRAG is a router over methods, not a replacement. A question whose answer sits in one passage gains nothing from a graph.

Extraction errors travel. If the model extracts a wrong relationship, every answer that walks that edge inherits the error. Test the extraction on a sample before indexing everything.

Freshness. The graph and reports describe the corpus as it was when they were built; when documents change, the affected chunks, entities and reports have to be rebuilt, so check how your tool handles updates.

Evidence limits. The paper's results come from two datasets of one to two million tokens and questions the authors generated; treat them as evidence for global questions on that scale, not a general ranking. Related: the cross-encoder reranker improves plain retrieval cheaply and is worth trying before a graph, and AI agents can call local and global search as tools.

How to answer it in an interview

The spoken structure is: the failure of plain RAG, the pipeline, the two query modes, then the cost.

Common mistakes

"GraphRAG replaces the vector database." (wrong) The pipeline still creates embeddings, and basic search is plain vector search. (right) "GraphRAG adds structure on top of retrieval and routes between modes."

"A knowledge graph is the same as a graph database." (wrong) The knowledge graph is the extracted data; where you store it is a separate choice. (right) "The graph is an index built by a model, whatever it is stored in."

"GraphRAG is always better." (wrong) The documentation keeps a basic mode for questions plain RAG serves well. (right) "It wins on connected and corpus-wide questions and costs more to build."

"The extracted graph is ground truth." (wrong) A model wrote it and can be wrong. (right) "I would sample and check extraction, and keep source links."

Follow-up questions

How does GraphRAG work?

A language model splits documents into chunks, extracts entities and relationships into a graph, clusters the graph into communities and writes a report for each. Questions are then answered from the graph (local search) or from the community reports (global search).

How do you build and implement GraphRAG?

Follow the four indexing steps above (chunk, extract, cluster, summarise) and the embedding stage, using Microsoft's open-source indexing pipeline; its documentation warns that graphrag init with the force option overwrites your configuration and prompts. Start with a small sample, check the extracted entities by hand, and compare with a plain RAG baseline. Add the graph only if the baseline fails on connected or corpus-wide questions; the embeddings and vector search hub covers the baseline, and hybrid search is a cheaper upgrade to try first. For where retrieval sits in a larger design, see machine learning system design and what an embedding is.

How does knowledge graph RAG work?

It retrieves by following relationships between entities rather than only by text similarity, so a question about a person's former employer can reach the employer's page. GraphRAG additionally summarises communities of the graph for broad questions.

What if the corpus is ten times larger, or changes daily?

Indexing cost grows with corpus size, because a model reads every chunk (Step 2), and the graph and reports go stale as documents change, so affected parts must be rebuilt. For a fast-changing collection, start with plain retrieval and add the graph only for the stable part.

GraphRAG vs traditional RAG: when does each win?

Traditional RAG wins when the answer is in one passage and cost matters. GraphRAG wins when the answer needs linked facts or a view of the whole collection and the collection is stable enough to justify indexing.

Sources

1. From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv 2404.16130 v2) — arXiv (Edge et al.), Microsoft Research, 19 Feb 2025
2. GraphRAG documentation: home page — Microsoft, 24 Oct 2026
3. GraphRAG documentation: local search — Microsoft, 24 Oct 2026
4. GraphRAG documentation: DRIFT search — Microsoft, 24 Oct 2026
5. GraphRAG documentation: default indexing dataflow — Microsoft, 24 Oct 2026
6. GraphRAG: LLM discovery on narrative private data (Microsoft Research blog) — Microsoft Research, 13 Feb 2024
7. LazyGraphRAG: Setting a new standard for quality and cost — Microsoft Research, 25 Nov 2024
8. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401 v4) — arXiv (Lewis et al.), 12 Apr 2021