Graph construction quality

The graph a GraphRAG pipeline builds is itself an artifact that can be good or bad, and this wiki had treated it as a given. Every method page here describes what happens after the knowledge-graph exists — communities, summaries, traversal, PageRank. memgraphrag is the first source that treats the extraction step as the thing to fix.

The symptom: recall up, relevance down

The measurement that frames the problem, from memgraphrag‘s preliminary study on G-Medical: GFM-RAG retrieves at 84.3% recall against vanilla RAG’s 71.8%, and lands 38.5% context relevance against RAG’s 62.9%. The graph finds more of the right evidence and drowns it in wrong evidence.

graphrag-bench had reported both halves separately — graphs reaching 87.9–90.9% recall on complex tasks “sometimes at worse context relevance than vector RAG,” and graph methods losing to plain RAG on fact retrieval. Naming it as one trade makes it a design target rather than a disappointment: the goal is not more retrieval, it is retrieval that stays clean.

The cause: extraction with no global state

Most GraphRAG pipelines run an LLM over document chunks independently. Nothing carries across chunks, so the merged graph inherits three defects (memgraphrag):

Thematic irrelevance — without a view of what the corpus is about, local extraction emits off-topic triples. The measurement that stings: dropping 40% of low-frequency triples slightly improves downstream accuracy (65.28% vs 64.85%). Two-fifths of the extractor’s output was not merely useless but faintly harmful.
Logical inconsistency — independently extracted chunks contradict each other. Three kinds are distinguished: mutually exclusive, temporal, and granularity conflicts. The merged graph offers reasoning paths built from claims that cannot all be true.
Structural fragmentation — with no global coreference resolution or schema alignment, one entity becomes several nodes scattered across disconnected subgraphs, which is precisely what breaks the multi-hop traversal graphs exist to enable.

What fixing it is worth

memgraphrag‘s answer is a shared-memory construction pass — frequency-thresholded schema filtering, conflict adjudication against passage evidence, and coreference bridging — and it does raise the numbers. But its own transplant experiment is the honest measure of how much the graph is worth on its own: handing the improved graph to four other frameworks’ retrievers gains between +0.19 and +0.71 points, against a full-system margin of about +2.9 over the best baseline.

So the current state of the evidence: construction quality is a real defect with a documented mechanism and a measurable cost, and the one attempt to fix it here shows most of the payoff coming from the matched retriever rather than from the cleaner graph. Whether better graphs help in general, or only help the retriever built alongside them, is unsettled — and one paper by one group is where the evidence stops.

Why it matters beyond GraphRAG

The failure modes are properties of LLM extraction over chunked text, not of graph retrieval. Any pipeline that builds a persistent structure from a corpus one window at a time — an ontology, a temporal-knowledge-graph, an agent’s accumulated memory — inherits the same three: it drifts off-topic, contradicts itself across documents, and splits one thing into several. The mitigations are correspondingly general: a global schema, an adjudication step, and coreference across the whole corpus rather than within a chunk.

The third failure mode gets its own number

link-kg and core-kg take the last of those three — one thing split into several — and measure it directly. Corpus-wide coreference resolution run before extraction, rather than dedup run after it, cuts duplicate nodes from 36% to 17.8% on long documents (27% to 10.6% on short ones). The direction agrees with memgraphrag‘s coreference bridging; the useful part is the baseline, because it says a third of an unattended GraphRAG graph is duplicate nodes.

Both papers are the same three authors on one legal corpus, so this is again one group’s evidence. But it is a different group from memgraphrag‘s, working a different domain, and reaching the same conclusion about where the defect lives.

graphrag · memgraphrag · graphrag-bench · knowledge-graph · retrieval-augmented-generation · temporal-knowledge-graph · link-kg · core-kg · entity-resolution · synthesis