MemGraphRAG

MemGraphRAG: Memory-based Multi-Agent System for Graph Retrieval-Augmented Generation — Wu, Xiang, Tang, Chen, Q. Zhang, Su (Xiamen University + Jilin University), arXiv:2606.00610 submitted 2026-05-30, accepted to KDD 2026. Code MIT-licensed, 139★, last pushed 2026-06-20.

Its subject is the part of GraphRAG this wiki had documented least: not how to retrieve over a knowledge graph, but whether the graph the pipeline builds is any good. See graph-construction-quality for the failure modes it names.

The diagnosis it starts from

The paper opens by reproducing the finding that graphrag-bench made this wiki’s headline — that graph RAG often loses to plain vector RAG — and it isolates the shape of the loss. On G-Medical, GFM-RAG reaches 84.3% retrieval recall against vanilla RAG’s 71.8%, and simultaneously drops to 38.5% context relevance against RAG’s 62.9%. Graph expansion buys coverage and pays in noise.

The cause it proposes: isolated, fragment-level extraction. Most GraphRAG pipelines run an LLM over document chunks independently, with no persistent global state, so the merged graph inherits three defects — thematic irrelevance, logical inconsistency, structural fragmentation.

One measurement makes the noise concrete. Filtering out 40% of low-frequency triples slightly improves downstream accuracy (65.28% vs 64.85%). Two-fifths of what the extractor produced was worth less than nothing.

The method

A shared-memory multi-agent construction pipeline producing a three-layer graph:

Schema layer — abstract ontology triples over entity types, induced from the corpus.
Fact layer — concrete relation triples extracted from documents.
Passage layer — the source text supporting each fact.

Layers are bidirectionally linked, so a retrieved fact traces to its passage and up to its schema. Three construction mechanisms sit on top: unified schema filtering (drop schemas below a frequency threshold — thematic denoising), global adjudication (a conflict detector resolves contradictions against passage evidence), and memory-guided bridging (coreference and schema alignment across chunks, against fragmentation).

Retrieval is multi-layer filtering, then node initialization, then Personalized PageRank over the heterogeneous graph. Two scoring details: a log-degree penalty suppressing generic hub types, and an IDF-weighted information-density term favouring passages with rare entities, damped by α=0.05 so passage nodes don’t dominate propagation.

Results

Five datasets (HotpotQA, 2WikiMultiHopQA, MuSiQue, plus G-Bench Medical and Novel), all methods on the same embedding model (NV-Embed-v2), k=5, GPT-4o-mini for both indexing and generation, temperature 0.

Generation, overall average: MemGraphRAG 59.25 against LinearRAG 57.15 (+2.10), GFM-RAG 55.27, HippoRAG2 55.79, LightRAG 47.80, MS-GraphRAG 41.43, and vanilla RAG top-5 47.97.

Retrieval on G-Bench(Medical): it holds recall while fixing relevance — complex reasoning recall 90.42 / relevance 82.64, against GFM-RAG’s 85.03 / 33.06 and LightRAG’s 82.91 / 42.79. The one baseline that already balanced both is LinearRAG (87.03 / 81.58).

Latency: 0.061s average per retrieval, against LinearRAG 0.123s, GFM-RAG 1.375s, HippoRAG 1.586s, LightRAG 11.052s. Attributed to PPR rather than per-query LLM filtering or iterative reasoning loops.

Ablations (HotpotQA): removing conflict resolution costs the most (69.40 → 66.95), then hub suppression (67.22), schema filter (68.10 on 2Wiki), information-density term (68.67).

The result the paper reports but doesn’t dwell on

Table 3 transplants MemGraphRAG’s constructed graph under four other frameworks’ retrievers. The gains are real, consistent, and small: HippoRAG 51.07 → 51.78, HippoRAG2 56.77 → 56.96, MS-GraphRAG 43.75 → 44.21, LazyGraphRAG 44.39 → 44.97. Between +0.19 and +0.71 points.

The full system scores 59.68 on the same aggregation. So the paper’s headline contribution — better graph construction through shared memory — carries under a fifth of the improvement when separated from the retrieval algorithm that was designed for it. The paper frames the table as evidence its constructor is a universal upgrade, which it is. Read the other way, it says most of the measured win comes from the retriever, and the two are not separable in the way the framing implies.

The paper does not claim otherwise; the hierarchical retriever is a stated contribution. But a reader taking away “memory-based construction is what makes this work” is taking away more than Table 3 supports.

Provenance: the benchmark is not third-party to this method

This wiki records graphrag-bench as “a neutral third-party benchmark… the authors propose no method of their own.” That was accurate when written. It no longer describes the relationship here.

GraphRAG-Bench’s authors are Zhishang Xiang, Chuanjie Wu, Qinggang Zhang, Shengyuan Chen, Zijin Hong, Xiao Huang, Jinsong Su. MemGraphRAG’s are Chuanjie Wu, Zhishang Xiang, Yunbo Tang, Zerui Chen, Qinggang Zhang, Jinsong Su. Four names are shared — including both equal-contribution first authors and the corresponding author. The same group built the benchmark, published the diagnosis that graph RAG underperforms, and has now published the method that tops the benchmark on G-Medical and G-Novel.

Nothing here suggests misconduct. The benchmark came first, genuinely proposed no method at the time, and its diagnosis has been cited by others. But the two independent-looking data points this wiki holds — a neutral benchmark, and a method winning on it — come from one lab, and the benchmark’s neutrality claim has to be scoped to other people’s methods from now on.

Tier and what to distrust

T1 — peer-reviewed (KDD 2026), full method, released MIT code, all systems run on one embedding model and one LLM at temperature 0.

Three things to hold loosely. The margin is thin: +2.10 average over LinearRAG, on LLM-judged accuracy with GPT-4o-mini as the judge and GPT-4o-mini as the generator — the same model family scoring its own outputs, which is the evaluation weakness graphrag-bench itself flagged across this literature. It does not win everywhere: on the llama-3-70b backbone (Appendix A), HippoRAG2 beats it on MuSiQue containment accuracy (33.90 vs 33.70) and on G-Novel (56.16 vs 55.76), with the overall margin +3.00. And the two tables disagree on one cell — G-Novel for the full system reads 57.41 in Table 1 and 54.41 in Table 3, while every other Table 3 entry reconciles exactly with Table 1. A transcription slip in a number that feeds a headline average.

graphrag · graphrag-bench · graph-construction-quality · retrieval-augmented-generation · knowledge-graph · hipporag2 · lightrag · from-local-to-global-graphrag · raptor · synthesis