Retrieval-Augmented Generation (RAG)
The mainstream pattern for using LLMs with document collections: documents are indexed, relevant chunks are retrieved at query time, and the LLM generates an answer from them. Per llm-wiki-gist, this is how NotebookLM, ChatGPT file uploads, and most RAG systems work.
The canonical definition (rag-original-paper)
The term and architecture come from Lewis et al., 2020 (Facebook AI) — rag-original-paper, now the wiki’s neutral primary source for RAG (the gap below had been flagged: prior framing came only from advocates of alternatives). It paired a parametric seq2seq generator with a non-parametric dense vector index of Wikipedia (DPR retriever), in RAG-Sequence and RAG-Token variants, and sold two durable properties: provenance and updatable knowledge without retraining. Crucially, the 2020 design never claimed accumulation or factual-graph connection — so the critiques below extend it rather than refute it.
How the source frames it
RAG is presented as the status quo that the llm-wiki pattern improves on. Its limitation, per the source: the LLM “rediscovers knowledge from scratch on every question” — nothing accumulates. A subtle question requiring synthesis across several documents forces the model to re-find and re-piece fragments each time.
Contrast with llm-wiki
- RAG: retrieval + generation at query time; no persistent intermediate artifact.
- LLM Wiki: knowledge compiled once into a maintained wiki; synthesis and cross-references already exist before the question is asked.
A second data point (gbrain)
gbrain supplies the first concrete comparison in this wiki. It benchmarks a hybrid retriever plus a typed-edge knowledge-graph against vector-only RAG and ripgrep-BM25, reporting +31.4 points P@5 from the graph (P@5 49.1% / R@5 97.9% on a 240-page rich-prose corpus). Its framing of the gap: “Vector search returns chunks that are semantically close. The graph returns chunks that are factually connected.” So the critique of RAG here is less “retrieval is bad” than “semantic retrieval alone misses factual connections that an explicit graph captures.”
Hybrid retrieval — the neutral source (hybrid-retrieval-rag)
The dedicated vendor-neutral source (InfoQ): vector-alone fails because embeddings are approximation engines that collapse distinguishing tokens (version numbers, error codes, flag names); production needs hybrid retrieval — dense vectors + BM25 (exact match) fused via Reciprocal Rank Fusion (k≈60), optionally cross-encoder reranking (top-50). Most production queries are hybrid (concept + identifier), which single-method retrieval systematically fails; validated at Perplexity/Glean. This independently corroborates gbrain‘s hybrid retriever, separating that engineering design from GBrain’s motivated benchmark. So the RAG critique splits cleanly: BM25 closes the exact-token gap, an explicit knowledge-graph closes the factual-connection gap, and the llm-wiki pattern targets the no-accumulation gap.
An upstream lever — chunking/segmentation (long-context-vs-chunking)
Before any of the gaps above bite, there is a choice the whole taxonomy assumes: how you segment text before you embed it. long-context-vs-chunking (ModernBERT experiments on HUPD/BigPatent) shows a chunk-boundary gap — a whole-document embedding averages a fact apart when its tokens straddle a boundary, scoring near-useless ~0.006–0.030 nDCG@10, while overlapping chunks (512 tokens, 128 overlap) recover it at 0.082. This is the segmentation-side cousin of the exact-token gap: hybrid-retrieval-rag fixes “embeddings collapse distinguishing tokens” with BM25 fusion; this fixes “embeddings average a boundary-crossing fact apart” with chunk overlap. It also carries the cluster’s cost lesson onto the context-window axis — a longer window is quadratically expensive (O(n²)) and usually loses to cheaper chunk-and-pool / overlap, the same “cheap segmentation beats heavy machinery” point graphrag-bench makes for graphs. So how you index is a lever distinct from what you retrieve.
A fourth gap — temporal validity (agent-memory-knowledge-graphs)
For persistent agent memory, a further gap appears: evolving facts. When a fact changes (the canonical example: a user moving cities), vector search returns the old and new versions as equally relevant — both are semantically close — so the agent can’t tell which is current. A temporal-knowledge-graph (bi-temporal modeling; built with graphiti) time-bounds superseded facts, answering “what is true now.” This adds the temporal-validity gap to the taxonomy above and is framed as graphs “quietly replacing RAG” specifically for agent systems — while vector RAG remains strong for static document retrieval.
A fifth gap — whole-corpus sensemaking (graphrag)
Vector RAG is strong on local questions (the answer sits in a few retrievable chunks) but fails on global ones — “what are the main themes across the entire corpus?” — because no chunk holds the answer and top-k never sees the whole. GraphRAG (Microsoft, 2024) closes this by building a knowledge-graph over the corpus, running Leiden community detection, summarizing each community, and map-reducing over those summaries for global queries. So it extends the taxonomy: BM25 closes exact-token, an explicit knowledge-graph closes factual-connection, a temporal-knowledge-graph closes temporal-validity, and GraphRAG closes global synthesis. Now grounded in the canonical primary source from-local-to-global-graphrag (Edge et al., Microsoft, 2024, T1 — 72–83% comprehensiveness wins over naive RAG, and a coarse community level answers global queries at 9×–43× lower token cost), alongside the Rust implementation graphrag-rs and the refinement leanrag.
The neutral check — graphs aren’t a blanket win (graphrag-bench). The first third-party benchmark (ICLR 2026, no method of its own) finds the gap-closing tools are task-conditional: GraphRAG beats vector RAG on multi-hop reasoning and summarization but loses on simple fact retrieval (~49–60% vs ~61–65%, where the graph adds redundant context), at 1–2 orders of magnitude more tokens per query. So the taxonomy above describes gaps graphs can close — not ones they always should. For discrete facts in single passages, plain (or hybrid, hybrid-retrieval-rag) retrieval is both better and far cheaper. The exception worth naming is HippoRAG2, the benchmark’s strongest and most token-efficient graph method — evidence the cost gap is a design problem, not intrinsic to graph RAG.
A refinement of the fifth, not a sixth gap (leanrag). LeanRAG (AAAI 2026, T2) keeps the global-synthesis target but argues plain GraphRAG leaves two weaknesses: semantic islands (the community summaries aren’t linked to each other, blocking cross-community reasoning) and structure-unaware retrieval (flat similarity search over summaries ignores graph topology and pulls redundant evidence). Its semantic aggregation wires the summary layer into a navigable network and its bottom-up hierarchical retrieval walks that structure, reportedly cutting retrieval redundancy ~46% and beating GraphRAG 78.1% on its own benchmarks. It improves the mechanism for the global-synthesis gap — it doesn’t add a new gap to the taxonomy — and is the thread’s first peer-reviewed anchor.
Note: the llm-wiki/gbrain “RAG doesn’t accumulate” framing still comes from advocates of an alternative; but the retrieval-mechanics claims are now corroborated neutrally by hybrid-retrieval-rag, and the baseline itself is now grounded in the primary source rag-original-paper (Lewis et al., 2020) rather than only in critics’ summaries.