Retrieval-Augmented Generation (RAG)
The mainstream pattern for using LLMs with document collections: documents are indexed, relevant chunks are retrieved at query time, and the LLM generates an answer from them. Per llm-wiki-gist, this is how NotebookLM, ChatGPT file uploads, and most RAG systems work.
The canonical definition (rag-original-paper)
The term and architecture come from Lewis et al., 2020 (Facebook AI) — rag-original-paper, now the wiki's neutral primary source for RAG (the gap below had been flagged: prior framing came only from advocates of alternatives). It paired a parametric seq2seq generator with a non-parametric dense vector index of Wikipedia (DPR retriever), in RAG-Sequence and RAG-Token variants, and sold two durable properties: provenance and updatable knowledge without retraining. Crucially, the 2020 design never claimed accumulation or factual-graph connection — so the critiques below extend it rather than refute it.
How the source frames it
RAG is presented as the status quo that the llm-wiki pattern improves on. Its limitation, per the source: the LLM "rediscovers knowledge from scratch on every question" — nothing accumulates. A subtle question requiring synthesis across several documents forces the model to re-find and re-piece fragments each time.
Contrast with llm-wiki
- RAG: retrieval + generation at query time; no persistent intermediate artifact.
- LLM Wiki: knowledge compiled once into a maintained wiki; synthesis and cross-references already exist before the question is asked.
A second data point (gbrain)
gbrain supplies the first concrete comparison in this wiki. It benchmarks a hybrid retriever plus a typed-edge knowledge-graph against vector-only RAG and ripgrep-BM25, reporting +31.4 points P@5 from the graph (P@5 49.1% / R@5 97.9% on a 240-page rich-prose corpus). Its framing of the gap: "Vector search returns chunks that are semantically close. The graph returns chunks that are factually connected." So the critique of RAG here is less "retrieval is bad" than "semantic retrieval alone misses factual connections that an explicit graph captures."
Hybrid retrieval — the neutral source (hybrid-retrieval-rag)
The dedicated vendor-neutral source (InfoQ): vector-alone fails because embeddings are approximation engines that collapse distinguishing tokens (version numbers, error codes, flag names); production needs hybrid retrieval — dense vectors + BM25 (exact match) fused via Reciprocal Rank Fusion (k≈60), optionally cross-encoder reranking (top-50). Most production queries are hybrid (concept + identifier), which single-method retrieval systematically fails; validated at Perplexity/Glean. This independently corroborates gbrain's hybrid retriever, separating that engineering design from GBrain's motivated benchmark. So the RAG critique splits cleanly: BM25 closes the exact-token gap, an explicit knowledge-graph closes the factual-connection gap, and the llm-wiki pattern targets the no-accumulation gap.
An upstream lever — chunking/segmentation (long-context-vs-chunking)
Before any of the gaps above bite, there is a choice the whole taxonomy assumes: how you segment text before you embed it. long-context-vs-chunking (ModernBERT experiments on HUPD/BigPatent) shows a chunk-boundary gap — a whole-document embedding averages a fact apart when its tokens straddle a boundary, scoring near-useless ~0.006–0.030 nDCG@10, while overlapping chunks (512 tokens, 128 overlap) recover it at 0.082. This is the segmentation-side cousin of the exact-token gap: hybrid-retrieval-rag fixes "embeddings collapse distinguishing tokens" with BM25 fusion; this fixes "embeddings average a boundary-crossing fact apart" with chunk overlap. It also carries the cluster's cost lesson onto the context-window axis — a longer window is quadratically expensive (O(n²)) and usually loses to cheaper chunk-and-pool / overlap, the same "cheap segmentation beats heavy machinery" point graphrag-bench makes for graphs. So how you index is a lever distinct from what you retrieve.
A fourth gap — temporal validity (agent-memory-knowledge-graphs)
For persistent agent memory, a further gap appears: evolving facts. When a fact changes (the canonical example: a user moving cities), vector search returns the old and new versions as equally relevant — both are semantically close — so the agent can't tell which is current. A temporal-knowledge-graph (bi-temporal modeling; built with graphiti) time-bounds superseded facts, answering "what is true now." This adds the temporal-validity gap to the taxonomy above and is framed as graphs "quietly replacing RAG" specifically for agent systems — while vector RAG remains strong for static document retrieval.
A fifth gap — whole-corpus sensemaking (graphrag)
Vector RAG is strong on local questions (the answer sits in a few retrievable chunks) but fails on global ones — "what are the main themes across the entire corpus?" — because no chunk holds the answer and top-k never sees the whole. GraphRAG (Microsoft, 2024) closes this by building a knowledge-graph over the corpus, running Leiden community detection, summarizing each community, and map-reducing over those summaries for global queries. So it extends the taxonomy: BM25 closes exact-token, an explicit knowledge-graph closes factual-connection, a temporal-knowledge-graph closes temporal-validity, and GraphRAG closes global synthesis. Now grounded in the canonical primary source from-local-to-global-graphrag (Edge et al., Microsoft, 2024, T1 — 72–83% comprehensiveness wins over naive RAG, and a coarse community level answers global queries at 9×–43× lower token cost), alongside the Rust implementation graphrag-rs and the refinement leanrag.
The neutral check — graphs aren't a blanket win (graphrag-bench). The first third-party benchmark (ICLR 2026, no method of its own) finds the gap-closing tools are task-conditional: GraphRAG beats vector RAG on multi-hop reasoning and summarization but loses on simple fact retrieval (~49–60% vs ~61–65%, where the graph adds redundant context), at 1–2 orders of magnitude more tokens per query. So the taxonomy above describes gaps graphs can close — not ones they always should. For discrete facts in single passages, plain (or hybrid, hybrid-retrieval-rag) retrieval is both better and far cheaper. The exception worth naming is HippoRAG2, the benchmark's strongest and most token-efficient graph method — evidence the cost gap is a design problem, not intrinsic to graph RAG.
A refinement of the fifth, not a sixth gap (leanrag). LeanRAG (AAAI 2026, T2) keeps the global-synthesis target but argues plain GraphRAG leaves two weaknesses: semantic islands (the community summaries aren't linked to each other, blocking cross-community reasoning) and structure-unaware retrieval (flat similarity search over summaries ignores graph topology and pulls redundant evidence). Its semantic aggregation wires the summary layer into a navigable network and its bottom-up hierarchical retrieval walks that structure, reportedly cutting retrieval redundancy ~46% and beating GraphRAG 78.1% on its own benchmarks. It improves the mechanism for the global-synthesis gap — it doesn't add a new gap to the taxonomy — and is the thread's first peer-reviewed anchor.
Not a sixth gap — an attack on the index itself (pageindex)
Every entry above keeps the vector index and adds something beside it: BM25, a typed graph, community summaries, temporal edges. pageindex (vectifyai, MIT, ~35.4k★) removes it. Its argument is one line: "Vector-based RAG retrieves by semantic similarity. But similarity ≠ relevance — what retrieval actually needs is relevance, and relevance requires reasoning." Similarity search "misses what is relevant but not similar, and returns what is similar but not relevant."
The replacement, explicitly modelled on AlphaGo's tree search: index each document as a hierarchical tree following its own sections, then have an LLM reason down that tree to the right section, the way a person opens a long report. No chunks, no embeddings, no vector DB; the retrieval unit is the document's natural section and the retrieval trace is a list of explicit references rather than a similarity score. This is the design underneath openkb's handling of PDFs of 20+ pages.
It relocates a cost rather than removing one. Vector lookup is a matrix multiply; tree search is an LLM call per hop, and the maker's own model advice — a basic model to build the index, "the best model you can afford" to search it — concedes that retrieval is now the expensive step. Claimed result: 98.7% on FinanceBench against ~50% for vector RAG, self-run (T3; evaluation code public, no independent replication). See pageindex for the detail and the caveats.
Where this sits relative to long-context-vs-chunking: that source asks how to segment text before embedding it. PageIndex answers the question by refusing it. The two are the moderate and radical ends of the same complaint — that chunking destroys document structure before retrieval ever runs.
Note: the llm-wiki/gbrain "RAG doesn't accumulate" framing still comes from advocates of an alternative; but the retrieval-mechanics claims are now corroborated neutrally by hybrid-retrieval-rag, and the baseline itself is now grounded in the primary source rag-original-paper (Lewis et al., 2020) rather than only in critics' summaries.
Ancestry note (2026-08-23): RAG is an information-retrieval system with a generator attached, and IR is a named information-science subfield with its own century of evaluation practice — relevance judged by people, with assessor disagreement counted as part of the measurement. Nothing in this cluster evaluates that way information-science-wikipedia.