rahulnyk/knowledge_graph

A Jupyter notebook that turns a text corpus into a knowledge-graph with a local 7B model, pandas, and NetworkX. MIT, ~3.5k★ / 561 forks, by rahulnyk, with a companion Towards Data Science article. It’s the minimal end of this wiki’s graph-builder shelf: no CLI, no schema layer, no graph database, one notebook (extract_graph.ipynb) and a flowchart.

Its stated purpose is GRAG (graph retrieval-augmented generation) — the same argument graphrag makes, from before the packaged tools existed.

The pipeline

Split the corpus into chunks, each with a chunk_id.
Per chunk, an LLM extracts concepts and the semantic relations between them. Each relation is an edge with weight W1.
Concepts co-occurring in the same chunk get an edge by contextual proximity, weight W2 — on the assumption that things mentioned near each other are related.
Group duplicate pairs: sum the weights, concatenate the relation names. One edge per distinct concept pair, carrying a weight and a list of relations as its label.
Compute node degree (sizing) and communities (coloring), render with Pyvis to an interactive JS page.

Everything runs locally: Mistral 7B OpenOrca via Ollama, chosen for following the system prompt and for costing nothing. (The tech-stack section’s setup step says ollama run zephyr, which contradicts the Mistral choice stated everywhere else — a README slip, not a second model.) Pandas dataframes are the graph schema, with a graph DB named as a later-stage option. Docker on port 8888 serves the notebook.

Concepts, not entities

The design choice worth taking from it. The README explicitly declines NER: “‘Bangalore’ is an entity, and ‘Pleasant weather in Bangalore’ is a concept” — and claims concepts make more meaningful graphs. Every other builder here extracts entities — sift-kg‘s typed PERSON/ORGANIZATION nodes, gbrain‘s wikilink targets, GraphRAG‘s entity pass. Nodes here are propositions instead, which buys expressiveness at the direct cost of the merging problem below: two entities are the same or they aren’t, while two phrasings of a proposition rarely match.

Co-occurrence as a second, separately-weighted edge type

The other transferable idea. Existing build strategies here all extract asserted relations: GBrain matches wikilinks, llm-wiki-agent runs a deterministic then a semantic pass, sift-kg has the LLM propose typed edges with confidence. This project adds a purely structural signal — you were in the same chunk — as its own weighted edge class, then lets the two accumulate on the same pair. It’s cheap (no extra LLM call), and it makes edge weight mean “how often, and by how many routes, did these two turn up together” rather than “how confident is the model.”

It also makes provenance structural: an edge is the chunk the two concepts shared, so tracing a claim back to its text is a lookup rather than a stored citation.

What it admits it doesn’t do

The contributor wishlist is an honest gap list, and its first item is concept deduplication: use embeddings so the LLM’s “doctor” and “doctors” stop being separate nodes, and so “doctor” and “medical practitioner” cluster. Then filtering generic high-frequency concepts, better proximity weighting, and a navigation frontend. Read beside the rest of the shelf, that list is a map of what the packaged tools built next.

Tier

T3 — an author’s README for a demo notebook. No evaluation, no benchmark, no accuracy claim; the graph on GitHub Pages is the whole evidence. Its value here is as a legible baseline: the smallest thing that produces a usable knowledge graph, with each design decision stated plainly enough to compare against sift-kg, lightrag, and graphrag.

knowledge-graph · entity-resolution · graphrag · sift-kg · graphrag-rs · lightrag · retrieval-augmented-generation · leiden-algorithm · synthesis