Log — Operational Databases Wiki

Append-only history. Entries start with ## [YYYY-MM-DD] <op> | <title>. <op> ∈ {split, ingest, query, lint}.

[2026-07-16] split | operational-databases-wiki created from _inbox (3 sources)

Spun out of the hub _inbox databases-beyond-olap cluster once it reached 3 — turso (embedded OLTP engine, parked 2026-06-23), awesome-db-tools (general DB-tooling catalog, parked 2026-06-24), and planetscale-768-servers (distributed-SQL sharding explainer, the 2026-07-16 trigger). Both prior park notes predicted a 3rd general-DB source would trip the spin-out; the human chose a new spoke over broadening the OLAP-only ../analytical-databases-wiki (keeping that spoke's columnar identity intact). Domain: transactional/OLTP + distributed-SQL databases and the tooling around them, across the embedded↔distributed spectrum — the operational sibling to ../analytical-databases-wiki.

[2026-07-16] ingest | Turso (in-process SQLite-compatible OLTP engine)

Founding source. turso — a Rust rewrite of SQLite, in-process OLTP; MVCC BEGIN CONCURRENT writes, io_uring async I/O, CDC, exact vector search, tantivy FTS, 7-language bindings, built-in MCP server (9 tools). v0.7.0 (Jul 2026), MIT, ~22.9k★, beta but in production (Turso Cloud/Kin AI/Spice.ai). Typed SoftwareApplication + source (T1, primary repo). Anchors the embedded pole of operational-databases; noted the MCP/agent seam (seekdb). Maker-org deferred (thin).

[2026-07-16] ingest | Making 768 servers look like 1 (PlanetScale)

Founding source + spin-out trigger. planetscale-768-servers — sharding a transactional SQL DB so 256 shards × 3 = 768 servers (~4 TB each, ~1 PB, millions QPS) present as one logical DB. Motivates sharding via the single-writer WAL / replicas-duplicate-not-distribute / multi-day-backup bottlenecks; mechanism = proxy router (Vitess/MySQL, Neki/Postgres) + hash shard key + cross-shard aggregation behind an NLB. Typed TechArticle + source (T2, vendor blog). New concept database-sharding; new Org planetscale (maintains Vitess, built Neki). Anchors the distributed pole.

[2026-07-16] ingest | awesome-db-tools (curated DB-tooling catalog)

Founding source. awesome-db-tools (mgramin) — ~200+ tools for working with databases (not engines), 20+ categories (clients, migration, backup, monitoring, HA/sharding, security, SQL utils…), spanning OLTP+OLAP+NoSQL; CC0, ~5.2k★. Typed Collection + source (T3, link catalog / page-candidate source). The tooling member of the spoke; flagged the open question of whether cross-type DB tooling coheres with the OLTP-engine core.

Also created concepts online-transaction-processing (the OLTP-vs-OLAP boundary that defines the spoke) and the operational-databases umbrella. Spine files (index/synthesis/this log) written. avoid-ai-writing self-pass (clean). Site rebuild + verify run at split (required for structural page moves).

[2026-07-29] ingest | Rethinking Data: three-tier stack → client-side event sourcing (InfoQ podcast)

Routed from the hub (Telegram). Five pages: the source summary rethinking-data-client-event-sourcing (PodcastEpisode, T2), two concepts (local-first-architecture, event-sourcing), the artifact livestore, and the person johannes-schickling (Prisma founder, LiveStore maintainer, the episode's guest).

The substance the spoke didn't have: this is the first source about where the OLTP store lives relative to the user, and it extends the founding embedded↔distributed axis past turso into the client. It's also the first material on merge semantics — event log vs CRDT vs query-sync — which the "where's the distributed-consistency substance?" open question had been waiting for, arriving from the client end rather than the NewSQL end that question anticipated. Both were noted in synthesis rather than quietly closed: the server-side consensus story is still missing.

Recorded as a real tension rather than smoothing it: planetscale-768-servers shards to preserve one globally consistent database, and local-first abandons global consistency on purpose. Both OLTP, opposite convictions about what OLTP protects. Schickling draws his own boundary (financial systems, social graphs = wrong fit), which is what makes it a trade rather than a contradiction.

T2 with a stated caveat: independent publisher, interested guest — the event-sourcing-beats-CRDTs claim comes from the person maintaining an event-sourcing framework, with no benchmark. Added as its own open question, and a CRDT-side source (Automerge/Yjs/Loro) is now the named gap.

Not paged, one source each, per the ENTITIES recursion discipline: the sync engines (ElectricSQL, PowerSync, Zero), the CRDT libraries, Overtone, Prisma-as-org, and the Local-First Conference. Nicholas Schiefer and Geoffrey Litt are credited on livestore but not paged — Litt in particular is a ../research-wiki-adjacent name (tools-for-thought) and would be worth a node if he recurs. avoid-ai-writing run. Verify deferred per hub policy (content-only, no page moves).

[2026-08-03] ingest | Pagination across shards (Medium, Umesh Kumar Yadav) — PAYWALLED STUB

Routed from the hub (Telegram). New: sharded-pagination-interview-post (T4 stub). Updated: database-sharding (the aggregation bullet now names its expensive cases), synthesis (open question), index. Body not retrieved. WebFetch returned the member-only preamble; a Firecrawl scrape returned the same paywall page. Per ../HUB.md edge handling this is the "Firecrawl also fails" case, so the source is recorded as an honest stub rather than dropped or written from guesswork. Nothing about the solution — keyset vs offset, scatter-gather merge, cursor encoding — is attributed to it. What the preview does establish: the framing question (order table across many shards, user opens page 100,000) and the anti-pattern SELECT * FROM orders ORDER BY create_time DESC LIMIT 1000000, 10, called "one of the slowest ways to retrieve data". Genre is interview prep, not documentation. Kept because it names a real gap: database-sharding compressed cross-shard aggregation into one bullet and planetscale-768-servers never says what an ORDER BY/LIMIT costs once rows sit on 256 shards. Deep pagination is the sharpest ordinary case — the router can't know which shard holds rows 1,000,000-1,000,010 without asking every shard for its first 1,000,010. T4 justified: paywalled, unread, single-author interview-prep content, no editor, no cited sources. Resolution path recorded on the page and in synthesis — a friend link, or Vitess docs on ORDER BY/LIMIT handling (planetscale already owns two nodes here). Entity discovery skipped (author unknown beyond a byline; nothing to page on evidence). Verify deferred per hub policy (content-only). avoid-ai-writing run.

[2026-08-03] ingest | Vitess query-serving documentation (v23.0)

Pulled at the human's instruction ("pull the Vitess docs") as the readable substitute for the paywalled sharded-pagination-interview-post. New: vitess-query-serving-docs (T1 source summary), cross-shard-queries (DefinedTerm/mechanism), vitess (SoftwareApplication). Updated: database-sharding (the aggregation bullet now points at the mechanism), sharded-pagination-interview-post (records that the substitute landed and what it didn't supply), synthesis (open question moved from unanswered to narrowed), index. Read: concepts/execution-plans, concepts/vtgate, user-guides/sql/vexplain, reference/features/ sharding, reference/compatibility/mysql-compatibility — v23.0, on 2026-08-03. Docs pages were thin through WebFetch summarisation, so the primary markdown was pulled from raw.githubusercontent (vitessio/website) and read directly. Substance: the planning rule is "push down as much work as possible to the underlying MySQL instances... when this is not possible... collect input from multiple sources and merge the results". Plans are operator trees evaluated leaf-to-root. A routing operator names keyspace, sharded-ness and vindex; scatter is defined as a sharded keyspace addressed without a vindex, with the explicit caveat that not every multi-shard query counts as scatter. VTGate is the merge point and speaks the MySQL wire protocol "as if it is a MySQL Server". The instrument the corpus was missing: VEXPLAIN TRACE annotates operators with NoOfCalls, Avg/MedianNumberOfRows and ShardsQueried; VEXPLAIN KEYS reports grouping/join/filter columns as candidate sharding keys without executing. Together they answer "how badly does this query scatter" and "which vindex would stop it". Honest limit recorded in three places: none of the pages read gives a cost model for deep LIMIT/OFFSET or recommends keyset/cursor pagination. The keyset remedy is the obvious consequence of the mechanism and is deliberately not written as sourced — cross-shard-queries states the problem and says plainly that the fix is uncited here. T1 justified: first-party reference docs for shipped software, version-pinned, describing behaviour VEXPLAIN can confirm against a live cluster. Caution: intent and mechanism, not measurement; VEXPLAIN exists only from v16 and operator names move between releases. Verify deferred per hub policy (content-only). avoid-ai-writing run.

[2026-08-08] ingest | Principles of Distributed Computing (Wattenhofer, ETH Zürich)

Routed from the hub (Telegram). T1 — 321-page graduate course text, freely published by ETH's DISCO group, theorems with proofs. New pages: podc-lecture-notes (source), consensus, consistency-models, plus entities roger-wattenhofer and eth-zurich.

Closes the spoke's #1 growth edge, from the other side. The edge asked for engine internals; this is the theory those engines were built against. Three results reframe what the corpus already held:

FLP, stated exactly (Thm 16.14): "There is no deterministic algorithm which always achieves consensus in the asynchronous model, with f > 0." So a store committing across shards is not solving consensus — it is buying partial synchrony (a timeout) or randomisation. The corpus can now ask which premise a vendor weakened.
Byzantine agreement needs f < n/3, the origin of 3f+1 replica counts; authentication moves the bound.
CAP is a theorem with a model — Fox & Brewer 1999, proven by Gilbert & Lynch 2002 for the asynchronous model, relaxable under partial synchrony. That replaces pick-two-of-three with the real move: weaken consistency by a named amount. Causal consistency is the vocabulary the spoke was missing when rethinking-data-client-event-sourcing arrived arguing merge semantics from an event-sourcing maintainer's position.

Recorded honestly: fifteen of twenty-two chapters are out of scope here — graph colouring, distributed sorting, MIS, locality lower bounds, social networks, synchronizers, communication complexity, wireless, self-stabilisation, labeling schemes, game theory. The source's centre of gravity sits outside the spoke holding it, which is a real routing tension and is written on the source page, in synthesis, and in the hub log. If more distributed-theory sources arrive, a distributed-systems-theory spoke is the natural home and this should be revisited rather than defended.

Edge refreshed: the consistency edge is now partly closed — the engine half stands, and Raft (what production systems actually run) is in neither the book nor this wiki. Added as edge 4.

Entities: 2 created.

[2026-08-09] ingest | Postgres and MySQL — the base case the spoke never wrote (via research pass)

Coverage edge 5 closed. Sources are the projects' own documentation, both T1: PostgreSQL's Architectural Fundamentals and Introduction to MVCC (postgresql.org/docs/current), and the MySQL 8.4 Reference Manual's Introduction to InnoDB (dev.mysql.com).

New pages: postgresql, mysql, and multiversion-concurrency-control (the mechanism both describe, which also takes a quarter off edge 6).

What was wrong before. Every page here is defined against single-node OLTP — sharding is what you do when one node stops being enough, local-first is what you do when the round trip to it hurts — and the wiki never said what that node does. MySQL was worse than absent: it existed only as the thing vitess shards, so all of its properties arrived filtered through planetscale's product.

Two facts earned their place immediately. Postgres forks a backend process per connection, which is why pooling is not optional and which measures the distance to turso (no server at all) and to vitess (the endpoint is a proxy). And InnoDB's clustered index puts table data inside the primary key index, making the primary key and the shard key one decision — the same decision cross-shard-queries describes at the fleet level, where vitess's vindexes make it again.

Both engines land on MVCC from different implementations — new heap versions plus vacuum against an undo log — and the shared guarantee is the single-node form of what consistency-models asks of distributed systems.

Scope recorded honestly: these two doc pages stop before WAL, checkpointing, crash recovery, replication and the planner. That is edge 5's successor, and it is where planetscale-768-servers already reports the real bottlenecks.

Entities: 0 created — no vendor org pages; both sources are project documentation, and the Turso-maker deferral note in index.md applies here too.

[2026-08-10] ingest | The Byzantine Generals Problem (Lamport, Shostak & Pease, 1982)

Routed here by the hub (runner-up: ../research-wiki, which owns tla-plus and the formal-methods lineage). Source: https://lamport.azurewebsites.net/pubs/byz.pdf, T1, ACM TOPLAS Vol. 4 No. 3, July 1982, pp. 382–401, 20 pages, SRI International. The arriving link carried a Facebook fbclid parameter; stripped.

Pages: byzantine-generals-problem (source summary), leslie-lamport (Person, canonical node). consensus updated with a correction; synthesis gained the same under "The limits the mechanisms were built against"; ../research-wiki's tla-plus cross-links to the new Person node.

The primary source corrects the secondary one. consensus said, from podc-lecture-notes, that "authentication changes the bound." The paper's Theorem 2 says algorithm SM(m) solves the problem for any number of generals and any number of traitors, and Section 5 drops the connectivity requirement to the weakest one that keeps the loyal subgraph connected. The bound is not moved, it is gone. Both statements are now on the page — the summary's and the paper's — per the record-don't- overwrite rule, with the primary marked as the authority.

Read locally with pypdf, not summarised from a secondary account. Extracted the full 20 pages to text first, which is where the two admissions came from that no summary of this paper carries: detecting an absent message (assumption A3) needs synchronized clocks, and clock synchronization under faults is "as difficult a problem as the Byzantine Generals Problem itself"; and A4's unforgeable signature "can never be guaranteed" — only made improbable, and against a malicious operator rather than a random malfunction it becomes a cryptography problem.

The finding that pays rent here. The closing sentence — "The only way to reduce the cost is to make assumptions about the type of failure that may occur" — is what every engine in this spoke does by assuming crash-stop. Raft and Paxos are affordable because a node is assumed to die rather than lie. That is a purchase, and it is the wrong one when a node is corrupted rather than dead. It joins the FLP framing already in consensus: a production store claiming consensus is always asserting something, and the question is what.

Also recorded: Section 6 shows the generals were never a metaphor about malice. The motivating system is majority voting over redundant processors in avionics and missile defence, and the canonical failure is physical — a faulty input unit putting a marginal signal on a wire that some processors read as 0 and others as 1, or two processors reading a clock as it ticks.

Entity: leslie-lamport paged on the recurrence rule (../ENTITIES.md) — named by this paper here and by tla-plus in ../research-wiki. SRI International deferred at one mention.

Does not close the standing gap in consensus: Raft and the engines' own accounts (Spanner TrueTime, CockroachDB, Vitess failover) are still absent. This is more theory, which was already the well-covered side.

[2026-08-10] ingest | The Part-Time Parliament (Lamport, 1998) — Paxos

Routed here by the hub (runner-up: ../research-wiki, same as the byzantine paper an hour earlier). Source: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/12/The-Part-Time-Parliament.pdf, T1, ACM TOCS Vol. 16 No. 2, May 1998, pp. 133–169, 33 pages, Digital Equipment Corporation, minor corrections 2000. The arriving link carried an fbclid parameter; stripped. Read locally with pypdf.

Pages: part-time-parliament. consensus gained a "Why majorities, exactly" section and a confirmation on the add-timing branch; leslie-lamport rewritten to hold both papers; synthesis gained "Lamport, twice, an hour apart".

The pair is the finding, not either paper. byzantine-generals-problem closes by saying the only way to make arbitrary-fault tolerance affordable is to assume something about how components fail. This paper's §4 takes that discount and states both halves: Paxos "does not tolerate arbitrary, malicious failures, nor does it guarantee bounded-time response. However, consistency is maintained despite the (benign) failure of any number of processes and communication paths." Crash-stop is a purchase, and it is why the engines in this spoke carry no 3f+1 replica count.

Answered a question the spoke was using without asking. Why majorities? Condition B2 — any two quorums intersect — plus B3, which forces a new ballot to adopt the latest earlier vote its quorum knows about. The shared member carries a decision forward. Majority is the cheapest quorum system, not the principle.

Confirmed rather than inferred. consensus had Paxos filed under "add timing" from Wattenhofer. The paper says it outright: any protocol meeting the progress condition "must involve measuring the passage of time," footnoted to Fischer, Lynch and Paterson (1985) with the remark that "many centuries were to pass before a rigorous proof of this result was given."

Recorded honestly: the paper declines to specify leader selection — the section ends with the debate's outcome unknown and "no record exists of the presidential selection protocol that was ultimately used." The part production systems find hardest is the part left open.

Paged as one page rather than splitting a paxos Thing page off from the paper; split when a second Paxos source arrives. Entity leslie-lamport updated in place, not duplicated. Digital Equipment Corporation deferred at one mention, as SRI International was.

Routing note. Three theory sources now sit in a spoke defined by operational databases. The trigger the 2026-08-08 note set has been reached; recorded in synthesis as a live question for the hub, not settled here.

[2026-08-10] ingest | The Chubby lock service (Burrows, OSDI 2006)

Routed here by the hub (runner-up: ../platform-ops-wiki, for the operational half). Source: https://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf, T1, OSDI 2006, 16 pages, Google. fbclid stripped from the arriving link. Read locally with pypdf.

Pages: chubby-lock-service. consensus's "what this corpus still lacks" is half closed; synthesis gained "And then the implementation showed up".

Third Lamport-adjacent source in two hours, and the one that changes the block's shape. The two papers this morning were results; this is a system running them, with an outage budget. It is the production witness consensus had been asking for since 2026-08-08.

The master lease is the reusable idea. A master wins a majority plus promises that no rival will be elected for a few seconds, and can then serve reads alone because the promise proves no other master exists. Writes still cost a majority. Cheap linearizable reads bought with a time-bounded promise rather than a quorum — invisible from the algorithm, and the thing this spoke's engines actually need.

Consensus rented rather than linked. Google had a Paxos client library and shipped a central lock service anyway. The structural reason, not the organisational one, is the good one: a consensus library needs a quorum of the application's own servers, while a lock service lets a single client make progress safely. The quorum is relocated into a service that specialises in holding one — the same shape as part-time-parliament's trade, one level up.

The finding to hold against future engines. 61 outages over 700 cell-days, most under 15s; six data losses in a few dozen cell-years — four database software bugs, two operator errors, none from hardware, and the operator errors occurred during upgrades meant to fix the software bugs. Replication handled what it was built for; everything that was lost was lost to code and people.

And the paper corrects its own premise. Designed as a lock service, most popular use is naming (60% of open files), because consistent caching with explicit invalidation beats a DNS TTL when polling is expensive — 3k processes at a 60s TTL would need 150k lookups/s against a ~50k/s DNS server. Locks are 31 ppm of RPC traffic; KeepAlives are 93%.

Tier honesty: peer-reviewed and therefore T1, but a first-party account of one company's system with no comparison and no external replication, and the numbers are a 2006 snapshot.

Routing note. This weakens rather than strengthens the distributed-systems-theory split flagged earlier today: Chubby is an operational system, not a theorem, and it makes the consensus block look like a stratum this spoke needed rather than stray theory. Recorded in synthesis; a fifth theory source with no system attached should force the call.

Entity: Mike Burrows deferred at one mention, consistent with SRI International and Digital Equipment Corporation. google linked cross-wiki, not duplicated.

[2026-08-10] ingest | Paxos Made Simple (Lamport, 2001) — and the paxos page splits out

Routed here by the hub. Source: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/12/paxos-simple-Copy.pdf, T1, dated 01 Nov 2001, 14 pages (ACM SIGACT News 32(4)). fbclid stripped. Read locally with pypdf.

Pages: paxos-made-simple (source) and paxos — the algorithm as its own concept page, which part-time-parliament's ingest this morning said to create "when a second Paxos source arrives." It has. consensus, chubby-lock-service, part-time-parliament and leslie-lamport all re-pointed at it; synthesis gained a section.

Not a duplicate, and the difference is the value. The 1998 paper states B1–B3 and proves them. This one derives them — P1 → P2 → P2a → P2b → P2c, each strengthening forced by a counterexample — and claims the algorithm "follows almost unavoidably from the properties we want it to satisfy." Both are held: the result, and the reasoning that makes it inevitable.

It is also where the field's vocabulary comes from. Proposers / acceptors / learners, and prepare-promise / accept-accepted. Every implementation speaks this paper's language rather than the 1998 paper's.

Two ideas worth carrying past Paxos. When you cannot observe the future, forbid it — the phase-1 promise exists because a proposer cannot know what acceptors will later accept (Chubby's master lease is the same trick with a clock). And, stated flatly here where 1998 only implies it: a failed leader election costs availability, never correctness. The paper spells out the dueling-proposer livelock in full and then says safety holds "regardless of the success or failure of the election."

Routing note — this does not trigger the theory split. It is the same algorithm the spoke already holds, told again; it deepens one page instead of widening the theory footprint. Trigger unchanged: a source opening new distributed-systems territory with no system attached.

Cross-spoke lead, not a route. That Lamport had to republish his own paper in plain English before the field could use it — content unchanged, exposition fixed — is evidence for ../research-wiki's diffusion-and-adoption cluster. Noted on the source page; that spoke owns it if it wants it.

[2026-08-13] ingest | PEV2 — the second tooling source, and it answers the tooling question

Routed here by the hub router from a Telegram link (github.com/dalibo/pev2). T1: the repository plus its source, read with gh. The README is a usage document, so the analysis comes from src/enums.ts and the component list — that is where the tool's opinion about a plan is written down.

Two pages created. pev2 (source summary) and query-execution-plan — the artifact itself, which the spoke had been discussing from two directions without ever paging.

It closes the spoke's first open question, by reframing it. synthesis.md asked whether the tooling member coheres, and set an explicit trigger: if a second tooling source lands, "database tooling" may deserve its own corner. The second source landed and the answer is no corner. PEV2 is not another catalogue — it is one deep tool for Postgres, an engine already paged here, reading the artifact cross-shard-queries was already discussing from the sharded end. It binds two existing pages rather than sitting beside them. The loose member turns out to be awesome-db-tools specifically, because it is a directory spanning three database families. Recorded as reframed, with the condition that would reopen it (another broad catalogue).

The substantive claim. Metric is a six-item enum — time, rows, cost, buffers, estimate_factor, io — and estimate_factor (planner estimate vs actual, with an over/under direction) is not in Postgres's output. EXPLAIN ANALYZE prints both numbers on every node and leaves the arithmetic and the search to the reader; PEV2 computes the ratio, ranks by it and colours the worst. The tool's contribution is triage, not information — which generalises: the engines emit complete, unranked diagnostics, and the tooling layer decides what to look at first. Same move in the derived I/O speeds, which the code marks with a leading * to show Postgres did not report them.

Two details worth keeping. It is licensed under the PostgreSQL licence, not MIT. And it ships as a single offline HTML file — a plan carries query text, schema names and often literals, so the local build is what allows its use on work that cannot leave the building. Dalibo does not make that argument; the page marks the reasoning as mine.

Provenance note on the lineage: PEV2 is a rewrite of Alex Tatiyants' pev (2016), which the README credits and describes as abandoned — "no activity at all for more than 3 years." A consultancy maintaining an orphaned community tool for seven years is a third sustainability model beside vendor product and volunteer project.

Also updated. postgresql gains a "reading what it decided" section, and its closing open-items line is corrected: the plan a query got is now covered, how the planner picks it is not. New growth edge 9 records exactly that hole — statistics, histograms, the independence assumption, extended statistics, autovacuum staleness — the mechanism under the number both new pages treat as the main diagnostic.

Entities: Dalibo named as the maintainer but not paged — ../ENTITIES.md wants evidence beyond a name, and the repo gives no organisational detail. Alex Tatiyants likewise credited inline only.

Not measured, and said so: nobody here shows that plan visualisation changes outcomes. The page says it rather than assuming the diagram helps because it is pleasant.

[2026-08-20] ingest | Citus — distributed PostgreSQL as an extension

Source: https://github.com/citusdata/citus (T1, first-party README + GitHub releases/repo API). Arrived via Telegram, routed here by the hub.

Why it belongs here and not next door. Citus has columnar storage and sells itself for real-time analytics, so the ../analytical-databases-wiki boundary had to be decided rather than assumed. It routes here on centre of gravity: it is a transactional Postgres scaled out, and its columnar tables reject UPDATE, DELETE and foreign keys and want batch loads only. That is an archival tier under time partitioning, not an OLAP engine — the README's own 651 MB → 25 MB example is compression of old partitions. Logged the analytical spoke as runner-up and cross-linked analytical-databases, clickhouse, duckdb rather than duplicating anything there.

The finding. Everything this spoke knew about distributed OLTP came from the proxy design — planetscale-768-servers, vitess, a router in front of the engine holding topology metadata. Citus does the same job as an extension inside the engine, so shards are ordinary Postgres tables and the whole feature surface (PostGIS, JSONB, lateral joins, time partitioning) survives distribution with nothing reimplemented. The cost is being welded to one engine and one version cadence — visible in three release branches maintained simultaneously — plus AGPL-3.0. Written up in citus, folded into database-sharding and the operational-databases umbrella, and made the new narrative section in synthesis.md.

The negative finding, which is worth as much. Citus ships no high availability of its own. It defers to Patroni 3.0, which runs one Postgres replication group per shard group. So failover is per-group streaming replication and there is no Raft or Paxos in the system's own account of itself — after two ingests (podc-lecture-notes, paxos) establishing what consensus costs, the first production distributed store here routes around the question by decomposing into Postgres pairs. The README also says nothing about cross-shard commit. Recorded as still open, not assumed.

Two open questions moved. The HTAP seam with the sibling spoke, flagged since founding with no source, now has one — reframed rather than closed, because the mixed-workload claim is a design statement in a vendor README with no benchmark, and the SIGMOD '21 paper (doi 10.1145/3448016.3457551) that would substantiate it is cited and unread. The distributed-consistency question named Citus explicitly as a wanted source; it now stands more precisely — what is missing is an engine's own account of its commit protocol and failover path, not another product page. Growth edges 1 and 3 sharpened accordingly.

New pages. citus (source summary), shard-colocation (mechanism — same distribution column, same node, therefore local joins/FKs/transactions; the design lever whose absence is scatter), and citus-data (Organization; the Postgres-extension house — pg_cron, HLL, TopN, pg_auto_failover — and the in-engine counterpart to planetscale's proxy). Updated: database-sharding gains the in-engine router and the row-based-vs-schema-based split; cross-shard-queries gains the same push-down rule from a second independent engine; postgresql gains the scale-out-without-leaving path; operational-databases splits its distributed end in two.

Entities: citus-data paged (evidence: copyright line, Microsoft CoC, the sibling-extension list, the Azure managed product). microsoft reused cross-wiki from ../agentic-tooling-wiki — not duplicated. Marco Slot named in the README's CMU talk credit only, and not paged: a talk credit is the thin evidence ../ENTITIES.md tells us to skip.

Source-quality caveat recorded on the page and in synthesis. The README is stale against its own releases — banner and both install snippets say Citus 13.0 / Postgres 17, while v14.2.0 shipped 2026-08-06 and 14.0 brought Postgres 18.1 support. Version facts here come from the releases API; the architecture prose is unaffected. Worth remembering as a habit: a first-party README is T1 about design and unreliable about versions.

Not verified here: every performance and customer claim (Algolia, MixRank's petabyte, Salesloft, ConvertFlow, HSL), and the adaptive-executor mixed-workload claim itself.

[2026-08-20] ingest | awesome-postgres (dhamaniasad.github.io/awesome-postgres) — catalog mined for four primaries

Routed by the hub from a curator-supplied collection page: "ingest items from this collection page." So the catalog was ingested as a mining index rather than as content — awesome-postgres is paged as the source (T3, 12,052★, CC0, created 2015-08-02, last pushed 2026-05-21), and four of its entries were pulled into pages of their own, each written from the project's own repository or documentation rather than from the catalog's one-line description. That follows ../HUB.md's rule for aggregator-derived records: the list entry is a lead, the primary source is the finding.

Which four, and why those. Each closes something the spoke had already named as missing, not whichever entries were most popular:

patroni (T1) — the dangling thread from that morning's citus ingest, which recorded that a production distributed Postgres ships no HA of its own and defers to Patroni.
pgbackrest (T1) — growth edge 5's successor, "the write path," which postgresql and mysql both stop short of.
pgbouncer (T1) — a claim postgresql had been asserting with no source behind it, that process-per-connection makes pooling mandatory.
pgmq (T1) — the Work Queues section, and the spoke's first "Postgres instead of a dedicated system" subject.

Two concept pages came out of the batch rather than out of any single entry. write-ahead-logging was written from the Postgres documentation because both Patroni and pgBackRest are unreadable without it — it closes the WAL half of growth edge 5 and half of edge 6. postgresql-extensions was written from the catalog's own section sizes plus PGXN: five alternative servers against many hundreds of things that run inside a stock one.

The finding worth keeping. Following the Citus thread down answered the standing distributed-consistency question from an unexpected direction — nobody in this stack implements consensus. Patroni holds a TTL'd leader lock in an external DCS and reduces failover policy to three integers bound by loop_wait + 2 * retry_timeout <= ttl; the quorum and the elections live one layer below, in etcd/Consul/ZooKeeper. Raft is now located (Patroni's own repo topics name it) and still unread, which is a sharper version of growth edge 4 than it had before. Patroni also supplies the fencing step no theory source covers: a hardware watchdog armed to expire five seconds before the TTL, so a demoted primary that will not stop gets rebooted by the kernel. And chubby-lock-service turns out to be the paper this all instantiates — lock service over Paxos library, eighteen years on.

Second finding, ecosystem-level. The catalog's shape is an argument: Postgres grows by extension and sidecar, not by fork. Written up in postgresql-extensions with both costs the sources show without arguing — version coupling (Citus maintains three release branches for it) and managed-platform allowlists (pgmq ships a SQL-only install path for exactly this reason).

Third, and it reframes the spoke's own domain line. Read together the four tools describe one pattern — the engine ships the mechanism and declines the policy. Replication with no opinion about who leads, a WAL with no archiver, a process per connection with no multiplexer, SKIP LOCKED with no queue. Folded into the thesis and into operational-databases.

Honest caveats recorded on the pages. Not one performance claim in this batch is verified — pgmq has no throughput number anywhere, pgBackRest's compression-bottleneck claim is its own, PgBouncer's feature page carries no benchmark. Patroni's HA-loop diagram page returned 404, so the leader-race loop is assembled from the configuration reference and the watchdog page and the gap is stated rather than inferred over. pgmq's "exactly once within a visibility timeout" is recorded as at-least-once with a clock, which is what the qualifier and the read_ct column both say.

Entities: none newly paged. zalando was reused cross-wiki from ../platform-ops-wiki — and the governance fact recorded instead, that zalando/patroni now redirects to patroni/patroni. David Steele (pgBackRest), Marko Kreen (PgBouncer) and Asad Dhamani (the catalog) are named on their pages from copyright and repository evidence but not paged as Person nodes: one project each, which is the thin-evidence case ../ENTITIES.md says to skip until a name recurs.

Left unmined, deliberately: the catalog's Monitoring (20 entries) and Optimization (14) sections, the latter aimed straight at growth edge 9 — logged as new coverage edge 11 along with the fact that awesome-db-tools has never been mined this way at all. PaaS (18) and Kubernetes (9) recorded as cross-spoke context on the source page for ../cloud-wiki and ../platform-ops-wiki.

[2026-08-20] ingest | awesome-postgres, completed — every catalog item accounted for

The curator's follow-up was "what about other items?" and then a standing goal: until all items ingested. The catalog holds several hundred entries, so "ingest all of them" was read as every entry accounted for in the wiki with honest provenance — not a page per line. Writing ~400 pages from one-line catalog descriptions would have broken the spoke's own rules (T1 bar, dedup, every claim traced to a primary) and produced a link dump. What was done instead has two layers, and each page states which layer it belongs to.

Layer 1 — eleven subjects read at their primary source (T1). Four in the first pass (patroni, pgbackrest, pgbouncer, pgmq); seven more here:

yugabytedb — the big one. Growth edge 1, open since founding, asked for a distributed store's own account of its commit protocol and failover path. This is it: tablets as Raft groups, RF >= 2f + 1, provisional records "invisible to readers until the transaction commits", a transaction status table that is "just another sharded table in the system", commit timestamp taken at the Raft-log append, coordinator failure resolved by provisional-record expiry rather than blocking. Edge 1 closed.
postgresql-planner-statistics — growth edge 9 closed, from the docs it asked for. The independence assumption with the documentation's own worked example: two identical columns, a = 1 AND b = 1 estimated at 1 row, actual 100, "two orders of magnitude" out. That is the mechanism under pev2's estimate_factor, and it explains the skew toward underestimation.
hypopg, pg-stat-statements, pgbench, ferretdb, logical-replication — the query-instrument set completed, the Monitoring section's substrate, the measurement standard, the document neighbour (edge 8 narrowed, not closed), and the other half of replication.

Layer 2 — fourteen coverage pages, one per functional category, enumerating every remaining entry at the catalog's word and marked T3, unverified on the page: postgres-ha-tooling, postgres-backup-tooling, postgres-monitoring-tooling, postgres-query-optimization-tooling, postgres-gui-clients, postgres-cli-tooling, postgres-schema-migration, postgres-work-queues, postgres-data-movement, postgres-client-libraries, postgres-managed-services, postgres-kubernetes-operators, postgres-forks-and-derivatives, postgres-learning-resources. The Extensions roster went onto postgresql-extensions. Each coverage page ends by naming its highest-value unread entry, and those are collected as growth edge 15.

postgres-schema-migration is not a catalog section — it is ~25 entries the catalog files across CLI, Utilities and GUI by interface rather than by job. Assembling it was the single most useful structural move of the pass: it is the largest unpaged functional category in the corpus, and logical-replication ends on exactly the problem it solves (DDL is not replicated).

Findings beyond the individual pages.

Three positions on distributing SQL, not two. Extension (citus, inherits Postgres replication, therefore no consensus of its own), proxy (vitess, engine-agnostic, reimplements), and reimplementation below the SQL layer (yugabytedb, per-shard consensus, no longer Postgres beneath the parser). Only the third gets a commit protocol, and its licence file says why: src/postgres/ is kept as a "Modified PostgreSQL fork for YSQL compatibility" while src/yb/ replaces storage, consensus and transactions.

The extension thesis survives its own counterexample section. Of the catalog's five alternative servers, ferretdb is not a fork at all (Postgres + the DocumentDB extension + a protocol proxy). An extension can add a table access method, a router, a queue, even a storage engine — it cannot swap WAL-and-streaming replication for a Raft quorum. That is the boundary, stated precisely for the first time.

Raft: two production users, zero explanations. patroni rents it, yugabytedb runs it. No source here describes the algorithm. Deliberately did not create a raft page from two second-hand product descriptions — that is the aggregator-inherited-error failure ../HUB.md warns about. Growth edge 4 stands and is now conspicuous.

The corpus's evidence standard, written down. pgbench's documentation states what a credible measurement needs — multi-minute runs, scale above client count, tracked vacuum, an unsaturated load generator — and no performance claim anywhere in this spoke meets any of those conditions. Recorded as a standing caution rather than as a criticism of any one vendor.

New growth edges 12–15: hybrid logical clocks (which yugabytedb's protocol rests on and nobody here defines), autovacuum (three separate pages now end on it), a cross-shard benchmark, and the ranked next-reads from the coverage pages.

Honest limits of this pass. The fourteen coverage pages are T3 and unverified by construction — they record what the catalog says, and the catalog is undated, carries one staleness marker in its entire length, and was last pushed 2026-05-21. postgres-gui-clients says so explicitly: several commercial entries there are likely dead products, and checking forty of them was out of scope. Nobody should adopt a tool off a coverage page without reading its primary. That limitation is stated on each page rather than hidden in this log.

Entities: none newly paged this pass. zalando reused again for the operator lineage.

[2026-08-22] ingest | In Search of an Understandable Consensus Algorithm (Raft) — via research pass

raft-paper (Ongaro & Ousterhout, Stanford; extended tech report, 20 May 2014, T1), read in full. Closes growth edge 4, the spoke's most conspicuous gap: yugabytedb runs Raft per tablet, patroni rents consensus from a DCS that runs it, and nothing here explained the algorithm.

Also written: diego-ongaro and john-ousterhout as entity nodes; stanford-university reused from ../agentic-tooling-wiki rather than duplicated.

What the paper argues that no other design here does. The target was understandability, pursued by restriction — only the leader appends, randomized timers settle elections, joint consensus covers membership change. Raft is deliberately less general than paxos, and that is the pitch.

The timing rule was already in this spoke, as a config file. broadcastTime ≪ electionTimeout ≪ MTBF, with safety independent of all of it, is what patroni's loop_wait + 2 * retry_timeout <= ttl encodes. Noted on both pages.

Numbers, all the authors' own: five servers, ~15 ms broadcast time, leader crashed 1,000 times per configuration. No randomness → elections over 10 s from split votes. 5 ms of randomness → 287 ms median downtime. 12–24 ms timeouts → 35 ms average. Understandability: 43 students, 33 scored higher on Raft, means 25.7 vs 20.8 of 60, with the authors' own lecture videos and grading rubric.

Successor written as edge 16: no database in this corpus publishes what Raft costs it. Two of them run it.

[2026-08-30] ingest | Checkpoints, torn pages and the durability dials

Via the hub research pass, taking the residue of hub quick win 3 ("the write path"). write-ahead-logging had ended, since 2026-08-20, with a list of four things it said were undefined in this corpus: checkpoints, full-page writes and torn-page protection, synchronous_commit, and WAL write amplification. All four are now written, from the engine's own documentation: postgres-wal-configuration (ch. 28.5) and postgres-wal-reliability (ch. 28.1), both T1, plus a checkpoint mechanism page. The stale sentence on write-ahead-logging was struck in place with the date rather than deleted, per QUALITY.md dimension 5.

The finding is a coupling nobody would guess from the WAL page alone. Write amplification and recovery time are the same dial. Checkpoint more often and crash replay gets shorter — and the log gets bigger, because the first modification of a page after each checkpoint logs the whole 8 kB page, not the change. So the two obvious goals, quick recovery and a small log, pull against each other, and full-page writes are the linkage.

The reason for the full images is hardware, not software: Postgres writes 8192 bytes as 16 sectors of 512, and a power failure can land between any two of them, leaving a page that is neither the old one nor the new one. A redo record describing a change cannot repair that, because it assumes a known starting state. Where the filesystem rules torn writes out — the docs name ZFS — full_page_writes can be switched off and the amplification goes with it.

One distinction worth having on record, because it is easy to blur: fsync=off risks a corrupt database, synchronous_commit=off risks losing transactions already reported as committed. The docs recommend the second for noncritical work precisely because it buys much of the speed of the first without the corruption risk. Since it is settable per transaction, durability is a property of the transaction rather than of the server.

Edge 6's durability half closes; the B-tree-versus-LSM comparison it also asks for stays open.

[2026-09-19] ingest | The architecture of Neki — the router is not engine-agnostic

Routed from the hub: https://planetscale.com/blog/the-architecture-of-neki (Harshit Gangal, PlanetScale engineering, 2026-09-18). Arrived by Telegram. It closes a hole this wiki had written down: vitess said Neki was "PlanetScale's Postgres equivalent, which this wiki has only as a name" — and database-sharding, planetscale, operational-databases and planetscale-768-servers all named it without describing it. Six mentions, no page. Now there is one.

New pages: neki (the seven components), the-architecture-of-neki (T1), harshit-gangal. Refreshed: vitess, planetscale (second source — one sharding architecture, two engines), database-sharding.

T1 on a vendor blog, and the justification matters. ../QUALITY.md's rule tiers a first-party page by what its load-bearing claims rest on. This is architecture — what each component is, where it runs, what it talks to — which is the artifact's own fact and checkable against the artifact. There is no benchmark and no performance claim anywhere in the post, so there is nothing in it that T3 exists to discount.

The finding: which engine you shard changes the architecture. Postgres tracks objects by numeric OID and clients read a column's type OID off the wire and cache it. Independent shards assign a custom type different OIDs, so Neki designates one shard cluster-wide as authoritative, translating every other shard's OIDs on the way out. MySQL has no equivalent object identity to reconcile. This wiki has treated sharding as routing-and-merging for two months; this is the first case where the engine's identifier scheme puts a mandatory single point into a design built to avoid them. Two more of the same kind: session-scoped advisory locks force a third connection-pool tier (and the Router, not the Sidecar, decides which tier a query gets — pgbouncer makes that a deployment-wide setting), and logical-replication decoding plus pg_rewind are Postgres facilities doing the data-movement and failover work.

And the disclaimer it undercuts. The post opens by denying Neki is "a MySQL sharding idea wearing a Postgres label." Router↔VTGate, Sidecar↔VTTablet, and the workflow names MoveTables / Reshard / OnlineDDL are Vitess's, unchanged; the author is a top-five Vitess contributor. The fair reading is neither the denial nor its opposite: topology from Vitess, mechanisms from Postgres. Recorded on neki, vitess and in synthesis rather than taken at face value.

One design divergence worth watching. vitess's page credits it with not becoming a distributed query engine. Neki's Router executes cross-shard joins itself, choosing between nested-loop, hash and merge on cost estimates. That is a planner.

What the source does not carry, and both new pages say so: no latency, no throughput, no shard count, no comparison — no number at all — and nothing about atomicity across shards. The durability policies are per shard and describe replication, not distributed commit. Growth edges 17 (atomicity across shards, which neither Vitess nor Neki has a source for here) and 18 (does Neki work) added.

Status kept on every page: Platform Preview, not GA.

[2026-09-23] ingest | MatrixOne (github.com/matrixorigin/matrixone + arXiv 2604.03927)

Routed from the hub (Telegram). matrixone (T1 for architecture and design; benchmarks maker-run and flagged). Read the README and the paper PDF (pypdf). HTAP placement decided here by centre of gravity, recorded in synthesis beside citus as the opposite design. The paper's claim that no other database offers diff/merge between forks is contradicted by Dolt (2019) and by seekdb in this corpus; recorded on the page, not silently dropped. Cross-link to analytical-databases-wiki noted.

[2026-09-28] research | a CRDT-side source (edge 2) — half closed, via research pass

Added eg-walker (Gentle & Kleppmann, EuroSys 2025, T1), read in full. The best text design it measures keeps a persisted event graph and builds a CRDT only while merging. That complicates the event-sourcing versus CRDT framing from rethinking-data-client-event-sourcing rather than settling it. Text only, and authors' own numbers. Successor edge 2a: the same comparison on structured, schema-changing app data.