Spokes.wiki Search About

Synthesis — Operational Databases

The evolving thesis. Spun out of the hub _inbox databases-beyond-olap cluster on 2026-07-16, once a third source cohered: an embedded OLTP engine (turso, parked 2026-06-23), a general DB-tooling catalog (awesome-db-tools, parked 2026-06-24), and a distributed-SQL sharding explainer (planetscale-768-servers, the trigger). Both prior park notes had predicted that a third general-DB source would trip this spin-out; the human chose a new spoke over broadening the OLAP-only ../analytical-databases-wiki, keeping that spoke's columnar identity sharp.

Current thesis

This spoke owns OLTP / operational databases — the row-oriented, transactional stores in an application's request path — as the deliberate sibling to the OLAP columnar stores in ../analytical-databases-wiki. That spoke explicitly carves out "transactional/OLTP databases"; this is the spoke that fills the carve-out. The boundary is engineering-real: the columnar scan that makes OLAP fast is the wrong layout for the single-row update OLTP does all day.

The founding trio traces one axis — how far a single logical database stretches — plus the tooling that keeps it running:

  1. Database-as-a-library — turso collapses the OLTP engine into the process: an in-process, SQLite-compatible store (a Rust rewrite of SQLite) that adds MVCC concurrent writes, async I/O, CDC, and vector/FTS. The embedded pole.
  2. Database-as-a-fleet — planetscale-768-servers explodes the OLTP engine across machines: sharding 768 servers behind a Vitess/Neki proxy router so they present as one database. The distributed pole.
  3. The tooling around them — awesome-db-tools catalogs ~200+ tools for working with databases (clients, migration, backup, monitoring, HA) — the operational layer between "it runs" and "it runs in production." Sharpened 2026-08-20: that layer has a governing principle, not just a population. The engine ships the mechanism and declines the policy — replication without an opinion about who leads (patroni), a WAL without an archiver (pgbackrest), a process per connection without a multiplexer (pgbouncer), SKIP LOCKED without a queue (pgmq). Each tool is a declined decision, made by somebody else.

So the two engines are the same job (serve transactional SQL) at opposite scales, and the axis between them — embedded ↔ distributed — is the spoke's founding spine. It deliberately mirrors the sibling analytical spoke's own embedded↔distributed axis (duckdb ↔ clickhouse), one line down in the OLTP register.

The shape of the field (early)

The axis has a third pole: the client

The founding spine was embedded ↔ distributed, bracketed by turso (database-as-a-library) and planetscale-768-servers (database-as-a-fleet). rethinking-data-client-event-sourcing extends the embedded end past the server: in a local-first app the OLTP store runs in the user's process, serves every read and write locally, and the server becomes a sync log. So the axis now runs client → process → fleet, and the same question ("how far does one logical database stretch?") gets a third answer.

Two consequences for how this spoke reads its own subject:

The interesting component stops being the database. Local-first stacks are built on plain SQLite; the engineering has moved to the sync engine, and the field has three competing answers about what should travel over the wire — an event log (livestore), a mutable auto-merging structure (CRDTs: Automerge, Yjs, Loro), or query-scoped rows (ElectricSQL, PowerSync, Zero). This is the corpus's first real material on replication and merge semantics, which the open questions below had flagged as the top gap — arriving from the client end rather than the NewSQL end that was expected.

Consistency becomes an explicit product trade, not a default. planetscale-768-servers shards to preserve one consistent logical database; local-first gives up global consistency on purpose to buy responsiveness and offline capability. Schickling draws the line himself: financial systems and social graphs are the wrong fit. Both poles are OLTP, and they disagree about what OLTP is for — the first genuine tension in the corpus, and a useful one.

Adjacent claim worth keeping: data ownership falls out of the architecture, since the user can hold the event log or the materialized SQLite file. The spoke's other engines treat export as a feature.

The limits the mechanisms were built against

Everything this spoke held past the embedded pole described mechanism — hash routing, proxy aggregation, scatter-gather, failover — and nothing described the results that say which mechanisms are possible. podc-lecture-notes (Wattenhofer, ETH Zürich, T1) supplies them, and closes the spoke's #1 growth edge from the theory side rather than the engine side the edge asked for.

Three results reframe what the corpus already had.

Consensus is impossible in the general case, and every system is a way of dodging that. FLP, stated exactly: "There is no deterministic algorithm which always achieves consensus in the asynchronous model, with f > 0." One crash-faulty node and no timing assumptions is enough. So a store that commits across shards is not solving consensus — it is buying an assumption, either partial synchrony (which is what a timeout is) or randomisation. When a distributed database claims consensus, the question the corpus can now ask is which premise did you weaken, and what happens when it does not hold (consensus).

Arbitrary failure has a price in nodes. Byzantine agreement needs f < n/3 — the origin of the 3f+1 replica counts — and authentication moves the bound, which is why signatures get their own chapter rather than being an implementation detail.

The primary source landed 2026-08-10 (byzantine-generals-problem, Lamport, Shostak and Pease, ACM TOPLAS 1982), and it corrects the summary in one place and deepens it in two. Authentication does not move the bound, it removes it: with unforgeable signatures the paper's SM(m) works for any number of nodes and any number of faults, at the weakest connectivity that keeps the correct nodes connected. The 3f+1 count buys exactly one thing — protection against a node telling different lies to different peers — and a signature buys it outright. Both claims are the paper's own theorems.

The paper is also honest about what its assumptions cost, in a way secondary accounts drop. Detecting an absent message needs synchronized clocks, and synchronizing clocks under faults is "as difficult a problem as the Byzantine Generals Problem itself" — the assumption is the problem again. And an unforgeable signature is only probabilistically unforgeable; against a malicious operator rather than a random malfunction it becomes a cryptography problem, cited to Diffie–Hellman. Byzantine fault tolerance and adversarial security separate at that line, which is the distinction most modern uses of the word "byzantine" blur.

And the closing sentence generalises past its own subject: "The only way to reduce the cost is to make assumptions about the type of failure that may occur." Every engine in this spoke takes that discount by assuming crash-stop. Raft and Paxos are affordable because a node is assumed to die rather than lie. That is a purchase, and it is the wrong one when a node is corrupted rather than dead.

CAP is a theorem with a model, not a slogan. Introduced by Fox and Brewer (1999), proven by Gilbert and Lynch (2002) for the asynchronous model, and relaxable under partial synchrony to keep availability and partition tolerance. That kills the pick-two framing and replaces it with the real engineering move: weaken consistency by a named amount (consistency-models). Causal consistency is the name the spoke was missing when rethinking-data-client-event-sourcing arrived arguing merge semantics from an event-sourcing maintainer's position with no neutral treatment behind it.

Lamport, twice, an hour apart

part-time-parliament arrived directly after byzantine-generals-problem, and the pair is worth more than either alone because they are the two sides of one trade, made by the same author sixteen years apart.

1982 prices the general case: tolerating components that fail arbitrarily costs 3f+1 nodes, message paths of length f+1, and factorial message counts — and its closing sentence says the only way to reduce that is to assume something about how things fail. 1998 takes that discount explicitly. Paxos "does not tolerate arbitrary, malicious failures, nor does it guarantee bounded-time response. However, consistency is maintained despite the (benign) failure of any number of processes and communication paths."

That is the whole reason production databases can afford consensus at all. Crash-stop is not a simplification, it is the purchase, and it is why Raft-backed and Paxos-backed engines carry no 3f+1 replica count — they are defending against absence, not against lies. The corresponding exposure is now nameable: a node that is corrupted rather than dead is outside the model every store in this spoke runs on.

Paxos also answers a question the spoke had been using without asking. Why majorities? Because quorum intersection is the mechanism (condition B2), and any two majorities of a set share a member; that member is what carries a decided value into every later ballot (B3). Majority is the cheapest quorum system, not the principle.

And the FLP framing above is confirmed from the primary source rather than inferred: the paper states that any protocol meeting its progress condition "must involve measuring the passage of time," footnoted to Fischer, Lynch and Paterson. Safety needs no clock; only termination does. The paper is candid that leader selection — the part production systems find hardest — is the part it declines to specify, ending that section with the note that no record of the presidential selection protocol survives.

And then the implementation showed up

chubby-lock-service (Burrows, OSDI 2006) arrived third, an hour after the two Lamport papers, and it changes the shape of the theory block rather than adding to it. This is a system, not a result — Paxos running at Google with an outage budget attached — and it is the production witness consensus had been asking for since 2026-08-08.

Three things it contributes that no theory paper can.

The master lease, which is how consensus becomes affordable to read from. A master wins a majority plus promises that nobody will elect a rival for a few seconds, and can then serve reads alone, because the promise proves no other master exists. Writes still cost a majority. That asymmetry — cheap reads bought with a time-bounded promise rather than a quorum — is the single most reusable idea in the paper, and it is invisible from the algorithm.

Consensus as a rented service rather than a library. Google had the Paxos library and shipped a central lock service anyway. The stated reason is organisational — applications acquire high availability after they have users, and a lock is a two-statement change where a replicated state machine is a rewrite — but the load-bearing reason is structural: a consensus library needs a quorum of the application's own servers to make progress, while a lock service lets a single client proceed safely. The quorum is relocated, not removed. Read beside part-time-parliament, that is the same trade one level up: 1998 makes agreement affordable by assuming benign failure; 2006 makes it affordable by making it somebody else's service.

What actually breaks. 61 outages over 700 cell-days, most under 15s. Six data losses in a few dozen cell-years: four database software bugs, two operator errors, none from hardware — and the operator errors happened during upgrades intended to fix the software bugs. In a system built for reliability, redundancy handled the failure mode it was designed for and every loss came from the code or the people. This spoke should hold that against any future engine that sells replication as a data-safety story.

One correction to its own premise, worth keeping. Chubby was designed as a lock service and its most popular use is naming — 60% of open files — because consistent caching with explicit invalidation beats DNS's time-based TTL when polling is expensive. Locks are 31 ppm of its RPC traffic; KeepAlives are 93%. A system's dominant use was not the one it was built for, and the paper says so plainly.

This also weakens the split case below. Chubby is an operational system, not theory — it belongs to a spoke about running databases more comfortably than to a hypothetical theory spoke.

The same algorithm, told twice, and the second telling is what worked

paxos-made-simple arrived fourth. It is not new territory — it is part-time-parliament again, with the island removed — and that is what makes it useful.

It derives rather than asserts. The 1998 paper states conditions B1–B3 and proves them; the 2001 paper builds them, P1 → P2 → P2a → P2b → P2c, each strengthening forced by a counterexample against the last. Reading them in order shows a result and then the reasoning that makes it inevitable, and the spoke now has both.

It is the source of the working vocabulary. Proposers, acceptors, learners; prepare/promise and accept/accepted. Every implementation in the field speaks this paper's language, not the 1998 one's. The algorithm now has its own page, paxos, because three sources cover it from three angles.

Two ideas worth keeping loose from Paxos. First: when you cannot observe the future, forbid it — a proposer cannot know what acceptors will accept, so phase 1 extracts a promise that they will not. Chubby's master lease is the same trick with a clock on it. Second, stated flatly where the 1998 paper only implies it: a failed or contested leader election costs availability and never correctness. That is the licence every leader-based system in this spoke operates under, and it is why a brief two-leader window is survivable.

And a fact about how ideas travel, not about consensus. Lamport published a rewrite of his own paper with the content unchanged and only the exposition fixed, and the rewrite is the one the field learned from. The idea was complete in 1998 and did not spread until 2001. That belongs to ../research-wiki's diffusion-and-adoption cluster more than here; noted, not argued.

The routing caveat got a second data point. The byzantine paper routed here on 2026-08-10 over ../research-wiki (which owns Lamport's tla-plus and the formal-methods lineage) because this spoke holds consensus. Three theory sources now — Wattenhofer's notes, the 1982 byzantine paper and the 1998 Paxos paper — all in a spoke defined by operational databases. The trigger the earlier note set has been reached and is recorded as a live question for the hub rather than settled here.

The fourth source argued against the split. chubby-lock-service is a production system with an outage budget, not a theorem, and it makes the block look less like stray theory and more like a consensus stratum this spoke was always going to need: the mechanism, its limits, and one system that runs it. The case for a distributed-systems-theory spoke is weaker at four sources than it was at three. Left open, and paxos-made-simple does not move it: a second telling of an algorithm the spoke already holds deepens one page rather than widening the footprint. The trigger stays what it was — a source opening new distributed-systems territory with no system attached.

A routing caveat worth keeping visible. Fifteen of the book's twenty-two chapters — graph colouring, distributed sorting, wireless protocols, self-stabilisation, game theory — are outside this spoke and are recorded rather than paged. The source's centre of gravity sits outside the spoke that holds it. If more distributed-systems theory arrives, a distributed-systems-theory spoke is the natural home and this routing should be revisited rather than defended.

The base case, written down at last

The spoke spent a month describing departures from a thing it had never described. postgresql and mysql are now here from their own documentation, and two details do work immediately.

Postgres forks a process per connection. Not a thread — an OS process, with the supervisor staying up to accept the next one. That single fact is behind connection pooling being mandatory rather than an optimisation, and it marks how far turso moves at the embedded end (no server process exists) and vitess at the distributed end (the process you connect to is a proxy that speaks the MySQL wire protocol).

InnoDB's clustered index makes the shard key and the primary key one decision. Table data lives inside the primary key index, and secondary index entries carry the primary key rather than a row pointer, so a non-primary-key lookup is two traversals. cross-shard-queries describes the same choice one level up, where the wrong key turns a query into a scatter. The cost structure that decides a query's fate on one node is the cost structure that decides whether a fleet can route it — vitess's vindexes are that decision made twice.

And both engines run MVCC, reached independently, with different bills: Postgres writes new heap versions and pays in bloat and vacuum, InnoDB keeps an undo log and pays under long transactions. The shared guarantee — reading never blocks writing, writing never blocks reading — is the single-node version of the question consistency-models asks of distributed systems. On one node a snapshot is exact and cheap. The moment there is more than one node, "as it was some time ago" needs a clock nobody gets for free, and you are back in consensus.

The router moved inside the engine

citus arrived 2026-08-20 and it rearranges the distributed end rather than adding to it. Everything this wiki knew about scaling a transactional database across machines came from planetscale-768-servers and vitess-query-serving-docs, and both describe the same shape: a proxy in front of the engine, holding topology metadata, parsing SQL, routing by a vindex, merging what it could not route. Citus does the identical job from the opposite side of the process boundary — it is a Postgres extension, the coordinator is a Postgres server, and every shard is an ordinary Postgres table on an ordinary Postgres worker.

The consequence is a capability claim rather than a performance one. A proxy can only route what it can parse and only merge what it can compute, so every engine feature it wants to support across shards has to be reimplemented in the proxy. An extension inherits the whole engine: PostGIS, JSONB, lateral joins, HyperLogLog and time partitioning keep working on a distributed table because the thing executing the fragment is Postgres. The bill comes due elsewhere — Citus is welded to one engine and one major-version cadence, which is why it maintains three release branches simultaneously, and it is AGPL-3.0 where the proxy vendors are not.

What both agree on is more useful than where they differ. cross-shard-queries now holds the same rule from two independent systems: push as much as possible into the underlying engine, fan out and merge only what you must, and make the routable path cover your real queries. Vitess's lever for that is the vindex; Citus's is shard-colocation — same distribution column, same node, therefore local joins, foreign keys and transactions. Two vocabularies, one instruction to the application developer. And neither source prices the fallback: two engines now describe scatter-gather and not one of them measures it.

Then the part the vendor does not advertise. Citus does not implement its own high availability. It defers to Patroni 3.0, which models the cluster as one replication group per shard group, each with a Postgres leader and sync standby. So the failover story is per-group streaming replication, not a cluster-wide consensus protocol — there is no Raft or Paxos anywhere in this system's own account of itself. The spoke spent two ingests establishing what consensus costs (podc-lecture-notes, paxos) and the first production distributed store it pages turns out to route around the question by decomposing into independent Postgres pairs. That is a real design position, and it is also the reason the README says nothing about cross-shard transaction commit — Citus distributes the data plane and borrows the control plane.

Where the consensus actually was, and the shape of everything around the engine

(added 2026-08-20, second ingest of the day — the awesome-postgres catalog, mined for four primary sources)

The citus ingest that morning ended on a negative: a production distributed Postgres that ships no failover of its own and hands the job to Patroni. Following that thread is what this batch did, and it resolves a question the spoke has carried since podc-lecture-notes — not by finding the consensus algorithm inside a database, but by finding out that it is nowhere near one.

patroni implements no consensus either. It holds a leader lock with a TTL in an external distributed configuration store — etcd, Consul, ZooKeeper, or the Kubernetes API — and derives its entire failover policy from three integers and the inequality binding them: loop_wait + 2 * retry_timeout <= ttl. The primary renews the key; if it cannot, the key expires and someone else takes it. So the stack is: Postgres has no opinion about who is primary. Patroni decides, but only by asking. The quorum, the election and the replicated log all live one layer further down, in a system nobody in this corpus has yet read. Raft has finally been located — Patroni's own repository topics list it beside etcd, consul and zookeeper — and it is still unread, which is now a sharper statement of growth edge 4 than "no engine's own account exists."

That relocation makes chubby-lock-service retroactively the most load-bearing source here. Its argument was that a lock service beats a Paxos library because a library needs a quorum of your servers while a lock service lets one client proceed. Patroni is that argument instantiated for Postgres eighteen years later: one primary, not a quorum member, made safe by a lease held in somebody else's quorum.

And Patroni adds the thing the theory does not cover. Consensus can decide who should lead; it cannot make a demoted primary stop accepting commits. Patroni's answer is a hardware watchdog armed to expire five seconds before the TTL — if the process cannot shut Postgres down cleanly, the kernel reboots the machine. Fencing is where distributed-systems theory hands off to a device file, and no source here had said so before.

The second finding is about the ecosystem's shape and it is a ratio. awesome-postgres holds several hundred entries, and five of them are an alternative server. Everything else runs inside a stock Postgres or beside it. postgresql-extensions writes that up: a distributed database (citus), a time-series database, a spatial database, several search engines, a GPU executor, a message queue (pgmq) and an entire storage engine are all shipped as extensions rather than forks. Citus argued from one vendor's side that the extension seam was wide enough; the catalog is the population-scale version of the same claim. The cost is visible in the same sources — a C extension is compiled against a server version (citus maintains three release branches for it), and loading code into a server is a privilege managed platforms withhold, which is why pgmq ships a second SQL-only install path for "someplace that does not natively support" the extension.

The third is the one the tooling half of the domain was always promising. Read together, patroni, pgbackrest, pgbouncer and pgmq describe one pattern: the engine ships the mechanism and declines the policy. Postgres has streaming replication and no opinion about who should be primary; a WAL and no archiver; one forked backend per connection and no multiplexer; SKIP LOCKED and no queue. Each of those four tools is a declined decision, made by somebody else and shipped separately. That is what "and the tooling around them" in this spoke's domain line turns out to mean, and it is a better answer than the founding corpus's — which was simply that tools exist.

The commit protocol, and the catalog read to the end

(added 2026-08-20, third pass — the curator asked for the rest of the catalog's items, so awesome-postgres was mined to completion: eleven subjects at their primary source, every remaining entry recorded in fourteen coverage pages)

Growth edge 1 closes here, and it took a fork to close it. yugabytedb is the first source in this corpus that describes a distributed store's own commit protocol. Tablets are Raft groups; a write is durable when "replicated to a majority of tablet-peers and successfully persisted on the majority"; fault tolerance is stated as RF >= 2f + 1, which is the quorum bound part-time-parliament proves and podc-lecture-notes derives, restated as a config parameter. Cross-shard transactions write provisional records that are "invisible to readers until the transaction commits", coordinated through a transaction status table that is "just another sharded table in the system", with the commit timestamp "chosen as the current hybrid time at the transaction status tablet at the moment of appending the transaction committed entry to its Raft log." Two-phase commit whose coordinator decision is made durable by a Raft append, with coordinator failure handled by provisional-record expiry rather than by blocking.

So the spoke now holds three positions on distributing a transactional SQL database, not two: extension (citus — inherits Postgres's replication, therefore no consensus of its own), proxy (vitess/planetscale — engine-agnostic, reimplements what it needs), and reimplementation below the SQL layer (yugabytedb — the only one that gets per-shard consensus, at the price of no longer being Postgres beneath the parser).

That third position is also the limit of the extension thesis. postgresql-extensions argues from the catalog that Postgres grows by extension; postgres-forks-and-derivatives enumerates the five exceptions and asks what each needed. YugabyteDB's answer is exact: an extension can add a table access method, a router, a queue, even a storage engine, but it cannot replace WAL-and-streaming replication with a Raft quorum, because Postgres exposes no such seam. The two claims fit; the fork count stays at five, and ferretdb turns out not to be one of them.

Raft is now attested twice and still unread. patroni rents it from an external store, yugabytedb runs it as its own replication layer. What the corpus does not have is any account of the algorithm — leader election, log matching, membership change. Growth edge 4 narrows to the paper itself, and paging a raft concept from two second-hand product descriptions would be the exact mistake ../HUB.md warns about.

Growth edge 9 closes too, from the engine's documentation. postgresql-planner-statistics supplies the mechanism under the number pev2 ranks by. The planner estimates selectivity per column from pg_statistic's MCV list and histogram — statistics that "are always approximate even when freshly updated" — and then "assumes that the conditions are independent, and so it multiplies their selectivities." The docs' own worked example: two identical columns, WHERE a = 1 AND b = 1 estimated at 1 row, actual 100, "two orders of magnitude" out. That is why estimate_factor errors skew toward underestimation — multiplying selectivities can only shrink the estimate — and it is why a nested loop gets chosen for a join that will run 400,000 times. CREATE STATISTICS fixes it in three flavours, all opt-in per column pair, which is what makes hypopg's what-if testing and pev2's ranking the practical entry points rather than the catalog views.

And the spoke finally paged its own standard of evidence. pgbench exists in this corpus because of a refrain: turso unverified, planetscale-768-servers a vendor blog, citus' adaptive-executor claim unbenchmarked, pgmq silent on throughput. The instrument's documentation turns out to be mostly warnings — "never believe any test that runs for only a few seconds", scale factor must exceed client count "else you'll mostly be measuring update contention", results drift with dead-tuple accumulation and autovacuum, and the tool "can itself become the bottleneck." None of the vendor claims recorded in this spoke state any of those conditions. That does not make them false. It means the spoke's habit of marking performance claims unverified is now backed by the engine's own account of what verification costs.

Open questions

Growth edges

Ranked; each names the kind of source that would close it (see ../QUALITY.md → Growth edges).

  1. The distributed-consistency substance. Partly CLOSED 2026-08-08 by podc-lecture-notes — consensus, FLP, byzantine bounds, quorum systems, CAP and the weak-consistency vocabulary (consensus, consistency-models). The engine half stands: no production system's own account is here — Spanner's TrueTime, CockroachDB on Raft, Vitess's failover path — and Raft itself, the algorithm most of them run, appears in neither the book nor this wiki. — needs: an engine-internals doc or the Raft paper. Sharpened 2026-08-20: citus is a production distributed store and its README supplies no such account — it defers HA to Patroni and never mentions commit protocol. The edge needs an internals document specifically, not another product page. Relocated 2026-08-20 by patroni: the consensus this stack depends on is real and lives in the DCS — etcd, Consul, ZooKeeper — one layer below the database. The internals document this edge wants may not be a database's at all. — now needs: etcd's or Consul's own account of its Raft implementation, or the Raft paper. CLOSED 2026-08-20 by yugabytedb, which supplies the internals document from inside a database rather than beside one. Superseded by edge 4 (the algorithm itself) and by the new edge 12 (hybrid logical clocks).
  2. A CRDT-side source. The event-sourcing-beats-CRDTs claim comes from someone who maintains an event-sourcing framework, with no benchmark. — needs: T1/T2 from Automerge/Yjs/Loro, or a neutral comparison. HALF CLOSED 2026-09-28 (research pass) by eg-walker — the CRDT side, measured, but for text only. Successor, edge 2a: a measured comparison of sync mechanisms on structured, schema-changing application data, which is where the event-sourcing claim was made. — needs: a T1/T2 benchmark or engine account covering Automerge/Loro/Yjs maps or a sync engine against an event log on app data.
  3. What a non-routable query costs. Opened 2026-08-03 and still unanswered after a second engine: scatter-gather cost when the shard key cannot route. citus (2026-08-20) describes parallel fan-out in the same terms vitess does and prices it no better. — needs: a measurement or an engine doc that states the penalty.
  4. Raft, and an engine's own account. (new 2026-08-08) podc-lecture-notes gives Paxos and the impossibility result; the algorithm most production stores actually run is in neither the book nor this wiki. — needs: the Raft paper, or an engine-internals doc describing a real failover path. Half closed 2026-08-20. patroni supplies the real failover path, in first-party detail. Raft is located — Patroni's repo topics name it, and its DCS options are consensus systems — and still unread. This edge is now specifically the Raft paper (or etcd's implementation notes), and nothing else will close it. Attested a second time the same day — yugabytedb runs Raft per tablet — which raises the count of production systems in this corpus using Raft to two and the count of sources explaining it to zero. The edge is unchanged and now conspicuous. CLOSED 2026-08-22 (research pass) by raft-paper — Ongaro & Ousterhout's extended tech report, read in full. Two production systems here run it and now one page explains it: strong leader, randomized election timers, joint consensus for membership change, five safety properties, and the availability inequality broadcastTime ≪ electionTimeout ≪ MTBF that patroni's loop_wait + 2 * retry_timeout <= ttl is a configuration form of. Successor, edge 16: every number in it — 287 ms median failover, 35 ms at a 12–24 ms timeout — comes from the authors' own five-node implementation, and the understandability claim rests on a 43-student quiz the authors wrote and graded. No database in this corpus has published what Raft costs it. — needs: an operator's or engine's own measurement of Raft failover and commit latency in production.

Coverage edges (added 2026-08-08, at the curator's request for a wider backlog). These widen what the spoke covers instead of answering an open question above; one ordinary solid source closes any.

  1. The two engines everyone actually runs. CLOSED 2026-08-09 (research pass) — postgresql and mysql written from the projects' own documentation (both T1), plus multiversion-concurrency-control as the mechanism they share. Successor: the write path. Both pages stop where the interesting operational failures start — WAL/redo, checkpointing, crash recovery, and replication, which is exactly what planetscale-768-servers describes hitting. — needs: the engines' internals docs on WAL and replication, or a chapter on write-ahead logging. The WAL half CLOSED 2026-08-20 — write-ahead-logging written from the Postgres documentation (T1): the ordering rule, REDO/roll-forward recovery, the sequential-append-versus- random-flush bargain, and "one fsync of the WAL file may suffice to commit many transactions" as the arithmetic under OLTP throughput. pgbackrest supplies the operational half — physical backup plus archived segments is what PITR consists of. Still open: replication itself (streaming, logical, lag) and checkpoints / full-page writes, which the intro doc does not reach and which patroni configures without describing.

  2. Storage-engine mechanics — the durability half closed 2026-08-30. MVCC was defined from two engines' docs on 2026-08-09 (multiversion-concurrency-control), the write-ahead log on 2026-08-20 (write-ahead-logging), and on 2026-08-30 the four things that page listed as undefined — checkpoints, full-page writes and torn pages, the synchronous_commit durability levels, and WAL write amplification — from postgres-wal-configuration and postgres-wal-reliability (both T1). What that pass established is that write amplification and recovery time are one dial: checkpointing more often shortens replay and enlarges the log, because each page re-pays its full 8 kB image on the far side of every checkpoint. B-trees and LSM trees are still undefined, and the LSM half matters most — it is the design the spoke's OLAP siblings and every embedded key-value store use, and nothing here states the read/write-amplification trade against the B-tree. — needs: a textbook chapter or an engine's internals docs.

  3. Isolation levels, stated properly. consistency-models covers the distributed side; the single-node side (read committed, repeatable read, snapshot isolation, the anomalies each allows) is unwritten. — needs: the ANSI definitions plus a critique such as the Berenson et al. paper.

  4. The document and key-value neighbours. MongoDB and Redis are operational stores by any reading of the domain and appear nowhere. — needs: first-party documentation, one page each. Partly opened 2026-08-20, and the distinction matters. ferretdb is a source about reproducing MongoDB's wire protocol on Postgres — including the licensing argument that produced it, MongoDB's move to SSPL — and not a source about how a document store is designed. Storage engine, indexing, and consistency model for the document family are still unpaged, and Redis is still absent entirely. The edge narrows rather than closes.

  5. How the planner estimates, and how the estimates go wrong. (added 2026-08-13) pev2 and query-execution-plan give the spoke the symptom — estimate_factor, the planner's guess against the actual row count, over or under — and the instruments for spotting it. Neither says where the guess comes from. Missing: table statistics and how they are collected, histograms and n-distinct, the independence assumption that makes correlated predicates blow up, extended statistics as the remedy, and autovacuum's role in letting them go stale. This is the mechanism under the one number both new pages treat as the main diagnostic. — needs: the Postgres planner/statistics documentation, or a query-optimizer chapter. CLOSED 2026-08-20 by postgresql-planner-statistics, written from the documentation this bullet asked for: pg_statistic/pg_stats, MCV lists and histograms, default_statistics_target, the admission that estimates "are always approximate even when freshly updated", the independence assumption with the docs' own two-orders-of-magnitude worked example, and CREATE STATISTICS in three flavours. Paired with hypopg for the what-if side. Two named pieces survive: autovacuum's role in letting statistics go stale, and what ANALYZE sampling costs on a large table. Carried into edge 13.

  6. The queue-as-database question, unmeasured. (new 2026-08-20) pgmq gives the spoke its first "Postgres instead of a dedicated system" subject, and neither its README nor this wiki has a single throughput number, a comparison against a real broker, or any account of what continuous UPDATE churn does to a heap under MVCC vacuum pressure. The same silence covers the whole extension ecosystem's performance claims. — needs: a benchmark, or a production write-up from one of the named users (Supabase, Tembo).

  7. The unmined catalogs. (new 2026-08-20) Two broad directories now sit in this spoke and only one has been mined. awesome-postgres' densest untouched sections are Monitoring (20 entries) and Optimization (14) — the latter aimed straight at growth edge 9, since pgtune, PoWA and PgHero are about the statistics pev2 can only show the symptom of. awesome-db-tools has never been mined at all. — needs: nothing new; this is work, not a gap. awesome-postgres DONE the same day, at the curator's instruction to ingest all of it: eleven subjects read at primary sources and every remaining entry recorded across fourteen coverage pages (postgres-ha-tooling, postgres-backup-tooling, postgres-monitoring-tooling, postgres-query-optimization-tooling, postgres-gui-clients, postgres-cli-tooling, postgres-schema-migration, postgres-work-queues, postgres-data-movement, postgres-client-libraries, postgres-managed-services, postgres-kubernetes-operators, postgres-forks-and-derivatives, postgres-learning-resources), plus the Extensions roster on postgresql-extensions. awesome-db-tools remains unmined — that is what is left of this edge.

  8. Hybrid logical clocks. (new 2026-08-20) yugabytedb's commit protocol rests on them at every step — provisional records carry a hybrid time, the commit timestamp is the status tablet's current hybrid time — and no source here defines what one is or what it guarantees without synchronised clocks. This is the piece that decides whether that commit timestamp means what Spanner's TrueTime means. — needs: the HLC paper, or Spanner's.

  9. Autovacuum, and the cost of being wrong about it. (new 2026-08-20) Three separate pages now end on it. postgresql-planner-statistics leaves autovacuum's role in stale statistics as its named remainder. pgbench quotes the docs saying autovacuum "can result in unpredictable changes in measured performance." pgmq's update-per-read design makes vacuum pressure its unmeasured risk, and the catalog's pg_squeeze exists to fight bloat with minimal locking. It is the same mechanism wearing three costumes and the corpus has no page for it. — needs: the routine-vacuuming documentation, or an internals write-up.

  10. A cross-shard benchmark. (new 2026-08-20, successor to edge 3) pgbench gives the spoke a measurement standard and a single-node OLTP transaction. Growth edge 3 asks what a query costs when the shard key cannot route it, which pgbench cannot answer at all. — needs: a distributed benchmark, or an engine's published scatter-gather numbers.

  11. The coverage pages' own next reads. (new 2026-08-20) The fourteen T3 coverage pages each end by naming their highest-value unread entry. Ranked across all of them: aqo (feedback-driven query optimization — the deepest available answer to what is left of edge 9), AgensGraph vs AGE (the same capability shipped once as a fork and once as an extension — a direct measurement of the seam), pgroll (two live schema versions for zero-downtime migration), OrioleDB (a storage engine as an extension), Spock or BDR (multi-master conflict resolution), PostgREST (row-level security as the authorization layer), and CloudNativePG (failover implemented against the Kubernetes API rather than embedding patroni).

  12. What Raft costs in production. (new 2026-08-22, successor to edge 4) raft-paper supplies the algorithm and every number in it is the authors' own: a five-node cluster of unstated hardware, 287 ms median failover, and an understandability claim resting on a 43-student quiz the authors wrote and graded. yugabytedb and the DCS under patroni both run Raft in production and neither publishes a failover or commit-latency figure. — needs: an engine's or operator's own measurement of Raft failover and commit latency under load.

  13. Atomicity across shards. (new 2026-09-19, from the-architecture-of-neki) The spoke now holds two sharded-SQL systems in architectural detail, vitess and neki, and neither source says what happens to a transaction that writes to two shards. Neki's durability policies (async / sync / cross-zone sync) are per shard and describe replication, not distributed commit. This is the first question consistency-models asks of any sharded OLTP store, the spoke has the vocabulary for it (consensus, paxos, raft-paper, byzantine-generals-problem) and no source that applies it here. — needs: Vitess's or Neki's own documentation on two-phase commit and cross-shard transaction semantics, or an independent evaluation.

  14. Does Neki work? (new 2026-09-19) the-architecture-of-neki carries no number at all — no latency, no throughput, no shard count, no Vitess comparison — and the product is in Platform Preview. The architecture is described and the behaviour is unknown, which is edge 14's problem on a specific system. — needs: PlanetScale's own published figures once it reaches GA, or a third-party evaluation.

The algorithm two of these systems run was designed to be understood (added 2026-08-22)

raft-paper closes the spoke's most conspicuous gap — two production systems here run Raft and nothing explained it — and the interesting part is not the mechanism.

The design target was a property of readers. Ongaro and Ousterhout set out to define consensus "in a way that is significantly easier to learn than Paxos," and the techniques are restrictions: only the leader appends, elections are settled by randomized timers, membership changes go through overlapping majorities. Nothing here is more powerful than paxos; it is less general on purpose. No other design in this corpus argues for itself that way, and it is the reason Raft is what yugabytedb and the DCS under patroni actually run.

The timing rule was already in this spoke as a config file. Raft states availability as broadcastTime ≪ electionTimeout ≪ MTBF, safety independent of all of it. patroni's loop_wait + 2 * retry_timeout <= ttl is that inequality with the constants filled in — the spoke had the operational form six weeks before the theory, which is the reverse of how it usually reads sources.

And the evidence is thinner than the algorithm's standing suggests. 287 ms median failover comes from the authors' own five-node implementation on unstated hardware; the understandability claim rests on a 43-student quiz written and graded by the authors, measuring quiz scores rather than anyone's ability to build a correct system. It is more evidence than any competitor offers and it is still one course-sized sample. Recorded as edge 16 rather than as a caveat, because two databases in this corpus could answer it and neither has.

The router is not engine-agnostic (2026-09-19)

neki has been a name in this wiki since July — vitess said so on its own page — and the-architecture-of-neki finally describes it. The useful finding is not that PlanetScale built a Postgres sharder. It is how much of the architecture the engine dictated.

The topology is Vitess's, and the post's denial of that is the weakest thing on it. Router↔VTGate, Sidecar↔VTTablet, a control plane, an operator — and the reshaping workflows keep Vitess's exact names: MoveTables, Reshard, OnlineDDL. harshit-gangal wrote the post and a top-five share of Vitess's commits. "Not a MySQL sharding idea wearing a Postgres label" overstates the distance.

The mechanisms are Postgres's, and that is where the real content is:

One design divergence from Vitess, understated by both pages. vitess is credited here with not becoming a distributed query engine — push down to MySQL, merge only what cannot be pushed. Neki's Router executes genuine cross-shard joins itself, choosing between nested-loop, hash and merge on cost estimates. That is a planner. Whether it stays one modest step further or becomes the thing Vitess avoided being is the interesting question, and nothing in the source answers it.

What the wiki still lacks, and should say plainly. No number of any kind: no latency, no throughput, no shard count, no comparison. Platform Preview, not GA. And — the gap that matters most for consistency-models — the source describes durability within a shard in three policies and says nothing about atomicity across shards. A sharded OLTP system's hardest question is whether a transaction touching two shards commits or aborts as one, and this corpus cannot answer it for the system it just described in seven components.

Contradictions / tensions

One live tension since 2026-07-29: planetscale-768-servers shards precisely to keep one globally consistent logical database, while local-first-architecture gives up global consistency on purpose to put the store in the client. Not a factual contradiction — both are right about their own workloads — but the two ends of this spoke now hold opposite views on whether consistency or responsiveness is the property an OLTP store exists to protect. Recorded, not resolved.

A second tension, recorded 2026-08-20 and not a contradiction either. citus and planetscale-768-servers solve one problem — make a fleet look like one transactional database — with designs that cannot both be the right general answer: routing inside the engine as an extension, or routing in front of it as a proxy. Each vendor states only its own side's advantages, and no source here compares them. What can be said from the two documents is narrow and worth keeping separate from the marketing: the extension keeps the engine's full feature surface and inherits its version cadence; the proxy is engine-agnostic and reimplements what it needs. Which cost dominates is unmeasured here.

The founding trio remain complementary (embedded engine / distributed engine / tooling) rather than competing. The standing external caution: planetscale-768-servers is a vendor blog (T2, product to sell) and turso is pre-1.0 with self-described incomplete SQLite compatibility — neither performance nor maturity claims are independently verified here. citus joins that list on a different axis: it is T1 first-party and technically detailed, but its README is stale against its own releases — the banner and both install snippets still name Citus 13.0 on Postgres 17 while 14.2.0 shipped 2026-08-06. The architecture prose is unaffected; the commands are not to be copied, and the version numbers on the page come from the releases API rather than the README.

Its performance and named-customer claims (Algolia, MixRank's "well over a petabyte" of time series, Salesloft, ConvertFlow, HSL) are the vendor's own and unverified here.

A third caution, recorded 2026-08-20, about how a claim is worded. pgmq advertises "guaranteed 'exactly once' delivery of messages to a consumer within a visibility timeout." Read whole, the qualifier is the claim: within the window, exactly once; past it, the message returns and another consumer gets it. That is at-least-once with a redelivery clock, and the schema concedes it — read_ct counts redeliveries, a column that would be meaningless under a true exactly-once guarantee. The project is not being dishonest; it reproduces SQS's contract faithfully, including the part readers routinely drop. Recorded because it is the second time in this corpus that the precise reading of a vendor's sentence was the whole finding, after citus's "no HA of its own."

And a structural tension the ecosystem carries. postgresql-extensions establishes that Postgres grows by extension rather than by fork — while awesome-postgres' own PaaS section lists eighteen managed services, each of which decides for itself which extensions may load. An open ecosystem of a thousand modules meets a closed per-vendor allowlist at deployment. pgmq's SQL-only install path exists because of it. No source here states the tension; both halves of it are sourced.

A fourth caution, and this one is about the corpus rather than a source. pgbench's documentation states what a credible Postgres measurement requires: a multi-minute run, scale factor above client count, tracked vacuum activity, and a load generator that is not itself the bottleneck. No performance claim recorded anywhere in this spoke states any of those conditions — not turso's, not planetscale-768-servers', not citus', and pgmq states no number at all. Recorded as a standing property of the corpus: this wiki holds many performance claims and zero measurements, and now has the engine's own document explaining why that distinction is not pedantry.

Cross-spoke adjacency

Index — Operational Databases Wiki

Catalog of every page, grouped by schema.org @type. Spine: synthesis (thesis), log.md (history), this file (catalog). Spun out of the hub _inbox databases-beyond-olap cluster on 2026-07-16 (3 sources). Some wiki-links resolve to sibling ../analytical-databases-wiki nodes (intentional cross-wiki links).

DefinedTerm (concepts)

SoftwareApplication (engines / sources)

TechArticle (sources)

ScholarlyArticle (sources)

Book (sources)

Collection (sources)

PodcastEpisode (sources)

Organization (entities)

Person (entities)

Synthesis

Notes

Bridge nodes (live in sibling wikis, linked cross-wiki)

analytical-databases · clickhouse · duckdb (../analytical-databases-wiki — the OLAP siblings) · seekdb (../agentic-tooling-wiki — agent state-store, the OLTP-reaches-for-agents seam) · microsoft (../agentic-tooling-wiki — citus-data's owner) · azure (../cloud-wiki — where the managed Citus is sold)

Related pages

Links to

Linked from