pgbench

The benchmarking program shipped with PostgreSQL: it runs the same sequence of SQL repeatedly, in some number of concurrent sessions, and reports transactions per second (T1, first-party).

This page exists because of a refrain running through the whole spoke. turso's performance claims are unverified. planetscale-768-servers is a vendor blog. citus's adaptive-executor claim is a design statement with the substantiating paper unread. pgmq publishes no throughput number at all. Growth edge 3 has been asking since 2026-08-03 for a measurement of scatter-gather cost and has been answered twice with more prose. The corpus has never paged the instrument that would settle any of it — and the instrument's own documentation turns out to be mostly a warning about how easy it is to measure the wrong thing.

What the default test is

A transaction "loosely based on TPC-B", seven statements: BEGIN, an UPDATE of a random row in pgbench_accounts, a SELECT of that balance, UPDATEs of pgbench_tellers and pgbench_branches, an INSERT into pgbench_history, END. Small random writes with a read — the OLTP shape this spoke is defined around, which is why the number it produces is comparable to anything at all.

Scale factor -s sets the data size: at scale 1, pgbench_accounts holds 100,000 rows, one branch, ten tellers; -s 100 gives ten million accounts. Output is tps, latency average, latency stddev, and failed transactions.

The caveats are the content

The documentation spends more effort on how to be wrong than on how to run it.

Short runs are noise. "In the first place, never believe any test that runs for only a few seconds. Use the -t or -T option to make the run last at least a few minutes, so as to average out noise."

Scale must exceed concurrency, or you are measuring contention. "The initialization scale factor (-s) should be at least as large as the largest number of clients you intend to test (-c); else you'll mostly be measuring update contention… -c values in excess of -s will undoubtedly result in lots of transactions blocked waiting for other transactions." There is one branch row per unit of scale and every transaction updates one, so at low scale the benchmark measures row locks rather than the database.

The result drifts with the state of the tables, and the reason is MVCC: "accumulation of dead rows and dead space in the tables changes the results. To understand the results you must keep track of the total number of updates and when vacuuming happens. If autovacuum is enabled it can result in unpredictable changes in measured performance." The same dead-tuple churn this spoke flagged as pgmq's unmeasured risk is here stated by the engine's own docs as a thing that makes benchmarks unrepeatable.

The tool can be the bottleneck. "A limitation of pgbench is that it can itself become the bottleneck when trying to test a large number of client sessions."

And exact reproducibility is the wrong goal. "From a statistical viewpoint reproducing runs exactly is a bad idea because it can hide the performance variability or improve performance unduly, e.g., by hitting the same pages as a previous run."

What this settles for the spoke

Not a number — this ingest ran no benchmark. What it settles is the standard of evidence. Every unverified claim recorded across these pages would need, at minimum, a multi-minute run at a scale above the client count, with vacuum activity tracked, from a load generator that is not itself saturated. None of the vendor claims in this corpus state any of those conditions. That does not make them false; it means "faster" in a README is not a measurement, and the spoke's habit of recording performance claims as unverified is now backed by the engine's own account of what verification takes.

The gap that remains is the interesting one: pgbench measures a single-node OLTP transaction. Nothing in it addresses the question growth edge 3 actually asks — what a cross-shard query costs when the shard key cannot route it. A distributed benchmark is a different instrument, and the corpus does not have one.

Related

postgresql · pg-stat-statements · multiversion-concurrency-control · online-transaction-processing · citus · pgmq · planetscale-768-servers · awesome-postgres · synthesis