Database sharding

Distributing a database’s data and writes across multiple primary servers — rather than only adding read replicas — so the whole fleet acts as one logical database. Sharding is the distributed end of the operational-databases spectrum, and planetscale-768-servers is the founding source: 256 shards × 3 servers = 768 machines, ~4 TB each, presented as a single database.

Why replicas aren’t enough

planetscale-768-servers frames three bottlenecks that read replicas can’t fix, which is what forces sharding:

1. The single-writer WAL — every write still funnels through one primary’s write-ahead log, so adding read replicas does nothing for write throughput.
2. Replicas duplicate, they don’t distribute — every replica holds the whole dataset, so capacity doesn’t grow by adding them.
3. Monolithic backups — backing up one enormous database to object storage can take days.

Sharding attacks all three by splitting the data itself across many primaries.

The abstraction — making N servers look like 1

The trick is a proxy router that hides the fan-out from the application:

• Parse the incoming SQL and consult topology metadata (often JSON) describing where data lives.
• Route each query to the right shard by a sharding key — commonly a hash of a column (e.g. user.id), which spreads rows evenly.
• Aggregate results from multi-shard queries back into one answer; pool and buffer connections. This bullet hides the expensive part. Any query the shard key can’t route — an ORDER BY across all rows, a COUNT(*), a deep LIMIT/OFFSET — becomes a scatter-gather: every shard runs it, and the router merges. sharded-pagination-interview-post poses the worst ordinary case (page 100,000 of an order table); cross-shard-queries holds the mechanism, from Vitess’s own documentation — push down what the vindex can route, fan out and merge at the proxy when it can’t.
• Sit behind a network load balancer so the application connects to a single hostname (mydb.pscale.com) and never sees the 768 servers underneath.

The named routers: Vitess for MySQL and Neki for Postgres, both from planetscale. This proxy-router layer is the operational counterpart to a single-node engine like turso — where Turso collapses the database into the process, sharding explodes it across a fleet while keeping the same one-database interface.

Related

operational-databases · online-transaction-processing · planetscale-768-servers · sharded-pagination-interview-post · cross-shard-queries · vitess · planetscale · turso · synthesis