Container scheduling

Deciding which worker runs a container, and getting it running there. The placement layer of a production fleet: a request arrives, something picks a machine with room for it, the machine starts it. The spoke’s interest is what that decision costs at scale — because the scheduler is where a fleet’s coordination bill is paid.

The axis: how fresh must the scheduler’s view be?

The corpus now holds both ends of one trade-off.

Consistent placement. Kubernetes scores every pod against every node — O(n × p), and serialized by default — over a cluster view kept consistent in etcd. Node heartbeats alone impose a baseline O(nodes) write load, and pod writes push etcd toward being the bottleneck (modal-1m-sandboxes, arguing the case against). What you buy is a scheduler that knows the true state of the cluster and can reason about it — bin packing, affinity, constraints.

Stale placement. Modal removes the data store from the creation path entirely: workers publish their own state asynchronously to a Redis stream, scheduling servers load-balance against an in-memory cache of it, and a server contacts a worker directly to place a sandbox. The view is stale on purpose; the worker rejects what it can’t take, and that rejection is the only consistency in the loop. Modal’s own framing — scheduling that is “more like load balancing than traditional container scheduling” — is the concession that this is a different job, not a better Kubernetes (modal-1m-sandboxes).

Which one is right depends on the workload. Long-lived services with placement constraints want the consistent view; they can afford it, and they need what it buys. A million sandboxes that live for seconds want the coordination gone — a rejected placement there costs one retry, which is cheaper than a consensus round by any measure.

Three layers, easy to conflate

The spoke’s scheduling sources sit at different altitudes and should not be read as one thread:

• Arrival — when work shows up. Synchronized rate(5 minutes) crons self-inflicting a DDoS, fixed with jitter (scaling-to-1m-lambda).
• Placement — where it runs. This page: Kubernetes vs. Modal.
• CPU — which thread runs next on a host, once placed. sched-ext and Meta’s ads-fleet policy.

The mechanisms have nothing in common; the lesson does. At fleet scale, scheduling stops being infrastructure you inherit and becomes a lever you engineer.

The thing under the scheduler

Both Modal’s rewrite and Meta’s kernel work found their real wall below the scheduler — for Modal, the Linux rtnl lock serializing container network setup, which stalled startups by tens of seconds no matter how fast placement got (modal-1m-sandboxes). A scheduler that hands out work faster than the host can instantiate it has moved the queue, not removed it.

Related

modal-1m-sandboxes · modal · compute-sandbox · kubernetes · serverless · scaling-to-1m-lambda · sched-ext · meta-sched-ext-ads · platform-ops