Modernizing the Meta Ads Service with an Open-Source Kernel Scheduler

Engineering at Meta (2026-07-13): how Meta cut tail latency on its ads-serving fleet (>5M requests/sec) by writing a custom Linux CPU scheduler with sched-ext — the BPF-based extensible scheduler framework that landed upstream in kernel v6.12 (Meta co-designed it with Google’s ghOSt authors). A production instance of eBPF’s core promise — programmability decoupled from the kernel release cycle — applied to the scheduler itself.

The problem

Upgrading to kernel v6.9, the new default EEVDF scheduler introduced latency regressions that reduced the number of ads ranked per response (a revenue signal). To dodge it, some hosts stayed pinned on the old v6.4 kernel — creating fleet fragmentation and kernel tech-debt. So the operational bind was classic platform-ops: a kernel-version dependency (the same currency constraint eBPF has) pitting a revenue metric against fleet hygiene.

The approach

A custom sched-ext policy soft-partitions CPUs into two pools — one for latency-critical request-path threads, one for less-sensitive work — and encodes domain knowledge of thread importance directly into scheduling decisions, handling the event-driven callbacks (wake-up, enqueue, dispatch, idle). The operationally decisive move: the policy runs as a user-space binary loading a BPF program, so “to roll out a change, we can simply restart the scheduler process” — no kernel rebuild or reinstall. That collapses a kernel-patch-cycle change into a process restart, the scheduler equivalent of a hot deploy.

Results (Meta-reported)

• −28% p99 latency on the ads-retrieval stage; +1.1% weighted-ads-ranked.
• 3.28 MW power saved across the fleet.
• Follow-on user-space policy iterations added −60% latency and −18% timeout errors — the payoff of the fast iteration loop (tune the policy, restart, measure) rather than a one-shot win.

Why it matters here

It extends ebpf past telemetry/networking/security into a fourth domain — scheduling — and gives the spoke a concrete instance of kernel-level performance engineering as a production-ops discipline: a revenue-relevant SLI (p99, ads-ranked) moved by a hot-swappable in-kernel policy, with the eBPF verifier keeping the untrusted scheduler safe. It also lands hard production numbers (latency %, megawatts) against the spoke’s standing qualitative-only caveat. T2: first-party Meta engineering blog — primary but self-reported, no independent replication.

Related

sched-ext · ebpf · site-reliability-engineering · service-level-objectives · platform-ops · scaling-to-1m-lambda