ML system design

Designing the system around a model: what to predict, from which features, trained how often, served under what latency, and measured against which business outcome. The model is one component of it, and usually not the hard one.

What the field actually builds

The one empirical source here is a catalog of 300+ production case studies from 80+ companies ml-system-design-case-studies. Its recurring problems are narrow and repeat across industries: recommendation and personalization (Netflix, Pinterest, Spotify), fraud and risk (Stripe, PayPal, Nubank), logistics and ETA prediction (Uber, DoorDash, Swiggy), and search and ranking (LinkedIn, Etsy, Zillow).

Ranking, matching and forecasting over behavioural data — that is what shipped, in the sectors where a small accuracy gain pays out immediately. It is a description of the written-up corpus rather than a census of the field, and the two are easy to confuse.

Design, not operation

The boundary with ../platform-ops-wiki is worth stating: that spoke owns operating production distributed systems — SRE, observability, incident response, the reliability of the thing once it runs. This page owns the design decision — the choice of target, features, model and serving shape. A feature pipeline's uptime is ops; what the feature is, is here.

What the corpus can't tell you

Every entry in the catalog is a company writing about a system that worked. The failures are absent by construction, and demo-to-production-gap is where the other half of the record would sit — currently held by a single unread headline ai-projects-fail-infrastructure-people. Until a source measures failure rather than asserting it, this wiki knows what succeeds and not how often.

Who calls the model

ml-models-as-agent-tools (MachineLearningMastery, 2026-09-10) adds one design claim, unmeasured: in an LLM-agent system the trained model stops being the pipeline and becomes a tool the agent calls — a fraud classifier returns a probability, and the agent does the fetching, thresholding, searching and routing around it. The case studies above are systems built around a model; this is the model as one step inside something else's loop. What that costs, and what a misroute does, the source does not say.

A decision contract: keeping policy outside the model

jev-typesafe-typed-decisions-guide (2026-09-25, T4 — an SEO account's how-to, not a case study) still gives this page something the 300+ catalog above never spells out: a concrete pattern for the calling side of a model, not just the model's own design. Its decision contract — minimal state per call, one stable ID per decision, a described answer space with a catch-all, and authorization/ payment/rate-limit logic left in deterministic code that only reads the model's typed answer — is a design choice about where the model's authority stops, which belongs in this page's scope (what the system computes) rather than in ../platform-ops-wiki (how it stays up). See typed-decision for the vocabulary this rests on.

The same source's WorkflowResult<T> pattern — a result is decision, review, or unavailable, never a silent default — and its staged rollout (playground → offline eval → shadow mode → assist mode → selective automation behind a pause switch → continuous review) are both generalisable past the one vendor it describes. Neither is measured; both are the kind of texture demo-to-production-gap already flags as the part demo-grade write-ups tend to skip.

ml-models-as-agent-tools · ml-system-design-case-studies · demo-to-production-gap · typed-decision · jev-typesafe-typed-decisions-guide · machine-learning · synthesis