How does vLLM work? (explainer)
A plain-language walkthrough of vllm‘s core ideas by Amit Shekhar (Outcome School, 2026-06-17). It adds no new claims or numbers over what the spoke already holds — its value is pedagogical: it frames the kv-cache memory problem and PagedAttention in terms a newcomer can follow, so it’s filed as an accessible secondary to the primary vllm / paged-attention-paper pages.
What it explains
- The waste it attacks. The vivid framing: a naïve server reserves one big contiguous block per request (say room for 2,000 tokens), but “most answers are short — if a user’s answer is only 50 tokens long, the space for the other 1,950 just sits there, reserved but unused.” That is the kv-cache fragmentation problem the spoke states abstractly, made concrete.
- PagedAttention as OS paging. vLLM splits the KV cache into small fixed-size blocks (~16 tokens), allocated on demand as a request generates tokens, with a block table mapping blocks to requests — exactly the OS virtual-memory analogy on vllm, told from scratch. This removes both over-reservation and fragmentation.
- Memory sharing. Because requests are block-indirected, multiple requests can point at the same blocks: shared identical prefixes (one cached copy of a common system prompt) and beam search paths that share early blocks and diverge later.
- continuous-batching. Rather than waiting for a whole static batch to finish, vLLM removes completed requests and admits new ones at every decode step, keeping the GPU full — the article’s framing of why short requests no longer idle a slot.
- Prefill vs. decode. Prefill processes the whole prompt and seeds the KV cache; decode then extends it one token at a time. Standard, restated cleanly.
Why it’s filed, and its weight
Tier T3 — a single-author educational blog post (Outcome School), no benchmarks and no first-party claims; every mechanism it describes is already covered, and grounded with measured numbers, by paged-attention-paper (Kwon et al., SOSP 2023, 2–4× throughput). So it does not advance the spoke’s standing open question (the which-lever-bought-what benchmark decomposition). Its contribution is accessibility — a clean on-ramp to vllm for a reader who hasn’t met PagedAttention. Author/publisher (Amit Shekhar / Outcome School) recorded inline rather than as entity nodes (low graph signal for an inference-mechanics spoke). Claims here trace to the article.
Related
vllm · paged-attention-paper · kv-cache · continuous-batching · prefill-decode-kv-cache · llm-inference