Grouped Query Attention (GQA)

The simplest and most widely deployed answer to the memory cost of the KV cache: let several query heads share one key/value head. Fewer distinct K and V vectors exist, so fewer get cached.

Multi-Query Attention (MQA) is the limit case — a single key/value head serving every query head. serret-attention-primer describes GQA as "one of the simplest, and most ubiquitous" such methods, which the model table on that page bears out: both Llama 3 70B and Gemma 3 27B use it.

What it changes and what it does not

Nothing in attention-mechanism changes. The kernel, the normalization and the weighted sum are identical; there are simply fewer W_K and W_V matrices than W_Q matrices, and the K and V a query head uses belong to its group rather than to itself. In the cache-size arithmetic — 2 · N_layers · N_heads · N_tokens · d floats — the head count in the K and V terms becomes the number of KV heads, and the query heads keep their own count.

That is the whole mechanism, and its bluntness is the point: it needs no change to the attention operation, no change to the serving path, and it composes with everything else.

The ratio is a model-design decision, and it varies a lot

From serret-attention-primer's table:

• Llama 3 70B — 64 query heads over 8 KV heads. 8:1.
• Gemma 3 27B — 32 query heads over 16 KV heads. 2:1.

Same d_head (128) in both, four times the sharing in one. Nothing in this corpus explains why two frontier open-weight models landed four-fold apart on the same lever, and the source does not attempt it. The trade is stated but not measured anywhere the spoke holds: more sharing means a smaller cache and longer feasible context, at some cost in what the heads can represent independently. No quality figure, no throughput figure, no ablation is available here.

Where it sits among the answers

Three levers reduce cache memory in this corpus and they work at different layers. GQA reduces the number of cached vectors by sharing heads. latent-attention reduces what a cached vector is, replacing per-head K and V with one shared low-rank latent per token. Quantization (quantization, and cloudflare-kimi-glm-serving's FP8 cache) reduces the bytes per element without touching the structure. They are independent and stack.

What the spoke still lacks

This page rests on an expository secondary source. The originating paper — Ainslie et al., GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints — is not held, and it is where the uptraining procedure and the quality-versus-speed measurements live. That remains the open half of the growth edge in synthesis. Sliding-window attention, named in the same edge, still has no page or source at all.

Related

attention-mechanism · kv-cache · latent-attention · serret-attention-primer · quantization · llm-inference · synthesis