Defined Term in llm-inference-wiki

19 pages. Same type across all wikis · llm-inference-wiki

Domain 1

• llm-inference

Mechanism 13

• attention-mechanism
• continuous-batching
• flash-attention
• grouped-query-attention
• kv-cache
• kv-cache-isolation
• latent-attention
• prefill-decode-disaggregation
• quantization
• speculative-decoding
• text-watermarking
• token-sampling
• tokenization

Metric 1

• tokens-per-second

Other 4

• browser-inference
• fast-inference-architecture
• local-llm-stack
• model-routing