Tag
kv-cache-transfer
- Apr 6, 2026 Disaggregated Prefill and Decode Pools Separating prompt processing from token generation into distinct pools, the cache handoff that joins them, and how the capacity ratio between pools is set. inference systems
- Jun 23, 2025 Prefill and Decode Disaggregation Prefill saturates arithmetic and decode saturates bandwidth. Putting them on separate machines lets each be sized for its own bottleneck, at the cost of moving the KV cache over a network. inference systems