Tag
memory-fragmentation
- Feb 10, 2026 Paged KV Cache Allocation and the Block Table Why reserving contiguous cache for a worst-case output length caps batch size, and how fixed-size blocks with an indirection table remove most of that waste. inference systems
- Dec 5, 2025 Paged Key-Value Residency and Block Tables How fixed-size cache blocks and per-sequence block tables remove the allocation and fragmentation penalties of contiguous cache extents. inference systems