Tag
chunked-prefill
- Apr 22, 2026 Chunked Prefill and the Interference It Removes Prefill is compute-bound and decode is bandwidth-bound, so running them in the same iteration makes each wait on the other. Splitting prompts into chunks is the standard remedy. inference systems
- Apr 13, 2025 Splitting Prefill From Decode Inside a Step Prompt processing is compute-bound and token generation is bandwidth-bound; how mixing them in one step costs latency, and how chunked prefill bounds the interference. inference systems