Tag
arithmetic-intensity
- Sep 29, 2026 Weight-Only Quantization and the Memory-Bandwidth Bound Why 4-bit weights speed up token generation while leaving prefill almost untouched, how the dequantize-on-load kernel works, and where the arithmetic intensity ceiling sits. inference systems
- Jul 24, 2026 The Roofline Bound and Arithmetic Intensity How the ratio of arithmetic to bytes moved places a kernel under either a compute ceiling or a bandwidth ceiling, and why the ridge point differs between devices. gpu computing
- May 8, 2026 Bandwidth-Bound and Latency-Bound Work How to tell which of the two memory limits a kernel is actually hitting, why the remedy differs completely, and how the arithmetic intensity decides. performance engineering
- Feb 25, 2026 Depthwise and Grouped Convolutions and the Cost per Channel How splitting channels into groups changes the parameter count, the arithmetic intensity, and the access pattern, and why a cheaper layer is often the slower one. deep learning
- Feb 4, 2026 Prefill and Decode: The Two Cost Curves Behind One Token Price Why the same model spends prefill time on arithmetic and decode time on memory bandwidth, and how that split decides what a token costs to serve. inference systems
- Feb 3, 2025 Why a Training Step Costs Three Times Its Forward Pass The parameter-times-token accounting behind training compute budgets, where the factor of three comes from, and when attention terms make the estimate wrong. deep learning