Tag
occupancy
- Sep 1, 2026 Latency Hiding and the Arithmetic of Warp Counts Deriving how many resident warps are needed to cover a memory stall from instruction count and latency, and where the Little's law model overstates what interleaving can deliver. gpu computing
- Aug 26, 2026 Register Pressure and the Spill into Local Memory Why the register file caps how many threads an SM can host, how the compiler trades registers for scattered memory traffic when it runs short, and why the spilled addresses still coalesce. gpu computing
- Mar 5, 2026 Occupancy as a Latency-Hiding Budget How resident warps are rationed by registers, shared memory, and block granularity, and how to compute the number of concurrent requests a given occupancy can sustain. gpu computing
- Nov 11, 2025 How Key-Value Cache Bytes Set the Batch Ceiling The byte arithmetic that decides how many sequences fit beside the weights, why the ceiling moves with context length, and what remains after it is reached. inference systems
- Jul 30, 2025 What Occupancy Actually Counts Why occupancy is a ratio of resident warps to a hardware maximum rather than a utilization measure, and how registers, shared memory, and block limits each cap it. gpu computing
- May 13, 2025 Batch Size, Occupancy, and the Marginal Cost of a Token How the cost of one more concurrent request falls as the batch grows, where it stops falling, and why the knee of that curve sets the price floor. inference systems
- Mar 20, 2025 Register Allocation and the Occupancy Ceiling How per-thread register counts are rounded into per-warp reservations, why a few extra registers can cost half the resident warps, and what forcing a lower count actually costs. gpu computing
- Feb 20, 2025 Grid-Stride Loops and Launch Size Independence Why the grid-stride loop survives shrunken launches and changing problem sizes, what it costs against a one-element-per-thread kernel, and when it is the wrong shape. gpu computing
- Sep 24, 2024 Launch Geometry and the Arithmetic of Thread Indexing How a grid extent becomes a linear identifier, what the block and grid dimension limits are, and why a non-multiple block size leaves a guard predicate in every kernel. gpu computing
- Dec 13, 2022 Sizing the Window: Why More Entries Stop Paying The diminishing returns of a larger instruction window, the queueing model that explains them, and the second-order effects that make big windows expensive. computer architecture