Tag
integer-arithmetic
- Jun 21, 2026 Affine Quantization: Scale, Zero Point, and the Error They Introduce How a real-valued tensor is mapped onto a small integer grid, why the zero point exists, and where the rounding error lands inside a dot product. inference systems
- Apr 8, 2025 Requantization Inside the Kernel: From Int32 Accumulator to Int8 Output How a convolution kernel turns a 32-bit integer accumulator into the next layer's 8-bit input, why the multiplier is a fixed-point pair, and where saturation and rounding diverge from the reference. inference systems
- Oct 29, 2024 Range Coding and rANS as Practical Entropy Coders How byte-oriented range coders and asymmetric numeral systems implement the arithmetic-coding ideal with integer state, and why interleaving makes the rANS variant parallelizable. computation theory