Tag
interconnect
- Dec 27, 2025 Fat Trees, Toruses, and the Bisection Bandwidth Ceiling How the worst-case cut through a network bounds global traffic, why a torus gives up bisection bandwidth for constant degree, and where each topology fits. scientific computing
- May 29, 2025 Tensor-Parallel Residency and Interconnect Pressure Sharding a model across devices splits weights and key-value heads, but every decode step becomes a collective; how that changes the memory and latency balance. inference systems