Tag
tensor-parallelism
- Feb 26, 2026 Replica Packing and Topology Constraints Tensor parallelism turns a model into a group of devices that must be co-located and upgraded together. That group, not the individual replica, is the unit a scheduler can place. inference systems
- May 29, 2025 Tensor-Parallel Residency and Interconnect Pressure Sharding a model across devices splits weights and key-value heads, but every decode step becomes a collective; how that changes the memory and latency balance. inference systems