Tag
model-parallelism
- Oct 8, 2025 Pipeline Parallelism: Filling the Bubble and Paying for It Micro-batch pipelining lets layers be split across devices, but the schedule leaves idle slots. How bubble fraction scales with depth and why interleaved schedules help. deep learning
- Jul 29, 2024 Composing a Parallelism Layout From a Memory and Bandwidth Budget Data, tensor, pipeline, and shard dimensions interact non-linearly. A procedure for choosing their sizes from per-device memory, interconnect tiers, and batch constraints. deep learning