Tag
scaling
- Aug 21, 2026 Compute-Optimal Allocation Between Attention and Feed-Forward Width How parameter and flop budgets split between attention and MLP blocks, why the split shifts with sequence length, and which allocations survive a change in context size. deep learning
- Jun 22, 2026 Halo Exchange and the Surface-to-Volume Tax on Domain Decomposition How ghost layers turn a stencil update into communication, why block decompositions beat slabs at scale, and where the ratio stops paying. scientific computing
- Nov 28, 2024 Amdahl, Gustafson, and the Fixed-Problem Fallacy Why the two scaling laws disagree, which quantity each holds constant, and how the choice of fixed time versus fixed problem size changes the predicted ceiling. algorithms
- Feb 5, 2024 Strong and Weak Scaling: What Amdahl and Gustafson Actually Bound The two scaling questions, the arithmetic behind their ceilings, and how to read a measured curve without attributing every bend to the algorithm. scientific computing