Tag
initialization
- Aug 14, 2025 Variance Preservation in Weight Initialization How the Xavier and He scaling factors are derived from the fan-in of a layer, what breaks when the constant is wrong, and why depth turns a small gain error into a vanished signal. deep learning
- May 9, 2024 Gradient Signal Propagation Through Depth at Initialization How the product of layer Jacobians determines whether a deep stack can be trained at all, why that product is easier to keep benign in residual architectures, and what the propagation analysis predicts about depth and width. deep learning