Tag
piecewise-linear-functions
- Apr 28, 2026 Depth, Width, and the Linear Regions of ReLU Networks Every rectified network is a partition of its input space into cells where it computes an affine map, and the count of those cells grows polynomially in width and exponentially in depth. deep learning
- Aug 25, 2023 Pointwise Nonlinearity and the ReLU Family of Activations Why one elementwise nonlinear function is enough to break the collapse of stacked affine layers, how the rectifier behaves at its kink, and what makes a unit permanently dead. deep learning