Tag
condition-number
- Oct 12, 2026 Condition Number, Residual, and Backward Error Why a small residual does not imply a small error, how the condition number splits the blame between problem and algorithm, and how to estimate it without forming an inverse. scientific computing
- Jul 5, 2026 Feature Scaling and the Conditioning of Gradient Descent How per-feature magnitude sets the condition number of a loss surface, why that number fixes the iteration count of first-order methods, and what each scaler costs. machine learning
- Jun 27, 2026 Forward Error, Backward Error, and Condition Numbers Separating what the algorithm got wrong from what the problem amplifies, why the rounding model hands over a backward error for free, and how a small residual can still mean a wrong answer. scientific computing
- Aug 15, 2025 Step Sizes and the Descent Lemma in Gradient Descent How the Lipschitz constant of the gradient bounds a usable step size, why 1/L is the natural choice for quadratic objectives, and which constants the convergence rates hide. machine learning
- Jul 3, 2025 Momentum and Nesterov Acceleration in Gradient Descent How velocity terms turn the gradient sequence into a filtered signal, why the quadratic case admits a closed-form optimal damping, and what breaks when gradients are noisy or the objective is not convex. machine learning
- Apr 18, 2025 Catastrophic Cancellation and the Cases Where Subtraction Is Exact Why subtracting two nearly equal numbers amplifies existing error rather than creating it, when the difference is exact by Sterbenz, and how formulas are rewritten to avoid the loss. scientific computing