Tag
regularization
- Jun 16, 2026 Cost-Complexity Pruning and the Weakest-Link Sequence How a maximal tree is collapsed into a nested family of subtrees indexed by one penalty parameter, and why the sequence makes cross-validation cheap. machine learning
- Feb 27, 2026 Decoupled Weight Decay and Adaptive Scaling Why adding a shrinkage term to the gradient is not the same as shrinking the parameter, how the adaptive denominator changes the effective decay per coordinate, and what the decoupled update fixes. deep learning
- Feb 14, 2024 Second-Order Boosting and the Regularized Split Gain A quadratic expansion of the loss gives closed-form leaf weights from gradient and curvature sums, turning each split into a comparison of two ratios and making regularization part of the objective. machine learning
- Jun 20, 2023 Monotonic and Interaction Constraints in Boosted Ensembles Bounding each node interval and restricting which features may share a path turn domain assumptions into structural properties of every tree, at a measurable cost in fit. machine learning