Tag
gradient-boosting
- Jul 21, 2026 Gradient Boosting as Gradient Descent in Function Space Each boosting round fits a small tree to the negative gradient of the loss at the current predictions, which makes the algorithm a steepest-descent step whose direction comes from a restricted model class. machine learning
- Jun 6, 2026 Histogram Split Finding and the Weighted Quantile Sketch Binning a feature into a few hundred buckets turns repeated per-node sorts into linear passes, and choosing the boundaries by weighted quantiles keeps the buckets where the data actually is. machine learning
- Feb 14, 2024 Second-Order Boosting and the Regularized Split Gain A quadratic expansion of the loss gives closed-form leaf weights from gradient and curvature sums, turning each split into a comparison of two ratios and making regularization part of the objective. machine learning
- Nov 15, 2023 LambdaMART and Metric-Driven Gradient Boosting How lambda gradients weight document pairs by their effect on NDCG, why the resulting tree ensemble is trained on pseudo-residuals, and where the heuristic gradient fails. information retrieval
- Jun 20, 2023 Monotonic and Interaction Constraints in Boosted Ensembles Bounding each node interval and restricting which features may share a path turn domain assumptions into structural properties of every tree, at a measurable cost in fit. machine learning