Machine Learning · Cluster
Supervised Learning
Regression, classification, and loss design.
- 01 Least Squares as Orthogonal Projection onto a Design Matrix Why the normal equations are a geometric condition rather than an algebraic trick, what rank deficiency does to the answer, and why forming the Gram matrix costs half the digits.
- 02 Logistic Regression as Conditional Bernoulli Likelihood How the sigmoid falls out of a log-odds parameterisation, why the negative log-likelihood is convex and unbounded in the separable case, and what the gradient actually computes.
- 03 Ridge Shrinkage in Ill-Conditioned Design Matrices How adding a multiple of the identity to the Gram matrix reweights singular directions, what the bias-variance trade actually buys, and why the penalty must not include the intercept.
- 04 Squared, Absolute, and Quantile Loss Compared by Their Gradients How the shape of a regression loss decides which observations move the fit, why the absolute loss has no derivative at zero, and what the pinball loss estimates instead of a mean.
- 05 Class Imbalance, Class Priors, and Cost-Sensitive Thresholds Why resampling and thresholding both leave the fitted probabilities unchanged or wrong, how a prior shift translates into an intercept adjustment, and where the expected-cost optimum sits.
- 06 Newton and IRLS Fitting for Logistic Regression Why logistic regression is fitted by repeated weighted least squares, how the Hessian decides the step, and what happens when the weights collapse under separation.
- 07 Softmax Coupling versus One-vs-Rest Probabilities What the normalising constant does to multiclass estimates, why one-vs-rest scores need not sum to one, and how the two parameterisations differ under class imbalance.
- 08 Hinge Loss, Maximum Margin, and the Points That Decide the Boundary Why the hinge loss produces a sparse solution supported only on near-boundary points, what the margin constraint actually constrains, and how C trades margin width against violations.
- 09 Proper Scoring Rules and Why They Cannot Be Gamed Why some losses reward honest probability forecasts while others reward sharpness alone, what decompositions reveal about calibration, and why the logarithmic score is the only local strictly proper rule.
- 10 Link Functions, Variance Functions, and Poisson Regression with Exposure How the log link keeps fitted rates positive and turns coefficients into multiplicative effects, why the mean-variance relation is a modelling assumption, and what an offset does that a predictor cannot.