Machine Learning · Cluster
Feature Engineering
Representation, scaling, and leakage.
- 01 Feature Scaling and the Conditioning of Gradient Descent How per-feature magnitude sets the condition number of a loss surface, why that number fixes the iteration count of first-order methods, and what each scaler costs.
- 02 One-Hot Encoding as Geometry and the Dummy Variable Trap Why one-hot columns span a subspace whose dimensions are the category count minus one, what collinearity the redundant column introduces, and when the penalty makes it harmless.
- 03 How a Feature Becomes a Leak Leakage defined as a dependency between the training representation and the evaluation labels, the three mechanisms that create it, and why it inflates validation without touching training error.
- 04 Out-of-Fold Target Encoding for High-Cardinality Categoricals How smoothing trades bias for variance in target encoding, why the encoding must be computed out of fold, and what the left-out fold estimate costs in noise.
- 05 Transformer State and the Fold Boundary Why every fitted preprocessing object must live inside the cross-validation loop, which statistics actually matter, and how refitting at the end reintroduces the leak.
- 06 Mean Imputation and the Variance It Removes Substituting the column mean for a missing value preserves the mean exactly and shrinks the variance and the covariance, changing coefficients, standard errors, and fit.
- 07 Variance-Stabilising Transforms for Counts and Proportions Why the log, square root, and arcsine transforms equalise variance across a mean-variance relation, and what changes when the counts include zeros.
- 08 Feature Hashing and the Collision Bound That Sets Its Width Why signed hashing leaves inner products unbiased, how many collisions a fixed width produces, and the error term that appears in place of a vocabulary lookup.
- 09 Group and Duplicate Leakage in Cross-Validation Splits Why row-level splitting fails when rows are not independent, how grouped and blocked folds restore the estimate, and what the fold count costs in group imbalance.
- 10 Point-in-Time Feature Assembly and the As-Of Join How to compute a feature as it stood at a labelled event, why an as-of join is not a filtered join, and where late-arriving records break the reconstruction.