Machine Learning · Cluster
Unsupervised Learning
Clustering, dimensionality reduction, and density.
- 01 Lloyd's Algorithm and the Local Minima of k-Means The assignment and update steps are two exact minimizations of the same objective, which is why the loop always terminates, why it stops at a local minimum, and what seeding changes about the answer.
- 02 What a Distance Function Decides About Cluster Shape Squared Euclidean, cosine, Mahalanobis, and set distances each impose a different notion of similarity, and the choice fixes which clusters are available before any algorithm runs.
- 03 Agglomerative Clustering and the Lance-Williams Recurrence Bottom-up merging needs one rule for the distance between two groups of points; the Lance-Williams update is the shared arithmetic behind single, complete, average, and Ward linkage, and it decides which hierarchies are order preserving.
- 04 Gaussian Mixture Models and the EM Lower Bound Soft cluster membership follows from a latent-variable model whose log-likelihood has no closed-form maximizer; the EM iteration maximizes a lower bound that is tight at the posterior, and it can also drive components to degenerate solutions.
- 05 DBSCAN, Density Reachability, and the Parameter Cliff A cluster as a maximal set of density-connected points needs only a radius and a count, produces non-convex shapes and explicit noise, and is order sensitive only at the boundary of a cluster.
- 06 Principal Component Analysis Through the Singular Value Decomposition Maximizing projected variance, minimizing reconstruction error, and decorrelating features all select the same subspace, and the SVD computes it without ever forming the covariance matrix.
- 07 Spectral Clustering and the Relaxation of Normalized Cut Turning points into a weighted graph, writing the partition cost as a cut, and relaxing the discrete indicator vector into an eigenvector problem explains why the method recovers non-convex groups and why the affinity scale decides the answer.
- 08 Why t-SNE Preserves Neighborhoods but Distorts Distances The embedding minimizes one asymmetric divergence between two similarity distributions, so it pulls true neighbours together while almost ignoring far pairs, which makes cluster sizes and inter-cluster gaps artefacts of the optimization.
- 09 Cluster Validity, Stability, and Choosing the Number of Clusters Internal indices measure geometry that a clustering may not assume, stability is not a consistent selector, and chance-corrected agreement between resampled partitions answers a narrower question than it appears to.
- 10 Kernel Density Estimation and the Bandwidth Trade-off Placing a kernel on every observation gives a density estimate with one smoothing parameter, and the rate at which that parameter must shrink with sample size explains why the method becomes almost unusable in high dimensions.