Deep Learning · Cluster
Generative Models
Latent variable and diffusion approaches.
- 01 The Evidence Lower Bound as a Latent Variable Objective How the intractable marginal likelihood of a latent variable model is split into a reconstruction term and a KL term, and what each term pushes the encoder to do.
- 02 The Reparameterization Trick and Gradient Flow Through Sampling Why a sampling operation blocks backpropagation, how the score-function estimator works around it, and what the reparameterized estimator changes about gradient variance.
- 03 Posterior Collapse and the KL Term That Wins Why a latent variable model can learn to ignore its latents entirely, how the KL term competes with reconstruction, and which interventions change the balance.
- 04 Forward Diffusion as Closed-Form Gaussian Corruption How a fixed Markov chain turns any sample into noise in a fixed number of steps, why the cumulative transform is Gaussian in closed form, and what that buys for training.
- 05 Denoising Score Matching and the Noise-Prediction Objective How regressing on the noise that was added to a sample yields an estimate of the data score, and why the equivalence holds without any normalising constant.
- 06 Classifier-Free Guidance as an Interpolation of Two Predictions How a single conditional model can be pushed toward its conditional distribution by extrapolating away from its unconditional prediction, and what that costs.
- 07 Reverse-Time SDEs and the Probability-Flow ODE How the time-reversed diffusion process is defined by the score, why a deterministic ODE shares its marginals, and what the two formulations trade against each other.
- 08 Noise Schedule Shape and the Signal-to-Noise Ratio Curve Why the choice of β schedule changes where a diffusion model spends its capacity, how variance-preserving and variance-exploding conventions differ, and when a cosine schedule beats a linear one.
- 09 Latent Diffusion and the Cost of Compressing Before Generating How moving the diffusion process into a compressed latent space changes the compute budget, what the autoencoder bottleneck removes, and which losses it silently introduces.
- 10 Bits per Dimension and Likelihood on Discretised Data Why a continuous density model of quantised images has a likelihood that depends on the quantisation step, how the change of variables works, and what the number does and does not compare.