Machine Learning · Cluster
Probabilistic Models
Graphical models and Bayesian inference.
- 01 Conjugate Priors and the Arithmetic of Posterior Updating How a likelihood and a prior combine into a posterior in closed form, what that closure preserves under sequential updating, and where the closed form stops being available.
- 02 Conditional Independence and the Factorization of Joint Distributions Why a joint distribution over many variables cannot be stored as a table, how independence assertions turn it into a product of small factors, and what the factored form can never represent.
- 03 Directed Graphical Models: D-Separation and the Markov Blanket How a directed acyclic graph selects a factorization, why three path shapes reduce to one blocking rule, and what the Markov blanket lets an algorithm ignore.
- 04 Exact Inference by Variable Elimination and the Cost of Treewidth How factors are multiplied and summed out, why the elimination order determines the size of every intermediate table, and why exact inference is exponential in treewidth.
- 05 Markov Random Fields and the Intractability of the Partition Function Why undirected models factorize over cliques, how the normalizing sum becomes a partition function, and why that single constant governs both inference and learning.
- 06 Belief Propagation on Trees and Why Loopy Message Passing Fails How sum-product messages re-derive exact marginals on a tree, and what a fixed point on a graph with cycles does and does not guarantee.
- 07 Metropolis-Hastings: Detailed Balance and Proposal Design How a Markov chain is built so that an unnormalized density becomes its stationary distribution, what the acceptance ratio compensates for, and when a correctly implemented sampler still mixes badly.
- 08 Variational Inference and the Evidence Lower Bound Why maximizing a lower bound replaces integrating a posterior, how mean-field updates are derived coordinate by coordinate, and what the bound systematically understates.
- 09 Expectation-Maximization as Inference with Latent Variables Why the log outside a sum over hidden variables blocks direct maximization, how an exact E step makes a lower bound tight enough to guarantee progress, and where the fixed points end up.
- 10 Structure Learning and the Limits of Markov Equivalence How independence tests and scoring functions recover a graph from data, why observation alone cannot orient every edge, and what the search has to assume to terminate.