Detailed Balance and the Stationary Distribution of a Markov Chain

A Markov chain Monte Carlo sampler is not a sequence of independent draws but a single trajectory whose next state depends on its last. The target distribution appears nowhere in the code except as a ratio of densities; the chain is told how to move and must be proven to settle into the right distribution. Detailed balance is the equation that does the proving, and it is deliberately stronger than what is required, because the stronger condition is the one that is easy to verify.

What stationarity requires, and what it does not

A distribution $\pi$ is stationary for a transition kernel $P$ when it satisfies the global balance equation

$$\pi(y) = \sum_x \pi(x) P(x, y)$$

for every state $y$, or $\pi = \pi P$ in matrix form. This is the true requirement: start the chain at $\pi$ and one step later it is still $\pi$. It says nothing about convergence from an arbitrary start, and nothing about how long convergence takes; it only pins the invariant measure.

Detailed balance is the pairwise condition

$$\pi(x) P(x, y) = \pi(y) P(y, x)$$

for every pair $(x, y)$. It asserts that the probability flow from $x$ to $y$ equals the flow from $y$ to $x$, not merely that the totals balance at each state. Summing over $x$ on both sides recovers global balance, so detailed balance implies stationarity. The reverse is false: a chain can be stationary without being reversible, and the extra strength is paid for in the difficulty of designing the moves.

The advantage of the stronger condition is that it is local. Verifying global balance requires summing over the whole state space, which is exactly the computation that MCMC exists to avoid. Verifying detailed balance requires checking the ratio at one pair at a time, and a move that satisfies it at every pair does not need to know the normalizing constant of $\pi$, because that constant appears on both sides and cancels.

The Metropolis-Hastings ratio is built to force it

The construction takes a proposal kernel $Q(x, \cdot)$ that is easy to sample and easy to evaluate, and accepts a proposed move from $x$ to $y$ with probability

$$\alpha(x, y) = \min!\left(1, \frac{\pi(y) Q(y, x)}{\pi(x) Q(x, y)}\right).$$

The resulting transition is $P(x,y) = Q(x,y)\alpha(x,y)$ for $x \neq y$. Multiplying, the flow from $x$ to $y$ is $\pi(x) Q(x,y) \alpha(x,y)$, and the flow back is $\pi(y) Q(y,x) \alpha(y,x)$. Because the acceptance ratio is a minimum with 1, one of the two directions has $\alpha = 1$ and the other is the reciprocal ratio, so the products are equal and detailed balance holds against $\pi$ exactly.

The design freedom is the whole point. Any proposal kernel gives a chain with $\pi$ as its stationary distribution, so the proposal can be chosen for its mixing, its cost, or its implementability, and the acceptance step corrects the discrepancy. The ratio needs $\pi$ only through the quotient $\pi(y)/\pi(x)$, so an unnormalized density is sufficient, which is the reason the method handles Bayesian posteriors and partition functions that cannot be computed.

import math, random

def log_target(x):
    return -0.5 * x * x            # unnormalized log standard normal

def metropolis(n, start, step):
    x = start
    log_px = log_target(x)
    for _ in range(n):
        y = x + step * random.gauss(0, 1)   # symmetric proposal
        log_py = log_target(y)
        if math.log(random.random()) < log_py - log_px:
            x, log_px = y, log_py
    return x

With a symmetric proposal $Q(x,y) = Q(y,x)$ the ratio collapses to $\pi(y)/\pi(x)$, which is why the Gaussian random walk is the canonical example. The acceptance rule compares log densities and never needs the normalizing constant $\sqrt{2\pi}$.

The asymmetry the ratio must account for

When the proposal is asymmetric, the correction term $Q(y,x)/Q(x,y)$ is not optional. It is known as the Hastings ratio, and forgetting it biases the chain toward whichever direction the proposal favors. A proposal that steps right more often than left, without the correction, makes the chain drift toward the right regardless of the target, and the stationary distribution is no longer $\pi$ but $\pi$ deformed by the proposal’s asymmetry.

The ratio also has a computational consequence. Evaluating $Q(y,x)/Q(x,y)$ requires the proposal density at two points, not just sampling from it, so the proposal must be one whose density is available in closed form or cheap to approximate. A Gaussian random walk has a symmetric density and the term is 1; a log-normal proposal, or a proposal in a transformed coordinate system, carries a non-trivial Jacobian that must be included in the ratio, and an omitted Jacobian is a silent bias of the same kind.

The acceptance probability exposes the tension between exploration and staying put. A proposal that moves far gets rejected often, and one that moves little gets accepted often, and both are failures of a different kind. The optimal acceptance rate is a property of the proposal’s shape and the target’s dimension, not a universal constant: for a Gaussian random walk the rule of thumb is to tune the step so that roughly a quarter to a third of moves are accepted in low-to-moderate dimension, falling toward the low end as the dimension grows. The tuning is a way of reading the ratio: an acceptance rate that is too high means the proposal is not exploring, and one that is too low means it is exploring and then discarding.

Reversibility is a constraint, not a requirement

Detailed balance forces the chain to be reversible: a chain run forward and a chain run backward are statistically indistinguishable. Reversibility rules out certain kinds of efficient movement. A systematic sweep that always proposes in the same cyclic order can mix faster than a reversible chain, because it never proposes the move that would undo its own progress, but such a sweep violates detailed balance and must be checked against global balance instead.

The practical examples are deterministic-scan Gibbs samplers and the Hamiltonian and slice samplers. Hamiltonian Monte Carlo proposes a long coherent move by integrating a mechanical system, and the proposal is reversible in the physical sense of momentum flip, which preserves detailed balance only because the integration is symplectic and the momentum is negated at the end. A non-symplectic integrator breaks the reversibility and with it the guarantee, which is why the integrator choice is a correctness issue in that setting rather than an accuracy one.

For the methods that keep detailed balance, the guarantee is strong but silent about convergence. The chain converges to $\pi$ under irreducibility and aperiodicity, and the rate of convergence is set by the spectral gap of $P$, which detailed balance helps make real but does not make fast. A reversible chain can have an arbitrarily slow mixing time, and the condition itself gives no bound on it. The failure mode is a chain that has the right stationary distribution but has not reached it in any feasible number of steps, and the autocorrelation of its output is the observable symptom.

The cost of the guarantee

The acceptance-rejection step is the price of forcing detailed balance. Every rejected move wastes the proposal, and the fraction of moves wasted is one minus the acceptance rate. A chain in high dimension, where most directions are uphill or downhill in a way the Gaussian proposal cannot anticipate, spends most of its time rejecting, and the wasted work grows with dimension. The alternative constructions trade the simplicity of the ratio for efficiency, but they must replace detailed balance with a condition of equivalent strength, and proving that replacement is the harder task.

The ratio is also where numerical care concentrates. Densities are multiplied in the numerator and denominator, and both underflow in the tails of a high-dimensional target, so the computation is done in log space. The test $\log u < \log\pi(y) - \log\pi(x)$ is exact when the log densities are accurate, but a subtraction of two large log densities in the tails loses the precision that the acceptance decision depends on. The tails are exactly where a proposal is most likely to move to a new mode, so a numerical loss in the tail is a sampler that silently refuses to mix between modes.

The stationary distribution is the guarantee; the mixing time is the reality. A sampler that satisfies detailed balance against $\pi$ and mixes in a hundred steps is a different instrument from one that mixes in a million, and the equation by itself does not tell them apart. The effective sample size of a reweighted importance sampler plays the same diagnostic role there: both measures describe how much independent information a correlated or weighted sample actually contains.