Classifier-Free Guidance as an Interpolation of Two Predictions

A conditional generative model is trained to approximate p(x | c). Sampling from that distribution produces outputs that are consistent with the condition but often only weakly so: the model hedges, because the conditional distribution over natural images given a short text description is broad, and the average of a broad distribution is a plausible but bland sample. Guidance is a modification to the sampling procedure that trades diversity for adherence to the condition, and it does so without retraining the model.

The original formulation required a separately trained classifier whose gradient steered samples toward a class. Classifier-free guidance removes the classifier by observing that the same effect can be produced from the difference between two predictions of the same model, one conditioned and one not. The arithmetic is a few multiplications per step, which is why it became the default.

The score of a conditional distribution, decomposed

Bayes’ rule applied to densities gives p(x | c) = p(c | x) p(x) / p(c). Taking the gradient with respect to x makes the marginal p(c) vanish, because it does not depend on x:

∇_x log p(x | c) = ∇_x log p(x) + ∇_x log p(c | x)

The first term is the unconditional score, which a diffusion model already estimates. The second term is the gradient of the classifier’s likelihood of the condition given the sample. Classifier guidance computes the second term with a separately trained classifier and adds it with a scale w:

∇_x log p(x | c) + w · ∇_x log p(c | x) = ∇_x log p(x) + (1 + w) · ∇_x log p(c | x)

The scale w > 0 sharpens the distribution toward samples the classifier finds strongly consistent with c. The cost is that the classifier must be trained on noisy inputs at every timestep, because the guidance is applied throughout the reverse process, and a classifier trained on clean data gives gradients that are wrong at high noise. Training that classifier is a second model and a second training run.

Eliminating the classifier by subtraction

The observation behind classifier-free guidance is that the classifier’s gradient can be written in terms of two scores that the generative model itself can produce. From the decomposition above,

∇_x log p(c | x) = ∇_x log p(x | c) − ∇_x log p(x)

so the guided score becomes

∇_x log p(x) + (1 + w)( ∇_x log p(x | c) − ∇_x log p(x) )
= (1 + w) · ∇_x log p(x | c) − w · ∇_x log p(x)

The classifier has disappeared. What remains is a linear combination of the conditional and unconditional scores, with coefficients that sum to one. The model must be able to produce both, which is achieved by training a single network with the condition dropped at random: for some fraction of training examples the condition is replaced by a null token or an empty embedding, and the network learns to model the unconditional distribution in that case.

The drop rate is a real hyperparameter. A rate of zero produces a model that cannot do guidance at all, because it has never seen the null condition. A rate near one produces a model that is mostly unconditional and weak at following conditions. Rates around 0.1 to 0.2 are the range in which both behaviours are learned competently, and the cost is that a fraction of training capacity goes to the unconditional task.

def guided_eps(eps_cond, eps_uncond, w):
    # linear extrapolation away from the unconditional prediction
    return eps_uncond + (1.0 + w) * (eps_cond - eps_uncond)

def guided_eps_equiv(eps_cond, eps_uncond, s):
    # same operation, written as s * cond + (1 - s) * uncond
    return s * eps_cond + (1.0 - s) * eps_uncond

The two functions compute the same thing with different parameterisations: s = 1 + w. The second form makes the geometry obvious, because the weights are a convex combination only when s is between zero and one. For s above one the combination is an extrapolation, and the result is no longer a score of any normalised distribution.

What extrapolation does to the distribution

The guided score corresponds to a distribution that is the conditional distribution raised to a power and renormalised, approximately:

p_guided(x | c) ∝ p(x | c) · ( p(x | c) / p(x) )^w

The exact statement is only approximate because the score is modified at every noise level independently, and the modified score field need not be the score of any single distribution. In the region where it is, raising a distribution to a power above one concentrates it on its modes: mass moves toward the points where the conditional density most exceeds the unconditional density, which are the points that are most specifically explained by the condition.

The consequence is the trade the method is named for. As w grows, samples become more clearly consistent with the condition and the set of distinct outputs shrinks. At w around 1 the effect is strong; at w of 10 or more the samples saturate, with oversaturated colours and repeated compositions, because the guidance has pushed the sample into a narrow region where the conditional model is confident and the unconditional model is not. The saturation is not a numerical artefact; it is what happens when a distribution is concentrated onto the extreme tail of its own density ratio.

The diversity loss is measurable in a way that does not require a benchmark: sample a batch at w = 0 and another at w = 3 with the same condition and compare the variance across the batch. The guided batch will have lower variance in both pixel space and in any semantic embedding. The condition is better satisfied, and the model explores less of the space of valid answers.

Why this is not the same as sharpening the sampler

Guidance modifies the score at every step, so the trajectory itself changes rather than only its endpoint. A sampler with a smaller step size or a different noise schedule also changes the trajectory, but in a way that converges to the same distribution as the step size goes to zero. Guidance converges to a different distribution, and that difference is the point.

The distinction matters for the choice of sampler. Because the guided field is not the score of the trained distribution, the standard convergence guarantees for the reverse process do not apply. Deterministic samplers, which integrate the probability-flow ODE, can be particularly sensitive: an ODE solver assumes the field is smooth and that its discretisation error is controlled by the step size, while a guided field with a large w has regions of high curvature where the conditional and unconditional predictions diverge sharply. The effect is that a step count that works at w = 1 can produce visible artefacts at w = 7, and the fix is more steps or a solver with a higher order, not a different schedule.

The alternative, guidance applied only in the middle of the chain, is a compromise that many implementations use. At high noise the guided field determines the global composition; at low noise it mainly amplifies colour and contrast. Applying guidance at all steps and then annealing it to zero at the last few steps keeps the composition while avoiding the saturation, because the final steps are where the extrapolation is least meaningful.

Interactions with the conditioning mechanism

The null condition must be a genuine token the model was trained on, not a zero vector substituted at sampling time. A network trained without seeing the null token interprets a zero embedding as a valid but unseen condition, and the resulting unconditional prediction is arbitrary. The standard construction is a learned embedding for the null case, added as one extra entry to the conditioning vocabulary, so the network has a defined behaviour for it.

For text conditioning, the guidance comparison is between the full prompt encoding and the encoding of an empty prompt. The two differ in sequence length as well as content in some architectures, and padding must be handled consistently, or the difference between the predictions includes an artefact of the padding rather than only the effect of the condition.

For class conditioning, the null case is a dedicated class index, and the model’s output layer has one extra row. This is the cheapest form and the one where the drop rate is easiest to reason about, because the unconditional prediction is a well-defined learned function rather than the output of a network fed a degenerate input.

The cost accounting

Guidance doubles the number of network evaluations per sampling step, because each step needs both the conditional and the unconditional prediction. This is the dominant cost of the method and the reason implementations batch the two together: a batch of size 2N containing N conditional and N unconditional inputs costs one forward pass of twice the batch size, which is cheaper than two passes of size N on hardware where the batch dimension is not the bottleneck.

Caching helps when the condition is constant across steps, which it is for the null branch. The unconditional prediction still depends on the noisy sample, so it cannot be cached across steps; it can only be batched. Some implementations approximate by reusing the unconditional prediction for several consecutive steps, which introduces a bias that grows with the step size and is invisible at high step counts.

The memory cost is one extra batch of activations, and the training cost is the fraction of examples spent on the null condition. Neither is large relative to the base model, which is why the method displaced classifier guidance almost completely. The alternative that remains competitive is guidance through a separately trained model that predicts the condition from the noisy sample, and its advantage is that the guidance signal comes from a model trained specifically for that task rather than from the difference between two outputs of the generative model.

The relationship between the score and the sampler’s drift is developed in the reverse-time formulation, and the noise levels at which guidance is most effective are the ones where the signal-to-noise ratio is near one, discussed in schedule design.