Calibrating Scores Before Choosing an Operating Point

A classifier score is a ranking device. It orders rows by the model’s internal evidence, and that ordering can be excellent while the numbers attached to it mean nothing in probability terms. Calibration is the property that makes the numbers meaningful, and it is a separate property from discrimination: a model can rank perfectly and be miscalibrated, or rank poorly and be perfectly calibrated by outputting the base rate for every row.

What calibration asserts

A model is calibrated when, among all rows assigned score $s$, the fraction belonging to the positive class is $s$. Formally, for every $s$ in the score range, $\Pr(Y=1 \mid \hat p = s) = s$. The condition is a statement about conditional frequencies over a population, and it is checked by binning: group the rows by score, compare the observed positive rate in each bin to the mean score in that bin.

The reliability diagram plots the two against each other with a diagonal reference. Points above the diagonal mean the model understates the probability; points below mean it overstates. The binned version of the same comparison yields the expected calibration error, a weighted average of the absolute gap across bins, with weights equal to bin populations.

The expected calibration error is a coarse instrument with two well-known defects. Its value depends on the binning scheme, so two reports of the same model with different bin counts can disagree. And it can be small for a model that is badly calibrated within each bin, because averaging inside a bin hides the shape. The binned counts are the evidence; the scalar is a summary that should not be quoted without them.

Two other diagnostics behave differently. The Brier score is the mean squared error between the score and the outcome, which decomposes into a calibration term, a refinement term, and an irreducible term; it therefore penalises both miscalibration and poor discrimination, so it cannot isolate either. The log loss is unbounded and punishes confident errors severely, which makes it a harsh test of the tails: a model that assigns 0.99 to a negative row pays about 4.6 nats for that single row.

Platt scaling and its assumptions

Platt scaling fits a one-dimensional logistic regression on the raw scores, producing $\hat p = 1/(1 + \exp(A s + B))$ with $A$ and $B$ estimated by maximum likelihood on held-out data. It assumes the relationship between score and log-odds is linear, which holds when the score is itself a monotone function of a log-likelihood ratio; for a support vector machine’s margin this is a reasonable approximation, and it was the original motivation.

The fit needs data the model was not trained on. Fitting $A$ and $B$ on the training scores gives a mapping tuned to the training distribution, which is exactly the distribution where the model is overconfident; the mapping then under-corrects at deployment. Platt’s original procedure used an internal cross-validation to produce out-of-fold scores for the fit, which is the correct arrangement and is often skipped.

Two parameters cannot correct a non-monotone distortion. If the model overstates at both ends and understates in the middle, a logistic mapping of the score cannot fix the middle without breaking the ends, because the mapping is monotone and has only a slope and an intercept. The residual after Platt scaling is therefore diagnostic: a reliability diagram that remains S-shaped after fitting indicates that the distortion is not of the form the method assumes.

Isotonic regression and its sample cost

Isotonic regression fits a non-decreasing step function to the score-outcome pairs by minimising squared error under the monotonicity constraint. It makes no assumption about the shape, so it corrects arbitrary monotone distortion, including the S-shape that defeats Platt scaling. The algorithm is pool-adjacent-violators: scan the sorted scores, and whenever the observed rate decreases between adjacent points, merge the two into a block with the pooled rate; repeat until the sequence is non-decreasing.

def pava(y, w):
    """Pool adjacent violators: non-decreasing weighted least-squares fit."""
    blocks = []                       # each: [sum_wy, sum_w, count]
    for yi, wi in zip(y, w):
        blocks.append([yi * wi, wi, 1])
        while len(blocks) > 1 and (
            blocks[-2][0] / blocks[-2][1] > blocks[-1][0] / blocks[-1][1]
        ):
            swy, sw, c = blocks.pop()
            blocks[-1][0] += swy
            blocks[-1][1] += sw
            blocks[-1][2] += c
    out = []
    for swy, sw, c in blocks:
        out.extend([swy / sw] * c)
    return out

The output is piecewise constant, which means the calibrated score takes only as many distinct values as the fitted function has blocks. That is the method’s cost: with few calibration rows, the blocks are few, and the mapping is coarse enough to collapse distinctions the model made. Isotonic regression is consistent but converges slowly, at a rate that requires on the order of the square root of the sample size in the number of blocks, so it needs a calibration set in the thousands to be useful and can overfit badly on a few hundred rows. The standard mitigation is to fit it with an internal cross-validation and average the resulting step functions.

Which method under which condition

The choice follows from the shape of the distortion and the size of the calibration set. Platt scaling has two parameters and is stable on small sets; isotonic has effectively as many parameters as blocks and needs a large set. When the distortion is roughly sigmoidal, which is the common case for margin-based and neural models, Platt scaling is adequate and cheaper.

For multiclass problems the one-versus-all approach fits a calibrator per class and then normalises the results to sum to one, which does not preserve calibration exactly and can be improved by fitting a single multinomial model over the score vector, as Zadrozny and Elkan describe. The per-class approach is the common implementation and its normalisation step is the reason multiclass calibration is usually worse than binary.

Calibration is fitted on a set disjoint from training, which consumes data. The alternative is to use out-of-fold scores from the training procedure itself, which costs nothing extra when cross-validation is already running and produces a calibration set of the full training size. The catch is that out-of-fold scores come from models trained on less data than the final model, so the mapping is fitted to a slightly worse score distribution; the bias is usually smaller than the variance saved by the larger calibration set.

The multiclass case and the normalisation step

One-versus-all calibration fits a binary calibrator per class against the rest and then divides by the sum so the outputs add to one. The division is not a calibrated operation: each per-class output was calibrated as a marginal probability, and the conditionals do not generally sum to one, so rescaling changes every value by a factor that varies by row. The result is usually better calibrated than the raw scores and worse than a joint fit.

A joint approach treats the score vector as the input to a single model with as many parameters as classes, fitted by multinomial likelihood. That preserves the sum-to-one constraint by construction and uses the information in the relative scores, which the per-class approach discards. The cost is more parameters and a greater sensitivity to the calibration set’s size.

The choice matters less than the size of the calibration set. With a few hundred rows, the per-class approach is the only stable option, and its normalisation error is small relative to the noise. With tens of thousands, the joint model is worth the extra fitting.

What calibration does not repair

Calibration is a property of the score distribution conditional on the population. Change the prevalence and a calibrated model becomes miscalibrated: if the model was calibrated at 10 percent prevalence and is deployed at 1 percent, its scores overstate the probability by roughly the ratio of the odds. The correction is to adjust the intercept of the log-odds by the log ratio of the prior odds, which follows from Bayes’ rule and requires only the two prevalences. Refitting the calibrator on data from the deployment population is the empirical alternative.

Calibration also does not repair discrimination. A model whose scores are all near the base rate can be perfectly calibrated and useless; the Brier decomposition makes this explicit by separating the calibration term from the refinement term, and the refinement term is the one that measures the model’s ability to separate. Reporting only a calibration metric invites the degenerate solution of predicting the base rate everywhere, which is perfectly calibrated and has no refinement at all.

Finally, calibration is not transferable across a change in the feature distribution even at fixed prevalence. If the input distribution shifts, the score distribution shifts, and the mapping fitted on the old distribution no longer maps scores to frequencies. Monitoring the reliability diagram on recent data is the only direct check, and the drift it detects is a signal about the inputs rather than about the calibrator.

The interaction with threshold choice is the practical reason to care. The cost-optimal threshold in Selecting a Decision Threshold from the Confusion Matrix is a function of the calibrated probability and the cost ratio; on uncalibrated scores it is a number with no interpretation. The threshold-free rankings in ROC-AUC and PR-AUC Under Class Imbalance are unaffected by calibration, which is precisely why a model can score well on them and still be unusable at a decision point.