Macro, Micro, and Weighted Averaging in Multiclass Metrics

Precision and recall are defined per class. A model over ten classes produces ten pairs, and collapsing them into one number requires a decision about weighting that is not present in the binary case, where there is only one class to score. The three conventional modes differ in exactly that weighting, and the difference is large enough that two reports of the same model can disagree by tens of points while both are arithmetically correct.

Per-class counts before any averaging

Fix the notation. For class $c$, let $TP_c$ be the rows of class $c$ predicted as $c$, $FP_c$ the rows of other classes predicted as $c$, and $FN_c$ the rows of class $c$ predicted as something else. Precision is $P_c = TP_c/(TP_c+FP_c)$, recall is $R_c = TP_c/(TP_c+FN_c)$, and $F1_c$ is their harmonic mean. Each of the three averaging modes is a weighted mean over $c$ of these per-class values, with a different weight.

Macro-averaging uses weight $1/K$ for each of the $K$ classes. Every class counts equally regardless of how many rows it contains. A class with four rows contributes as much to the score as a class with forty thousand, so a rare class that the model handles badly moves the macro average by a large amount.

Micro-averaging sums the counts first and computes the ratio afterwards: $P_{\text{micro}} = \sum_c TP_c / \sum_c (TP_c + FP_c)$. Since every prediction belongs to exactly one class in a single-label problem, $\sum_c (TP_c + FP_c)$ is the number of rows, and $\sum_c TP_c$ is the number of correct predictions, so micro precision equals accuracy. The same collapse applies to micro recall and therefore to micro F1. Micro-averaging is accuracy under another name, which makes it uninformative about per-class behaviour and informative only about aggregate correctness.

Weighted averaging uses $n_c/n$ as the weight for class $c$, where $n_c$ is the number of rows of that class. It interpolates between the two: classes are weighted by their frequency, so it behaves like micro when one class dominates and like macro when the classes are balanced.

Why macro F1 is not the harmonic mean of macro precision and macro recall

The F1 of an average is not the average of the F1s. Compute macro precision as the mean of the per-class precisions, macro recall as the mean of the per-class recalls, and the harmonic mean of those two numbers; the result generally differs from the mean of the per-class F1 values. Both quantities are reported under the same name in different libraries.

The mean-of-F1 definition is a weighted mean of per-class harmonic means, so each class contributes a quantity that penalises imbalance between its own precision and recall. The harmonic-mean-of-means definition mixes the averages first, which lets a class with high precision and low recall be compensated by another class with the opposite profile. Neither is wrong; they answer different questions, and a number that moves when a library is swapped is not a property of the model.

For multiclass problems the per-class definition is the more common one, because it preserves the per-class trade between precision and recall. The difference between the two is largest when classes have very different precision-recall profiles, which is exactly the situation in which the aggregate is worth reporting at all.

The question each mode answers

Macro-averaging answers: how well does the model perform on a typical class. It is the right summary when every class matters equally, which is the case when the classes are the object of interest rather than incidental labels: a diagnostic system where each condition is equally important, a classification task where the long tail is the point. It is the wrong summary when rare classes are rare because they are rare, and a model that ignores them entirely is acceptable.

Micro-averaging answers: how often is the model correct. It is the right summary when the rows are the unit of cost, which is the case when errors are counted one at a time regardless of class. It is a poor summary of class-level behaviour because a dominant class can carry the number on its own: with 95 percent of rows in one class, a model that predicts that class always reaches micro F1 equal to 0.95 while achieving zero recall on every other class.

Weighted averaging answers: how often is the model correct, with per-class ratios rather than raw counts. It is the compromise used when neither equal class weighting nor equal row weighting is defensible, and it has the awkward property that it is dominated by the large classes while still being computed from per-class ratios, so it can differ from micro F1 in ways that are hard to explain to a reader.

from collections import Counter

def averages(y_true, y_pred, k):
    tp = Counter(); fp = Counter(); fn = Counter()
    for t, p in zip(y_true, y_pred):
        if t == p:
            tp[t] += 1
        else:
            fp[p] += 1
            fn[t] += 1
    support = Counter(y_true)
    n = len(y_true)
    prec = {c: tp[c] / (tp[c] + fp[c]) if tp[c] + fp[c] else 0.0 for c in range(k)}
    rec = {c: tp[c] / support[c] if support[c] else 0.0 for c in range(k)}
    macro_p = sum(prec.values()) / k
    micro_p = sum(tp.values()) / n          # equals accuracy when every row has one label
    weighted_r = sum(support[c] * rec[c] for c in range(k)) / n
    return macro_p, micro_p, weighted_r

The k passed in must include classes absent from both lists; a class with no true rows and no predictions has undefined precision, and the usual convention is to score it zero, which drags the macro average down for a class the model was never asked about.

Where the modes diverge sharply

Consider three classes with 900, 90, and 10 rows, and a model that is perfect on the first, half right on the second, and always wrong on the third. Per-class recall is 1.0, 0.5, and 0.0. Macro recall is 0.5; weighted recall is $(900 + 45 + 0)/1000 = 0.945$; micro recall equals accuracy, also 0.945. The macro number says the model handles half of the typical class; the micro number says it is right on almost every row. Both are true, and a report containing only one of them is a report about a different question than the reader assumes.

The divergence is monotone in the class imbalance. As the largest class grows relative to the others, weighted and micro converge to the majority-class accuracy while macro stays sensitive to the tail. The ratio between macro and micro is therefore a usable diagnostic in itself: a large gap between them means the model’s quality is uneven across classes, and the gap’s size is a rough measure of how much of the aggregate is carried by the head.

Multilabel problems break the micro collapse. When a row can carry several labels, $\sum_c (TP_c + FP_c)$ counts predictions rather than rows, so micro precision is no longer accuracy; it is the fraction of predicted labels that are correct. The relationship between micro precision and micro recall still holds in the usual form, but the identity with row accuracy is gone, and a model that predicts fewer labels can raise micro precision while lowering micro recall without any change in per-row correctness.

Aggregates under a shifting class distribution

An aggregate is a scalar summary of a vector, and the mapping from vector to scalar is where information is lost. The loss is acceptable when the class distribution is stable and the aggregate is compared only against other aggregates computed the same way on the same distribution. It becomes a defect when the distribution shifts between the two measurements being compared, which happens whenever a model is evaluated on a new period, a new population, or a resampled test set.

Consider the same model measured twice, on two test sets that differ only in the proportion of a rare class. Its per-class precision and recall are identical on both, because those quantities are conditional on the class. Its macro average is also identical, since the per-class values are unchanged. Its micro and weighted averages differ, because the weights differ. A report that tracked only micro would read the change as a change in model quality; the per-class table shows that nothing about the model changed.

This is the argument for reporting the per-class table as the primary artefact and the aggregate as an index into it. The table is invariant to the weighting question and carries the support counts that determine how much each row of the table can be trusted. The aggregate is a convenience for sorting a list of candidates, and a sorting key that changes with the test set’s composition is a poor key unless the composition is fixed.

Reporting practice that survives a change of library

The aggregate is a convenience for ranking and for dashboards; the per-class table is the evidence. A report that contains the confusion matrix, the per-class precision and recall with support counts, and one aggregate whose weighting is named explicitly is unambiguous. A report that contains “F1: 0.87” is not, because three defensible computations produce three different numbers.

Two further details matter for reproducibility. Zero-division conventions differ: some implementations return zero for a class with no predicted positives, others raise, and the choice changes the macro average whenever a class is never predicted. And the class list must be fixed explicitly rather than derived from the labels present, or a model that never predicts a class will be scored on a shorter list than the model that does, which inflates the macro average of the worse model.

For model selection, the aggregate must match the decision. If the operating decision is a single threshold on a calibrated score, the binary machinery in Selecting a Decision Threshold from the Confusion Matrix applies per class and the aggregate is a summary of those decisions. If the classes are being ranked rather than decided, the threshold-free measures in ROC-AUC and PR-AUC Under Class Imbalance are the relevant ones, and the averaging question reduces to how to combine per-class areas, which has the same three answers and the same ambiguity.