Delta and Cumulative Temporality in Collection Pipelines

A counter can be reported in two ways. The cumulative form sends the total since the process started, so successive reports are increasing. The delta form sends the change since the previous report, so successive reports are increments that carry no history. Both describe the same underlying event stream, and the choice between them propagates through an entire pipeline: it decides how resets are detected, how much data crosses the network, and which aggregations are possible at which stage.

The two conventions are usually named cumulative temporality and delta temporality. The distinction is a data-model decision rather than a wire-format detail, and mixing the two in one pipeline without conversion produces results that are wrong in a way that is hard to diagnose, because the numbers remain plausible.

What each form actually carries

Under cumulative temporality, the reported value at time $t$ is $\sum_{i \le t} \Delta_i$, the total since the counter was created. Two consecutive reports from the same process satisfy $v_{t_2} \ge v_{t_1}$, and the increment over the interval is the difference. The value is a function of the process’s entire history, so the report is self-contained: a consumer that receives only one report knows the total, and a consumer that misses a report can still compute an increment by differencing across the gap.

Under delta temporality, the reported value at time $t$ is $\Delta_t$, the increment since the last report. The reports sum to the cumulative total, but no individual report carries it. A missed report loses that increment permanently unless the producer retries, and the consumer cannot distinguish a genuine zero-increment interval from a missing one.

The reset behaviour differs accordingly. A cumulative counter that restarts reports a smaller value than the previous one, which the consumer must detect and correct as described in Counter Resets and the Extrapolation Inside rate(). A delta counter that restarts reports an increment that includes whatever accumulated before the restart, so the reset is invisible and no correction is needed. This is the strongest argument for delta temporality: it moves the reset problem from the consumer to the producer, where the information about what happened is still available.

The transmission cost and the alignment cost

The trade runs in opposite directions at the two ends of the pipeline.

Cumulative reports grow without bound in magnitude, but the encoded size does not: time-series compression stores the difference between consecutive values, and for a counter those differences are small regardless of the absolute total. The first value of a series costs its full magnitude in bits; every subsequent one costs the size of the increment. Cumulative temporality is therefore cheap on the wire and expensive only at the first sample after a series is created or restarted.

Delta reports are small by construction, each carrying only the increment. Their cost is that every report must be delivered for the sum to be correct, so the pipeline needs a reliable handoff, and the aggregation step needs to keep a running sum of what it has received. A collector that aggregates deltas must add them into a local cumulative total if anything downstream expects a running value, which means the collector now carries state proportional to the number of series it handles.

Alignment is where delta wins. Two delta reports covering the same interval can be added directly, even if they came from processes that started at different times. Two cumulative reports can be added only if both cover the same interval since their respective creations, and processes started at different times have different origins, so their totals are not comparable until they are differenced against their own previous values. Summing raw cumulative values from two processes with different start times produces a number that is the sum of two unrelated histories, which is meaningless as a rate but looks fine as a total.

Converting between the two

Conversion is mechanical but has one place where information can be lost.

Delta to cumulative is a running sum: keep a per-series accumulator and add each incoming delta. The accumulator must be keyed by the full series identity, and it must be reset when the series disappears, or it will accumulate forever and report a total that no process ever produced. The cost is memory proportional to active series, and the failure mode is a stale accumulator that keeps growing after its source is gone.

Cumulative to delta is a subtraction against the previous value, with a reset rule. If the current value is less than the previous one, the process restarted, and the delta is the current value rather than the difference. If there is no previous value, the delta is the current value, which is only correct if the series was created at or after the previous collection interval. A series that appears for the first time with a large total, because the collector was restarted and lost its previous-value cache, will emit that entire total as one delta, producing a spike in the derived rate that did not occur in the workload.

That last case is the common operational failure of cumulative-to-delta conversion: the conversion state is fragile, and losing it converts a lifetime total into a single interval’s increment. A producer that emits deltas natively does not have this problem, which is why formats designed for aggregation-first pipelines default to delta and formats designed for scraping default to cumulative.

Temporality must be uniform within an aggregation

The rule that makes pipelines correct is that a single aggregation must see one temporality. Adding a delta report to a cumulative report is a category error, and the result depends on the magnitude of the cumulative value rather than on the events, so it can be arbitrarily wrong.

This is not only a mixing problem between producers. A collector that aggregates deltas and forwards cumulative output has created a boundary, and everything downstream of that boundary must treat the output as cumulative. A recording rule that reads cumulative series and writes delta series moves the boundary again. Each boundary is a place where the reset semantics change, and a pipeline with many boundaries is a pipeline where the temporality of any given series is not obvious from its name.

The practical discipline is to attach the temporality to the series metadata rather than to the pipeline stage, so that a consumer can check rather than assume. Formats that carry temporality as an explicit field make this possible; formats that do not leave the consumer to infer it from the values, which fails exactly in the cases where the inference matters: a new series, a restarted producer, or a gap.

Reset detection differs in one important way

Under cumulative temporality, a decrease is the signal, and a decrease can also be a genuine negative increment if the instrument is not monotonic. Under delta temporality, there is no decrease to observe, so a producer that emits a negative delta is reporting a real decrease rather than a reset, and a consumer that applies reset handling to it will corrupt the sum.

This is why non-monotonic sums are a distinct type in the data model rather than a flag on a counter. A gauge that happens to be a sum, such as the number of items in a queue accumulated across shards, can go down, and it must not be treated as a counter with resets. The type declaration is what tells the pipeline which rule to apply, and inferring the type from the values is not reliable: a counter that never decreases and a gauge that happens to be monotonic over the observed window are indistinguishable from samples alone.

Choosing a default

For a scraping collector that pulls from many short-lived targets, cumulative temporality is the natural fit, because each pull is independent and a missed pull costs one interval of resolution rather than one interval of data. For an SDK that pushes to a collector, delta is the natural fit, because the SDK knows when its own counters reset and can emit a correct increment at that moment without relying on the consumer to detect it.

A pipeline that spans both should convert once, at the point where the two meet, and should carry the resulting temporality explicitly from there onward. Converting more than once is not incorrect, but each conversion is a stateful operation with its own loss mode, and the losses do not cancel. The choice also interacts with the aggregation boundaries discussed in Aggregation Laws: Which Metric Operations Commute: a pipeline that sums before converting to delta must have a consistent view of which series exist, because a series that vanishes from the sum removes its entire history from the next delta.