Counter Resets and the Extrapolation Inside rate()

A counter records the total number of times something happened since the process started. The absolute value is almost never interesting; what matters is the increment over a time window. The trouble is that the instrument is a process-local variable, so it returns to zero whenever the process does. A query that subtracts the first sample from the last will report a large negative number at exactly the moment when traffic is most likely to be abnormal, which is when the reading is needed most.

Handling that correctly requires three separate mechanisms working together: a reset detector, a compensation rule that reconstructs the lost increment, and an extrapolation rule that stretches the observed samples to cover the full query window. Each one is an approximation, and each one fails in a different way.

The counter contract and its two violations

A counter is defined by a single invariant: successive readings from one series are non-decreasing. Everything downstream depends on it. Rate calculations assume it, aggregation assumes it, and alerting thresholds are written against rates that would be meaningless without it.

Two events break the invariant. The first is a restart, where the underlying storage is reinitialised and the value drops to zero or to whatever the process has accumulated since it came up. The second is a wrap, where a fixed-width integer overflows. Wraps are rare in practice because counters are stored as 64-bit floats in most collection formats, and a float64 represents integers exactly only up to $2^{53}$, after which increments of one are silently lost. A counter that increments a million times per second reaches that boundary in roughly 285 years, so the practical concern is restart, not overflow.

There is a subtler violation that the format itself permits: a counter that can decrease. Some instrumentation libraries expose a “counter” that also accepts negative deltas, or an “up-down counter” whose name suggests monotonicity it does not have. Such a series cannot be rate-converted at all, because a decrease is indistinguishable from a reset. The distinction is carried in the metric type, which is why the type has to be declared rather than inferred from the values. The OpenTelemetry data model separates monotonic sums from non-monotonic sums for exactly this reason, and the same separation appears in the exposition formats that predate it.

Reconstructing the increment across a reset

Given a window of samples $v_0, v_1, \dots, v_{n-1}$ taken at times $t_0, \dots, t_{n-1}$, the naive increment is $v_{n-1} - v_0$. The reset-aware version walks the samples and adds a correction at every point where the sequence decreases:

total = last.value - first.value
for i in 1 .. n-1:
    if samples[i].value < samples[i-1].value:
        total += samples[i-1].value

The correction term is the value just before the drop, not the size of the drop. That is the whole trick, and it follows from the assumption that the counter was at $v_{i-1}$ and then restarted, so the true increment across that step is $v_i$ (the amount accumulated since the restart), not $v_i - v_{i-1}$. Adding $v_{i-1}$ to the running total turns $v_i - v_{i-1}$ into $v_i$. The reconstructed total is therefore the sum of all post-reset values plus the final value minus the initial one.

The assumption embedded here is that at most one reset occurred between two adjacent samples. If a process restarts twice within one scrape interval, the intermediate value is invisible and the increment is undercounted. That is not detectable from the samples, and no amount of arithmetic recovers it. It is the reason short-lived batch jobs are poor candidates for counters: a job that lives for less than one scrape interval may produce no usable increment at all, which is a separate problem from resets and is why such jobs are usually instrumented with a histogram or a summary that survives the process.

A producer that knows when its own counter resets can avoid the whole problem by reporting increments rather than totals, which is the choice described in Delta and Cumulative Temporality in Collection Pipelines. The reconstruction described here applies to the form that reports totals.

The reconstruction also assumes the reset was to zero. A counter that resets to a non-zero value, for instance because the initial value is seeded from a persisted total, produces an increment that is correct only if the seeding happened at the same instant as the drop. Any gap between the two is attributed to the wrong window.

Why the result is scaled to the window edges

Samples almost never land exactly on the boundaries of the query range. A one-minute rate evaluated over the range $[12{:}00{:}00, 12{:}01{:}00]$ with a fifteen-second scrape interval typically has four samples inside it, the first a few seconds after the start and the last a few seconds before the end. The observed interval $t_{n-1} - t_0$ is therefore shorter than the requested window, and dividing the increment by the observed interval yields a rate that is correct for the sampled span but not comparable to rates computed over other windows with different alignment.

The fix is to extrapolate the increment to cover the missing edges. The published algorithm computes the distance from the first sample back to the window start and from the last sample forward to the window end, then extends the interval by those amounts, subject to two clamps:

• If a gap is longer than about 1.1 average sample intervals, it is not treated as a partial interval. The algorithm substitutes half an average interval instead, on the reasoning that a long gap means the series was absent or the scrape was missed, and attributing the whole gap to the counter would invent traffic that may not have occurred.
• For counters only, the back-extrapolation is additionally limited by the time it would take the counter to reach zero at the observed rate. If the first sample is small and the observed increment is large, the counter cannot have been running for the full distance back to the window start, because it would have had to start below zero. The limit is $t_{\text{span}} \cdot v_0 / \Delta v$, which is the zero-crossing time.

The scaling factor is then the extended interval divided by the observed interval, applied to the increment. The rate divides once more by the nominal window length. Both adjustments are multiplicative, so a rate over a window whose samples are all clustered at one end can be scaled by a factor noticeably larger than one.

This is where the most common surprise comes from: a rate computed over a range shorter than two scrape intervals has one sample or none, and the query returns nothing at all rather than zero. A range of exactly two intervals yields two samples and a very noisy estimate, because the reset detector has a single comparison to work with and the edge extrapolation can dominate the result. As a rule of thumb, a rate window should span at least four scrape intervals before its output is worth graphing, which is why the relationship between window length and sample spacing is worth understanding on its own; Scrape Interval, Lookback Delta, and the Resolution of a Rate works through the aliasing that appears when it is violated.

What extrapolation cannot fix

Extrapolation assumes the process behaved, on average, like it did during the sampled span. For a counter that rises in bursts, that assumption is wrong at every boundary. A service that receives a batch of requests every thirty seconds, sampled every fifteen, will show a rate that alternates depending on where the samples fall relative to the batches. The total over a long window converges, but any individual point estimate inherits the phase of the sampling.

The same applies to the reset correction. Adding the pre-reset value assumes the increment lost in the reset is zero, which is to say the counter was scraped at the instant of the restart. In practice the lost increment is up to one full scrape interval of traffic, and it is lost silently: the reconstructed total is always an underestimate, never an overestimate. Over many restarts the deficit accumulates as a systematic bias in the long-run rate, which matters when a counter feeds a billing or quota calculation rather than a dashboard. The bias is also one-directional, unlike the errors introduced by the aggregation step, which are discussed in Aggregation Laws: Which Metric Operations Commute.

Floating-point accumulation adds a second, smaller bias, and it is the same one that makes the on-disk encoding of these values a matter of bit layout rather than of arithmetic, as Delta-of-Delta Timestamps and XOR Float Encoding on Disk sets out. Summing many small increments into a float64 loses low-order bits once the running total exceeds $2^{53}$ in magnitude relative to the increment size, and the loss is not symmetric across aggregation paths. The mitigation is the same one used for any long sum: accumulate in integer types where the format permits, or use compensated summation when the values must be floats.

None of these approximations are optional refinements. A counter without reset handling is unusable across restarts, and a counter without extrapolation is not comparable across windows. The cost is that every rate carries a small, bounded, one-directional error, and knowing which direction it points is the difference between reading a graph correctly and reading it confidently wrong.