Multiwindow Burn-Rate Alerts and the Short Window as a Reset

A burn-rate rule has one threshold and one window, and the window controls two things at once. It controls how much evidence the rule requires before firing, and it controls how quickly the rule clears after the condition ends. A long window gives stable evidence and slow recovery. A short window gives fast recovery and noisy evidence. Setting one value forces a choice between paging on brief blips and leaving a resolved alert firing for an hour after the service has recovered.

The construction that avoids the choice evaluates two windows of different lengths against the same threshold and fires only when both exceed it. The long window supplies the evidence that the budget is genuinely being spent at a dangerous rate. The short window supplies the current state of the service, so the alert clears as soon as the short window’s contents have passed out of it.

What each window contributes

Take a rule requiring a burn rate of 14.4, which over a thirty-day window corresponds to spending 2 percent of the budget in one hour, a figure derived in Error Budgets and the Burn Rate That Consumes Them. Evaluated over a one-hour window, the rule fires when the cumulative error ratio over the past hour reaches 14.4 times the budget ratio. That is a strong statement about the last hour as a whole, and it is also the reason the rule is slow: the first minute of an incident contributes one sixtieth of the evidence, so an incident that starts at full severity takes most of an hour to bring the one-hour burn rate to the threshold.

Evaluated over a five-minute window instead, the same threshold is reached after five minutes of the same severity. The measurement is real, but it is also fragile. Five minutes of a thirty-day budget is a very small sample of the traffic, and any short burst that happens to concentrate failures clears the threshold. A rolling deployment that briefly fails a portion of requests, a cache flush, or a single slow dependency can all produce a five-minute burn rate above 14.4 without any sustained degradation.

The pair resolves this because the failure modes of the two windows are different. Before an incident, the long window holds the historical average and stays below the threshold, so a transient burst in the short window cannot fire the rule alone. After an incident ends, the short window empties within five minutes, so the alert resolves promptly even though the long window still holds the incident’s errors. The long window answers whether this is serious; the short window answers whether it is still happening.

Budget consumed	Long window	Short window	Burn rate
2%	1 hour	5 minutes	14.4
5%	6 hours	30 minutes	6
10%	3 days	6 hours	1

Each row is a policy about how much budget is allowed to disappear before the service pages for it. The first row catches a severe failure quickly and spends 2 percent of the month to do so. The third row catches a slow drift that would consume a tenth of the month before it fired, and it is the row that catches the failures too small to notice in any single hour.

Writing the rule as a conjunction

The two conditions are separate PromQL expressions whose results must both be non-empty for the alert instance. The following is the body of a rule file, and the labels on the expression carry the interface and the objective into the alert:

groups:
  - name: slo-burn
    rules:
      - alert: CheckoutErrorBudgetFastBurn
        expr: |
          (
            sum(rate(requests_total{job="checkout", code=~"5.."}[1h]))
            / sum(rate(requests_total{job="checkout"}[1h]))
            > (14.4 * 0.001)
          )
          and
          (
            sum(rate(requests_total{job="checkout", code=~"5.."}[5m]))
            / sum(rate(requests_total{job="checkout"}[5m]))
            > (14.4 * 0.001)
          )
        for: 2m
        labels:
          severity: page
          slo: checkout-availability
        annotations:
          summary: Checkout is burning its availability budget at 14.4x

The and operator retains the left-hand series only where a matching right-hand series exists, so the output is the intersection of the two conditions. Both sides carry the same labels, which is what makes them matchable. A subtlety that produces silent breakage: the two expressions must return the same label sets, otherwise the intersection is empty and the rule never fires. Adding a label to one side and not the other removes every matching pair.

The for duration is still useful with two windows. It guards against the case where both windows cross the threshold in the same evaluation and then fall back immediately, which is possible when the short window is short relative to the evaluation interval.

What the conjunction does to false positives and to detection

Independence between the two windows is not available, since the short window’s samples are a subset of the long window’s. The probabilities cannot be multiplied, and any argument that treats them as independent overstates the reduction in false positives. What the pair actually buys is a different requirement: the condition must be present in the recent past and in the accumulated past simultaneously. A burst that is over before the long window moves cannot satisfy both. A sustained condition satisfies both almost immediately after the short window has filled, so detection is close to the short window’s length rather than the long window’s.

The cost appears in a specific scenario. Traffic that alternates between healthy and failing on a period shorter than the long window can keep the long window’s burn rate above the threshold while the short window repeatedly empties. Such a service spends a large part of its budget in aggregate and never pages, because the two windows are never simultaneously over the line. This is the intended behaviour of the short window as a reset, and it is a miss rather than a false positive. A rule that must catch it needs a third row with a long short-window, which is what the 3-day and 6-hour pair is for.

The second cost is resolution behaviour that surprises responders. Because the long window still holds the incident when the short window clears, the alert’s evidence is asymmetric at resolution: it clears on five minutes of data, while the condition that fired it required an hour. A responder who checks the one-hour graph after a resolution sees a burn rate still above the threshold and reasonably asks why the page cleared. The answer is that the rule’s firing condition was a conjunction and only one conjunct failed.

Choosing the shorter window

The short window must contain enough samples to make its own measurement trustworthy. With a fifteen-second scrape interval, five minutes is twenty samples per series, which is workable; with a scrape interval of one minute, five minutes is five samples, and the ratio computed from five samples can be dominated by a single unlucky request. The rule of thumb is the short window should be at least four scrape intervals, and larger where the traffic is bursty.

The short window must also not be shorter than the rule’s evaluation interval. If the rule is evaluated once a minute and the short window is thirty seconds, then most evaluations recompute the same single sample and the two-window construction degenerates: the short condition becomes a function of the last sample rather than of recent history. Setting the short window to a small multiple of the evaluation interval is the simplest guard.

The third constraint is the relationship between the short window and the page’s purpose. A short window that is too long reintroduces the slow resolution the pairing was meant to fix, and a short window that is too short makes the conjunction fire only on conditions that are still fully active at the moment of evaluation, which delays detection of conditions that fluctuate. The useful range is roughly five to thirty minutes for a paging rule, with the long window at least ten times longer.

Interaction with the alert state machine

The two-window rule interacts with alert state in a way that single-window rules do not, because its firing condition can be satisfied by an instantaneous configuration rather than by a trend. The for duration, the router’s grouping and repeat timers, and the alert’s own pending state all consume the same wall-clock time, and their sum is the minimum time between the onset of a condition and a human learning about it. The next question in the chain, how the state machine turns a persistent condition into a page and a cleared condition into a resolution, is the subject of Alert State, Hysteresis, and the Cost of Flapping.

Two further variants are worth knowing. The first replaces the short window with a “current” window of one or two evaluation intervals, which makes the alert behave almost like an instantaneous test and is appropriate only for conditions that never fluctuate. The second adds a third rule at a much higher burn rate with no long window at all, intended for conditions that are catastrophic from the first second, such as a total outage; that rule accepts false positives in exchange for sub-minute detection, and it is the only place where that trade is justified.