Observability · SLO Practice
The SLI Ratio: Numerator, Denominator, and Measurement Point
Why an availability target fixes nothing until the fraction behind it is defined, and how moving the counting location changes the reported number.
An availability target of 99.9% is a claim about a ratio, and the ratio is where nearly all of the ambiguity lives. The number alone settles nothing. It does not say what counts as an event, which events are eligible, or where in the request path the counting happens. Two services can both report 99.9% over the same thirty days while one dropped a thousand requests and the other dropped none, because they counted different things at different places. The target is the last decision in the sequence, not the first.
A service level objective is useful only if the indicator beneath it is defined tightly enough that two engineers computing it independently arrive at the same number, and tightly enough that a change in the number means a change in the system rather than a change in the query. That is a stricter requirement than it sounds, because most of the interesting failure modes of an indicator are ambiguities that only surface when someone disputes a result.
A ratio has a numerator, a denominator, and a scope
Every event-based indicator has the shape good / valid, where the numerator is a subset of the denominator. Three definitions are needed, not one:
- Population. Which events the objective claims to cover: a set of endpoints, a tenant tier, a protocol version, a region. A ratio computed over all traffic is dominated by whatever endpoint has the most volume, which is rarely the one that matters.
- Criterion. What makes a single event good. Completion within a latency threshold, a status code outside the 5xx range, a payload that validates. For HTTP, the status code alone is a lossy criterion: RFC 9110 distinguishes client errors from server errors, and a 404 returned for a resource that should exist is a failure that no status-based numerator will catch.
- Boundary. Which events are excluded before the ratio is formed: health probes, synthetic checks, requests from clients on retired protocol versions, traffic rejected before authentication.
The boundary is where most disputes live, and it is the one most often left implicit. An exclusion list is a policy decision with the same force as the target itself, because removing a class of traffic from the denominator removes its failures from the budget.
Where the count happens decides the answer
The same request can be counted at the client, at the load balancer, at the ingress proxy, or inside the server, and each location observes a different set of events. The differences are not rounding.
A request that fails at DNS resolution never reaches any server-side counter. A request that completes the TCP handshake but fails during the TLS handshake is visible to the load balancer and invisible to the application. A request the load balancer rejects because no backend is healthy produces an error response with no corresponding application log line. A request the application answers in 2.5 seconds while the client gave up at 2.0 seconds is a failure at the client and a success at the server. A response truncated by a connection reset after the headers were sent is counted as complete by any instrumentation that records status at first byte.
These compose. If 0.2% of connection attempts fail during the TLS handshake and 0.05% of the requests that arrive receive a 5xx, a server-side indicator reads 99.95% while a client-side indicator reads roughly 99.75%. The second number is closer to what a user experiences, and the first is closer to what the service’s own code controls. Neither is wrong; they answer different questions. An indicator that does not state which one it is has not been defined.
The general rule is that the measurement point should sit as close to the user as the instrumentation allows, and the ownership boundary should be stated separately. Placing the count at the edge makes the number reflect user impact but makes it useless for attributing a regression to a component. Placing it inside the server makes attribution easy and hides every failure that prevented a request from arriving. Most mature setups keep both and treat the difference between them as a signal in its own right, since a widening gap between edge and origin error rates is itself an incident.
The exclusion list is policy, not housekeeping
Health probes are the canonical example. On a service with modest traffic, a liveness probe every ten seconds across a dozen instances contributes more requests per minute than real users do during a quiet period, and those requests essentially never fail. Including them raises the denominator, dilutes the error ratio, and lets a genuine outage hide inside a large population of synthetic successes.
The same applies to traffic that cannot be served correctly for reasons outside the service’s control: requests for API versions that were retired, clients that cannot negotiate a supported cipher, scanners probing for administrative paths. Excluding them is legitimate. The legitimacy depends on three properties, and each is testable:
- Small. The excluded set is a small fraction of the total, so the indicator still describes the system. An exclusion list that covers a third of traffic is a redesign of the objective in disguise.
- Enumerable. Each exclusion is a named rule with a reason, not a filter expression buried in a dashboard query.
- Stable. The rules do not change while an incident is in progress. An indicator whose denominator can be edited during the incident it is measuring is not a measurement.
The third property has a subtle consequence for stored data. If the exclusion is applied at query time rather than at collection time, then editing the rule retroactively changes historical indicators, including indicators that were used to justify a past decision. Applying exclusions at the point where the ratio is computed, and versioning the rule alongside the objective, keeps history honest.
Latency is also a ratio, and that choice has costs
Latency objectives are usually written as a ratio of counts rather than as a percentile: the fraction of requests served in under 300 milliseconds must be at least 99%. This form is chosen because a count ratio aggregates correctly across instances, while a percentile does not. Averaging per-instance p99 values produces a number that is not the p99 of anything.
The threshold itself becomes a hidden parameter. A threshold set at a latency the system cannot reliably meet produces a permanently exhausted budget and teaches everyone to ignore it. A threshold set far above typical behaviour produces a budget that never binds and therefore never constrains anything. The useful choice is a threshold near the point where user behaviour changes: the timeout in a client library, the point at which a page’s rendering blocks, the deadline a caller propagates.
Quantization matters too. When the latency distribution arrives as a histogram, the ratio is exact only at bucket boundaries. A threshold of 300 milliseconds against buckets ending at 250 and 500 milliseconds requires either interpolation, which assumes a shape inside the bucket, or a threshold moved to a boundary. Prometheus’s histogram documentation is explicit that interpolation is an approximation and that bucket layout determines the error. An indicator whose threshold sits just below a wide bucket’s upper bound can be wrong by a large factor in the fraction it reports.
What the arithmetic hides
A thirty-day window at 99.9% corresponds to 43 minutes and 12 seconds of complete unavailability if the budget is spent as time. The event-based budget is 0.1% of requests, and the two figures coincide only when traffic is flat. At ten requests per second, thirty days is 25,920,000 requests and the budget is 25,920 failures. A one-minute total outage at that rate costs 600 requests, about 2.3% of the budget. The same minute during a peak at ten times the rate costs 6,000 requests, about 23% of it. The event-based form charges for user impact; the time-based form charges for clock time. Neither is a bug, but mixing them in one document produces targets that cannot be met simultaneously.
What the ratio form cannot express is anything that is not a countable event. Partial correctness, degraded result quality, silent data loss discovered later, and failures that only appear when a client combines several responses all fall outside it. Those need their own indicators, usually with a different shape entirely, and the temptation to fold them into the availability ratio is what produces indicators nobody can explain.
The proxy nature of the ratio is worth stating as a limit rather than a caveat. A request that succeeded after four retries inside the client contributes one success to the numerator, and the retries themselves may never be recorded anywhere. A request served from a cache that had been stale for six hours is a success under an availability criterion and a failure under a freshness one. The ratio answers exactly the question its population and criterion name, and reading it as a general statement about quality is the same mistake as reading a packet-loss counter as a measure of network usability.
There is also a discrete counting problem whenever the criterion depends on a ratio of two measurements taken at different times. A denominator that counts requests started and a numerator that counts requests completed do not necessarily describe the same set of requests: a request in flight when the window closes is counted in the first and not in the second, and a request that completes after a long delay is counted in the second and not the first. Over a long window the discrepancy is negligible; over a one-minute window with long-running requests it is not, and it biases the reported ratio downward in a way that looks like an improvement as load increases.
The budget derived from this ratio is the subject of error budget units and burn rate, and the choice of window over which the ratio is formed is treated in objective windows and time equivalence. Both depend on the definitions above being fixed first. Whether the resulting number stays attached to the system it describes is a separate question, taken up in measurement drift and attribution error.
References
- Site Reliability Engineering, Chapter 4: Service Level Objectives — Google
- RFC 9110: HTTP Semantics — IETF
- OpenTelemetry Specification: Metrics Data Model — OpenTelemetry project