DevOps · Deployment Strategies

Feature Flags Versus Traffic Routing as Rollout Controls

Why a flag and a routing weight expose different populations, what it costs to keep both code paths in one binary, and how evaluation consistency and stale defaults cause failures.

Two mechanisms are commonly described as ways to release gradually. A routing weight sends a fraction of traffic to a new version. A feature flag lets one version contain both behaviours and decide per request which to execute. They look interchangeable in a dashboard and are not: they expose different populations, they fail in different ways, and they leave different amounts of work in the codebase when the rollout is over.

The distinction that matters operationally is where the decision is made. Routing decides the destination of a request before any application code runs, so a request that hits the old version exercises no new code at all. A flag decides inside the application, after the request has been routed to an instance that necessarily contains the new code.

Two different geometries of exposure

Routing exposure is proportional to traffic. A weight of 10 means roughly a tenth of requests reach the new version, and the blast radius of a defect is bounded by that fraction. Since the two versions are separate processes, they also have separate resource footprints, separate logs, and independent failure domains: if the new version panics on start-up, only its own share of traffic is affected.

Flag exposure is proportional to the code paths that the flag guards. If the flag wraps a new report generation routine that runs for one request in five hundred, then enabling it for everyone affects a two-hundredth of requests, whatever percentage of users the flag targets. Flags also allow targeting that routing cannot express cheaply: an internal-account list, a single tenant, a specific request attribute, a user in a particular region. Conversely, a flag gives no protection against defects outside the guarded region — a change in shared initialisation code that the new version ships will affect every request the new binary serves, exactly as it would if the version had been rolled out by weight.

Because a flag’s exposure is a property of the code, not of the deployment, the two mechanisms compose. A canary rollout gives a small fraction of traffic to the new version, and a flag inside that version selects an even smaller fraction of requests for the risky behaviour; the risk surface is the intersection. Systems that use both tend to route first (which gives a cheap abort, since the old version is still running) and enable flags later, once a version is known not to be broken in its common paths.

Both code paths ship in the same binary

The cost of a flag is that the disabled path is still deployed. It is compiled, it is present in the artefact, and it is reachable if the flag changes its mind. That has three consequences that routing does not have.

Testing must cover both states, and in a service with several flags the states multiply: ten independent flags define a thousand combinations, and the tested set is a handful. The practical discipline is to keep flags independent and short-lived rather than to attempt exhaustive coverage, and to test the disabling path explicitly, because the path taken by most requests during a rollout is the one that is least exercised in the author’s development environment.

Resource use is the sum of both paths in some cases. Initialisation that happens at start-up regardless of the flag (registering a handler, opening a connection pool for a subsystem only the new path uses, loading a model or a template) costs every instance even while the new behaviour is disabled. A flag cannot defer that cost; only a different binary can.

Rollback changes character. Reverting a flag is a configuration write, effective within the flag system’s propagation delay, and requires no new artefact, no capacity for a second fleet, and no image pull. Reverting a deployment means shifting traffic back to instances that must still be running, with all the capacity and warm-state considerations described in blue-green cutover mechanics. Against that, a flag rollback leaves the new code running, so any damage the new code does outside the guarded region is unaffected by flipping the flag.

The taxonomy in common use separates flags by longevity and by who controls them: release flags that live for the duration of one rollout, operational toggles that stay permanently as kill switches, experiment flags whose whole purpose is comparison, and permissioning flags that express entitlement. The categories have different lifetimes and different standards for removal, and conflating them is how flag systems accumulate dozens of permanently enabled toggles nobody dares delete.

Evaluation must agree everywhere

A flag’s decision is a function of the flag key, the rules configured for it, and an evaluation context — the attributes of the subject, typically a user or account identifier plus request metadata. For percentages and consistent targeting, implementations hash the identifier together with the flag key and map the hash space onto the allowed variations. This gives stickiness: the same subject receives the same variation on every request, on every instance, without shared state.

The hash input is the source of subtle inconsistency. Two services that evaluate the same flag must agree, and they will not if one passes a user identifier and the other passes a session identifier, or if their hash implementations differ in seed or in which fields are concatenated. The observable failure is a client that sees the new user interface while the API behind it rejects the request because its own evaluation of the same flag said the feature was off. Passing the identifier and the attributes as part of the request context (a propagated header is the usual mechanism) avoids the divergence, at the cost of making the context part of every service’s contract.

Where evaluation happens matters as well. Remote evaluation through a central service makes rule changes immediate and centralises the logic, but puts a network hop and an availability dependency in the request path. Local evaluation with rules distributed to the SDK removes the hop and keeps working during a control-plane outage; it introduces a propagation delay and a stale window during which different instances use different rules. The two behave differently in exactly the situation where flags matter most (a flag flip during an incident) because remote evaluation is one write and local evaluation is a write with an asynchronous fan-out whose completion cannot be observed from the control plane.

Defaults decide what happens when the flag system is unavailable

Every flag has a value for the case where the rule set cannot be fetched: the SDK’s configured default, or the value compiled into the call site. That default is a policy decision that is usually made accidentally.

Failing closed (disabled) is right for a flag guarding a new behaviour, since the previous behaviour is the known-good one. Failing open (enabled) is right when the flag guards the old path — the shape that emerges when a flag is retrofitted to preserve the previous implementation while the new one becomes the default, at which point the flag’s disabled state is the untested branch. The same flag can be wrong in both directions over its lifetime, which is why the release-flag lifecycle ends with deletion rather than with leaving the flag enabled.

A flag that gates a destructive or irreversible operation is a special case: the default must be the safe state regardless of the flag system’s health, and the operation should be structured so that being wrong at the boundary is recoverable. The set of operations that cannot be undone is the subject of rollback boundaries and irreversibility.

Where the two mechanisms disagree about correctness

A final distinction is the one that decides which tool is appropriate. Routing tests a version; a flag tests a code path inside a version. A version can be wrong in ways that no flag exposes: incompatible serialisation on the wire, a changed resource footprint, a migration executed at start-up, a dependency version bumped. A code path can be wrong in ways that no routing exposes, because the new path may only be reachable for certain inputs, certain tenants, or after certain state accumulates, and the traffic that reaches it during a canary share may not contain those inputs at all.

That difference is why flags are the better instrument for a change that is only meaningful for a subset of subjects, and routing is the better instrument for a change whose risk is in the binary itself. Where both are used, the health signals that gate progress must be computed per version and per flag state, or the rollout machinery will be measuring the population average of two decisions — a measurement problem taken up in health signals for rollout gating.

References

  1. Feature Toggles (aka Feature Flags) — Pete Hodgson, Martin Fowler
  2. OpenFeature Specification — Cloud Native Computing Foundation
  3. Kubernetes Documentation: ConfigMaps — Kubernetes project
  4. RFC 9110: HTTP Semantics — IETF