Compatibility Promises as Borrowed Time

An internal function can be changed by changing its callers. A published endpoint, an exported library function, or a message format on a queue cannot, because its callers are outside the boundary of the change and their schedules are not negotiable. The moment that boundary is crossed, the cost of an edit stops being “update the call sites” and becomes “support both shapes, indefinitely, while asking everyone to move”.

That is the essential character of compatibility debt: it converts principal into a permanent stream of interest. Nothing is broken, but every future change to the contract must now be designed around the set of consumers who cannot or will not change. The obligation is often worth taking, since the alternative is a system nobody can integrate with, but it should be taken knowingly, because the interest is charged on every subsequent change to that surface.

What makes a change breaking

The intuition that a breaking change is “one that removes something” is too narrow. A change is breaking if any consumer that satisfied the previous contract can now fail or silently misbehave. Under that definition, several additions qualify.

A new required field. The producer adds a field it requires consumers to echo. Every existing consumer sends payloads without it. Rejection is immediate and total.

A new value in an enumeration. Adding a member looks additive, and is additive for consumers that treat unknown values as a fallback. Consumers that switch exhaustively on the value — a common pattern in languages with exhaustive matching, and a common pattern in hand-written dispatch tables — now have an unhandled case. Whether this breaks depends entirely on consumer behaviour.

A narrowed range or a raised limit. Accepting amounts up to a new maximum is compatible; rejecting amounts below a new minimum is not, even if no legitimate client sends them, because the contract said they were valid.

A type change in a serialized format. Integers becoming strings, or a single object becoming an array of one, breaks every typed client. In JSON there is no schema enforcement at the transport layer, so the break appears as a deserialization error or, worse, as a silent coercion.

A changed default. Changing the default page size, the default timezone, or the default sort order leaves the syntax intact and changes the results. Consumers rarely notice until an audit does.

A changed error contract. A new status code for a condition that previously returned a different one, or a field renamed inside an error body, breaks error handling that was written against the old shape.

The operational test is mechanical and worth running before every release: take the previous contract, enumerate every consumer-visible property, and for each one ask whether some client that was correct before is wrong now. Doing this by inspection fails predictably, which is why schema diffing tools exist; the tool’s rule set is a codification of the list above.

Client tolerance decides the server’s freedom

The power to make additive changes lives on the consumer side, not the producer side. A producer can only add fields if consumers ignore unknown fields, and many consumers do not. A strict decoder that rejects unexpected properties — the default in several serialization libraries under strict settings, and the natural result of generating types from a schema — turns every addition into a break.

The tolerant reader pattern is the consumer-side discipline that keeps evolution cheap: accept unknown fields, preserve them where the object is passed through, ignore unknown enum values, and never depend on field ordering or on the presence of optional fields. Its cost is real. Bugs caused by a misspelled field name become silent rather than loud, because the value is dropped instead of rejected, and the consumer proceeds with a default. That trade — compatibility bought with silent misconfiguration — is worth naming explicitly, because it is the reason strict validation exists.

import json

def merge_update(stored: dict, patch: dict) -> dict:
    """Patch known keys, leave unknown ones untouched so newer fields survive a round trip."""
    known = {"name", "quantity", "unit_price"}
    out = dict(stored)
    for key in known:
        if key in patch:
            out[key] = patch[key]
    for key, value in patch.items():
        if key not in known:
            out[key] = value          # preserve, do not interpret
    return out

Nothing in that function protects the consumer from a field it should have understood and did not. The protection comes from a test that fails when the producer adds a field the consumer was supposed to handle, which is a different mechanism, and one that has to be maintained deliberately.

Versions, and what a version number buys

Version identifiers are often treated as the compatibility mechanism. They are only a label; the mechanism is that different versions can coexist. A path-segment version (/v1/, /v2/) makes coexistence trivial and doubles the surface to maintain. A media-type version does the same with more precision per representation. A package major version lets consumers pin, which pushes the cost onto them and creates the long tail described below.

Semantic versioning formalises the contract: a major version change signals incompatible API changes, a minor version signals backwards-compatible additions, a patch signals backwards-compatible fixes. The useful part is the commitment that follows from it — if a project follows the rule, consumers can take minor updates without review. The part that is frequently missed is that the rule is about the API as consumers observe it, not about the API as the maintainer understands it. A bug fix that changes an observable output for inputs someone depended on is a major change under the rule, whether or not it is a fix.

Language ecosystems make the trade at different points. Go’s compatibility promise holds the language and standard library stable across the 1.x line, which means the cost of a mistake in a public API is paid by keeping it; the go directive plus module versions let a project depend on two major versions of the same module simultaneously, at the price of distinct import paths. Python’s PEP 387 sets out a deprecation process with warnings over multiple releases before removal, which spreads the interest over time and gives consumers a migration window. Both are policies about how much interest the maintainer is willing to pay to keep the ecosystem stable.

Hyrum’s law and the surface nobody declared

The declared contract is smaller than the real one. Any observable behaviour will eventually be depended on by somebody: a particular ordering of fields in a response, a specific error message string, the fact that a lookup is fast for small inputs, the fact that two calls in sequence behave differently from the same two calls in reverse. The mechanism is selection: consumers are written against whatever they observe, and the observations that happen to work get embedded in their code.

The practical consequence is that the compatibility surface grows without anyone deciding that it should. There is no clean remedy, only containment: keep observable behaviour narrow and stable in the parts that are cheap to keep stable, use opaque identifiers rather than exposing internal ones, avoid leaking database column names into responses, and prefer explicit ordering to incidental ordering. Every one of those choices reduces what a consumer can accidentally depend on, and none of them eliminates the phenomenon.

Where the law bites hardest is in serialized structures that mirror an internal model. Exposing the internal representation makes the internal model part of the contract, and then a refactoring that is purely internal becomes a breaking change. This is the same mechanism that makes stored data shapes expensive to change, examined in Data Shape Debt and the Migration That Outlives the Code.

The long tail and the deprecation accounting

Compatibility obligations are not uniform across consumers. A small number of active consumers track releases and migrate within weeks. A long tail of pinned clients, embedded integrations, and unreachable installations stays on old versions for years, and the tail determines how long the old path must be supported.

The accounting question is concrete: for each deprecated behaviour, how many requests still exercise it? Instrumentation answers this, and the answer changes the decision. A deprecated field with no observed use for a full reporting cycle can be removed after a warning period, because the remaining risk is bounded. A field with steady use from consumers who cannot be identified is a permanent obligation, and the honest response is to keep it and stop pretending it will go away.

Deprecation has a cost of its own that is easy to miss: a warning emitted to callers is noise they cannot act on if they do not control the caller, and warnings that are ignored train people to ignore warnings. Announcements, changelogs, and telemetry are more effective than runtime warnings for the part of the tail that reads anything at all.

When the obligation is worth cancelling

Two conditions justify breaking compatibility rather than paying interest forever. The first is that the old behaviour is unsupportable for a non-negotiable reason: a security defect that cannot be fixed within the contract, a dependency that no longer exists, a protocol the counterparty has withdrawn. The second is that the old behaviour has no remaining consumers, in which case removal is not a break at all but a deletion.

Everything else is a judgment about rates. Maintaining a second version costs a constant fraction of the surface’s development capacity, plus the cognitive cost of remembering which version a given fix belongs in. Against that, forcing migration costs a concentrated hit to every consumer at once, including consumers whose own change budgets are already committed. The usual resolution — run both, instrument the old path, remove when the traffic reaches zero or the support window closes — is a way of paying interest deliberately for a bounded period rather than indefinitely.

The worst outcome is neither choice but drift: a version that exists in name only, with fixes backported inconsistently and no record of which consumers are on it. At that point the compatibility debt is indistinguishable from the structural problem described in Divergent Duplicates and the Cost of Two Truths, because the two versions have quietly become two implementations of a rule that nobody owns.