DevOps · Platform Automation

Versioning the Platform Interface Under Existing Tenants

How a self-service interface changes without breaking its consumers: explicit versions, compatibility guarantees, and why the parts nobody uses break first.

A self-service platform’s interface is the set of types a tenant writes: the custom resource schemas, the parameters of the service template, the policy language the platform accepts, the labels and annotations the platform reads. Each of those is a contract, and the tenants are the counterparties. Changing a contract that has counterparties is a compatibility problem, and the platform’s difficulty is that its counterparties are not a single versioned client library but a population of repositories, each pinned to whatever the template produced on the day it was created.

The tooling for this is mature. The discipline is not. Versioning fails in practice not because the mechanics are hard but because the platform never decides which parts of its interface are part of the contract.

The contract is what tenants can write, not what the platform reads

The useful question for a versioning policy is: can a tenant write something that the platform will accept, such that the platform’s behaviour changes without the tenant changing anything? If yes, that something is part of the interface and changing it is a compatibility event.

Labels and annotations are the classic example. They are unversioned strings that tenants set, and the platform reads them to decide behaviour: which namespace gets a sidecar, which deployment gets a larger request limit, which repository is owned by which team. A platform that renames one of these labels, or changes the value it recognises, has changed behaviour for tenants who wrote the old value, without any schema change to warn them. The schema version number did not move, and nothing in the tenant’s world reported a deprecation.

The same applies to policy. A platform that accepts Rego policies from tenants has made the policy language and the set of data documents it exposes part of the interface. Removing a data attribute, or changing the meaning of an input field, breaks tenant policies that reference it, and the breakage appears as a rule that stops matching rather than as an error. This is the undefinedness behaviour described in policy evaluation semantics, and it means a policy-language compatibility break is often silent.

The inverse question matters too: which parts of the platform are not the contract. The platform’s internal resources, the shape of its own storage, the implementation of its controllers — these change freely, and the versioning policy should say so, because otherwise every internal refactor is argued over as if it were a compatibility event. Drawing the line explicitly is what makes a versioning policy operable rather than aspirational.

The three guarantees a version can offer

A compatibility statement for a versioned resource usually promises a subset of the following, and the subset should be written down.

Schema compatibility. Fields are not removed or re-typed within a version. New fields may be added, and a tenant who sets an unknown field in a request has it rejected rather than silently dropped. This is the guarantee that lets a tenant’s old manifests keep working against a new server.

Semantic compatibility. The same declaration produces the same outcome. A field that changes meaning without changing shape — a limit that is now enforced where it was advisory, a label that now selects where it did not — is a semantic break and cannot be detected from the schema alone.

Conversion compatibility. When a resource has multiple versions, the API server must be able to convert objects between them without losing fields. Kubernetes makes this concrete with a rule that no field may be dropped in a conversion from a newer version to an older one: the API server refuses a write that would lose data. This is what makes “the tenant writes v1, the storage is v2” safe, and it is the property that most platforms, having not implemented conversion, instead avoid by keeping a single version alive for a long time.

The choice a platform makes is usually between two of these: it can offer semantic stability within one long-lived version and make schema changes by introducing a new version, or it can offer schema stability and accept that semantic changes will be made in place with a deprecation warning. The first is more honest; the second is more common, because it requires less machinery.

Deprecation is a scheduled incompatibility

A field or a version that the platform intends to remove goes through a declared sequence, and the sequence has to include a date or it is not a promise. The pattern is standard: announce the deprecation, keep the old form working through a release boundary, emit a warning when it is used, then remove it in a release that follows the announced minimum interval.

The warning is the part that is usually missing, and it is the part that makes the schedule real. A tenant cannot migrate away from something they were never told they were using. Emitting a warning at the point of use, in the request response or the decision log, turns “the field stopped working” into “the field warned for six months and then stopped working”, which are different experiences with different remedies.

The removal itself is a behaviour change, and the same reasoning about labels applies: removing a field from the schema is visible as a rejection, while changing the value a label recognises is silent. Both should go through the same deprecation sequence, but only the first is enforced by the type system. The second has to be enforced by policy, which is what admission-time evaluation provides: a rule that rejects the old label value, with a message naming the new one, converts a silent semantic change into an explicit migration.

The obligation that follows from deprecation is the counterparty’s burden. A platform that announces a deprecation has not migrated anyone; it has moved the work to the tenants, and the tenants are the people the platform exists to serve. The honest version of a deprecation policy includes an estimate of how many tenants still use the old form and a plan for the long tail, because the long tail is the part that blocks the removal.

Why the rarely used parts break first

The parts of an interface that break are not the parts that are used often; those have tests, dashboards, and tenants who notice. The breaks happen at the edges: the resource kind that one team uses, the annotation that a legacy migration script sets, the field that the template writes and no human has ever read.

The mechanism is the absence of a consumer who would report the break. A change to a heavily used field is caught by the first tenant who deploys. A change to a field used by one repository that deploys quarterly is discovered a quarter later, in a context far removed from the change that caused it, and attributed to the wrong thing. The platform’s testing cannot help here, because the test would have to know that the field was used at all.

This is the argument for treating every tenant-written value as part of the contract until proven otherwise, and for keeping the inventory of what is actually used. A label search over the tenant repositories, a decision-log query for which policy data attributes are referenced, a count of which template parameters are non-default — these are cheap to run and turn a blind compatibility change into an informed one. The inventory is not a substitute for the versioning policy; it is the input that tells the policy which removals are real.

The discipline cuts the other way on the platform’s own side. The platform’s templates, its default policies, and its sync configuration are all consumers of the same interface, and they are often the last to be updated. A platform that deprecates a field in its tenant-facing schema while its own template still writes the field has not completed the deprecation; it has created a situation where the platform’s own scaffolding produces warnings, or produces nothing after the removal.

The end state: a guarantee that can be stated

A platform that has done all of this can state its guarantee in one sentence: a tenant declaration written against version N of the interface continues to be accepted and to mean the same thing for the lifetime of version N, and the platform will announce a removal a minimum of so many releases before it happens. Everything else — the schema tests, the conversion rules, the deprecation warnings, the inventory — is machinery that makes the sentence true.

The sentence is worth writing down because it is the platform’s side of the implicit bargain that made self-service work in the first place. The tenant accepted the template’s generated configuration, and the ownership of that configuration, on the assumption that it would keep working. Template ownership is the mechanism that keeps generated code current; versioning is the mechanism that keeps the contract behind it stable. When the two disagree, the failure is usually visible as a tenant who stops upgrading, and the reason they stopped is that an upgrade once broke them. The versioning policy exists so that the platform can say, with evidence, that the next upgrade will not.

References

  1. Kubernetes Documentation: API Versioning — Kubernetes project
  2. Kubernetes Community: API Conventions — Kubernetes community
  3. Kubernetes Documentation: Deprecated API Migration Guide — Kubernetes project