Consumer-Driven Contract Testing and the Cost of Many Consumers

An integration test that starts both services proves that they work together today. It proves nothing about whether the next provider release will keep working, because the provider cannot run every consumer’s tests, and the consumer cannot see the provider’s changes before they ship. The gap is organisational as much as technical: each side knows half of the truth about the interface, and neither half is written down.

Consumer-driven contract testing closes the gap by moving the assertion to the consumer and the verification to the provider. The consumer declares what it needs from the provider in a form that can be replayed, and the provider runs those declarations against its own build in its release process. Failures then appear on the side that can fix them, before deployment rather than after.

The expectation only the consumer knows

A provider’s own tests assert that responses match its schema. That is not the same as asserting that responses satisfy any particular caller. Consumers typically use a fraction of the response: three fields out of twenty, one status code out of six, and a specific error shape on one path.

A contract records that fraction. It consists of interactions, each with a request the consumer will make and the response it expects, including the status, headers, and body. The body is usually matched loosely: a consumer that reads id and total_cents does not care what else is present, and a contract that demanded an exact match would fail on every additive provider change, which defeats the purpose.

{
  "consumer": { "name": "reporting-service" },
  "provider": { "name": "billing-api" },
  "interactions": [
    {
      "description": "fetch one invoice by id",
      "request": { "method": "GET", "path": "/invoices/INV-1" },
      "response": {
        "status": 200,
        "headers": { "Content-Type": "application/json" },
        "body": { "id": "INV-1", "total_cents": 4200, "status": "paid" },
        "matchingRules": {
          "body": { "$.total_cents": { "matchers": [ { "match": "integer" } ] } }
        }
      }
    }
  ]
}

The matching rules are the substance. A literal 4200 would make the contract a fixture, not a contract; an integer matcher says the consumer needs a number there and does not care which. Choosing the matcher is where the consumer states its real dependency, and getting it wrong produces either a brittle suite or a permissive one that misses changes.

The consumer side of the toolchain replaces the real provider with a mock that replays the contract, which lets the consumer’s tests run without the provider deployed. That is the immediate benefit. The verification benefit is larger and arrives later.

What the broker adds

Contracts are versioned artifacts that belong to the consumer, and they have to reach the provider’s build. A shared filesystem or a git submodule works for a small number of teams and fails as the count grows, because the provider then needs to know which contract versions are current and which consumers still matter.

A broker stores contracts keyed by consumer, provider, and version, and exposes them to the provider’s build for verification. Its real function is the mapping between contract versions, application versions, and deployments. That mapping is what allows a build to answer whether a particular provider build satisfies every consumer that is currently in an environment.

The broker also holds the answer to a question that teams forget to ask: who are my consumers. A provider that does not know its consumers cannot know which contracts should exist, and a consumer that publishes no contract is invisible. The registry of participants is as valuable as the contracts themselves.

Verification is not integration testing

During verification, the provider runs against the recorded requests. This is usually done with the provider’s real HTTP stack and a controlled data layer, or with a running instance in a test environment and seeded data. Two properties follow from that setup.

First, verification exercises the provider’s routing, serialisation, validation, and whatever data layer the test instance is wired to, but not its production data, its external dependencies, or its deployment. A change that breaks the response only when a particular record exists will not be caught. Contract tests are a fast, deterministic complement to integration tests, not a replacement.

Second, the provider must be able to satisfy the request. If a contract asks for GET /invoices/INV-1, verification needs that invoice to exist in whatever state the provider runs against. Provider states, the mechanism by which a contract names a precondition such as “an invoice exists with id INV-1”, exist precisely to solve this. The consumer declares the state, and the provider supplies a hook that establishes it. Without states, contracts either fail spuriously or drift into asserting on whatever data happens to be present.

The asymmetry that makes the pattern work is that verification is one-directional. The consumer never runs the provider. It records an expectation; the provider proves it can meet it. That inverts the usual coupling of integration tests, where every consumer would need the provider running, and it is why verification scales to many consumers while integration testing does not.

Deployment ordering and the can-i-deploy gate

Verification alone leaves a hole. A provider can pass every contract in the broker and still deploy a version that breaks a consumer whose latest contract has not been verified against that build. The gate that closes the hole is a query, run before deployment, asking whether a given application version is compatible with the versions of everything it depends on and everything that depends on it, in the environments that matter.

The answer depends on deployed versions rather than on published ones, which is the whole point. A consumer that has published a new contract but not yet deployed it does not block the provider. A consumer that is running against the live environment with an old contract does. Encoding “what is actually running where” is the broker’s harder job, and it requires the release process to record deployments rather than only builds.

When this works, provider deployment becomes safe by construction: the gate refuses to ship a version that any live consumer would reject. When it is missing, contract testing degrades into a suite of tests that produce warnings nobody acts on.

Where the pattern thins out

Contract testing works when consumers are known, few, and internal. Each of those conditions can fail.

Public APIs have consumers who will never publish a contract. For those, the contract is the documentation, and the compatibility discipline has to come from the provider’s own analysis of what changed, as described in breaking change detection in schemas.

Many consumers multiply the cost linearly. A provider with forty consumers maintains forty contract suites in its verification run, and every provider change is evaluated against all of them. The suite becomes slow and the failures become noisy, and the team starts excluding consumers from the gate. The usual mitigation is to consolidate: a few representative contracts per usage pattern, chosen so that the union of assertions covers the response surface. That is a coverage argument rather than a correctness one, and it should be stated as such.

Behavioural dependencies are out of reach. A contract asserts that one request produces one response. It cannot express “the status must change to overdue within a day of the due date”, or “two concurrent payments must not both succeed”. Those are properties of the system over time, tested by different means.

Stateful flows are awkward. A contract that requires a resource to exist before it can be fetched encodes an ordering assumption, and long flows become chains of contracts whose states are hard to establish independently.

Messages need the same discipline

Asynchronous interfaces have the same problem with an extra degree of difficulty: the consumer of a message is often unknown to the publisher, and the message is not a request-response pair. The contract becomes a message expectation, and verification becomes the publisher proving that the messages it emits satisfy the consumers’ declared expectations.

Two things change. The contract must describe the message envelope and payload, including the headers that carry routing and correlation, and the provider states become “a message of type X was published” rather than “a resource exists”. Everything else, including the deployment gate, applies with the same shape. The failure mode is also the same: publishers who never learn who subscribes to their topics cannot tell whether a change is compatible, which is an argument for a schema registry and a compatibility policy enforced at publish time rather than at deploy time, as covered in schema evolution and wire compatibility.

The honest cost

A contract suite is code owned by the consumer and run by the provider. It requires both teams to adopt tooling, a broker to run, release steps that record deployments, and a policy about what happens when the gate fails. For two services with one team, a single integration test is cheaper and nearly as informative.

The pattern earns its cost when the number of consumer-provider pairs grows past what a person can track, and when the alternative is discovering breakage after deployment. Below that threshold, the discipline worth keeping is smaller: consumers writing down which fields they read, and providers treating those notes as part of the interface. The writing down is the valuable part, and it can be done without any tooling at all.