Release Gates: Verifying the Cut Before It Ships

A release gate is a check that runs between “the artifact exists” and “the artifact is published”, and whose failure prevents publication. The definition is narrow on purpose. Tests that run on every change are not gates; they are continuous checks whose failure is an inconvenience. A gate is the last check that can still stop something, which means its design has a different cost model: a false positive blocks a release, and a false negative ships a defect.

Everything else about gating follows from that asymmetry.

The checks that are actually mechanical

A gate has to be decidable by a machine from inputs it has, and the set of such checks is smaller than release process documents usually imply.

Existence and completeness. Every artifact the release claims to contain is present, and each has an expected digest recorded. A missing platform build is caught here rather than by a user on a platform nobody tested.

Coordinate availability. The version being published does not already exist in the target repository. This is the check that enforces immutability, and it can only be done against the registry at publish time, not earlier, since another process may have claimed the coordinate in between.

Manifest agreement. The declared version equals the version stamped into the artifact’s metadata, and equals the tag. Three places record the version, and they desynchronise regularly: the tag is applied to a commit, the build reads a manifest file, and the repository path encodes a directory name.

Lockfile stability. Re-resolving from the manifest in a clean environment produces the same resolved set as the committed lockfile. This detects both an unpinned dependency and a lockfile that was hand-edited into a state no resolver would produce.

Provenance presence. The attestation for the artifact exists, is signed by an expected builder identity, and names the digest being published.

Reproducibility. A second build from the same inputs on a different machine produces the same digest.

Each of these fails for a reason that names a specific repair. That property, not the check’s sophistication, is what makes a gate usable.

Reproducibility is the claim a second machine can settle

A single build produces a digest and no information about whether that digest is determined by the inputs. Building twice and comparing is the only direct test, and it is the one gate whose result cannot be faked by the build system reporting success. The artifact either hashes to the same value or it does not.

What breaks first is timestamps. Archive formats embed modification times, and a compiler that stamps the current time into an object produces a different binary each run. The standard remedy is a fixed epoch derived from a declared source — the commit timestamp is not stable across rebases, so a declared constant in the build definition is — plus normalisation of file modes and ordering.

Second is path dependence. Compilers and linkers embed absolute paths in debug information, and build directories differ between machines. Prefix remapping, or building at a fixed path inside a container, removes the difference.

Third is ordering. Directory traversal order, map iteration order in generated code, and parallel link order all vary between runs unless the producer sorts. A generated file listing symbols in hash order produces a different digest on every invocation even when nothing changed.

Fourth, and hardest, is the environment: compiler patch version, system headers, and anything fetched from the network during the build. An unpinned toolchain means the input set is not the same across the two builds, and the comparison measures the environment rather than the build’s determinism. Hermetic builds, where the build has no network access and a declared dependency set, reduce this to a checkable property.

The diagnostic value of the comparison is worth stating: when the two digests differ, the difference is a byte-level diff away from naming the varying input. diffoscope exists for exactly that, recursing into archives and object formats to report what differs. A gate that only reports “digests differ” wastes the most useful signal available.

Why a gate must not rebuild

The temptation is to make the gate build the release again and ship the gate’s output, on the theory that the gate’s environment is cleaner. That inverts the purpose. The artifact that was tested is the artifact that must ship, and a rebuild produces a second artifact whose verification status is unknown until it too is verified.

Where reproducibility holds, the two builds produce the same digest and the question is moot: either output is the same bytes. Where it does not, the rebuild tells you the build is not reproducible — which is a finding to fix, not a step to add — and shipping the second output means shipping something that no prior test saw.

The corresponding gate shape is: publish the byte string that was verified, and verify the claim that the byte string is reproducible, as a separate check. Both properties are testable and neither requires reordering the pipeline.

Failing without being bypassed

A gate that blocks a genuine release gets bypassed once and then permanently, because the bypass becomes the documented path. Three design rules reduce that pressure.

Separate blocking from reporting. Checks with a known false-positive rate, such as a flaky integration suite or a linter with style opinions, report and do not block. A gate list with one flaky entry trains everyone to ignore failures.

Make every failure name its repair. “Reproducibility check failed” is not actionable. “Rebuild produced sha256:a1b2… expected sha256:c3d4…, differing files: /usr/lib/libfoo.so.1 (see diffoscope output)” is. The gate’s output is the interface, and a gate whose output requires reading the gate’s source is a gate that will be bypassed.

Keep the check fast enough to run on every candidate. A gate that takes an hour is run once, at the end, when the pressure to ship is highest. A gate that takes minutes can run on every candidate build, which means failures are found while the author is still working on the change.

The complement is that a bypass must be an explicit, recorded action rather than an edit to the pipeline. A flag on the release record, requiring a second identity to approve, keeps the gate’s authority while admitting that judgement sometimes overrides it. What breaks is the silent bypass: a conditional in the pipeline that skips a step when a variable is set, which becomes the normal path and is invisible in the release record.

The boundary against tests

Gates check properties of the release as an object: existence, identity, agreement, provenance. Tests check properties of the software’s behaviour. The two are frequently conflated, and the conflation produces gates that duplicate test suites and test suites that assert release mechanics.

The reason the distinction matters is scope. A gate runs once per release; a test runs once per change. A property that can only be evaluated on the final artifact, such as “the published version string matches the tag”, has no meaning per change and belongs in the gate. A property that describes behaviour belongs in the test suite, where it runs hundreds of times before the gate is reached. Moving a behavioural test into the gate reduces its coverage while increasing the gate’s duration, which is a strictly bad trade.

There is a genuine middle category: checks on the assembled artifact that are cheap but only meaningful once. A binary’s dynamic dependencies against an allowlist, the absence of a debug build flag, the presence of a license file in the package. These belong in the gate because they describe the artifact’s composition rather than its behaviour, and they are the checks that catch a build configuration mistake that no unit test can see.

What the gate cannot establish

No gate determines whether the software is correct, and none determines whether a consumer’s environment will work. A gate establishes that the release is internally consistent and traceable: that the bytes are what was built, that what was built came from the recorded inputs, that the version agrees across every place it is written, and that nothing has been published over.

Those four claims are worth having because each corresponds to a class of incident that is otherwise invisible until it is expensive. Which artifact is running, which source produced it, which version it claims to be, and whether anything has changed underneath since. The last of these is where the gate meets the immutability rules described in immutable artifact coordinates and provenance, and the version agreement it enforces is the same promise that semantic versioning as a compatibility contract extracts from consumers.