DevOps · Build Pipelines

Trust Boundaries in Shared Build Caches

A cache entry written by an untrusted build is an input to a trusted one. How key collisions become code execution, why content addressing does not help, and how namespaces separate the two.

A build cache is usually described as an optimization. That description is accurate about intent and wrong about effect. A cache hit replaces the execution of a command with a fetch of its supposed result, and the fetched bytes then enter the build exactly as if the command had produced them. If someone other than the intended producer can write a cache entry that a later build will look up, the cache is a code injection channel into every build that hits it.

The lookup criteria are the attack surface

A cache lookup succeeds on a key match. The key is derived from the inputs the build system believes the action has, which is the same information an attacker sees. An attacker who commits a source file that is an input to a privileged action knows that action’s key, and therefore knows which entries will be looked up when the trusted build runs.

Producing a colliding entry requires no cryptographic break. The attacker arranges for the same key to describe a different output. Since the key is computed from declared inputs, and the trusted and untrusted builds are running the same declared inputs — that is exactly why the keys match — the attacker needs the output to differ while the key does not. The simplest way is an under-declared input: an action that reads something the key does not cover. A build step that consults an environment variable the key omits, a tool resolved through PATH, a file outside the declared list, or a service on the network all provide a free variable that the attacker controls in their own build and that the key will not notice.

The result is that the trusted build skips its own action and uses the attacker’s output, then links, signs, and publishes it. Nothing in the mechanism is broken. The cache did what it was told; the key was incomplete.

Content addressing does not prevent this. Verification that a retrieved blob matches the digest it is stored under proves only that the blob was not corrupted in transit. The digest came from the cache entry the attacker wrote, so the blob matches it perfectly. What the build needs is the answer to a different question: whether the entry was produced by a build with the same inputs under the same trust assumptions as this one. Content hashes cannot answer it, which is why integrity and authenticity are separate properties and why the provenance mechanisms in Artifact Provenance from Commit to Deployed Digest exist.

Which builds are untrusted

The dividing line is not “our code” versus “their code”. It is whether a person with commit rights has reviewed what the build will execute.

Builds of the default branch, of a release tag, or of any revision a maintainer pushed are trusted, because a maintainer decided to run them. Builds of a proposed change from someone without commit rights are untrusted, and this includes most branch builds in a project that accepts external contributions. Continuous integration systems differ in the details of what secrets and credentials such builds receive, but the cache is available to them by default in most designs, because the cache is keyed on content and the content comes from the same repository.

Builds of a proposed change from a branch inside the repository are the ambiguous case. The author may have commit rights and the branch may not be reviewed. Whether it counts as trusted is a policy decision, and the safe default is to treat anything that is not the default branch or an explicit release reference as untrusted.

There is a second, subtler category: a trusted build whose inputs include something an untrusted build controls. A build of the default branch that consumes a dependency published from the project’s own untrusted pipelines is in this category, and so is a build that merges an untrusted branch into a trusted one before running.

Scoping the cache mechanically

There are four mechanisms for separating trust levels, and they compose.

Separate stores. Untrusted builds get a different cache instance entirely, with its own credentials, and entries never cross. This is the only mechanism with no shared-key surface, and its cost is the loss of sharing: a dependency rebuild never warms the trusted cache, so trusted builds pay for the dependency every time. For projects where the expensive dependencies are in the trusted path anyway, the cost is often acceptable.

Namespaced keys. The key includes a namespace derived from the trust level, so the same inputs under different trust levels produce different keys and cannot collide. The namespace must be derived from something the untrusted build cannot spoof — the identity the orchestrator assigns, not a value from the repository. A namespace taken from a branch name or a file in the checkout is attacker-controlled and provides no separation.

import hashlib

def scoped_key(action_key: str, trust_domain: str) -> str:
    # trust_domain comes from the orchestrator's record of who is running the
    # build, not from anything the build itself can write.
    h = hashlib.sha256()
    h.update(trust_domain.encode())
    h.update(b'\x00')          # separator: no two pairs concatenate alike
    h.update(action_key.encode())
    return 'scoped:' + h.hexdigest()

Read-only untrusted access. Untrusted builds may read the trusted cache and may not write to it. This preserves the useful direction: a dependency built once on the trusted path is available to proposals. It still allows the reverse leak described below, and it requires the storage layer to enforce read-only access for untrusted credentials rather than relying on the build to behave.

Separate keys per trust level with promotion. An entry produced by an untrusted build is written under a distinct key, and a verified rebuild on the trusted path promotes the same content to the trusted key. The promotion step is where verification happens, and it costs a rebuild unless the two builds are known to be equivalent.

The leaks run in both directions

The obvious direction is untrusted writes poisoning trusted reads. The reverse direction gets less attention and is often more damaging: a trusted cache entry read by an untrusted build can carry data the untrusted build was not meant to see.

The mechanism is ordinary. A trusted build compiles code with a credential in the environment, or downloads a private package with a token, and the result is cached. The credential itself is usually not in the artifact, but derived data can be: a generated configuration file, a decompiled representation, an error message with a URL containing a token, a compiled-in string. The untrusted build requests the same key — it has the same source, or it fabricates the inputs to match — and receives an entry the trusted job produced under conditions it cannot reproduce and is not permitted to observe.

A cache is therefore a channel, not only a store, and treating it as trusted infrastructure shared across a boundary means accepting that channel. Projects that run untrusted code at all generally accept that the untrusted build can read anything in the repository, which bounds the leak to data outside the repository. The bound is the thing to state explicitly, because it is what the risk assessment depends on.

Making the cache survive a hostile producer

The general posture is to treat every cache entry as an input from an untrusted source, and to verify as much as the mechanism allows.

Verify the output digest against the manifest the entry claims. This catches corruption and truncation and does nothing against an adversary who defined the manifest.

Verify that the key covers what it claims. This is a property of the build definition rather than of the retrieval, and it is the reason hermetic actions matter: an action that runs in a sandbox with a fixed environment and no network has a key that can plausibly be complete, because there is nowhere else for an input to hide. An action that runs in the developer’s shell with inherited environment and network access has a key that is a guess, and a cache over it is a mechanism for distributing guesses.

Isolate the consequences. Run the build of untrusted contributions in a job with no access to publish credentials, no access to signing keys, and no access to production registries, so that a poisoned entry can at worst cause a wrong test result rather than a signed release. The cache boundary is one layer of this, and it is not sufficient alone.

Sign entries where the storage layer supports it, and verify the signature against an identity the orchestrator controls. This is what turns the question from “was this entry produced by someone with the same inputs” into “was this entry produced by the builder I expect”. The design space here overlaps with update frameworks: authenticity, freshness, and rollback resistance are separate properties, and a cache that verifies signatures but accepts an arbitrarily old entry is still vulnerable to being served a stale, validly signed result. Systems that care about this borrow the role-and-delegation structure from projects that have thought it through, rather than inventing a scheme per cache.

What remains uncovered

Even with all of this, the cache introduces a dependency that a rebuild does not have: the availability and correctness of the cache service itself. An entry can be lost, and the pipeline must rebuild rather than fail, which is a design requirement covered in Shared Build Cache Design.

There is also no general way to detect that a poisoned entry was used after the fact, unless the entry’s provenance was recorded with the build. A pipeline that logs the key and the output digest for every action has a record that can be checked later; one that logs only “cache hit” does not, and the question “which artifacts were built from a poisoned entry” becomes unanswerable. Recording the key and digest costs a few lines per action and is the only forensic trail the mechanism offers.

References

  1. The Update Framework Specification — TUF project
  2. CWE-345: Insufficient Verification of Data Authenticity — MITRE
  3. CWE-494: Download of Code Without Integrity Check — MITRE
  4. SLSA Specification — OpenSSF