Vendored Source Trees and Offline Artifact Mirrors

Vendoring replaces a network lookup with a file in the repository. The dependency graph is still a graph, and the versions are still versions, but the artifacts and their metadata arrive with the source checkout rather than from a registry. This removes an entire class of failures and introduces a different one, and the trade is sharper than the usual “faster and more reliable” framing suggests.

The dependency problem becomes a source-control problem

When dependencies are fetched at build time, the set of inputs to a build includes whatever the registry served at that moment. When dependencies are vendored, the set of inputs is the checkout, and it is exactly as reproducible as the repository is.

The immediate consequence is that the build no longer needs a resolver. Go’s vendoring makes this explicit: with a vendor directory present and the module’s Go version at or above 1.14, build commands use the vendored packages and do not consult the module cache or the network. The vendor/modules.txt file records which module versions the directory contains, and the build verifies that the manifest and the vendored set agree rather than re-solving. Resolution has already happened; what remains is a consistency check.

This is the same asymmetry that lockfiles exploit, taken one step further. A lockfile pins versions and hashes but still requires the registry to serve the bytes. Vendoring pins the bytes themselves. The failure modes that remain are those of the repository: a merge that drops a vendored directory, a partial copy that omits a transitive dependency, or a vendored tree that was never updated after the manifest changed.

Why vendoring changes the resolution problem

The interesting effect is not that vendoring avoids the network. It is that vendoring makes version conflicts visible in a way that resolution hides.

A registry-based install permits two versions of the same package to exist in one build if the host language supports it, and forbids it otherwise by failing resolution. A vendored tree in a language with a single global package namespace has no such escape: the directory has one copy of each package, so two modules requiring incompatible versions of a third cannot both be satisfied. The build fails with a duplicate or missing import rather than with a resolver error, and the failure appears at compile time in a file the author did not edit.

Languages with per-module version selection avoid this. Go’s module system allows the graph to contain several major versions of a module at once, distinguished by the /vN path suffix, so a vendored tree can contain both example.com/lib and example.com/lib/v2 as separate directories. The path is the disambiguator, which is the design decision described in Version Precedence and the Semantics of Range Operators, and it is what makes vendoring viable in a language without a global resolver.

Languages that resolve to a single version per package pay a different price: the vendor operation must itself run the resolver, so the vendored tree is a snapshot of one resolution. Regenerating it after a manifest change can produce a large diff touching packages unrelated to the change, because resolver preference order is global.

Duplication and its compound costs

Vendoring duplicates code across every repository that depends on it. Two projects using the same library at the same version each store a copy, and a monorepo that vendors per-module stores the same library several times inside one repository.

The costs are not only disk. Review tooling sees vendored files as source, so a dependency bump that touches a hundred files appears in review as a hundred changed files unless the tool is configured to collapse them. Version-control history grows with content that the project does not maintain, and blame annotations become meaningless inside vendored paths. Repository size affects clone time for every developer and every CI job.

Mitigations are conventional rather than structural. Vendored directories are excluded from review by path rules, excluded from linting and formatting, and excluded from coverage measurement. None of these reduces the storage cost, and all of them mean the vendored code is not held to the project’s own standards, which is correct but worth stating: it is a copy of someone else’s artifact, not project source.

Mirrors as vendoring at the infrastructure layer

A mirror keeps the registry protocol and changes the location. Clients are configured to fetch from a different base URL, and the mirror proxies or caches upstream content. From the resolver’s perspective nothing has changed; from the network’s perspective everything has.

Cargo’s source replacement expresses this directly: a source can be replaced by another source of the same kind, and the replacement is declared in configuration rather than in code. Cargo’s documented assumption is that the source code is exactly the same at both sources, and that a replacement may not contain crates the original does not, so a mirror is constrained to be a faithful copy rather than a curated subset. That is a stronger requirement than a plain HTTP proxy, which can serve anything.

A mirror has three jobs, and they conflict.

Availability: serve artifacts when upstream is unreachable or slow. This argues for caching aggressively and for retaining artifacts indefinitely.

Freshness: serve new versions as soon as they are published. This argues for short cache lifetimes on index documents and for revalidating often.

Isolation: serve only vetted packages, or only packages that have passed an internal check. This argues for a curation step that upstream does not have, and it changes the trust model: the mirror becomes an authority rather than a cache.

The first two are ordinary HTTP caching problems, addressed by separating mutable index documents from immutable artifacts, as discussed in Registry Index Metadata and Resolution Without Downloading Artifacts. The third is not a caching problem at all, and mirrors that attempt it usually need a separate publication workflow.

A repository cache sits between the two: it keeps fetched external dependencies keyed by their declared digest, so a build that names the same dependency twice does not fetch it twice, and a build on a machine that already has the entry does not touch the network at all. Bazel’s external dependency machinery works this way, with a content-addressed cache directory that can be pre-populated and shipped alongside a build environment. The cache is not a mirror, because it does not speak the registry protocol and cannot answer a query it has not seen before; it is a memo table keyed by hash, which is enough when the dependency set is fully declared in build files.

Offline installation is the degenerate case: the mirror is a directory of artifacts plus index documents, and the client is configured to use it with the network unavailable. This works when the artifact set is complete for the target platform, and fails with a resolution error naming a package that exists upstream but was never mirrored. Completeness is the hard part, because the transitive closure of a dependency graph is not known until it is resolved, and the platform-specific subset is not known until installation.

What vendoring does not fix

Vendored code is not audited code. Copying a dependency into a repository does not review it, and the copy drifts from upstream silently: security fixes published upstream do not reach the vendored copy until someone updates it. A vendored tree therefore needs an explicit update cadence, and the cadence is usually the weakest part of the practice.

Vendoring also does not fix provenance. A vendored directory can be modified after copying, and nothing in the format records that the copy matches the published artifact. Systems that care about this store a hash per vendored package alongside the tree, so the copy can be verified against the registry’s published digest. Without such a record, a vendored tree is a fork with no upstream.

Finally, vendoring does not remove the need for a lockfile. The vendored set must be describable in terms the manifest understands, or a later regeneration cannot tell which vendored packages are still required. Go’s modules.txt and Cargo’s checksum entries exist for this reason: they are the lockfile’s content in the vendored tree’s own vocabulary.

A small consistency check captures the essential invariant:

import json, pathlib

def check_vendor(root, manifest):
    # manifest: {"requires": {name: version}}
    vendored = {}
    for line in (root / "vendor" / "modules.txt").read_text().splitlines():
        if line.startswith("# "):
            parts = line[2:].split()          # module path and version
            vendored[parts[0]] = parts[1]
    problems = []
    for name, version in manifest["requires"].items():
        if vendored.get(name) != version:
            problems.append(f"{name}: manifest {version}, vendored {vendored.get(name)}")
    return problems

The check is shallow: it compares declared versions and ignores whether the files inside each directory are complete or unmodified. That shallowness is the honest characterisation of vendoring in general. It guarantees that a build does not consult the network; it does not guarantee that what is on disk is what upstream published. Guaranteeing that requires hashes over the tree, which is a content-addressing problem, and the machinery for it is the same as for any other artifact.

The practice is worth using where the build environment is constrained, where network access during builds is unacceptable, or where the dependency set is small enough that duplication costs little. It is a poor fit for a project with a large transitive graph, frequent updates, and ordinary network access, where a lockfile plus an artifact cache achieves most of the reproducibility at a fraction of the repository cost.