Lockfiles as Reproducibility Contracts and Why They Drift

A manifest states intent and a lockfile states outcome. The manifest says which ranges are acceptable; the lockfile records the single version of each package that the resolver chose, together with enough provenance to fetch that exact artifact again. Installing from a lockfile should not require running the resolver, and the result should not depend on what the registry published after the lockfile was written.

That contract is weaker than it sounds, and the ways it weakens are systematic rather than accidental.

What has to be in the file

A lockfile that pins only names and version numbers pins less than it appears to. Version 1.4.2 of a package may have been republished, or the registry may serve a different artifact for the same version on a different platform. A lockfile that is serious about reproducibility records, for each package: the exact version, the resolved source or registry, a hash of the artifact bytes, and the dependency edges that version declares.

The dependency edges are the part people forget. Without them, installing from the lockfile requires asking the registry what 1.4.2 depends on, which is a network query against mutable data. A lockfile that carries the edges can be installed by walking its own graph. This is why lockfile formats generally store a flat list of package records, each with its own dependency list, rather than a tree: the flat list survives deduplication and re-parenting, and the tree can be reconstructed by following edges from the root.

Formats differ in how much of this they demand. package-lock.json records resolved URLs and integrity strings per entry, and nests dependency edges according to the on-disk layout the installer will produce. Cargo.lock records name, version, source, checksum, and each entry’s declared dependencies, with no URLs, because the source identifier plus the registry protocol determines the download location. poetry.lock and Pipfile.lock carry hashes per file for Python, which matters because a single Python distribution version can ship many wheel files and one source archive, each with a distinct hash.

PEP 751 defines a lockfile format for Python that records package name, version, and a list of archive entries with their hashes, plus optional markers and dependency metadata. Its existence is a response to a real problem: Python had several lockfile formats with incompatible guarantees, and each tool could only install its own.

The three inputs a lockfile cannot see

A lockfile is a function of the manifest, the resolver’s preference rules, and the registry state at the time it was generated. Only the first is in the repository. The second is implicit in the tool version. The third is invisible.

Tool version matters because resolver preferences are not standardised. A newer resolver that changes its version-selection heuristic, or that begins honouring a constraint it previously ignored, will produce a different lockfile from an unchanged manifest. Regenerating a lockfile with a newer tool can therefore change dozens of entries with no manifest change at all. The diff looks like churn; it is the heuristic changing.

Registry state matters at generation time, not install time. If a lockfile is generated today, it records what existed today. A package published tomorrow that satisfies every range will not appear in the lockfile until someone regenerates it. That is the intended behaviour and also the source of the “works on a fresh clone, fails in CI” pattern: a CI job that runs the resolver instead of installing from the lockfile sees a different world.

Platform-dependent resolution is the third gap, and it is structural. A lockfile produced on Linux may record only the Linux artifacts for a package that ships per-platform binaries. Installing that lockfile on macOS either fails or silently re-resolves, depending on the tool. Formats handle this by recording per-platform entries and markers, which is why a Python lockfile can list five wheels for one package version and why a lockfile for a project supporting three operating systems can be several times larger than the manifest.

Why a lockfile goes stale without anyone editing it

Staleness has several distinct causes, and conflating them produces bad tooling.

The manifest changed and the lockfile was not regenerated. This is detectable: the tool can compare the manifest’s ranges against the versions in the lockfile and report any requirement that no locked version satisfies. Adding a dependency to the manifest without touching the lockfile leaves the new package absent entirely.

The lockfile is internally inconsistent. A locked package’s declared dependencies may reference a version that is not in the lockfile, which happens when a lockfile is merged by hand after a conflict in version control. Merge conflicts in lockfiles are common because two branches both add entries to a flat list, and the textual merge can succeed while producing a graph with a dangling edge. Formats that sort entries deterministically reduce the conflict rate but do not eliminate the class of error.

The environment changed. A lockfile with platform markers evaluates those markers at install time, so the same file can select different subsets on different machines. This is correct and still surprising: two developers with identical lockfiles and identical manifests can have different installed sets because their Python version or operating system differs.

The lockfile is valid but the artifacts are gone. Registries occasionally remove versions, whether by policy or by accident, and a lockfile naming a removed version cannot be installed from that registry. Mirrors and caches exist partly to make this failure rarer, and vendoring exists to make it impossible, which is covered in Vendored Source Trees and Offline Artifact Mirrors.

Reading a lockfile as a proof

The useful mental model is that a lockfile is a witness for a satisfiability claim: this assignment satisfies these constraints, and here is the artifact that realises each part of it. Verification of that witness is cheap and local. It requires checking each locked version against the manifest ranges and each recorded hash against the fetched bytes. It does not require re-running the search.

This asymmetry is the point. The search described in Dependency Resolution as Backtracking Search Over Version Sets is expensive and its result depends on preference order; verifying a candidate assignment is linear in the graph size. Lockfiles move the expensive step to a controlled moment and make every subsequent install a verification.

A small checker demonstrates the two checks that matter most, range satisfaction and hash agreement.

import hashlib

def verify(manifest_ranges, lock, fetched, key):
    # manifest_ranges: {name: (lo, hi)}; key orders version strings
    problems = []
    for name, (lo, hi) in manifest_ranges.items():
        entry = lock.get(name)
        if entry is None:
            problems.append(f"{name}: absent from lockfile")
        elif not (key(lo) <= key(entry["version"]) < key(hi)):
            problems.append(f"{name}: {entry['version']} outside [{lo},{hi})")
    for name, entry in lock.items():
        digest = hashlib.sha256(fetched(name)).hexdigest()
        if digest != entry["sha256"]:
            problems.append(f"{name}: hash mismatch")
        for dep in entry["deps"]:
            if dep not in lock:
                problems.append(f"{name}: dangling edge to {dep}")
    return problems

The key parameter is the ecosystem’s version comparator, which is the point: range checking is meaningless without the ordering defined in Version Precedence and the Semantics of Range Operators. The function assumes the lockfile already contains the dependency edges, which is exactly the property that makes offline verification possible. Without those edges, the second loop has nothing to check and the verifier must query the registry, at which point the lockfile has stopped being a contract.

Update strategies and their blast radius

Tools expose a spectrum from “install exactly what is locked” to “resolve everything fresh”. Between them sit targeted updates: bump one package and let its own subtree move, keeping everything else fixed. The implementation of a targeted update is a resolve with the other versions pinned as exact constraints, which reuses the machinery of the general resolver and produces a small, reviewable diff.

Whole-graph updates are cheaper to implement and worse to review. Because resolver preference order is global, bumping one package can change the versions of packages that have no dependency relationship to it, simply because the search explored a different order. Reviewing such a diff for unintended changes requires understanding the entire graph.

The conservative default in most ecosystems is now: lockfiles are committed for applications, and omitted for libraries. The reason for libraries is that a library’s lockfile is not authoritative for its consumers, since the consumer’s resolver will choose versions anyway, and a committed lockfile invites the false belief that it constrains downstream installs. The reason for applications is that the application’s lockfile is authoritative, because nothing above it exists to override the choice.

There is no mechanism that makes a lockfile correct. It records a decision; the decision can be a poor one, such as a version with a known defect or one that pulls a large transitive tree. What the lockfile provides is that the decision is visible, diffable, and stable, which is the property that makes dependency changes reviewable at all.