Lockfiles, Hash Pinning, and Reconstructed Environments

A requirements file that says numpy>=1.20 describes a set of acceptable packages. It does not describe an environment. Between the constraint and the installed state sit a resolver, an index that changes over time, and a build step that may compile from source against whatever the host provides. A lockfile exists to collapse that distance into a fixed list, and hash pinning exists to make the collapse verifiable rather than merely plausible.

Resolution is a search that produces one answer from many

A resolver takes a set of constraints — the direct requirements, each with version specifiers, plus the transitive requirements declared by each candidate version — and searches for an assignment of one version per package that satisfies all of them simultaneously. The search space is exponential in the worst case, and the constraint language is not restricted to conjunctions of intervals: extras, environment markers such as sys_platform == "win32", and mutually exclusive version ranges from different dependents all interact.

Which solution the resolver returns depends on its ordering heuristics, and different resolvers can return different valid solutions from the same index at the same moment. That is not a bug in either; the problem is underdetermined. Backtracking order, the preference for newer versions, and the order in which requirements were presented all influence the result.

The consequence for reproducibility is direct: a constraint set does not identify an environment, because more than one environment satisfies it and the choice is made by software whose behaviour is not part of the specification. The lockfile’s job is to record the choice that was made, converting a search problem into a lookup.

What the lock record has to contain

A faithful lock record needs four things per package, and omitting any one of them leaves a hole.

The exact version. 1.26.4, not ~=1.26. The version is the primary key of the record.

The artifact identity. For each platform and interpreter combination the environment might be built on, the lock must name the specific distribution file — numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl, for example — and its hash. The filename encodes the interpreter ABI tag, the platform tag, and the ABI version, so it identifies the binary, not just the release.

The hash. A digest of the artifact bytes, checked before installation.

The dependency closure. Not just the direct requirements; every transitive package, so that installing does not re-enter the resolver.

The Python ecosystem splits these across two file conventions. A requirements file can carry --hash=sha256:... entries, one per acceptable artifact, and pip enforces them in hash-checking mode: any artifact not matching a listed hash is refused, and the mode extends to build-time dependencies for source distributions. A lock format may instead store its own resolved graph, with markers recording which entries apply only on particular platforms. Both approaches share the property that matters: the record names bytes, not a range of versions.

The hash is over the distribution archive, not over the extracted tree. That matters because installation can transform the artifact — compiling a source distribution, generating metadata, rewriting script shebangs — and the transformation is not covered by the digest.

What installation actually guarantees

Hash checking guarantees one thing precisely: the bytes fetched are the bytes recorded. It does not guarantee that the same bytes produce the same installed tree on two machines, and the gap between those two statements is where most reconstruction failures live.

Consider what happens between download and import.

A wheel is a ZIP archive whose filename declares its compatibility. Its contents include compiled extension modules as shared objects linked against particular runtimes, plus a .dist-info directory with metadata and a RECORD file listing each installed file with its hash. Installing a wheel is close to extraction plus a rename, so the installed tree is largely determined by the archive. Even here, the wheel’s own compilation was done elsewhere, against a particular C library and compiler; the .so carries that provenance in ways the digest does not expose.

A source distribution is compiled locally. The resulting extension module depends on the local compiler version, optimization flags, the presence of CPU features such as AVX-512 that the build may dispatch on at runtime, and the numerical library that the package’s build configuration linked against. Installing from source with the same sdist on two hosts can produce two binaries with different floating-point behaviour, and neither is wrong.

Post-install hooks may run arbitrary code. Entry points generate console scripts; some packages compile bytecode at install time, producing .pyc files whose contents depend on the interpreter version.

The interpreter itself is outside the lockfile entirely. A lock naming cp311 wheels installs nothing if the interpreter is 3.12, and there is no wheel to select. The ABI tag in the filename is the mechanism that turns this mismatch into a visible error rather than a subtle misbehaviour, and it works only because the tag is checked.

The environment is more than the package set

Three layers commonly sit outside any lock.

System libraries. A Python package’s extension module may link against a BLAS implementation, an MPI runtime, or a compression library provided by the operating system. Two hosts with the same wheels but different BLAS builds can produce different numerical results, and the lock records nothing about either.

Environment variables. Thread counts, cache directories, library search paths, and locale all change behaviour. OMP_NUM_THREADS alters how many threads a numerical kernel uses, which changes the order of a reduction and therefore the low bits of a sum. The variable is part of the environment in the sense that matters and is not in the lockfile.

Data and configuration outside the package tree. Model weights, lookup tables, license files, and per-user configuration directories are inputs to execution as much as any dataset.

The honest scope of a lockfile is therefore “the Python package layer”, and a reconstruction claim that reaches beyond it needs another mechanism. Container images address all three layers at once, at a cost discussed in Container Images as Frozen Environments and Their Limits.

Verifying a reconstruction

Two environments can be compared at increasing levels of strictness: the lockfile equality (trivial), the resolved package set (install both and compare pip freeze-style output), the installed file tree (compare per-file hashes against RECORD), and the behaviour (run the analysis and compare outputs to a tolerance).

import hashlib, pathlib, sys

site = pathlib.Path(sys.argv[1])          # a site-packages directory
record = {}
for rec in site.glob("*.dist-info/RECORD"):
    for line in rec.read_text(encoding="utf-8").splitlines():
        path, digest, _size = line.split(",")
        if digest:                        # "__pycache__" and .pyc entries may be blank
            algo, _, want = digest.partition("=")
            got = hashlib.new(algo, (site / path).read_bytes()).hexdigest()
            if got != want:
                record[path] = "mismatch"
print("checked", sum(1 for _ in site.glob("*.dist-info/RECORD")), "distributions")
print("mismatches:", sorted(record)[:10])

The RECORD file is part of the wheel specification, so this check verifies that the installed files are the ones the wheel declared — a statement about extraction fidelity, not about the wheel’s behaviour. If install-time code rewrote a file, the mismatch appears here, which is the intended detection.

The last level is the only one that answers the question anyone actually cares about, and it requires the analysis to be runnable and its output to be comparable. That comparison needs a stated tolerance for floating-point results, which Conditioning and the Justified Precision of a Reported Result sets out.

Failure modes in practice

Hash drift on an index. Some indexes permit replacing an artifact; the digest then changes while the version does not. A lock built before the replacement fails loudly at install, which is correct behaviour and also means the environment cannot be reconstructed from that index after the fact. Mirrors or an internal artifact store preserve the bytes; a lockfile alone cannot.

Platform markers silently expanding. A lock resolved on one platform may contain entries that only apply elsewhere; installing it on the original platform is fine, but the resolver’s platform-specific choices for the other platform were made without evidence. Locks that record per-platform closures avoid the guess.

Constraints leaking back in. Installing additional packages into a locked environment runs the resolver again, and it may move a pinned version to satisfy the new requirement. The result is no longer the locked environment, though the lockfile still describes one.

Source builds under changed toolchains. A lock with only a source distribution entry recompiles at install. If the project later requires a newer compiler feature, the build fails at a point where the lockfile offers no guidance, because the toolchain was never part of the record.

The pattern across all four is that a lockfile pins the layer it describes and nothing else. The engineering response is to make the enclosing layers explicit — an image digest, a recorded compiler, a captured environment — rather than to expect more from the lock than the install path can provide.