Lockfile Determinism, Diffs, and Merge Conflicts

A lockfile that changes when nothing changed is worse than no lockfile. It makes every diff suspect, turns merges into guesswork, and defeats the reason the file is committed. The property that prevents this is determinism: given the same manifests and the same registry metadata, the writer must emit the same bytes. That is a property of the serialiser and the resolution policy together, and it is easy to lose without noticing, because a non-deterministic writer still produces a correct file — just a different correct file each time.

Determinism is a property of the writer

The resolver produces an assignment, which is a set of mappings. A set has no order. The lockfile is a sequence of bytes, and the mapping from set to sequence is where determinism is won or lost — the mapping itself, and which fields belong in it at all, is set out in What a Lockfile Records. Every field that comes from an unordered source must be sorted by a key that is itself deterministic, and every field that is derived from the environment must be either excluded or normalised into a stable form.

Sorting by package name alone is insufficient when a name can appear at several versions, as it can in installers that permit duplicates. The sort key must be the full identity: name, then version under the ecosystem’s ordering, then source. Sorting by version as a string puts 1.10.0 before 1.9.0, which is a stable order and the wrong one; the order does not affect correctness of the content, but it does affect whether the file looks sane to a human reading a diff, and it affects the file’s bytes, so it must be fixed once and never changed.

Dependency lists inside an entry need the same treatment. A manifest’s dependency map is unordered, and a writer that iterates a hash map inserts them in whatever order the hash function produces. The fix is to sort the list before emitting it. This is a one-line change that is omitted in a surprising number of tools, and its symptom is a lockfile that reorders itself between runs on machines with different hash seeds.

Values that should not be in the file

Several categories of value inject environment into the artifact. Absolute paths appear when a dependency is a local directory; a lockfile generated on one machine then contains a path that does not exist on another. Registry credentials sometimes end up embedded in resolved URLs, which leaks them into version control. Timestamps and generator versions appear in headers and produce a diff on every run.

Each has a normalisation. Local paths can be recorded relative to the manifest’s directory, or replaced by a digest of the directory’s contents. URLs can be stripped of credentials and reduced to a canonical form. Timestamps can be omitted entirely, since the file’s content is what matters and version control already records when it changed. The generator’s version is harder: it is genuinely useful for debugging a resolution that a newer tool would perform differently, but recording it makes the file’s bytes depend on which tool wrote it last.

Git references are the subtlest case. A dependency declared against a branch resolves to whatever commit that branch pointed at during resolution. Recording the branch name makes the lockfile a record of an intent rather than of a result, and the next resolution may produce different content from the same file. Recording the commit hash fixes the content and loses the intent; most formats record both, with the hash as the authoritative field.

Solver drift is not nondeterminism

Two resolvers can both be deterministic and still disagree. A conflict-driven resolver and a greedy one will select different valid assignments from the same graph, and two versions of the same resolver can differ after a heuristic change. The lockfile is therefore deterministic only relative to a policy, and a tool upgrade that changes the policy will rewrite every lockfile in the repository even though no manifest changed.

This is a different failure from hash-order noise: the new file is not randomly different, it is systematically different, and the differences are as large as the policy change. Tools mitigate it by trying to preserve existing choices when re-resolving, which converts the resolution into a constrained problem — “find a solution close to this one” — and adds a dependency on the old file’s contents to the new file’s bytes. That is a reasonable trade for upgrades, and it means the lockfile is not purely a function of the manifests.

Why the union merge fails

When two branches both change dependencies, a textual merge of the lockfile usually succeeds, because the entries are independent records and the merge tool can combine them. The result is often wrong. The failure is structural rather than textual, and it takes a few forms.

The first is a shared transitive dependency. Branch one adds a package that requires the shared library at 2.x, so its lock entry for the library moves to 2.3. Branch two adds a different package that requires 1.x, and its lock entry stays at 1.9. A union merge keeps both entries, or keeps whichever side the merge tool preferred, but the dependency edges recorded in the newly added entries still point to the version from their own branch. The merged file now contains an edge to a version that is not present, or a package present at a version no requirement admits.

The second is a removal. Branch one deletes a dependency from the manifest and from the lock. Branch two adds a package that transitively requires it, adding a lock entry whose edge points at it. The union keeps the entry, so the lock references a package the manifest no longer declares — usually harmless for installation, since the entry is fetched anyway, but wrong as a statement of the graph, and it will be removed by the next tidy operation, producing another diff.

The third is an integrity mismatch. Both branches can contain an entry for the same name and version with different digests if the registry served different content, or if the two branches resolved through different mirrors. A union merge picks one, and the choice is not derivable from the manifests. This case should be rare, and when it occurs it is a signal worth investigating rather than merging.

None of these failures is caught by the merge tool, because each side’s file is internally consistent. The inconsistency appears only when the edges are checked against the entry set, which is a graph operation the merge tool does not perform.

Verifying a merged lock

A merged lockfile can be checked cheaply, and the check is worth running in continuous integration. The procedure has three parts, in increasing cost.

First, every recorded edge must point to an entry that exists. Walk each entry’s dependency list and confirm the referenced identity is present in the file. This catches the dangling-edge case and is linear in the number of edges.

Second, every root requirement must be satisfied by the locked version of that name, using the same range predicate the resolver uses. This catches the case where a manifest change was merged but the corresponding lock change was not, or where the merge kept the wrong side.

Third, the graph must be closed: every entry must be reachable from the root through recorded edges, and no entry may be reachable only through an edge that its own parent does not declare. The third condition is the expensive one, because it requires the manifests of the locked versions. Implementations that skip it accept locks that are internally consistent but describe a graph the installer would not build.

A lock that passes all three checks is very likely usable. It is not a proof that the resolver would have produced it, and the difference matters for review: a merged lock can be consistent and still not be the file any single resolution would generate, which means the next run of the resolver may reformat or rewrite it. Adopting the merged file and then regenerating it on the default branch resolves the discrepancy, at the cost of a follow-up commit.

Regenerate rather than merge

The alternative to merging is to treat the lockfile as a build product. On a merge conflict, take either side’s manifest merge, then run resolution once and commit the result. The lockfile conflict is resolved by ignoring both sides’ lock content, which is correct because the lock is a function of the manifests and the registry, not an independent source of truth.

The cost is time: a full resolution may fetch metadata for hundreds of packages, and on a large graph it is slower than editing a conflict by hand. The cost is also reviewability, since the regenerated file can differ from both sides in ways that are not obviously related to the merge. Both costs are real, and both are smaller than the cost of shipping a lock that does not match its manifests.

A middle path is to merge the textual content, verify it, and fall back to regeneration when verification fails. The verification is cheap enough to run on every merge, and it catches the cases where the union happens to be wrong. The cases where the union is right — most merges, since added dependencies rarely intersect — then avoid the full resolution.

Reading the diff

A lockfile diff is the most information-dense artifact in a dependency change, and its size is usually much larger than the manifest diff that caused it. Three patterns are worth recognising.

New entries with a source different from the registry indicate a dependency added by URL, which bypasses the registry’s integrity metadata and deserves attention. A changed digest for the same name and version indicates either a republish or a mirror inconsistency; neither should pass review silently. And a large block of version bumps with no manifest change indicates a resolver or policy change, which is the solver drift described above and should be reviewed as a tool upgrade rather than as a dependency update.

The entries themselves are only half the information. The edges explain why a package is present, and a diff that adds an entry without showing its parents is the part that reviewers most often skip. Tools that annotate a diff with the requirement path are doing the work the lockfile cannot: the file records the result, and the reasons live in the resolution that produced it. Those reasons are also what Integrity Hashes and Registry Trust relies on when deciding whether a changed entry is a routine update or a supply-chain event.