Bit Rot, Link Rot, and Replaying an Analysis Years Later

A computation that reproduces today is not guaranteed to reproduce in five years. The files do not change, and yet the result becomes unrecoverable, because reproducibility is a relationship between an analysis and the ecosystem around it, and the ecosystem is the part that moves. Understanding the failure modes is the difference between an archive that preserves a claim and an archive that preserves a memory of a claim.

The failures fall into two families with very different prospects: degradation of the bytes, which is mechanical and largely detectable, and degradation of the pointers, which is social and mostly not.

The bytes rot slowly and silently

Digital storage is the most reliable medium ever used and not at all permanent. Magnetic media lose magnetization over years to decades. Flash cells leak charge; the retention of an unpowered solid-state drive is measured in years and worsens with wear and temperature. Optical media degrade through dye fading and substrate breakdown, with some archival discs rated for decades and consumer discs for years. Hard disks fail outright at a rate that makes any single drive an unreliable long-term store.

Silent corruption is the case that matters for reproducibility, because it does not announce itself. A flipped bit in a data file produces a wrong number with no error. A flipped bit in an executable produces a crash, which at least is detectable. A flipped bit in a checksum or a provenance record produces a false verification or a false alarm, depending on direction.

The defense is checksums computed at creation and verified at read, which is exactly what content addressing provides, and redundancy, which is what any serious archive provides. A single copy of a file with no digest has no way to distinguish its current state from its original state; a digest distinguishes them, and multiple copies allow the archive to repair the difference. The combination — store under the content hash, keep at least two geographically separated copies, verify on a schedule — turns bit rot from an undetectable risk into a recoverable one.

The schedule matters. Verification that happens only when the file is next requested is verification that may happen after the only good copy is gone. A periodic scrub that reads and re-checks every object is the mechanism that finds corruption while redundancy can still repair it; the frequency is a decision about how much data the archive is willing to risk.

The pointers rot faster than the bytes

Most of what a computation depends on is not stored with it; it is named by a reference that has to resolve somewhere else. Packages are named by version and index. Container images are named by digest and registry. Papers are named by DOI. Data is named by URL. Each name is a promise made by an institution, and the institutions change.

Dependencies disappear. A package is unregistered, a version is yanked from an index, a binary wheel for a now-obsolete interpreter is no longer hosted. The resolver can no longer reconstruct the environment from the recorded lockfile, because the recorded version no longer exists anywhere it knows to look.

Registries disappear or change policy. An image registry is retired, or moves from free to paid, or deletes old tags. A digest is stable only as long as a registry still holds the blob; the Open Container Initiative format guarantees how to address an image, not how long anyone will serve it.

URLs break. A dataset hosted at a lab URL is moved when the lab’s website is reorganized, and the old URL serves a redirect or a 404. The average lifetime of a URL before it changes or dies is short on archival timescales; link rot studies report double-digit percentages of citations no longer resolving after a decade.

Software itself becomes unrunnable. A computation written against a Python 2 environment cannot run on any interpreter that is still maintained. The source is not lost; the interpreter is gone from the distribution, and the native extensions cannot be rebuilt against modern system libraries. The computation is preserved in the sense that a museum preserves an object, and not in the sense that it can be run.

Each of these is a failure of a pointer, not of the computation. The mitigation is the same in every case: capture the bytes rather than the reference, at the moment the reference still works. A pinned dependency is a reference; a vendored wheel or a software-preservation archive is a capture.

What has to be captured at creation time

The distinction that organizes all of this is between capturing state and recording pointers to state. A pointer is cheap and fragile; a capture is expensive and durable. The reproducible-by-design computation captures the things that are expensive to recreate and records the rest.

The environment has to be captured as an image digest that a registry still serves, or better, as an image exported to the archive itself. An image digest alone is a pointer to a registry; the exported image is the bytes. The two differ in what survives the registry’s retirement.

The code has to be captured in a form that is itself archived. Version control is a system of record, not an archive; a commit hash is a pointer into a history that lives on a forge and can be rewritten or taken down. Software preservation archives that store source with a persistent identifier — Software Heritage assigns intrinsic identifiers derived from content — turn the source into bytes addressed by digest rather than by forge location.

The data has to be captured with its own checksums and, where the data is the basis of a published claim, deposited in a repository that assigns a persistent identifier and promises retention. The FAIR principles make explicit what “captured” means: an identifier that resolves permanently, metadata that makes the data findable and its meaning clear, and a license that permits reuse.

The execution record — the provenance graph from Content-Addressed Provenance Graphs for Derived Outputs — has to be captured alongside the artifacts, because it is the map that ties an output back to its inputs, code, and environment. Without it, a replay has to reconstruct the map by inspection, which for a computation of any size is not a reconstruction but an archaeology.

The common thread is that capture happens at creation time or not at all. After the index stops serving the version, after the registry deletes the tag, after the lab moves the file, the bytes are gone from every reachable location and no later effort recovers them. The window during which a computation can be made durable closes, and it closes quietly.

Replaying against a degraded ecosystem

Given a complete capture, replaying years later is a different problem from reconstructing from scratch, and the main obstacle is the boundary where the captured environment touches the host.

The interpreter inside the image may depend on a kernel feature that a new host no longer provides, or on a glibc ABI that the host’s kernel supports only through a compatibility path. The GPU-accelerated part of a computation depends on a driver that has moved on, and a captured CUDA library may not load against a modern driver. The captured MPI runtime may not interoperate with the scheduler of whatever cluster the replay runs on. These are the limits that Container Images as Frozen Environments and Their Limits describes as the boundary of what an image freezes.

Where the boundary blocks a replay, the fallback is emulation or a preserved virtual machine, which runs the original environment on a software layer that itself has to be maintained. Emulation trades one dependency on the host for a dependency on the emulator, and an emulator is a program that also rots; preserving the ability to run the preserved environment is a recursive problem with no bottom, only successively smaller audiences.

The practical response is to rank by cost. The data and the code are worth capturing at full fidelity, because they are the irreplaceable part and they are small relative to the environment. The environment is worth capturing as an image where its behaviour could matter; the image is large but redundant parts deduplicate. The execution record is worth capturing because it is small and it is what makes the rest usable. The host-side components — kernel, driver, scheduler — are not captured at all, and the honest claim for a late replay is that it ran on a similar host, not the original one.

What an archive can actually promise

An archive that stores bytes under content hashes, verifies them on a schedule, and keeps redundant copies can promise that the bytes will be there and will be what they were. It cannot promise that the software to interpret them will exist, that the host will run that software, or that the institutions whose names appear in the record will still mean the same thing.

That boundary is the reason a reproducibility claim is strongest when it is exercised early and repeatedly, not when it is stored against a future emergency. A computation replayed every month against its captured environment is a computation whose environment has not yet drifted out of reach; the same computation first replayed after five years is an experiment in archaeology whose outcome is uncertain. The practice of continuous replay is the subject of Executing Provenance Records to Certify That a Result Still Reconstructs, and it is the only arrangement in which the archive’s promise is tested rather than assumed.

The larger lesson is that durability is not a property of the files but of the relationship between the files and the world that interprets them. A computation that matters should be treated the way an archive treats a manuscript: the bytes are the easy part, and the meaning is what requires a community that keeps translating it into whatever the present can still run.