Content Addressing: Why a Layer Is Named by Its Digest

A name like sha256:2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae is not a label someone chose. It is the output of hashing the bytes it refers to, which makes the name and the content the same fact stated twice. Every property that makes container image distribution work follows from that single decision, including several that surprise people the first time they meet them.

The digest is a claim about bytes

The format is algorithm:hex, where the hex is the lowercase encoding of the hash of the exact byte sequence stored at that address. For sha256 the encoding is 32 bytes, rendered as 64 characters. A client that fetches a blob, hashes it, and gets a different value has detected either corruption in transit or a registry serving the wrong object; both are treated as hard failures, not warnings.

The verification applies at every level, not just to layers. A descriptor is a small structure containing mediaType, digest, and size. Descriptors point at the config blob, at each layer, and at the manifest itself when one manifest is referenced by an index. Because the manifest is fetched by digest, and the manifest contains digests of its children, a single verified digest at the top authenticates the entire tree. A tag is only a pointer into that tree; the tree is self-verifying once the root digest is known. The structure of that tree, and the order in which its blobs are fetched, is described in From Image Manifest to Mounted Root Filesystem.

Sizes are part of the claim too, for a practical reason: they let a client reject an oversized response before buffering it, and they let a scheduler decide what to fetch first. A descriptor whose size does not match the received byte count is invalid even if the digest matches, because the two disagree about the same object.

Why sha256 and not a shorter hash

The security property needed is second-preimage resistance: given a digest, it must be infeasible to construct different content with the same digest. Collision resistance is needed as well, since an attacker who can produce two different layers with equal digests can substitute one for the other in a manifest they control.

sha256 produces 256 bits, and the generic collision cost is around 2^128 operations by the birthday bound. Truncating to 128 bits would halve the output and cut the birthday bound to roughly 2^64, which is within reach of well-funded adversaries and, more importantly, within reach of accidental collisions in very large content stores. The cost of the longer digest is 32 extra bytes per object in a manifest, which is irrelevant next to the megabytes of layer data it protects.

The algorithm is not fixed by the format. Descriptors carry the algorithm name, and registries have experimented with others. In practice sha256 is universal because changing it requires every client, registry, and stored image to agree, and the benefit is marginal.

Canonicalisation, or the same JSON twice

Hashing JSON is treacherous. Two documents that a parser considers equal can serialise to different bytes: key order, whitespace, Unicode escaping, and number formatting all vary between implementations. If the digest is computed over whatever bytes a tool happened to emit, two builds of the same logical image produce different manifest digests, and nothing deduplicates.

The OCI specification constrains this where it can: JSON documents are UTF-8, and the digest is taken over the exact bytes stored, not over the parsed value. A tool that rebuilds a manifest must therefore reproduce the byte sequence, not merely an equivalent document. RFC 8785 defines a general JSON canonicalisation scheme for that problem, sorting keys and fixing number representation so that independent implementations agree on one byte sequence.

A consequence worth internalising: editing a manifest by hand and re-uploading it under the old digest is impossible. Any change, including reformatting, produces a new address. Tools that “update a tag to point at a fixed image” are creating a new manifest, not patching the old one.

Deduplication is a free consequence

Because addresses are derived from content, identical content has one address. Two images built from the same base image share those layer blobs exactly, and a registry that stores blobs by digest stores them once. The same applies within a host’s local store: pulling ten images that share a base costs one copy of the base’s layers.

The sharing is at the granularity of whole layers, which is coarse. A layer is shared only if its entire tar stream is byte-identical, and tar streams include metadata that changes easily: file modification times, ownership, and the order in which directory entries were written. Two builds that install the same packages from the same repository can produce different layer digests because of a timestamp or a directory traversal order. Reproducible builds address this by normalising those fields, but the format itself only rewards byte equality.

Below the layer level there is no sharing at all in the standard model. A 200 MB layer and a 201 MB layer that differ in one file share nothing, because each is one blob with one address. Chunk-level deduplication exists in some storage backends, where a layer is split into content-defined chunks and each chunk is addressed separately, but that is a property of the store rather than the image format, and a client pulling from a registry still receives whole blobs.

Tags are mutable, digests are not

The registry API separates the two namespaces. A tag lives in a repository and is updated by pushing a manifest; resolving GET /v2/<name>/manifests/<tag> returns the manifest with a Docker-Content-Digest header naming the immutable address. Fetching by digest returns the same bytes forever, or a 404 if the blob has been deleted.

This is why pinning by digest is the only way to guarantee that what runs today is what ran yesterday. It also creates a garbage collection hazard: a tag can be moved off an image, leaving the old manifest unreferenced but still present, and a latest tag that is rebuilt nightly accumulates one manifest per build.

Deletion is therefore not reference-counted in the way a filesystem is. The registry stores blobs without knowing which manifests use them, because a blob can be referenced by any manifest in any repository that shares the store. Garbage collection has to invert the relationship: walk every manifest and index that is still reachable from a tag or an explicit digest, collect the set of digests they mention transitively, and delete blobs outside that set. The walk must be run with uploads and pushes quiesced, or a manifest written mid-walk can reference a blob the collector has already decided is unreachable. Registries expose this as an offline or read-only maintenance operation for exactly that reason.

A second hazard is the reverse: an image referenced only by digest, with no tag, is indistinguishable from garbage to a collector that starts from tags. Deployments that pin by digest and rely on the registry’s garbage collector will eventually find their image gone after a collection pass unless the digest is recorded somewhere the collector consults.

Addresses as cache keys

Local runtimes use the same addresses for their own bookkeeping. A layer already present in the content store is not fetched again, and unpacked layer directories can be keyed by the layer’s diff_id, so the expensive step (extracting a tar archive of thousands of files) happens once per layer per host. The mapping from layer digest to unpacked directory is what lets a hundred containers start from one image without a hundred extractions.

The failure mode here is partial state. An interrupted pull leaves a blob whose digest has not yet been verified, and a partially unpacked layer directory. Both must be distinguishable from complete objects, which is why stores write to a temporary name and rename into place, and why unpacked directories are marked complete only after the last entry is applied. A store that treats presence as completeness produces containers that fail at exec with missing files, and the error surfaces far from the cause.

Cross-repository sharing in a registry, covered in Registry Protocols: Blob Uploads, Ranges and Cross-Repository Mounts, uses the same mechanism: a manifest can declare that one of its layers already exists in another repository, and the registry links the existing blob rather than transferring it again, provided the client is authorised for both. The transfer is skipped precisely because the digest proves the content is identical.

What the model cannot express

Content addressing gives immutability and verifiable integrity, and it gives nothing else. It does not say who created a blob, when, or whether it is safe. A digest proves that the bytes are the bytes, not that they were built from the source they claim. Signatures and attestations are separate objects attached to a digest, and they are only as trustworthy as the key that signed them and the policy that checks it.

It also cannot express a delta. There is no “this layer is the previous layer plus a patch” in the format; every layer is self-contained. Systems that want delta transfer implement it above the format, computing the difference between two known blobs and shipping that, with the receiving end reconstructing the target and verifying its digest. The verification step is what keeps such schemes honest: reconstruction is untrusted until the resulting bytes hash to the address that was requested.

The address is the contract. Everything else in image distribution is an optimisation on how those bytes travel, and every optimisation is allowed only because the final hash check catches it when it goes wrong.