The Registry Pull Path From Manifest to Blob

The distribution API is a small HTTP surface (fewer than a dozen endpoints) and its complexity lives entirely in the paths a response can take. A pull is not one request but a sequence of them, each of which may be answered locally, redirected to object storage, challenged for credentials, or served in pieces. Understanding the sequence is what makes it possible to reason about where an image comes from and which parties can observe or influence the transfer.

The four endpoints that matter

A pull touches /v2/, the version check; /v2/<name>/manifests/<reference>, for resolving and fetching manifests; /v2/<name>/blobs/<digest>, for layer and config content; and /v2/<name>/tags/list, which a pull does not need but which many clients call for discovery. Everything else (blob upload sessions, deletions, cross-repository mounts) is push-side.

The version check is a plain GET /v2/ and a registry answers 200 OK. The value is in the failure: an unauthenticated registry returns 401 with a WWW-Authenticate header describing how to authenticate, and a client that skips the check will get a 401 on its first manifest request anyway. The check exists to fail early and to advertise capabilities.

Repository names are constrained: lowercase alphanumerics with separators of ., _, and -, matching a path-like pattern with up to two / characters for namespacing. A client that builds a request from user input without validating the name can be induced to send a request to an unintended path, since the name is interpolated directly into the URL path.

Token authentication is a separate service

Most registries do not authenticate the pull themselves. They answer with 401 and a WWW-Authenticate: Bearer realm="https://auth.example/token",service="registry.example",scope="repository:library/alpine:pull". The client then requests a token from the realm with those parameters, presenting whatever credentials it has (basic auth, an OAuth refresh token, cloud instance credentials) and receives a bearer token to include on subsequent requests.

Two properties of this design are worth knowing. First, the token is scoped to a repository and an action, so a token for pulling library/alpine is useless for pushing internal/app. Second, the realm is chosen by the registry and can be any HTTPS endpoint, which means a client configured for one registry will send its credentials to whatever realm that registry names. This is the reason registry configuration must be pinned to a known host rather than to a wildcard pattern: a compromised or impersonated registry can direct credential-bearing requests elsewhere. Clients mitigate this by refusing realms outside an allowlist, and by never sending credentials to a plaintext realm.

Tokens are short-lived by convention, on the order of minutes, and are cached per scope. A client that fetches a token per blob request will spend a substantial fraction of pull latency on authentication round trips; a client that caches tokens too aggressively hits 401 mid-pull and must retry. Both behaviours appear in real implementations, which is why the retry-on-401 path must be correct rather than incidental.

Manifests are fetched, hashed, and possibly refetched

The first substantive request is the manifest fetch. A client sends an Accept header listing the manifest and index media types it understands, in preference order. The registry returns the highest-preference type it can serve. This is where a client’s type support determines whether it sees an index or a single manifest, and a client that lists only Docker types against a registry serving OCI content gets an error rather than a fallback.

The response body is hashed. If the request was by tag, the resulting digest is recorded as the resolution of that tag and the body is accepted as-is; if the request was by digest, the body is compared against it. The distinction is important for caching: a tag resolution produces a digest that a client can then use for the manifest request itself, avoiding a second fetch, but only if the client trusts that the tag-to-digest mapping is current. Clients that re-resolve tags on every pull and clients that cache the mapping for a fixed interval are both common, and the choice changes the window in which a replaced tag goes unnoticed.

Manifest bodies are small (hundreds of bytes to a few kilobytes) so the fetch is dominated by round-trip time, not bandwidth. This is why pull latency for a small image is roughly the sum of a handful of RTTs, and why registries are placed close to their consumers.

Blobs arrive by redirect, often to another host

The blob endpoint is where the protocol stops being uniform. A registry may serve blob bytes itself, or it may answer 307 Temporary Redirect to a URL in object storage. Nearly all large registries do the latter, because streaming multi-megabyte blobs through an application server is wasteful when the storage backend can serve them directly.

The redirect target is usually pre-signed and short-lived. The client must follow it, and must not send the registry’s bearer token to it; the storage host did not ask for it and, under RFC 6750, a bearer token should only be sent to the party it was issued for. Implementations differ in whether they strip the Authorization header on cross-origin redirects; the safe behaviour is to strip it whenever the authority changes.

Verification happens after the bytes arrive, not before. The client hashes the stream as it writes it into the local content store, and on mismatch deletes the partial file. This means a corrupt or malicious response costs the bandwidth of the blob before it is rejected, which is an unavoidable consequence of content addressing: the hash can only be checked once the content exists.

func pullBlob(ctx context.Context, c *http.Client, url, digest string) (io.ReadCloser, error) {
    req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    req.Header.Set("Accept", "application/octet-stream")
    resp, err := c.Do(req)
    if err != nil {
        return nil, err
    }
    if resp.StatusCode != http.StatusOK {
        resp.Body.Close()
        return nil, fmt.Errorf("blob fetch: %s", resp.Status)
    }
    // Hash on the way to disk; the caller decides whether to commit.
    pr, pw := io.Pipe()
    go func() {
        h := sha256.New()
        n, err := io.Copy(io.MultiWriter(pw, h), resp.Body)
        resp.Body.Close()
        if err == nil && "sha256:"+hex.EncodeToString(h.Sum(nil)) != digest {
            err = errors.New("digest mismatch")
        }
        _ = n
        pw.CloseWithError(err)
    }()
    return pr, nil
}

The pattern is deliberately streaming: the blob never has to fit in memory, and the digest check is fused with the write so there is no separate verification pass over stored bytes. The cost is that a mismatched blob is written and then discarded rather than rejected before touching disk.

Ranges, resumption, and the cost of a restart

The distribution API supports Range requests on blobs, and clients use them to resume an interrupted download. The benefit is real for large layers over unreliable links, and the cost is that a resumption must still verify the whole blob at the end, because a range request does not attest to the bytes outside the range. A client that resumes into a partially written file and then verifies the final digest is correct; a client that verifies only the resumed segment is not.

Registries are not required to honour ranges, and a response may be a plain 200 with the entire body, which a naive resumption path will treat as a new full download and possibly append to an existing partial file. The client must compare the response status and the Content-Range header before deciding what to do with the bytes.

Parallelism is the other lever. Layer blobs are independent objects, so a client can fetch several concurrently, and the limit is set by connection pool size, registry rate limits, and the storage backend’s tolerance. The manifest must be fetched first to learn the layer list, so there is a strict serialisation point at the front of every pull, and a fan-out afterwards. Registries commonly rate-limit by request count rather than by bytes, which penalises clients that split a large layer into many ranged requests.

What each hop can see

The set of parties observing a pull is larger than the endpoint in the reference. The registry sees which repositories and digests a client requests, and when. The token service sees the identity presented and the scopes requested. Object storage sees the redirect target being used, and the source address. A mirror or proxy in between sees all of it. Each of these is a place where the client’s assumption about where content came from can be wrong.

The protocol offers no confidentiality for the fact that a pull happened, and no integrity for the tag-to-digest binding beyond TLS. What it does offer is that the bytes are checked against a digest the client chose, which bounds the damage any of those parties can do: they can deny service, they can observe, but they cannot substitute content for a digest without breaking the hash.

Where the digest itself comes from is the question the rest of this cluster answers. A client that pins digests removes the registry’s discretion over content; a client that verifies signatures removes the registry’s discretion over which digest is acceptable. Both depend on the pull path described here being implemented correctly, following redirects without leaking credentials, decoding transport encodings before hashing, and verifying the complete blob rather than a segment. The layer caching that sits on top of this path, and the way cache keys determine whether two pulls share work, is covered in Layer Cache Keys and What Makes a Build Reproducible.