When a Snapshot Is Taken and How Commit Order Is Assigned

A snapshot is a decision about which committed transactions are visible, made at one instant and then frozen. Two questions determine what that instant is: when the engine takes it, and how it orders transactions that commit concurrently. The first is an API-level choice with observable consequences. The second is a shared-memory problem that constrains how the engine can be built.

The snapshot is a set of exceptions

The naive formulation of a snapshot is a timestamp: everything committed before time T is visible. This works only if commit order is total and every participant agrees on it. The formulation actually used is a set of exceptions: a low watermark, a high watermark, and the list of transactions that were in progress between them.

The high watermark is the next transaction identifier to be assigned, captured when the snapshot is taken. Anything with an identifier at or above it had not started yet and is invisible. The low watermark is the oldest identifier that any in-progress transaction might still need, and everything below it is either committed or aborted with no ambiguity. The in-progress list holds the identifiers between the two.

The list is the expensive part. Every visibility check against a snapshot must determine whether a given identifier appears in it, which is a lookup into a variable-length structure. Engines therefore keep the list in a compact form — a sorted array of identifiers, or a bitmap over identifier ranges — and accept that the check is a binary search rather than a comparison.

/* The read view, reduced to the three fields that decide visibility. */
typedef struct ReadView {
    TransactionId low;        /* below this: status is resolved, no xip scan */
    TransactionId high;       /* at or above: started after the snapshot */
    TransactionId *running;   /* identifiers in progress, sorted */
    int           nrunning;
} ReadView;

static int
xid_visible(ReadView *rv, TransactionId xid)
{
    if (xid >= rv->high)
        return 0;                       /* had not started yet */
    if (xid >= rv->low &&
        bsearch(&xid, rv->running, rv->nrunning,
                sizeof(TransactionId), cmp_xid) != NULL)
        return 0;                       /* running at snapshot time */
    return did_commit(xid);             /* finished: consult the commit log */
}

An identifier below low is not automatically visible; it is resolved, meaning its fate is already recorded and no in-progress list needs to be consulted. The commit log still decides whether it was committed or aborted, and an aborted transaction’s version is invisible. Above low, the sorted in-progress array decides whether the in-progress list must be consulted at all: present means the transaction was running when the snapshot was taken, so its work is invisible; absent means it had already finished, and the commit log resolves it.

The three-way structure is what makes the check cheap in the common case. Most identifiers a reader encounters are either well below low or were assigned after the snapshot, so the search in the narrow window is what the reader pays for.

Transaction-start versus first-statement

Taking the snapshot at BEGIN gives the transaction a view anchored to the moment it started waiting. Taking it at the first statement that reads gives a view anchored to when work actually began.

The difference is observable and sometimes surprising. A transaction that begins, performs administrative work for several seconds, and then reads will, under the first-statement policy, see everything committed during that interval. Under the transaction-start policy it will not. Neither is wrong; they define different levels. The first-statement policy is friendlier to connection pools and to interactive sessions where the client opens a transaction early and issues statements later. The transaction-start policy gives a guarantee that is easier to reason about, because the snapshot boundary is the same for every statement in the transaction.

There is a third variant, per-statement snapshots, which is how read committed is usually implemented. Each statement gets its own read view, so a statement is internally consistent and successive statements are not. This is the mechanism that makes read committed non-repeatable across statements while keeping individual scans free of torn reads.

Engines mix these. A system may take the snapshot at the first read for its snapshot level and per statement for read committed, which means the level’s identity is partly a policy about when the read view is captured rather than a property of the version storage. The version storage is identical in both cases; only the boundary moves.

Why the snapshot must be exported

A snapshot that is internal to one transaction cannot be shared with another process, which makes it useless for the common pattern of a parallel query or a set of worker processes that must all see the same state.

The mechanism that solves this is to serialize the read view into a form another session can import: the three fields, plus a validity interval so that the importing session knows when the snapshot stops being safe. The importing session can then read with the same visibility as the exporter. The cost is that the exporting transaction must remain open, since closing it may allow version collection to remove data the imported snapshot still needs. This is why an exported snapshot pins the same resources as a long-running transaction, and why the pin is a source of the storage pressure described in vacuum horizons and dead version collection.

The serialized form must also carry the identifier of the transaction that exported it, because the importer’s own identifier differs and its own writes must remain visible to itself. That is a detail with a consequence: an imported snapshot is not simply a copy of the visibility set, it is a visibility set plus a substitution for the reader’s own identity.

Commit order as a shared sequence

First-committer-wins validation needs to know which transactions committed after a given snapshot. That requires a total order over commits, assigned at the moment of commit, and agreed upon by everyone who might validate against it.

The straightforward implementation is an atomic counter incremented at commit under a short critical section. Every committing transaction takes a sequence number, and a validating transaction compares its snapshot boundary against the sequence numbers of the write sets it must check. The critical section is tiny — increment and publish — but it is a single point of serialization for all commits in the system.

import threading

class CommitSequencer:
    def __init__(self):
        self._lock = threading.Lock()
        self._seq = 0
        self._by_seq = {}            # seq -> write set, for recent commits

    def commit(self, write_set):
        with self._lock:
            self._seq += 1
            seq = self._seq
            self._by_seq[seq] = write_set
            return seq               # total order, agreed by all validators

    def conflicts(self, start_seq, write_set):
        with self._lock:
            return [s for s, ws in self._by_seq.items()
                    if s > start_seq and ws & write_set]

The lock is held for the duration of the increment and the publication, not for the validation scan, which is why the counter scales to high commit rates even though it is global. What does not scale is the second part: the validator must examine the write sets of all transactions that committed since its snapshot, so a long-running transaction with a large write set pays a scan proportional to the commit rate times its own lifetime.

The alternative to a global counter is a distributed timestamp service, and the alternative to that is physical clocks. Both reintroduce the problem the counter solves. A timestamp service is a network round trip on the commit path; physical clocks disagree, and disagreement means two transactions can each believe they committed before the other’s snapshot, producing exactly the anomaly validation is meant to prevent. This is why systems that partition writes still route commit ordering through a single authority for the items they must mutually exclude.

The gap between commit and visibility

A transaction’s commit sequence is assigned before its effects are visible to other transactions. Between the assignment and the moment another reader’s snapshot includes it, there is a window in which the transaction has committed but is not yet visible.

The window is unavoidable because making the effects visible requires publishing to a shared structure — a commit log, a status table, a set of in-memory flags — and the publication takes time proportional to the number of participants. During the window, a reader that takes a snapshot will not include the transaction, even though the commit happened first in real time.

This is not a correctness problem as long as the snapshot boundary is defined in terms of the published state rather than the sequence number. The snapshot captures the set of transactions whose commits have been published; a transaction whose sequence was assigned but whose publication is incomplete is simply not in that set, and the reader treats it as still in progress. The transaction’s own subsequent operations must handle the case where its commit becomes visible to a snapshot taken after its publication but with a boundary below its sequence, which is why the publication protocol must make the transaction’s state and its sequence number consistent at the same instant.

The consequence for the application is a small delay between a commit returning and another session being able to see the committed rows. In a single-node engine the delay is microseconds and invisible. In a distributed engine it is a network hop, and applications that read their own writes through a different connection must be aware that the read may not see them yet.

Choosing the boundary deliberately

The design decisions here are not independent. Taking the snapshot at first statement makes exported snapshots more useful, because the exporting session holds the view for less time. A global commit counter makes validation exact but puts every commit through one lock. An in-progress list that grows with concurrency makes visibility checks longer during peak load, which is a load-dependent cost that shows up as CPU rather than as waiting.

The observable behaviour that follows from these choices is what distinguishes isolation levels in practice, more than the abstract definitions do. Two engines can both claim snapshot isolation and differ on whether a transaction that begins and waits sees writes committed during the wait, on whether an exported snapshot can be shared across sessions, and on how commit latency changes as concurrency rises. The definitions in the isolation level ladder describe the anomaly set; the choices described here describe the timing, and the timing is what applications notice.