Two-Phase Commit and the Coordinator Decision Record

A transaction that touches two nodes has no local commit point. Each node can make its own changes durable, but no single node can make the pair of them durable together, because the second node’s disk is not reachable from the first node’s fsync. The protocol that supplies the missing commit point is two-phase commit, and the part of it that people underestimate is not the messaging but the record the coordinator writes before it sends anything.

The vote is a promise, not a preference

In the first phase the coordinator sends a prepare message to every participant. A participant that receives it does the local work a commit would require: it writes the transaction’s changes, writes a prepare record in its own log, takes the locks it needs to keep those changes invisible or unrevertible, and replies yes or no. A no is unconditional and final. A yes is a contract with three parts:

1. The participant has the transaction’s effects durable enough to survive its own crash.
2. The participant will not unilaterally abort afterwards.
3. The participant will hold the locks until it learns the outcome.

The second clause is what makes the protocol work and what makes it expensive. Before voting, the participant could have chosen to abort at any moment. After voting, it has surrendered that choice to a decision it cannot see yet. That is the whole point: a commit is atomic only if every participant’s changes become durable at the same logical instant, and a participant that could still abort after voting would break the instant.

So the prepare record in the participant’s log is not an optimisation. It is the state that lets the participant answer, after a restart, “I voted yes and I am waiting”, which is a state from which no local rule can produce an outcome.

The coordinator log as the single source of truth

The coordinator’s job reduces to producing one durable fact: the decision. Before it sends the decision to anyone, it writes it to its own log. Commit records and abort records are both decisions; what matters is that exactly one of them exists and that it survives a crash.

The ordering constraint is the entire protocol:

• Write the decision record.
• Then send it.
• Then, and only then, forget the transaction.

If the coordinator crashes after collecting all votes but before writing the decision, recovery reads the log, finds no decision, and is free to choose. Abort is the safe choice, and every mainstream implementation chooses it, for reasons covered in presumed abort and presumed commit. If the coordinator crashes after writing commit but before sending it, recovery reads the log, finds commit, and resends. Participants that already received it ignore the duplicate; participants that did not receive it get their answer. The decision record converts an uncertain situation into a retransmission.

This is why the coordinator is often described as the single point of failure of two-phase commit. The description is accurate but slightly off-centre. The failure is not that the coordinator can die; it is that the decision exists in exactly one place, so while that place is unreachable, no participant can learn the outcome. Replicating the decision elsewhere removes most of that, which is the subject of replicated commit decisions.

Locks that outlive the client

Between the vote and the decision, every participant holds row locks, or their equivalent, on everything the transaction touched. Those locks are held on behalf of a transaction whose client may have disconnected, whose coordinator may be unreachable, and whose fate is unknown to the participant. From the perspective of any other transaction that wants those rows, this is indistinguishable from a very slow commit.

The duration is bounded by the coordinator’s availability, not by any timeout the participant controls. A participant cannot time out and abort, because the coordinator may already have written commit and may be about to tell everyone else. This is the blocking property of two-phase commit, and it is a direct consequence of the vote’s second clause rather than a defect in any particular implementation. It is examined from the participant’s side in in-doubt transactions and blocking.

Implementations expose the held state explicitly. PostgreSQL’s PREPARE TRANSACTION moves a transaction out of the session that ran it and into a state recorded on disk, so the prepared transaction survives both the client disconnecting and the backend process exiting. The prepared transaction keeps its locks and its snapshot; COMMIT PREPARED and ROLLBACK PREPARED are issued later, possibly from a different session. That command pair is the participant half of two-phase commit exposed to the operator, and its existence is a fair warning about the cost: a prepared transaction is a lock holder with no owner.

BEGIN;
UPDATE accounts SET balance = balance - 100 WHERE id = 7;
PREPARE TRANSACTION 'xfer-9f2c';
-- session may now end; locks and snapshot remain on disk
-- later, from any session:
COMMIT PREPARED 'xfer-9f2c';

The max_prepared_transactions setting exists because these objects are not free: each consumes a slot in shared memory and holds locks that other sessions may be waiting on. A server configured with the default of zero refuses to prepare anything at all, which is a deliberate posture for deployments that do not want the operational burden.

What the two phases cost

Counting messages for n participants: n prepares out, n votes back, n decisions out, n acknowledgements back. That is four messages per participant, two of which are round trips on the critical path before the client can be told the outcome. Add the coordinator’s decision write, which must be flushed, and each participant’s prepare write, which must also be flushed.

The latency floor is therefore one network round trip for the vote plus one for the decision, plus two durable writes. In a single data centre the network terms are small and the flush terms dominate; across regions the round trips dominate, and a commit that spans two continents pays the distance twice. No amount of pipelining removes those two sequential waits, because the decision cannot be sent before the votes are counted, and the votes cannot be counted before the participants have prepared.

Optimisations reduce the constant rather than the structure. A coordinator can send prepares in parallel rather than in sequence, which collapses the first round trip to the slowest participant rather than the sum. It can omit the final acknowledgement round when the participants’ logs already make the decision recoverable, which is what presumed abort does. It can co-locate the coordinator with one participant so that participant needs no prepare message at all, a trick that turns n participants into n-1 remote ones.

Failure cases the record resolves, and one it does not

The decision record handles coordinator crashes cleanly in one direction: a participant that cannot reach the coordinator can ask any other participant, and if any of them has received the decision, the answer propagates. It handles participant crashes cleanly in the other: on restart the participant finds its prepare record and waits, and the coordinator resends.

What it does not handle is the case where the coordinator’s log is lost. If the coordinator’s storage is destroyed after some participants voted yes and before any of them learned the decision, the transaction is unresolvable by the protocol alone. The participants hold locks, cannot abort, and cannot commit. Recovery becomes a human or an external tool deciding, and the decision must be consistent across all participants or atomicity is broken. This is the failure that motivates every attempt to move the decision onto a replicated log, and the reason “the coordinator is a single point of failure” is a statement about durability of a record rather than about the process that writes it.

A second case is subtler: a participant that votes yes and then crashes and loses its prepare record through a storage failure rather than a process failure. On restart it has no memory of the transaction and will report nothing when the coordinator asks. The coordinator may then record commit for a participant that has no durable copy of the changes. Two-phase commit assumes that a yes vote implies the record survives the crash it was designed for; when the storage lies about durability, the protocol’s guarantee is exactly as strong as the storage’s.