Distributed Systems · Replication

Synchronous and Asynchronous Replication: Where the Commit Point Sits

How the moment a write is acknowledged determines durability, latency, and the amount of data a failover can lose, and why semi-synchronous replication is the usual compromise.

A replicated write passes through several distinct moments: it is received by a node, written to that node’s memory, appended to its log, flushed to that node’s disk, shipped over the network, received by another node, and flushed there. The client’s acknowledgement is sent after some subset of those moments. Which subset is the entire content of the synchronous/asynchronous distinction, and it decides both the latency the client observes and the amount of data a crash can erase.

The commit point is a protocol decision, not a disk setting

A single-node database has one commit point: the moment the log record is durable on that node’s storage. fsync returning is the usual proxy for that, and the durability guarantee is exactly as strong as the storage’s promise about its write cache.

With replication, there are several candidate commit points and the protocol must name one:

  1. The leader has the record in memory.
  2. The leader has the record in its log buffer, not yet flushed.
  3. The leader has flushed the record to its own disk.
  4. The leader has flushed, and at least one follower has received the record in memory.
  5. The leader has flushed, and at least one follower has flushed.
  6. A majority of the replica set has flushed.

Acknowledging at (1) gives the lowest latency and the weakest guarantee: a leader crash loses the write and the client was told it succeeded. Acknowledging at (6) gives the strongest guarantee available without a consensus protocol and costs at least one network round trip plus the followers’ flush time.

The distinction between (4) and (5) is the one most often blurred in descriptions. A follower that has the record in its receive buffer will lose it if it crashes before writing, and will lose it if its operating system crashes. Whether “synchronous” means “the follower has it in memory” or “the follower has it on disk” changes what a simultaneous power loss of leader and follower can destroy.

The latency arithmetic

In a single data centre, the round trip between two machines is on the order of 0.1–0.5 ms, and a log flush to a local SSD is on the order of tens to hundreds of microseconds. A synchronous commit at (5) therefore adds roughly the network round trip plus the follower’s flush to every write, which is commonly a small multiple of the local commit cost.

Across regions the network term dominates. A round trip between Europe and North America is on the order of 70–100 ms depending on the city pair. A synchronous commit at (5) to a cross-region follower adds that round trip to every write. At 100 ms per write, a single connection can complete at most ten writes per second, and the achievable rate scales only by adding concurrent connections. This is not a throughput limit of the hardware; it is the arithmetic consequence of making the client wait for a signal that must cross an ocean.

Asynchronous replication at (3) removes the network term from the client’s path entirely. The leader acknowledges after its own flush and ships the log in the background. Writes cost what they cost locally, and the replication lag becomes a buffer of data that exists only on the leader.

The trade is not latency against throughput in the abstract. It is latency against the size of the window that a leader failure can erase.

What a failover loses

Under asynchronous replication, the leader’s log tail has been acknowledged to clients but not yet applied on any follower. When the leader fails and a follower is promoted, that tail is gone. The clients were told the writes succeeded. This is not a theoretical window: it is bounded by replication lag, which under load can be seconds or, when a follower is saturated, minutes.

The standard mitigation is to compare log positions before promoting. A follower that is furthest along is chosen, and the amount of lost tail is the difference between the old leader’s last position and that follower’s. Some systems expose this difference as a metric precisely because it is the quantity operators need to know.

Under synchronous replication the window is empty by construction: a write was acknowledged only after a follower had it, so any promoted follower has every acknowledged write. That statement holds only if the set of nodes that acknowledged is not itself lost. With one synchronous follower, losing the leader and that follower simultaneously loses everything the pair held. Synchronous replication to one node converts “a leader crash loses the tail” into “a correlated crash of leader and follower loses the tail”, which is a smaller but not empty window.

This is why quorum-based commit at (6) exists. A majority quorum intersects every other majority, so any node that participates in a future leader election has seen every acknowledged write. The price is that a majority must be reachable for writes to proceed, which is the availability cost examined in quorum intersection and stale reads.

Semi-synchronous: acknowledge on receipt, promote on flush

The usual compromise acknowledges as soon as one follower has received the record, and treats the record as durable once it reaches disk. PostgreSQL’s synchronous replication has this shape: a commit that requires a synchronous standby waits for confirmation that the standby has written and flushed the WAL record, and the degree of waiting is governed by synchronous_commit. MySQL’s semi-synchronous replication takes the weaker form, where the leader waits only for a replica to acknowledge receipt of the event rather than its flush.

The important subtlety is what happens when the synchronous follower disappears. A leader that waits indefinitely for a specific follower becomes unavailable when that follower is down. Implementations therefore degrade: after a timeout, the leader reverts to asynchronous mode and continues accepting writes. That degradation is a silent change in the durability contract. The system keeps running, and the amount of data at risk quietly grows. Some implementations require an explicit configuration change to allow the degradation; others do it automatically. Either way, the guarantee during the degraded period is the asynchronous one.

A further distinction separates “wait for any one of a set” from “wait for this specific node”. Waiting for a specific node keeps the durability target predictable but couples availability to that node’s health. Waiting for any member of a set keeps writes flowing when one member dies but means the identity of the durable copy changes over time, which matters for the failure-correlation argument above: the node holding the acknowledged copy may be in the same rack as the leader.

Reading the guarantee from the write path

Because the commit point is a protocol property, it can be checked from the outside by asking what a client is promised at the moment of acknowledgement. Three questions settle it:

  • If every node in the cluster loses power immediately after the acknowledgement, is the write present after restart? If the answer is yes for arbitrary subsets, the write was committed to a majority. If it is yes only when the leader survives, the write was local.
  • If the leader is destroyed and a follower is promoted, can the write be missing? If yes, replication was asynchronous with respect to the acknowledgement, regardless of how the log shipping is described.
  • If the network between leader and follower is cut, do writes continue? If yes, the acknowledgement did not depend on the follower, so the guarantee is asynchronous during the partition.

The answers do not always align. A system can ship logs synchronously and acknowledge asynchronously, or acknowledge on follower receipt and flush lazily. Reading the configuration names is unreliable; the ordering of acknowledgement against the durable write on the follower is what defines the contract.

Coupling with the read path

The commit point determines what a reader can see. A follower applying an asynchronously shipped log trails the leader by the shipping delay. A read served there can return a value older than one already acknowledged to another client. Whether that is acceptable depends on the session, not on the replication mode, which is why the guarantee a client needs is usually expressed per session rather than per cluster — the subject of session guarantees and read-your-writes consistency.

There is a related effect on the read path when the leader is the only node with the latest data: reads that must observe their own writes have to be routed to the leader, or the follower must be able to report how far it has applied and the reader must wait. Both options turn the read into a partially synchronous operation, which is a cost that the asynchronous write path appeared to have avoided.

Where the choice is not free

Asynchronous replication is the default in most single-leader systems because it makes the common case cheap and pushes the failure case into a rare event with a measurable, bounded cost. Synchronous replication is chosen when the cost of losing an acknowledged write exceeds the cost of adding a round trip to every write — typically financial records, coordination metadata, and anything where a lost write requires reconciliation with an external party.

The two can coexist per transaction. A database may commit most writes asynchronously and offer a per-statement durability setting that forces a synchronous wait for a specified class of writes. The result is a system where the durability contract is a property of the statement rather than of the cluster, which is more honest than a cluster-wide label but harder to reason about, because the guarantee now depends on which code path issued the write.

References

  1. PostgreSQL Documentation — Warm Standby and Synchronous Replication — PostgreSQL Global Development Group
  2. MySQL 8.0 Reference Manual — Replication — Oracle
  3. Paxos Made Simple — Leslie Lamport, ACM SIGACT News, 2001