Conflict Resolution When Two Regions Accept Writes

Two regions accepting writes for the same record means two writers can update it without either knowing about the other. The network partition or the propagation delay does not create the conflict; it only reveals it. The conflict exists the moment two updates are made concurrently in the technical sense — neither happened before the other — and every multi-region write design has to state what it does with that state. The choice determines which invariants the system keeps and which it silently discards.

Concurrency is not wall-clock order

The natural instinct is to order writes by timestamp and take the later one. This is last-writer-wins, and it fails for a reason that has nothing to do with clock accuracy: two writes can be genuinely concurrent, meaning no causal relationship exists between them, and any total order imposed on them is arbitrary. If a user in Frankfurt renames a record at 12:00:00.100 and a user in Oregon sets its status at 12:00:00.105, the timestamps suggest the status change came second. But if the Oregon writer had never seen the rename, the two operations are concurrent, and applying them in either order produces a state that neither user intended.

Last-writer-wins resolves this by fiat: it picks one and discards the other. For a single field, discarding the loser is defensible. For a record, the granularity matters enormously. If the unit of conflict is the whole record, the rename is lost when the status change wins, because the status change carried the old name. If the unit is the field, both survive, and the result is a record that no single writer ever saw — which is usually what the users actually wanted, but only if the fields are independent.

The consequence is that LWW is safe exactly when the data model’s fields are independent and the loss of a concurrent update is acceptable. It is unsafe for anything with an invariant across fields: an account balance and its ledger, an inventory count and its reservations, a state machine whose transitions must be legal. In those cases LWW does not resolve a conflict; it corrupts a record.

Detecting concurrency rather than ordering it

To know whether two writes are concurrent, the system needs causal metadata. A version vector is the minimal form: a map from replica identifier to the highest sequence number that replica has produced, carried with each write.

record: order-4471
value:  { status: shipped }
vector: { eu-west-1: 12, us-west-2: 7 }

# update from eu-west-1 after seeing vector above
vector: { eu-west-1: 13, us-west-2: 7 }   # dominates -> causally later

# concurrent update from us-west-2, unaware of eu-west-1:13
vector: { eu-west-1: 12, us-west-2: 8 }   # incomparable -> conflict

Comparison is element-wise. Vector A dominates B if every component of A is greater than or equal to the corresponding component of B and at least one is strictly greater; then A is causally after B and simply replaces it. If neither dominates, the writes are concurrent and the system must choose: keep both (a sibling set), merge them with a type-specific function, or invoke a resolver. Dynamo’s approach was to return all siblings to the application and let it merge, which pushes the decision into application code where the invariant is known.

The cost of version vectors is size. A vector with one entry per replica grows with the number of writers, and in a system with per-client replicas it grows without bound. Practical implementations truncate, which reintroduces the risk of missing a causal relationship: a truncated vector can make two causally ordered writes look concurrent. The truncation policy is therefore a correctness parameter, not an optimization detail.

Merge functions and what they preserve

When the system keeps both versions and merges, the merge function determines the guarantees. A commutative, associative, idempotent merge yields a convergent replicated data type: replicas that have received the same set of updates produce the same state regardless of order or duplication.

Three families cover most needs:

Counters. A grow-only counter is a vector of per-replica counts summed on read. Increments never conflict, and the merge is a per-component maximum. A decrement requires tracking positive and negative counts separately per replica, because a single per-replica maximum cannot represent “this replica decremented twice and incremented once” without losing the negative component.

Sets. A grow-only set merges by union, which is trivially convergent. A set that supports removal needs either a tombstone per element (an add-wins or remove-wins policy made explicit by the tombstone’s presence) or a two-phase structure that tracks adds and removes separately. The choice between add-wins and remove-wins is a semantic decision: for a shopping cart, add-wins is the usual choice because a removed item reappearing is less harmful than a concurrently added item disappearing; for a blocklist, remove-wins is the safe direction.

Registers. A last-writer-wins register needs a total order on writes, which brings back the clock problem. A multi-value register keeps the set of concurrent values and defers the choice, which is the version-vector approach above. Neither gives a single value without a policy; the difference is whether the policy is applied at write time (LWW, silent) or at read time (multi-value, visible).

The invariant that breaks first

Every merge policy preserves some invariants and breaks others, and the break is usually discovered in the domain logic rather than in the storage layer.

A concrete example: an inventory count replicated as a counter converges correctly on the number of items. It does not preserve the invariant that the number never goes below zero, because two regions can each sell the last unit concurrently, each decrementing from one to zero, and the merged count is negative. The storage layer did its job — the merge is correct — and the business rule is violated. Preventing that requires a reservation protocol that crosses regions before the sale, which is a synchronous operation and pays the inter-region latency floor on every purchase.

The same structure appears in unique constraints. Two regions can each create a user with the same email address, and the union of the two sets contains two records that violate the uniqueness the single-region system enforced. No merge function can fix this after the fact, because the information needed to pick a winner (which registration should be rejected) is not in the data. The options are to make uniqueness a cross-region synchronous check, to partition the key space so a given email can only be written in one region, or to accept duplicates and reconcile them with a later process that merges accounts.

Routing as an alternative to merging

The most effective way to avoid conflict resolution is to make concurrent writes to the same key impossible by construction. Three mechanisms do this.

Home-region assignment. Each key is assigned to a region, and all writes for that key go there. Writes are still fast for users near the home region and slow for users far from it, but the slow path is a routing decision rather than a conflict. The assignment can be by user, by tenant, or by hash of the key. The failure mode is that when the home region is unavailable, writes for its keys must either wait or be accepted elsewhere with a temporary reassignment that reintroduces the conflict for the duration.

Consistent hashing with failover ownership. Keys map to a ring of nodes, and a failed node’s range is taken over by its successor. This is the mechanism behind many sharded stores. It reduces conflicts to the failover window, during which the old owner may still be serving writes for its range if it has not yet learned it lost ownership. Fencing is what closes that window, and it is the subject of fencing tokens and lease expiry.

Per-key serialization through a single writer. If every write for a key is routed through one elected leader, there are no concurrent writers by definition. The leader can be per-key rather than global, which distributes the write load, but leader election per key has its own cost and its own failure modes.

Each of these converts a data problem into a routing problem, and routing problems have the advantage of being visible: a request that cannot reach its home region fails loudly instead of silently merging into a wrong state. That is usually the better failure, because a wrong merge is discovered later and by then has propagated.

Choosing honestly

The decision procedure is short. If the data has cross-field invariants, do not accept concurrent writes to it; route to a single owner and pay the latency. If the data is a set of independent fields, LWW per field is sufficient and cheap. If the data is naturally a counter or a set, use the corresponding convergent type and get convergence for free. If the data is a document with structure that users edit concurrently, keep both versions and surface the conflict to the user, because no automatic merge can know which of two edits to a paragraph was intended.

What is not defensible is a system that accepts concurrent writes, resolves them with LWW, and claims the data is consistent. It is convergent — all replicas eventually agree — and it is not consistent with the operations that were submitted. Those are different properties, and the difference is exactly the set of updates that were silently discarded.