Distributed Systems · Replication

Session Guarantees: Read-Your-Writes and Monotonic Reads

How four per-session guarantees are implemented with version tracking, why they are cheaper than global consistency, and where client redirection breaks them.

Global consistency properties are expensive because they constrain every client against every other client. Most applications do not need that. They need a property that holds within one client’s interaction with the system: the user who just saved a comment must see the comment, and the user who has been reading a thread must not see it travel backwards in time. These are session guarantees, and they can be implemented with metadata carried by the client rather than by coordination among replicas.

Four guarantees and the anomalies they forbid

The classic set is four, named in the session guarantees work of Terry and colleagues:

  • Read-your-writes. A read by a session observes all writes previously made by that session. Without it, a user saves an edit, reloads, and sees the old version.
  • Monotonic reads. Successive reads by a session observe successively newer states. Without it, a user refreshes and sees content from further in the past than the previous view, which happens when consecutive reads land on replicas with different lag.
  • Monotonic writes. Writes by a session are applied in the order the session issued them. Without it, a rename and then an edit can be applied in the wrong order at different replicas.
  • Writes-follow-reads. A write by a session is applied after all writes that the session had observed before issuing it. Without it, a user replies to a comment on a lagging replica and the reply lands in a history where the comment does not exist yet.

Each is a per-session property. None of them orders one client’s operations against another’s except where the session observed the other’s write, which is exactly what writes-follow-reads encodes.

The important observation is that all four are cheap to check if each session carries metadata about what it has seen, and expensive to guarantee by coordination. A system can offer all four without a quorum protocol, and can offer none of them while claiming strong consistency at the cluster level if reads are routed carelessly.

Tracking what a session has seen

The standard implementation attaches a version to every write and lets the session accumulate the versions it has observed. Two forms are common.

The first is a per-session vector: the session keeps the version vector of the last write it made, plus the vector of the last state it read. On a read, the client sends the vector it requires, and the replica either satisfies it or refuses. A replica satisfies the requirement if its own applied state dominates the requested vector, meaning it has seen at least everything the session has seen. If it does not, the read is redirected to a replica that does, or the client waits.

The second form is a token: an opaque handle identifying a point in the replication stream. A write returns a token; the read presents it and the serving replica checks whether it has applied past that token. Tokens are smaller than vectors and easier to pass through a load balancer, at the cost of being tied to the replication stream’s ordering rather than to per-key causality.

class Session:
    def __init__(self):
        self.read_pos = 0    # stream position of the last state this session observed
        self.write_pos = 0   # stream position of this session's last own write

    def required(self):
        # read-your-writes is the stronger of the two requirements
        return max(self.write_pos, self.read_pos)

    def on_read(self, pos, value):
        self.read_pos = max(self.read_pos, pos)
        return value

    def on_write(self, pos):
        self.write_pos = pos
        self.read_pos = max(self.read_pos, pos)

The details differ between systems, but the structure is stable: the session holds the maximum position it has observed, and every operation either satisfies that position or is refused.

Why a replica can answer without contacting anyone

The reason session guarantees are cheap is that the check is local to the replica. A replica knows its own applied position. Comparing that position to the token the client presents requires no network round trip and no agreement with other replicas. The replica either has the state or it does not.

This is the difference from a quorum read. A quorum read must contact R replicas to find the latest value. A session-consistent read contacts one replica and asks whether it is far enough along. If the answer is yes, the read completes at single-replica latency. If the answer is no, the client is redirected to a replica that is further along, which is a routing decision rather than a consensus decision.

The cost appears when the client’s required position is ahead of every reachable replica. That happens when the client has just written to a leader and then reads from a follower: the follower may not have applied the write yet. The options are to wait, to route the read to the leader, or to fail the read and let the client retry. Waiting converts the guarantee into a latency penalty proportional to replication lag; routing to the leader converts it into a load concentration on the leader. Both are real costs and neither is avoidable — the guarantee is being purchased, and the currency is either latency or leader capacity.

Where redirection breaks the guarantee

The guarantees are properties of a session, and a session is an abstraction that the infrastructure must preserve. Three things break it.

The first is a client that does not carry its token. A browser that reloads a page, a mobile app that is killed and restarted, or a request routed through a stateless proxy that strips headers will present no token, and the guarantee silently lapses. The fix is to persist the token in something the client controls — a cookie, a local store — and to accept that a user on a new device has a new session with no history.

The second is load balancing that ignores the token. If the router sends the next request to a replica that has not applied the session’s write, the replica refuses or serves stale data. A router that understands the token can send the request to a replica that satisfies it; one that does not will produce read-your-writes violations under normal operation.

The third is failover. When the replica that held the session’s writes is lost, the token refers to a position in a stream that may have been truncated. Whether the guarantee survives depends on whether the position is still meaningful on the new replica. A token that is a log position survives if the log was replicated; a token that is a local sequence number does not.

The interaction with causal metadata

Session guarantees and causal consistency overlap but are not the same. A causally consistent store tracks dependencies between writes and refuses to expose a write before its dependencies. A session guarantee tracks what one client has observed and refuses to expose a state older than that.

Causal consistency implies read-your-writes and monotonic reads for a client that observes its own writes, because the client’s own write is a dependency of anything that follows it. The converse does not hold: a store can give every session its four guarantees while allowing a client to see a comment without the post it replies to, if the client never observed the post.

The implementation cost differs accordingly. Session guarantees need per-session metadata and a local check; causal consistency needs per-write dependency metadata and a check at every replica for every write. The former scales with the number of sessions; the latter scales with the write rate. For a read-heavy workload with many sessions and few writes, session guarantees are the cheaper mechanism. The causal metadata that makes the stronger property possible is described in version vectors and conflict detection.

Monotonic reads is the one users notice

Of the four, monotonic reads is the anomaly that produces visible bugs in ordinary applications. A feed that refreshes and shows older content, a comment thread where a reply disappears, a profile page that alternates between two states as requests land on different replicas — all are monotonic-read violations and all are produced by round-robin routing across replicas with different lag.

The fix is the token described above: the session records the position of the last read and requires the next read to be at least as new. The cost is that a session pinned to a position cannot be served by a replica that is behind it, which reduces the effective read capacity when one replica lags. Under a load balancer that spreads reads evenly, that reduction is proportional to the lagging replica’s share of the traffic.

A weaker and cheaper approximation is to route a session to the same replica for a period, sometimes called session affinity or sticky routing. That gives monotonic reads as long as the replica keeps up and the session stays on it, and gives nothing when the replica fails or the session is moved. It is a routing heuristic, not a guarantee, and the difference matters during failover: the guarantee holds until the moment it is most needed, and then it does not.

Choosing a level per operation

A system does not have to apply one guarantee to everything. The common arrangement is to require read-your-writes on operations where the user expects confirmation, allow monotonic reads on navigation, and permit reads with no session requirement on aggregate or cached views — the case usually described as eventual consistency, where the only promise is convergence in the absence of new writes. Making the choice per operation rather than per cluster is what allows a single deployment to serve both a strongly consistent settings page and a cheap feed.

The mechanism supports that granularity because the token is carried by the client and checked by the replica. An operation that does not need the guarantee simply omits the token, and the replica serves whatever it has. The operator’s job is to know which operations omit it, since those are the ones that will exhibit stale reads during replication lag and after failover.

References

  1. Session Guarantees for Weakly Consistent Replicated Data — Terry, Demers, Petersen, Spreitzer, Theimer, Welch; PDIS 1994
  2. Replicated Data Consistency Explained Through Baseball — Doug Terry, Microsoft Research Technical Report MSR-TR-2011-137
  3. Eventually Consistent — Werner Vogels, ACM Queue