Why a Replication Slot Pins Write-Ahead Log and How to Bound It

Write-ahead log is recycled continuously on a primary. Once a segment has been applied to every standby and is no longer needed for crash recovery, the segment is removed or renamed for reuse, and the log’s disk footprint stays near a configured size. A replication slot changes that rule: it is a durable promise that the server will retain every byte a consumer has not yet confirmed, regardless of how far behind the consumer is or whether the consumer is connected at all.

That promise is what makes logical replication resumable across a consumer’s restart. It is also the most common way a database fills its disk with log it cannot use, because the retention rule has no implicit expiry.

What the slot actually stores

A slot is a small catalogue object holding a name, a type, a plugin name for logical slots, and a position. The position is the point in the log up to which the consumer has confirmed processing. The slot does not store data itself; it stores a pointer, and the pointer’s effect on the log’s retention is the entire mechanism.

For a physical slot the position is a log sequence number, and the server will not remove any segment containing bytes at or after that number. For a logical slot the position is more complicated: the slot records both a confirmed log position and a catalogue xmin, the oldest transaction whose catalogue changes the decoder might still need to interpret. The second value is why a logical slot can pin storage that is not log at all — old versions of catalogue rows and, indirectly, dead tuples in system catalogues, which the vacuum horizon cannot reclaim.

The retention arithmetic is simple and unforgiving. If a primary generates log at a steady rate and a slot’s confirmed position does not advance, the retained volume grows at that rate without limit. A primary writing a gigabyte of log per hour with an idle slot accumulates twenty-four gigabytes a day, and the failure is not a slowdown but a full filesystem, at which point the primary stops accepting writes because it cannot write log.

The idle-slot case

The consumer does not need to be running for the slot to retain. This is deliberate: the whole point of a slot is that a consumer can be offline for a maintenance window and resume from where it stopped. But the same property means a decommissioned consumer, a misconfigured one, or one whose credentials expired will silently pin the log forever.

Two settings exist to bound the damage, and both are off by default.

max_slot_wal_keep_size sets a limit on the volume of log a slot may retain. When the retained volume would exceed it, the server invalidates the slot: the slot remains in the catalogue, but its restart position is lost and a consumer that reconnects must resynchronise from scratch. The default value of -1 means no limit, which is the setting that turns a stale slot into a disk-full outage.

idle_replication_slot_timeout invalidates slots that have not been advanced for a configured duration. It converts “nobody noticed the consumer died” into a bounded retention window, at the cost of invalidating a slot whose consumer was merely slow or paused for a legitimate reason. It is a newer setting than the size limit, absent from releases before it was added, so a server that has been upgraded from an older major version may not have it available at all.

The two are complementary rather than redundant. The size limit bounds the disk consequence of a fast log producer; the time limit bounds the consequence of a slow one. A system with both configured has a worst-case retention that is the smaller of the two bounds.

-- retained log per slot, and whether the slot can still be used
SELECT slot_name,
       slot_type,
       active,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained
  FROM pg_replication_slots
 ORDER BY pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) DESC;

The active column distinguishes a slot whose consumer is connected from one whose consumer is not. An inactive slot with a large retained volume is the dangerous case; an active one with the same volume is merely a lagging consumer, which is a different problem with a different remedy.

Invalidating a slot is not free

When a slot is invalidated, the log it was pinning becomes removable, and the immediate crisis passes. The consequence arrives later, at the consumer. A logical consumer that reconnects to an invalidated slot cannot resume; it must be recreated and must re-read the entire table it was replicating, which for a large table is a full copy plus the change stream generated during that copy. On a primary, that copy competes with production I/O for as long as it runs.

The resynchronisation cost is why the size limit is chosen with care rather than set to something small. A limit that is too tight converts a transient consumer outage into a repeated full resync, each of which produces its own log volume and its own load. A limit that is large enough to survive a normal maintenance window and small enough to leave the filesystem headroom is the target, and it depends on the log generation rate, which is the number to measure first.

Log generation rate, and why it is larger than the write rate

The rate at which log accumulates is not the rate at which data is written. Every page modification is logged at least once, and the same page modified several times produces several records unless full-page writes coalesce them. Bulk operations — index builds, table rewrites, backfills — generate log in proportion to the data they touch, and a single concurrent index build can produce more log in an hour than a week of ordinary traffic.

The setting that most affects the baseline is wal_level. At minimal, the server logs only what crash recovery needs and omits the information required for replication and for point-in-time recovery; at replica it logs enough for both physical replication and archive recovery; at logical it additionally logs the information the decoder needs to reconstruct row changes, which for a tuple-heavy workload is substantially more per change. A slot requires at least replica for physical and logical for logical, so the retention problem and the increased generation rate arrive together.

Full-page writes are the other large multiplier. When a page is first modified after a checkpoint, the entire page is written into the log rather than the delta, so that a torn page can be repaired during recovery. That makes the log volume for a workload that touches many distinct pages larger than the sum of its changes, and it is why a large scan-and-update job can generate log several times the size of the data it changes.

Slots and standbys

A physical slot used by a standby changes the arithmetic in a way that is easy to get wrong. The slot guarantees the primary retains log the standby has not received, which is what allows the standby to reconnect after a network partition without a full rebuild. But if the standby is down for a long time, the slot pins exactly the log the primary would otherwise have recycled, and the primary’s disk fills with log that will only ever be sent to a server that is not there.

The interaction with max_slot_wal_keep_size is the reason that setting exists in the physical case. Without it, a standby that is down for a week is indistinguishable, from the primary’s point of view, from a standby that is down for a minute. The operator’s decision is whether to preserve the standby’s ability to catch up at the cost of unbounded log retention, or to bound retention at the cost of requiring a rebuild after a long outage. A rebuild is a full copy of the cluster, which for a large database is hours of I/O; the retention bound should be set so that a rebuild is the rarer outcome.

Cascading replication adds a second hop where the same rule applies, and the intermediate server’s own log retention is governed by its downstream’s confirmed position. A slot on the intermediate server pins the intermediate’s log, not the primary’s, so the two bounds are independent and both must be configured.

The monitoring that prevents the outage

Three quantities, sampled together, describe the state: retained volume per slot, the age of the last advance per slot, and free space on the log’s filesystem. The first two come from the slot view; the third from the operating system.

The alarm thresholds should be derived rather than copied. Retained volume is dangerous relative to free space, not in absolute terms, so the useful metric is the ratio. The time since the last advance is dangerous relative to the maintenance window the consumer is expected to fit inside. And the filesystem’s free space should be alarmed well before it is exhausted, because the failure at exhaustion is a write refusal rather than a degradation.

The recovery procedure needs to be decided before it is needed. Dropping an idle slot frees the log immediately and is the right action when the consumer is gone for good. Leaving it and adding disk is the right action when the consumer will return and a resync is unacceptable. Both are legitimate, and which one applies is a property of the consumer’s role, which the database does not know — the slot’s name is often the only clue, which is why slot names should identify the consumer rather than being generated.