Credit-Based Flow Control and Consumer Prefetch Limits

Pull-based consumption has flow control built in: a consumer asks for records when it has room, and the broker cannot overrun it because nothing moves without a request. Push-based consumption has the opposite problem. The broker decides when to deliver, so it needs a rule that stops it from sending a message to a consumer that is not ready for one. Every push protocol solves this with a counter, and the counter’s initial value is the most consequential consumer setting in the system.

The counter goes by different names — prefetch count, link credit, receive maximum — but the mechanism is the same in outline. The consumer grants a bounded number of outstanding deliveries. The broker decrements as it sends and cannot send once the count reaches zero. In the prefetch-count form, each acknowledgement returns one unit of permission, reopening the window by exactly one message; AMQP 1.0 instead lets the receiver grant more credit in a flow frame at any time, independent of settlement.

Credit is permission, not a batch size

The distinction matters because credit is not a reservation of messages. A grant of fifty permits up to fifty unacknowledged deliveries at that moment, not the next fifty. If the consumer acknowledges quickly, the broker keeps sending; the window slides. If it acknowledges nothing, deliveries stop at fifty regardless of how deep the queue is.

In AMQP 1.0 the window is explicit on each link. A flow performative carries link-credit, which is the number of deliveries the sender may still send, and delivery-count, a sequence number that advances with each delivery. Credit is granted in units and consumed per delivery, and the specification requires the sender to stop once link-credit reaches zero. The important structural point is that credit flows in the opposite direction from messages, on the same link, without being tied to any particular message. A consumer can top up credit at any time without waiting to acknowledge.

MQTT 5.0 has the same concept under a different name. The Receive Maximum property is a two-byte integer whose maximum is 65,535; zero is a protocol error, and an absent property defaults to 65,535. A client that sets it to one guarantees it will never have more than one unacknowledged QoS 1 or QoS 2 message, which is the strictest form of stop-and-wait. Setting it to the default of 65,535 is effectively no flow control at all for a single client connection: the broker may push that many unacknowledged messages before pausing.

Prefetch as an unacknowledged count

RabbitMQ’s basic.qos sets the prefetch count per consumer or per channel. With a prefetch of ten, the broker will send at most ten messages that have not been acknowledged, and a new message is sent as each basic.ack arrives. A prefetch of zero means unlimited, which is the AMQP 0-9-1 default and a common source of unbounded consumer memory.

The per-consumer versus per-channel scope is not a detail. A shared prefetch across a channel with several consumers means the budget is divided among them, so a busy consumer can consume the entire window and leave its neighbours idle. Per-consumer prefetch gives each consumer an independent window, which is usually what a work queue wants.

The failure modes are symmetric and easy to reproduce:

• Prefetch too small. The consumer finishes its one in-flight message, acknowledges, and waits for the next to travel from broker to client. Throughput is bounded by one message per round trip, so a prefetch of one becomes a sequence of stop-and-wait exchanges. The symptom is low utilization and a consumer that looks idle while the queue backs up.
• Prefetch too large. The consumer holds a large window of messages in memory, some of them far from being processed. If it dies, all of them become redeliverable at once, and the group thrashes. In a multi-consumer queue, a large window also lets one consumer grab more than its share, producing the familiar pattern of one busy worker and several idle ones.

The useful sizing rule comes from Little’s law: the number of in-flight messages needed to keep a consumer busy equals its processing rate multiplied by the round-trip time to the broker. A consumer that processes two hundred messages per second over a link with a five-millisecond round trip needs about one outstanding message to stay saturated; latency, not queue depth, determines the window.

class CreditWindow:
    """Sender-side view of a link-credit window."""
    def __init__(self, credit: int):
        self.credit = credit

    def send(self) -> bool:
        if self.credit == 0:
            return False          # sender must wait for a flow frame
        self.credit -= 1
        return True

    def drain(self, count: int):
        if count == 0:
            return                # zero drain is not a top-up
        self.credit += count      # credit is granted, never refunded on send

Zero is a special value in several of these protocols, and for different reasons: basic.qos reads zero as unlimited, MQTT rejects zero as a protocol error while treating an absent property as 65,535, and AMQP 1.0 reads zero credit as “stop sending”. A client library that maps one convention onto another can invert the meaning of the safest-looking value.

Where the window sits in the delivery path

A push protocol can have several windows in series: one on the consumer link, one on the broker’s internal queue toward the dispatcher, and one on the connection to the disk. The smallest window governs the flow, and it is not necessarily the one that is configured. A prefetch of a thousand is meaningless if the broker’s egress buffer for that connection holds fifty messages and its writer blocks.

This is why raising prefetch sometimes produces no improvement. The consumer asked for a wider window, the broker filled the internal buffer, and the connection saturated. The observable is a queue with messages ready and a consumer whose connection is at its write limit. Diagnosing it means measuring at each stage rather than assuming the configured value is the operative one.

There is also an ordering interaction. A single consumer connection carries deliveries in order, so a slow handler at the head of the window delays every message behind it on that link even if the consumer has capacity elsewhere. A larger prefetch does not fix head-of-line blocking; it only gives the consumer more queued work to be blocked behind.

Pull, long polling, and the shape of the same problem

A pull-based system moves the decision to the consumer but does not eliminate the need for a bound. A consumer that fetches a thousand records into memory has the same problem as one with a prefetch of a thousand; it has just chosen the number at fetch time instead of at subscribe time. The advantage of pull is that the consumer cannot be surprised: it knows exactly how much it asked for.

Long polling sits between the two. The consumer issues a receive that blocks on the broker for up to a bounded interval, typically twenty seconds in SQS, and returns as soon as a message is available or the interval expires. Latency is low without a persistent push channel, and the in-flight count is per request, so the window is naturally the number of messages requested per call, which is capped by the protocol at ten for SQS.

Both designs converge on the same invariant: the number of messages a consumer can be holding must be bounded by something the consumer controls, and that bound must match the concurrency it can actually use. A bound below the useful concurrency wastes capacity; a bound above it converts the excess into memory and into a larger redelivery burst after a crash, which is how a single failure becomes the repeated reprocessing described in delivery semantics and duplicate windows, with the lease timing covered in acknowledgements, offsets, and visibility timeouts.

The bound also interacts with the broker’s own memory. A push broker that has promised credit to ten thousand connections must be prepared to hold the corresponding deliveries in its outbound buffers, even if every consumer is slow. Credit is a promise in both directions: the consumer promises to accept, and the broker reserves the capacity to send. Setting a very large prefetch on every consumer multiplies the broker’s per-connection buffer requirement by the same factor, which is why the setting is a capacity decision for the cluster and not only a tuning knob for the client.