Idempotency Keys and the Retry Window for Writes

A client sends a payment request. The connection drops after the server has committed the charge but before the response reaches the client. The client cannot distinguish this from a request that never arrived, so its only sane behaviour is to retry. If the retry creates a second charge, the failure has cost money; if it does not retry, and the original request was in fact lost, the order never completes. There is no information available to the client that resolves the ambiguity, and no amount of client-side cleverness supplies it.

RFC 9110 solves this for the methods it classifies as idempotent. PUT, DELETE, and the safe methods GET, HEAD, OPTIONS, and TRACE may be repeated with the expectation that the effect is the same as a single application. POST is explicitly not in that set, because a POST is defined by what it does rather than by what it addresses. Creating a resource, capturing a payment, sending a message: each is a fresh event every time it is submitted.

The idempotency key pattern extends the guarantee to those methods. The client generates an identifier before the first attempt and reuses it for every retry of the same logical operation. The server treats two requests carrying the same key as two deliveries of one request.

What the server must record, and when

The guarantee is not “deduplicate the request”. It is “return the same result to the retry that the original produced”. Those differ, and the difference is where implementations go wrong.

Three things have to be persisted, keyed by the client’s identifier: the state of the operation, a fingerprint of the request, and the response that was returned. The critical step is the ordering. The record must be written before the side effect becomes visible, and the side effect and the record must commit atomically if the effect lives in the same transactional store. A design that performs the write and then inserts the key leaves a window in which a crash loses the key while keeping the charge, so the retry charges again.

The usual shape is a unique constraint on the key column. The first request inserts a row in a pending state, performs its work in the same transaction where possible, and updates the row to completed with the serialised response. A retry hits the unique constraint and reads the existing row instead of doing the work. Stripe’s API documents the client-facing half of this contract: a client-supplied key and a stored response replayed for retries with that key.

// Handle charges one POST per idempotency key.
func (s *Server) charge(w http.ResponseWriter, r *http.Request) {
	key := r.Header.Get("Idempotency-Key")
	if key == "" {
		http.Error(w, "Idempotency-Key required", http.StatusBadRequest)
		return
	}
	fp := fingerprint(r) // hash of method, path, and body; must restore r.Body for process

	rec, created, err := s.store.Claim(r.Context(), key, fp)
	if err != nil {
		http.Error(w, "store unavailable", http.StatusServiceUnavailable)
		return
	}
	if !created {
		// Someone owns this key: same request replays, different request conflicts.
		if rec.Fingerprint != fp {
			http.Error(w, "key reused with different payload", http.StatusUnprocessableEntity)
			return
		}
		if rec.State == "completed" {
			w.Header().Set("Idempotent-Replayed", "true") // custom replay marker
			w.WriteHeader(rec.Status)
			w.Write(rec.Body)
			return
		}
		w.Header().Set("Retry-After", "1")
		http.Error(w, "request in progress", http.StatusConflict)
		return
	}

	resp := s.process(r)
	s.store.Complete(r.Context(), key, resp)
	resp.writeTo(w)
}

The Claim call must be atomic with respect to the key, which in SQL means an insert that fails on a unique index rather than a read followed by a write. Two concurrent first attempts then race on the index, and exactly one wins.

Scope, fingerprints, and reused keys

A key is only meaningful within a scope, and the scope has to be stated. The common choices are the authenticated account, the specific endpoint, or the whole API. Account scope is the safest default: it makes the key unique per tenant, so one tenant’s key cannot collide with or replay another tenant’s record, and it does not require the client to coordinate across endpoints. Endpoint scope is more permissive but allows the same key to be reused for unrelated operations, which weakens the fingerprint check into a coincidence detector.

The fingerprint matters because keys get reused by accident. A client library that generates a UUID (RFC 9562) per process rather than per operation will send the same key with different bodies, and the server must reject that rather than replay a response for the wrong request. Hashing the method, the path, and a canonical form of the body is sufficient. Canonicalisation has to be defined, since the same logical payload can be serialised with different member order or whitespace; RFC 8259 makes JSON object member order insignificant, so the fingerprint must normalise it or the same retry will be rejected as a mismatch.

A mismatch is a client bug, and the useful response is a client error that names the problem. Silently replaying the old response hides the bug and produces a caller that believes it created something it did not.

The concurrent duplicate

The hard case is not the retry after a timeout; it is the duplicate that arrives while the first request is still running. The client timed out at its own deadline, not at the server’s, and retried immediately. Now a second request hits the key while the first is mid-flight.

There are three defensible answers. Return 409 Conflict with a short Retry-After, telling the client to wait and ask again. Block until the first completes and then return its response, which requires the waiting request to hold a connection for an unbounded time and is fragile under load. Or return the pending state as a 202 Accepted with a status location, which changes the operation into an asynchronous one.

The first is the simplest. The important property is that none of them performs the side effect twice, and that the choice is documented, because clients write retry logic against it; whichever response is chosen, its payload should say whether a retry is safe, following the principles in error contracts and problem details. The interaction with pagination and other long-running reads is different: reads have no key, and a client that resubmits a read during a write may legitimately see either state.

Expiry is a correctness parameter

Storing every key forever is not viable; the table grows without bound. So keys expire, and the retention window is a promise about how long retries are safe.

Retention must exceed the maximum time a client could plausibly still retry. That bound is set by the client’s own retry policy, by any intermediate proxy timeouts, and by the operator who replays a failed batch by hand hours later. A window shorter than the client’s retry horizon means the guarantee silently lapses: the key is gone, the request is treated as new, and the effect happens twice.

The failure is invisible at the boundary. Nothing errors. The retry succeeds. The duplicate charge appears in reconciliation instead. This is why the window belongs in the API documentation and not only in a configuration file, and why the client’s retry budget should be shorter than the server’s retention, with margin.

Deletion is also where the record’s shape matters. Deleting completed rows on a timer reclaims space; deleting pending rows can resurrect a stuck operation, so pending rows need their own timeout and reconciliation path rather than blind expiry.

Side effects outside the transaction

The key insert and the database write can usually be made atomic. It is much harder to make the key insert atomic with an email, a message published to a broker, or a call to a payment processor. If the process crashes between the commit and the external effect, the operation is recorded as complete and the email was never sent; if the external call is made first, a crash before commit leaves the effect done and the key free.

The resolution is to move external effects out of the request path. The request commits a local record and an outbox row in the same transaction, and a separate process drains the outbox with its own at-least-once semantics. The external effect then needs its own deduplication, keyed by something stable, because at-least-once delivery means it will sometimes run twice. Idempotency at the API boundary protects the API’s own state; it does not extend automatically to systems the API calls.

Where the pattern stops working

A key protects one operation. It does not make a sequence of operations atomic, so a client that must create an order and then a shipment needs either a compound single operation or an explicit saga with compensation. It does not help with non-deterministic responses: if the first attempt’s result is lost and the replay recomputes a value that has moved, such as a balance, the replay must return the stored original rather than the current truth, which surprises clients that expected freshness. And it cannot repair a client that generates a new key per attempt, which is indistinguishable, at the server, from a client making many distinct requests.

The guarantee is narrow and precise: within the retention window, for a given key and a matching fingerprint, the effect occurs once and the response is replayed. Everything outside that sentence is somebody else’s problem, and the interface should say so.