Middleware Composition and Why Order Changes Behaviour

A handler that authenticates, logs, compresses, and recovers from panics is usually written as four handlers, each wrapping the next, and the chain is built once per route rather than once per request. The composition is plain function application, and the resulting call graph is a stack: every layer runs its opening code on the way in, calls the next layer, and runs its closing code on the way out, in reverse order. Everything that makes middleware order matter follows from that one fact. Which chain a request enters is decided by route matching, so a route registered outside the default chain silently loses the protections the rest of the table has.

The subtle part is that the reverse half runs after the response may already have been committed. A layer that wants to observe or modify the response has to do so before the first byte reaches the transport, and the chain does not enforce that. Ordering mistakes are therefore not type errors or even test failures; they are behaviours that appear only in the branch where the response is written early.

The stack, concretely

In Go, a middleware is a function from handler to handler. The outermost function is called when the request arrives, does its setup, calls the handler it was given, and returns. If each middleware is written as a closure with an inline function value, composing ten of them allocates ten closure objects per request unless the values are hoisted to package level, which is the usual reason a chain built inside a request-scoped function shows up in allocation profiles.

The Python gateway interface has the same shape with different mechanics: a middleware is a callable that takes the application and returns a replacement application, and the inner application calls start_response before yielding the first chunk of the body. The protocol requires start_response to be called exactly once, and the returned iterable is consumed by the server. A middleware that yields chunks lazily must therefore not depend on anything it planned to do after the inner application returned, because the inner application’s body may not have been consumed yet.

Django documents the contract explicitly: middleware listed first sees the request first and the response last, and each class has separate hooks for the two directions. That documentation exists because the reverse-order rule is not obvious from reading a list.

Orderings the mechanism forces

Some orderings are required rather than stylistic.

Panic recovery must be outside any layer that can panic and inside nothing that needs to run on a panic path it cannot observe. Placing recovery at the outermost position means a panic in authentication, in the handler, or in the closing half of a logging layer is caught and converted into a response. Placing it in the middle means panics from layers outside it escape to the server’s own handling, which usually kills the connection without a response.

Logging wants to be outside recovery so that it records the status recovery produced, and outside the timing layer so that it measures the whole chain. It wants to be inside the request identifier layer so that the identifier is available when the log line is written. Those three constraints fix its position relative to three other layers without any further argument.

Authentication must precede anything that does work on behalf of the caller, including body parsing. A server that reads and spools a large upload before checking credentials has spent disk and bandwidth on an anonymous request; the reason Expect: 100-continue exists is to let the server refuse before the body is sent, and it only helps if the authentication layer runs before the body is read. Streaming request bodies covers the reading side of that decision.

Cross-origin handling has a similar forced position: preflight requests carry no credentials and no body, so a preflight that reaches an authentication layer requiring a token is rejected before the policy that would have allowed it is consulted. The layer that answers preflights belongs outside authentication even though the layer that sets the response headers belongs inside the handler’s response path.

Short-circuiting and the committed response

Any layer may decline to call the next one. That is how authentication produces a 401, how a rate limiter produces 429, and how a cache serves a hit. The contract that makes it safe is that the layer must write a complete response, including a body or an explicit zero length, and must not return as though it had called through.

The dangerous case is partial commitment. Once the first body byte is written, the status line and headers are on the wire; a layer above cannot change them, and a layer below cannot either. Implementations expose this as a documented fact for their response writers: header maps are snapshotted when the response is flushed, and later mutation is discarded. A compression layer that sits above a handler which has already written therefore cannot add Content-Encoding, and a tracing layer that wants to record the final status must capture it from a wrapper rather than from the live header map.

Buffering layers exist to make this manageable, at a price. Go’s http.TimeoutHandler captures the response in a buffer so that it can replace it with 503 Service Unavailable if the deadline expires before the handler finishes; the consequence is that the handler’s output is held in memory until it completes, so a streaming endpoint wrapped in it stops streaming and a large response is duplicated. A middleware that captures the body to compute a checksum or to log it has the same property, and its memory cost scales with the largest response rather than with the number of connections.

func chain(h http.Handler, mw ...func(http.Handler) http.Handler) http.Handler {
    for i := len(mw) - 1; i >= 0; i-- {
        h = mw[i](h) // last listed runs closest to the handler
    }
    return h
}

func recoverer(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        defer func() {
            if p := recover(); p != nil {
                // headers may already be sent; nothing useful to write then
                w.WriteHeader(http.StatusInternalServerError)
            }
        }()
        next.ServeHTTP(w, r)
    })
}

Compression, lengths, and streaming

The interaction between compression and framing is the most common concrete ordering bug. A compression layer must remove Content-Length, because the compressed length is unknown until the body ends, and it must either switch the response to chunked framing or close the connection. If a layer inside it has already set Content-Length from the uncompressed length and the compression layer only appends bytes, the response declares a length that does not match, and the client either truncates the body or blocks waiting for bytes that will never come.

Compression should therefore be outside anything that computes lengths and inside anything that needs to see compressed bytes on the wire, which in practice means near the outer edge. It also has to cooperate with flushing. A server-sent event stream or a long-poll response relies on flushing each event to the client; a compression layer that buffers until its internal window fills will hold small events indefinitely unless it forwards flush calls. Whether it does is a property of the specific implementation, not of the idea of compression, and it is worth checking before enabling compression on a streaming endpoint.

Caching layers have the mirror-image problem. A cache that stores responses needs a key that includes everything the response varies on, and a middleware that adds a header after the cache has computed its key will produce two responses that differ but collide. The ordering rule is that anything contributing to the response’s variance must run before the cache and be visible to it.

State carried across layers

The chain is per-request but the layer objects usually are not. A middleware closure created once and reused for every request shares any variable it captures, so a per-request value stored in a field is visible to concurrent requests and will be overwritten at unpredictable times. The safe pattern is to pass values down explicitly, either in the request context or as arguments to the next layer, and to keep captured variables immutable after construction.

Context values deserve their own caution. A chain that stores a user, a tenant, a trace span, a start time, and a deadline as five context entries gives every lookup cost proportional to the number of entries added before it, and gives the reader no type safety. Keying entries by unexported struct types rather than strings prevents collision between packages, which is the failure that produces a value of the wrong type at the point of use. Where a layer needs several related values, one struct value under one key is cheaper and clearer than five.

None of this makes the chain expensive. Twenty layers of function calls and header checks amount to well under a microsecond against a handler that touches a database. The cost that shows up in profiles is the allocation of closures, the copying done by buffering layers, and the per-request map building that some frameworks perform to merge route parameters with context. Those are worth measuring; the call depth is not.