Read Models Across Service Boundaries

A single screen showing an order, its shipment status, and its invoice needs data owned by three services. The decomposition was made for write-side reasons, and none of those reasons imply that the read side should traverse three boundaries at request time. Most latency and availability problems in service architectures appear precisely here: the write model was split carefully, and the read path was left to compose the pieces on every request.

A screen that needs three services

The naive composition calls each owner in parallel and merges the results. Latency becomes the maximum of the three rather than the sum, which is the good news. Availability becomes the product, which is not: three dependencies at 99.9% give 99.7% for a read that has no writes of its own, and adding a fourth for a recommendation strip takes that to 99.6%. A read-only page is now less reliable than the writes it displays.

Partial failure handling complicates it further. If the invoice service fails, does the page render without the invoice, or fail entirely? If it renders, the composition layer must encode which fields are optional, and that knowledge is business logic living in a gateway. If it fails, the availability product stands.

Lists make it worse. A page of 25 orders requires the identities from the order service and then a lookup per distinct customer, or a batch call if one exists, and pagination has to be owned by one service because the join destroys the sort order the owner used. Composition works for detail pages with a small, fixed set of dependencies and breaks down for lists, search, and anything with cross-entity sorting.

Composition at the edge, and its limits

Composing at the edge is the cheapest option and the right default. A backend-for-frontend service, or an aggregation endpoint owned by the team that owns the screen, issues the calls in parallel, applies a deadline to the whole fan-out, and assembles a response shaped for the consumer. The client makes one call, so the availability product is computed once, on the server, where a fallback can be applied.

Three design details decide whether it holds up. The fan-out must have one deadline for the whole request, divided among the calls, rather than an independent timeout per call, or the slowest dependency sets the page latency through the maximum. Each optional dependency needs a defined degraded response, so a failure produces a missing panel rather than an error page. And the aggregation layer must not accumulate business rules; once it starts deciding which orders are visible, it has become a second owner of the order’s semantics without the transaction that would make it correct.

The limits are structural. A fan-out of ten dependencies has an availability product that no amount of retry discipline fixes, and the aggregation service becomes a place where every team’s schema changes land at once.

Projections that own the read side

The alternative is to materialise the answer. A read model subscribes to the events published by the owning services and maintains a denormalised record shaped for the query. The screen reads one row from one store, and the number of dependencies at request time collapses to one.

The mechanics are a consumer per projection with a durable offset, an upsert keyed by the entity identifier, and a schema that includes the fields the query needs rather than the fields the owner has. Each event carries the entity id, a version or sequence number, and the changed fields. The projection applies the event and advances its offset.

The offset is per consumer, not per entity, so it only tells the projection how far along the stream it is. It says nothing about whether a particular entity’s events arrived in order, because concurrent producers and multiple partitions mean that events for one entity can be spread across partitions unless the topic is keyed by entity id. Keying by entity id is what makes per-entity ordering a property the broker provides rather than something the projection has to repair.

Ordering is only half of what the projection needs. The other half is that the events exist at all, which depends on the owner publishing them in the same transaction as the state change, the arrangement described in The Dual-Write Problem and the Transactional Outbox.

Where ordering cannot be guaranteed, a version guard in the write closes the gap:

INSERT INTO order_summary (order_id, customer_name, status, total_cents, version)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (order_id) DO UPDATE
   SET customer_name = EXCLUDED.customer_name,
       status        = EXCLUDED.status,
       total_cents   = EXCLUDED.total_cents,
       version       = EXCLUDED.version
 WHERE order_summary.version < EXCLUDED.version;

A stale event that arrives late updates nothing, because its version is lower than the row’s. The WHERE clause is the entire out-of-order defence, and without it the projection’s correctness depends on a delivery-order property that most brokers do not offer across partitions.

Staleness as a stated contract

A projection is eventually consistent, and the delay is a number that has to be known, published, and monitored. Three things define it: the time between the write committing and its event being published, the time the event spends in the broker, and the time the projection takes to apply it. A relay that polls every 100 ms, a broker with single-digit millisecond delivery, and a consumer that batches 200 events per write produce a lag somewhere between tens of milliseconds and a second, and the honest statement is the measured distribution rather than the best case.

The harder problem is read-your-writes. A user submits a change and the interface immediately re-reads the list; the projection has not caught up, and the change appears to have been lost. Retrying a refresh eventually shows it, which teaches users that the interface is unreliable.

Four techniques address it, in increasing cost. The command response can return the new state, so the interface has no reason to re-read. The read can be routed to the owning service for a short window after a write by that user, which reintroduces the dependency for a small fraction of traffic. The client can carry a version token from the write and the projection can be asked to wait until it has applied that version, with a short timeout and a fallback to the owner when the wait expires. Or the interface can show the value it just submitted as a provisional state until the projection catches up, which is the cheapest option and the one that requires the most care in the display.

The version-token approach is the only one that is both correct and general, and it needs the projection to expose its applied version per entity. A token that names only a global offset is easier to implement and less useful, because a global offset does not guarantee that this entity’s event has been applied when events arrive out of order across partitions.

Rebuilding, deleting, and drifting

A projection is derived data, and derived data has to be rebuildable. A rebuild reads the event log from the beginning, or from a snapshot plus the tail, and writes into a fresh table. Rebuilding in place while serving traffic produces a window in which the projection is a mixture of old and new applications, and any non-idempotent transformation shows it.

Two operational facts shape the rebuild. First, the log must still contain the events, which means the retention period is a hard bound on how far back a projection can be rebuilt without a snapshot. Second, the rebuild writes at the rate the log can be read, so the time is proportional to the number of events rather than to the number of entities; a year of events for a small table can take longer than the table’s size suggests.

Deletes need explicit handling. An event that removes an entity must produce a tombstone in the projection, and a projection that only handles upserts leaves rows that no longer exist in the source. Tombstones also need their own retention, since a projection that has been offline longer than the tombstone retention will resurrect deleted rows when it catches up.

Schema drift is the quiet failure. A field renamed by the owner disappears from new events, and the projection keeps the last value it saw, which looks correct in the UI and is stale by an unbounded amount. Contracts between the event producer and the projection are the subject of Independent Deployability and Cross-Service Contracts, and a projection is a consumer whose compatibility requirements are easy to overlook because it has no API of its own.

The final cost is duplication. Every projection stores its own copy, and each copy must be backed up, monitored, and kept consistent with the schema it was built for. That cost is real and usually still smaller than the cost of composing five services on every page load, but it is a cost, and a projection whose query is served by one screen is rarely worth it.