The Inter-Region Latency Floor Set by Fiber Paths

A team that has only operated within a single region usually carries one implicit assumption: latency is a performance property that can be bought. More bandwidth, a better instance class, a closer peering point, and the number gets smaller. Cross-region traffic breaks that assumption. There is a distance at which no amount of provisioning helps, and every design decision downstream of the topology inherits it.

Light in glass is not light in vacuum

The constant that matters is not 299,792,458 m/s. Light in an optical fiber core travels through doped fused silica with a refractive index around 1.47 at the wavelengths used for long-haul transmission. Divide the vacuum speed by that index and the propagation velocity lands near 204,000 km/s, or about 5 km per 10 microseconds one way. A commonly used rule of thumb is 1 millisecond of one-way propagation per 200 km of fiber.

Two consequences follow directly. First, the round-trip time between two points has a hard lower bound of roughly 2 × distance / 204,000 km/s, plus the latency of every active device in the path. Second, the fiber does not run in a straight line between cities. Real long-haul routes follow rights of way, rivers, rail corridors, and submarine landing points, and the path is typically 1.2 to 1.5 times the great-circle distance. A route that looks like 4,000 km on a map behaves more like 5,000 to 6,000 km of glass.

A worked example for a transatlantic pair. Great-circle distance between the US east coast and western Europe is roughly 6,000 km. Applying a 1.3 route factor gives about 7,800 km of fiber, which is about 38 ms of one-way propagation and therefore about 76 ms of round-trip propagation before any equipment is counted. Measured inter-region RTTs on such a path land in the high 70s to low 90s of milliseconds, which is consistent with the model: the floor dominates, and the equipment term is single-digit milliseconds.

Compare a pair 800 km apart in the same continent: 800 km of great-circle distance becomes roughly 1,040 km of fiber, about 5.1 ms one way, about 10 ms round trip. The difference between 10 ms and 80 ms is not a difference in quality of the provider. It is a difference in where the two cities are.

What the equipment term contains

The propagation floor is joined by a set of smaller but non-negligible costs:

• Serialization. A 1,500-byte packet on a 10 Gb/s link occupies the wire for about 1.2 microseconds. On a 100 Gb/s link it is about 0.12 microseconds. At inter-region distances this term is irrelevant, but it becomes visible in switch queues when many flows share a port.
• Switching and routing. Each optical add/drop or router hop contributes queuing and forwarding delay. Long-haul paths traverse a modest number of these; the total is usually a few milliseconds, not tens.
• Regeneration and amplification. Optical amplifiers add small delays, and a route long enough to require regeneration adds more. Submarine systems with many spans accumulate a few milliseconds over the propagation floor.
• Congestion. This is the only term that scales with load rather than distance, and it is the one an operator can actually control.

The important structural point is that four of these five terms are fixed by geography and hardware. Only congestion responds to capacity planning. A design that budgets for 30 ms of inter-region RTT on a path whose propagation floor is 76 ms is not under-provisioned; it is impossible.

Turning the floor into a design constraint

Because the floor cannot be reduced, it has to be treated as an input to protocol and application design rather than a target to optimize. Three quantities convert it into concrete limits.

Request amplification. If a user-facing request requires k sequential round trips between regions, the latency contribution is at least k × RTT. A single cross-region call costs 80 ms; a chain of five costs 400 ms before any server-side work happens. This is the reason cross-region request chains degrade so sharply with distance: the per-call cost is multiplied by the call count, not amortized by concurrency.

Bandwidth-delay product. A path’s capacity to keep data in flight is bandwidth × RTT. A 1 Gb/s path with 80 ms RTT holds about 10 MB of data in transit at any moment. A single TCP connection that has not grown its window to fill that product cannot use the link, which is why bulk cross-region transfers behave poorly when they consist of many short-lived connections rather than a few long-lived ones. TCP’s congestion window growth is limited by the acknowledgment rate, and on a long path the acknowledgment rate is itself limited by RTT. RFC 1072 documented this class of problem for long-delay paths decades ago; the arithmetic has not changed.

Consistency cost. Any protocol that requires a synchronous round trip between regions before acknowledging a write pays the full floor on every write. That is the subject of replication lag as a recovery point bound, but the arithmetic starts here: 80 ms per synchronous commit caps a single writer at roughly 12 commits per second if it serializes them, which is a throughput ceiling derived purely from distance.

Choosing a topology against the floor

The floor makes some topologies viable and others not, and the selection is mostly arithmetic.

User base	Candidate topology	Propagation floor (approx.)
Single metro	One region, multi-AZ	under 2 ms
Single continent, 3,000 km span	Two regions, active-passive	30–40 ms RTT
Two continents	Two regions, active-active per-user	75–150 ms RTT
Global read-heavy	Many read replicas, single write home	floor paid on writes only

The third row is where most design arguments happen. Active-active across continents does not reduce the latency a user experiences for a request that must touch both sides; it only reduces the latency for requests that can be served entirely from the near side. If the data model permits a user’s data to be pinned to one region, the user sees single-region latency and the cross-region floor appears only in the replication path. If the data model requires every write to be visible everywhere before it is acknowledged, the floor appears in the user’s critical path and cannot be removed. Conflict resolution for two-region writes covers what happens when the second option is chosen anyway.

Where the model misleads

Two failure modes are common when this arithmetic is applied naively.

The first is using straight-line distance. Cloud providers publish region locations, and the great-circle distance between those locations is easy to compute, but the fiber path is not the straight line. Routes between some city pairs are indirect because the direct path crosses water without a landing station or a country without a right of way. Treating great-circle distance as the path length systematically understates RTT by 20 to 50 percent on such pairs.

The second is treating a measured RTT as a floor. A measurement taken during a quiet period may be close to the floor; a measurement taken during peak hours includes queueing and can be several times higher. The floor is a property of physics; the observed value is a property of physics plus load. Design that targets the observed value at peak has no headroom left, and design that targets the floor has no margin for the equipment term. The honest budget is the propagation floor plus a few milliseconds, with the difference between that and the observed peak treated as congestion to be managed.

Nothing in this arithmetic depends on the provider. Two regions 10,000 km apart will cost about 100 ms of round-trip propagation whether the fiber is owned by a hyperscaler or a consortium. That is the constraint every other article in this cluster works around: the failover time budget spends it, quorum placement trades against it, and cell-based isolation is largely an attempt to keep requests from crossing it at all.