Route Table Precedence and the Local Route

A route table in a cloud network looks like the output of ip route on a Linux host and behaves like it in one respect only: the most specific matching prefix wins. In every other respect it differs, and the differences are what make routing problems in a virtual network hard to reason about. There is an entry that cannot be removed, entries that can exist while pointing at nothing, and an ordering rule between static and learned routes that is the opposite of what a dynamic routing protocol would produce.

The entry nobody created

Every route table starts with a route whose destination is the VPC prefix itself and whose target is described as local. It is created with the table and cannot be deleted or edited. It expresses the guarantee that all subnets in a VPC reach each other, which is the entire point of putting them in one VPC.

The local route is not a special case bolted onto matching logic. It is the concrete form of the statement that the fabric owns the whole prefix: for any destination inside it, the provider already knows which host holds the address, so there is no decision for the tenant to make. A tenant route can be more specific than the local route and will win, because longest-prefix match applies uniformly. A tenant route cannot be equally specific and win, because an exact duplicate of the local prefix is either rejected at creation or ignored, depending on the target type. The practical consequence is that redirecting all intra-VPC traffic through an appliance requires enumerating the subnets behind it, one route per subnet, rather than pointing the whole block at the appliance.

This asymmetry explains a class of design that looks arbitrary until the local route is understood. An inspection VPC, a firewall appliance, or a gateway load balancer endpoint is reached by adding routes for the specific prefixes that must be inspected. When a new subnet is added later and nobody adds a route for it, its traffic goes direct and bypasses inspection with no error and no log entry on the appliance. The local route is doing exactly what it promises; the gap is in the enumeration.

Longest prefix, then the tie-break

Matching proceeds the way it does in any CIDR router. For a given destination address, every route whose prefix contains the address is a candidate; the candidate with the largest prefix length wins. A /24 route beats a /16 route for an address in both, regardless of target type or creation order. Only when two routes have identical prefixes does a second rule apply.

That second rule is where the model diverges from a conventional router. Among routes with the same prefix, a static route takes precedence over a route learned by propagation from an attached gateway or connection. A dynamic routing protocol would normally prefer the learned route, on the theory that it reflects current topology, and would use administrative distance or preference to express that. The cloud model inverts this: the tenant’s explicit statement beats the learned one, which keeps a tenant from being unable to override a peer that advertises a prefix the tenant also wants to use locally.

For prefixes that are not identical, there is no tie-break at all, and this catches people. A /25 static route and a /24 learned route do not compete: the /25 wins for its half of the space and the /24 handles the rest. Traffic for the two halves of one subnet can therefore take two entirely different paths, and nothing in either route’s configuration says so.

Destination	Target	Prefix length	Used for
10.0.0.0/16	local	16	all intra-VPC traffic not matched more specifically
10.0.5.0/24	appliance	24	inspection of one subnet
10.0.5.128/25	blackhole	25	the upper half of that subnet, dropped
0.0.0.0/0	internet gateway	0	everything else

The table above is not a configuration to copy. It is a demonstration that precedence is a property of prefix length alone, and that a single subnet can be split between two targets by prefix length without any explicit ordering statement.

How the lookup is actually performed

The provider does not run a general-purpose routing daemon per packet. The tenant’s table is compiled into the forwarding state of the virtual switch that hosts the source interface, and the lookup happens at the first hop, as described in VPC address space, subnets, and the implicit router. The result is a target, not a next-hop address: the switch either delivers locally, encapsulates to a host, or hands the packet to a gateway function.

The compilation step is why route table changes are eventually consistent rather than instantaneous. A table edit is accepted immediately and shows as available, but the forwarding state on each host is updated by a control plane that must reach every switch. During the interval, some sources use the new route and others use the old one, which produces paths that differ per flow. A change that redirects traffic to an appliance can therefore be partially applied, and a connection opened in that window may bypass the appliance for its entire lifetime, since the path is chosen once and subsequent packets follow the established flow.

Implementing the matching rule is a short exercise, and doing it makes the tie-break explicit.

def lookup(routes, addr):
    """routes: iterable of (prefix, length, target, is_static), IPv4 as int."""
    best_key, best_target = None, None
    for prefix, length, target, static in routes:
        mask = (0xFFFFFFFF << (32 - length)) & 0xFFFFFFFF if length else 0
        if (addr & mask) != (prefix & mask):
            continue
        key = (length, static)  # longer prefix first, then static
        if best_key is None or key > best_key:
            best_key, best_target = key, target
    return best_target

The tuple comparison is the whole policy: prefix length is the primary key and the static flag is the secondary one. Nothing else participates. There is no metric, no administrative distance, and no age.

Blackhole routes and the absence of an error

A route whose target has been deleted does not disappear. It becomes a blackhole: the destination still matches, the target still exists as a reference, and packets are discarded. No ICMP unreachable is generated toward the sender, and no counter is incremented anywhere the tenant can see.

That behaviour is deliberate and defensible. Generating unreachable messages from the fabric would require the fabric to synthesise packets from addresses it does not own, and would leak the existence of routes the sender may not be entitled to know about. The cost is that a blackhole is indistinguishable, from the sender’s side, from a destination that is simply not responding. A TCP connection attempt hangs until the client’s own retransmission timeout expires; a UDP flow disappears.

The failure mode to watch for is the one created by deleting a gateway while leaving its routes in place. The gateway is gone, the routes still match, and every flow to those prefixes silently drops. Since route tables are per-subnet and there may be many, the cleanup is not local to the deleted object, and nothing in the deletion reports which tables still reference it. Internet gateway, NAT gateway, and source address translation covers the specific case where the deleted target is the path to the public internet, which is the version of this failure that is noticed fastest.

Prefix lists and the cost of repetition

A provider typically offers a named list of prefixes that can be referenced from a route table entry, so that the same set of destinations is expressed once and reused. This matters because a table has a finite number of entries, and a design that enumerates dozens of subnets per table across dozens of tables hits that limit.

The limit is not the interesting part. What matters is that a prefix list is a reference, and a change to the list propagates to every table that uses it. That is convenient until it is not: adding a prefix to a shared list can silently change the path for a subnet whose table nobody was thinking about, and the change is not visible in that table’s own configuration. A route table that appears unchanged can begin sending traffic somewhere new because a list it references was edited elsewhere. When a routing change is being diagnosed, the entries that are references have to be expanded before the table can be read as a forwarding policy.

Reading a table as a policy

The mental model that makes all of this tractable is that a route table is not a description of topology but a decision procedure applied at the first hop. Three questions resolve any case: is the destination inside the VPC prefix, which means the local route applies unless something more specific exists; is there a more specific route, which wins regardless of target type; and if two routes tie on prefix, is one of them static, which wins.

Every surprising routing outcome in a virtual network reduces to one of those three answers being different from what was assumed. The route that was thought to override the local route was not more specific. The route that was thought to be overridden by a learned one was static. The route that was thought to be gone was a blackhole. None of these require a new mechanism to explain, and none of them produce an error message.