Networking · Network Virtualization
Flood and Learn Versus a BGP EVPN Control Plane
How a tunnel endpoint discovers the remote endpoint for a MAC address by flooding and learning from the reply, what that costs at scale, and how a routing protocol replaces the guesswork.
A tunnel endpoint forwarding a frame needs one fact: the underlay address of the endpoint that currently hosts the destination MAC. The division of labour between the two layers that makes that address necessary is set out in the overlay and underlay split. Nothing in the frame carries it, and no algorithm derives it from the MAC. Either the endpoint is told, or it finds out by asking everyone and watching who answers. The two control planes differ entirely on that point, and the difference shows up as a cost curve that crosses.
Learning by broadcast
The flooding approach reuses the mechanism a physical switch already uses. When a frame arrives for a MAC that is not in the local table, the endpoint sends it to every other endpoint in the same virtual network. Two implementations exist: unicast replication, where the sending endpoint makes one encapsulated copy per remote endpoint, and multicast, where the endpoints join an underlay group per virtual network and the underlay replicates.
Whichever is used, the frame is a broadcast from the overlay’s point of view, so every endpoint that receives it delivers it to every attached guest in that virtual network. Exactly one guest replies, because exactly one owns the MAC. The reply comes back through the tunnel, and the receiving endpoint records the outer source address against the inner source MAC. That is the “learn” half: the mapping was never announced, it was observed.
The entry has a timer. MAC tables age, because an entry that outlives a virtual machine’s migration sends traffic to a host that no longer has the guest. Aging is the mechanism that lets the system tolerate change without a control plane, and it is also the mechanism that guarantees the flood will happen again.
Two properties follow from the structure. The first is that the cost of an unknown destination scales with the size of the virtual network, not with the size of the fabric. A virtual network of a thousand endpoints pays a thousand copies for one broadcast frame, and broadcast traffic is not rare: ARP for IPv4, neighbour discovery for IPv6, DHCP, and any application that broadcasts. The second is that the learning is passive and local. An endpoint learns only from traffic it has seen, so the first packet to every destination is always flooded, and an endpoint that has just restarted has an empty table and floods everything until it fills.
The scaling limit is arithmetic. If a virtual network has E endpoints and generates B broadcast frames per second, the encapsulated copies per second are E times B in the unicast replication case. At E of a few thousand and B of a few per second, the number is in the thousands per second of additional packets, each of which must be generated, encapsulated, and transmitted by one host. The host’s CPU and the NIC’s transmit path become the bottleneck, and the bottleneck is on the sender, so the cost is not shared.
Announcing instead of asking
A routing protocol inverts the arrangement. Instead of a request that everyone must receive, each endpoint announces what it hosts, and the announcements are distributed to the endpoints that need them. The frame that arrives for an unknown MAC now has two possible outcomes: the table has an entry and the frame is sent directly, or the table has no entry and the frame is dropped rather than flooded, because the absence of an announcement is evidence that the destination does not exist.
EVPN, defined in RFC 7432 and applied to tunnels in RFC 8365, carries these announcements in BGP. The choice of BGP is not incidental: it is the one routing protocol designed to carry hundreds of thousands of prefixes between autonomous administrative domains, with policy, filtering, and a well-understood convergence model. The route types that matter for an overlay are few.
A Type 2 route announces a MAC address, optionally with an IP address, along with the endpoint that hosts it and the virtual network it belongs to. A Type 3 route announces that an endpoint is present in a virtual network and should receive broadcast and unknown-unicast traffic, which replaces the flood list with an explicit membership list. An IP prefix route, defined separately from the original specification, announces a prefix rather than a host, allowing the overlay to route between subnets without every endpoint learning every MAC.
The encapsulation type is carried in an extended community, so the same address family can describe VXLAN, Geneve, or MPLS, and the receiving endpoint knows which tunnel type the route implies. The virtual network identifier travels in the label field of the route, reused for that purpose when the encapsulation is not MPLS, and the route target identifies the EVI so that a route is imported only by the endpoints configured for that network. That import filter is what keeps the per-endpoint table proportional to the networks it participates in rather than to the fabric’s total.
What changes at the moment of forwarding
The data plane lookup is the same table lookup in both designs. The difference is in what happens on a miss and in how much the system knows before the first packet arrives.
With flooding, the first packet to a new destination is delayed by a round trip through the flood and the reply, and every endpoint in the network processes a frame it will discard. With a control plane, the route is present before the first packet because it was announced when the guest appeared, so the first packet is forwarded directly. A virtual machine that starts, announces, and is reachable before it sends anything is a different operational model from one that is reachable only after its first outbound frame.
The trade is state and coupling. Every endpoint now holds a route for every MAC in its virtual networks, which for a large network is the same table it would have built by learning, but built in advance and maintained by a protocol. The endpoints must run BGP sessions, which means a route reflector or a full mesh, configuration for the session endpoints, and a convergence story for when a session drops. A flood-and-learn fabric needs none of that: two endpoints with a shared VNI can talk with no coordination at all.
Withdrawal, and the migration case
The strongest argument for a control plane is not the first packet but the removal. A guest migrates from one host to another. Under flooding, the old mapping remains in every endpoint’s table until it ages out, so traffic continues to arrive at the old host. If the old host still has a path to the guest, through some other route, the traffic may be delivered by an indirect path; if it does not, the traffic is lost and the sender sees nothing until the entry expires. The window is the aging timer.
Under EVPN, the move is announced. The new endpoint advertises the MAC from its own address, and the receiving endpoints compare the sequence number in the route with the one they hold. A higher sequence number means the newer announcement, so the table is updated immediately, and the old route is superseded without waiting for a timer. A withdrawal is likewise explicit. The protocol therefore gives a bounded and short convergence window, at the cost of requiring that the announcement actually be sent and that sequence numbers be managed consistently.
That is the general shape of the comparison: flooding optimizes for the absence of coordination and pays with unbounded staleness and flood amplification, while a routing control plane pays for coordination and gains determinism. Neither dominates, and the crossover is a function of how many endpoints a virtual network has and how much broadcast it generates. Below a few dozen endpoints in a network with light broadcast, the flooding cost is negligible and the protocol is overhead. Above a few hundred, the amplification is the dominant term and the protocol is the cheaper option.
A mixed design, and its failure mode
Many deployments run both. The control plane distributes what it knows, and flooding remains as a fallback for the cases the control plane cannot cover, such as a destination that has never announced or an endpoint that has lost its session. That combination is reasonable, but it has a specific hazard: the fallback can hide a control-plane failure. If announcements stop arriving, the fabric continues to work by flooding, at reduced efficiency, and the degradation is invisible until the flood load becomes the problem. Monitoring therefore has to watch the route count and the flood rate, not just whether traffic is being delivered.
The other hazard is disagreement between the two sources. A learned entry and an announced entry for the same MAC can differ, and which one wins depends on implementation. Where the announcement is authoritative, a stale learned entry must not override it; where learning is authoritative, a withdrawn route must not immediately be replaced by a stale observation. The ordering rule has to be explicit, because the two mechanisms were designed separately and neither knows about the other’s timers. The mapping itself, and how an endpoint resolves an inner address to an outer endpoint, is treated in overlay endpoint resolution and its caching.