ARP and the Neighbour Cache Lifecycle

An IP packet leaving a host needs a destination MAC address before the frame can be built, and the routing decision has supplied only an IP address. The mapping between the two is not derivable from either address: the 32-bit IPv4 address has no bits reserved for a hardware address, and the 48-bit MAC address carries no IP information. Something has to ask. ARP is that something, and its design is a broadcast question with a unicast answer, cached and reused until the cache decides the answer has gone stale.

The packet format and why it is so general

An ARP packet is not tied to Ethernet or to IPv4. Its header declares the hardware address type, the protocol address type, and the length of each, so the same format could carry IPX over Token Ring. For Ethernet and IPv4 the values are fixed: hardware type 1, protocol type 0x0800, hardware length 6, protocol length 4. The operation field is 1 for request and 2 for reply.

Field	Bytes	Value for IPv4 over Ethernet
Hardware type	2	1 (Ethernet)
Protocol type	2	0x0800 (IPv4)
Hardware length	1	6
Protocol length	1	4
Operation	2	1 request, 2 reply
Sender hardware address	6	MAC of the sender
Sender protocol address	4	IPv4 address of the sender
Target hardware address	6	Zero in a request
Target protocol address	4	IPv4 address being resolved

The total is 28 bytes, which is why an ARP frame’s payload is padded to the 46-byte floor and why the padding must be ignored. The generality costs nothing in the common case: the lengths are constants, so a receiver can parse the fixed-offset fields without a variable-length scan.

A request is sent to the broadcast MAC address, because the sender does not know who holds the target address. The request contains the sender’s own mapping, so every station that sees it can update its cache with the requester’s address without asking separately. Only the station whose protocol address matches the target replies, and the reply is unicast to the requester’s hardware address taken from the request. Every other station discards the request after possibly learning from it.

The cache entry is a state machine, not a timer

A naive cache would store a mapping with an expiry and delete it when the expiry passed. ARP implementations instead keep entries in states, and the states exist because a stale entry and a missing entry have different failure costs. A missing entry causes a broadcast and a queued packet. A stale entry causes a packet sent to a station that has moved, which is silently lost. The second failure is worse, because nothing reports it, so the cache prefers to revalidate rather than delete.

The states in a typical implementation are:

• Reachable: the mapping is believed correct, and no verification is needed.
• Stale: the mapping is believed correct but unverified; the entry is used immediately, and the first use triggers a probe while the packet is sent.
• Delay: a brief wait after the first use of a stale entry, so that upper layers with a chance of confirming reachability can do so before a probe is sent.
• Probe: a unicast ARP request is sent to the cached hardware address, and the entry waits for a reply.
• Incomplete: no mapping exists; packets are queued and requests are retransmitted until a reply arrives or the queue is dropped.

The design intent is that a stale entry keeps working while it is being verified. The alternative, deleting the entry on expiry, would replace a silent failure with a broadcast storm on every idle-then-active transition, and would put the first packet of every burst behind a resolution delay. Keeping the entry and probing means the common case, where the station has not moved, costs nothing but one extra request.

The probe is unicast to the cached hardware address rather than broadcast. That is deliberate: a broadcast probe would be answered by whoever currently holds the address, which is what happens with a broadcast request anyway, but a unicast probe reaches only the cached station, so a reply proves that the specific cached mapping is still live.

/* Skeleton of a neighbour entry; state transitions are driven by the timer. */
enum nstate { N_INCOMPLETE, N_REACHABLE, N_STALE, N_DELAY, N_PROBE };

struct neighbour {
    uint32_t addr;            /* protocol address, network byte order */
    uint8_t  hw[6];           /* hardware address, valid unless INCOMPLETE */
    enum nstate state;
    unsigned probes;          /* unanswered probes so far */
    struct sk_buff_head queue; /* packets waiting for a mapping */
};

The queue is bounded. An unreachable destination that is probed repeatedly will accumulate packets until the limit is reached, at which point the oldest are dropped or the newest is refused, depending on implementation. This is why a host resolving an address on a segment where nothing answers sees its packets vanish rather than block indefinitely: the resolution fails, and the error surfaces at the socket layer as an unreachable host.

Gratuitous ARP and duplicate address detection

A station may send an ARP request or reply for its own address without being asked. The request form is broadcast with the target address set to its own, which both announces the mapping and, in the request case, asks whether anyone else holds the same address.

The second use is duplicate address detection. A station can broadcast a request for its own address with the sender’s protocol address field zeroed, so the message asks whether anyone else holds the address without teaching neighbours a mapping that may be wrong; if a reply arrives, another station on the segment claims the same IPv4 address. RFC 5227 specifies this probe and the responses: the defending station answers, and the probing station must not use the address. This is the only duplicate detection IPv4 has at the link layer, and it is optional; a host that never probes will not discover the conflict. The consequence of a duplicate is that traffic for the address goes to whichever station last answered, and the two stations fight over the mapping indefinitely.

The first use is cache correction. A station that has just changed its hardware address, for instance after a failover, broadcasts a gratuitous reply so that neighbours update their caches immediately rather than waiting for their entries to go stale. The broadcast reaches every station in the domain, and each updates the mapping for the announced address.

Spoofing, and why ARP has no defence

Nothing in ARP authenticates the sender. The reply’s source hardware address is whatever the sender put in the frame, and the protocol address is whatever the sender claims. Any station can send a reply claiming to be the gateway, and every station that receives it will update its cache and send its off-segment traffic to the attacker.

This is ARP cache poisoning, and the mitigation is not in ARP. Options include static cache entries, which do not scale and break when hardware changes; inspection at the switch that validates the pairing of source MAC and source IP against a learned binding, which is a switch feature rather than a protocol one; and cryptographic protection at higher layers, which prevents the attacker from doing anything useful with the redirected traffic but does not prevent the redirection. The absence of authentication is a property of the design, and IPv6’s Neighbor Discovery was specified separately, with an optional security extension, rather than as a copy of ARP.

IPv6’s Neighbor Discovery uses ICMPv6 rather than a separate protocol, carries the link-layer address in an option field so the message format does not depend on the link type, and uses multicast to a solicited-node group rather than broadcast, which limits the number of stations that must process a resolution. The state machine, however, is recognisably the same shape: incomplete, reachable, stale, probe. RFC 4861 defines it with the same revalidation logic, and the same absence of authentication in the base specification.

Proxy ARP and the boundaries it creates

A router may answer ARP requests for addresses it can reach but does not own. The reply gives the router’s hardware address, so the requester sends frames to the router, which forwards them. This is proxy ARP, and it lets a host believe a destination is on its own segment when it is not.

Proxy ARP is useful when a host has a wrong netmask or when subnets are being migrated, and it is dangerous for the same reason: it makes an off-link destination appear to be on the local segment, so a host’s belief that a resolved destination is directly reachable is false in a way it cannot detect. It also adds broadcast traffic, because every address the proxy covers produces an ARP request on the segment when a host first tries to reach it. Implementations usually have a per-interface switch to enable it, off by default, precisely because the failure it creates is invisible until something depends on the distinction.

The cache also interacts with the bridge. A bridge learns from the source addresses of the frames it forwards, so ARP traffic both refreshes the switch’s table and is flooded when its destination is a group address. The interaction is described in MAC learning and bridge forwarding, and the frame-level details of a 28-byte payload inside a 64-byte frame are in Ethernet frame layout.