Networking · Network Operations

Interface Drop Counters and What Each One Actually Attributes

How kernels classify discarded packets into drop, error, and overrun counters, and why a clean counter set can coexist with a link that loses traffic.

An interface counter is an assertion about where a packet was lost and, by implication, about which subsystem could have prevented it. The set of counters exposed for an interface is not a taxonomy of network faults but a record of the points in one particular implementation where a discard is counted. Reading them as a diagnosis requires knowing which point each counter corresponds to, because several of the points are inside the host and one of them is inside the hardware.

The discard points inside a receive path

A received frame passes several stages before it becomes a packet the stack can process. The adapter writes the frame into a descriptor ring in host memory and signals completion. The driver allocates a buffer to replace the one handed up, wraps the frame in a kernel structure, and passes it to the network stack. The stack then runs the protocol handlers, which may discard the packet for reasons that have nothing to do with the hardware.

Each stage has its own counter and its own name. The conventional set across Unix-derived systems, inherited from the interfaces group of the MIB and given concrete names in the Linux statistics structure, separates packets discarded for a reason the stack could not classify from packets the device dropped because it had nowhere to put them, and it separates both from errors the hardware reported. The distinction matters because the three have different causes and different remedies.

The receive counter itself is worth reading carefully before any of the others, because it counts packets the host later discarded. A frame that arrived intact and was then dropped for lack of a buffer, or because no protocol handler claimed it, still increments the receive packet count; the drop is recorded separately. A receive count that matches what the sender transmitted therefore does not mean the traffic was delivered, only that it reached the host’s receive path.

The counters then split by who dropped the packet. The general discard counter is the stack’s, and it increments for packets received but not processed: a protocol the host does not handle, a shortage of resources during processing, or a frame discarded by link-layer address filtering. A packet dropped because one socket’s receive queue is full is counted against that socket rather than against the interface, which is why the interface total can read zero while a single listener loses traffic. The missed counter is the device’s, and it increments when the device had no buffer for an arriving frame, which is the case where the host’s stack never saw the packet at all. A separate overrun counter covers FIFO overflow inside the device, which is a hardware buffering failure rather than a host scheduling one.

Device-side buffer exhaustion is a scheduling problem: the host is not servicing the receive ring quickly enough, either because the interrupt is coalesced too aggressively, because one core handles too many queues, or because the ring is smaller than the burst it must absorb. Stack-side discards are a consumer problem, and they are attributable to a specific socket or a specific rule rather than to the interface as a whole.

The categories are distinguishable by a second observation. Buffer exhaustion is correlated with burst arrival and disappears when the traffic becomes smooth, because the ring drains between bursts. A software discard is correlated with a specific consumer, such as one socket whose application is not reading, and it persists at a steady rate as long as that consumer is slow. Looking at the per-queue and per-socket state separates them faster than looking at the aggregate counter.

Where the count is kept decides when it moves

Interface counters are per interface, and on a device with multiple receive queues they are the sum across queues. The aggregation hides a distribution that matters. If eight queues exist and one of them is starved because the interrupt for it is pinned to a core that is busy elsewhere, the aggregate overrun counter rises while the other seven queues are idle. A per-queue view, where the driver exposes one, shows the asymmetry immediately.

Counters also reset at different times depending on how they are read. The kernel’s own counters are monotonic since the interface was created; counters maintained by the hardware are cleared when the driver reads them or when the interface is reset. A driver that reads and clears hardware counters and adds them to a software total produces a number that behaves monotonically but whose increments are delayed by the polling interval. A monitoring system that samples at a shorter interval than the driver’s polling sees a staircase rather than a slope, and any rate computed over a single sample interval from such a counter is wrong.

Virtual interfaces add a further complication. A bridge, a bond, or a tunnel has its own counters in addition to the counters of the physical interfaces beneath it, and a packet discarded at the bridge layer may be counted by the bridge and not by the physical device. The same packet can therefore appear in the physical interface’s receive count and in the bridge’s discard count, which is not a double count but a record of two different points. Summing counters across the interface hierarchy produces a total larger than the traffic, and the error grows with the depth of the stack.

The error counters describe frames that arrived damaged or that the hardware could not transmit. A frame with a bad frame check sequence is counted as a receive error and discarded; a frame that the adapter could not transmit because the medium was busy past the retry limit is counted as a transmit error. Neither is a queue overflow, and neither is a software decision.

The distinction is operationally important because errors point at the medium and discards point at the host. A rising receive error count on copper usually means a cable, a connector, or a duplex mismatch; a rising receive error count on an optical link usually means a marginal signal, which in turn often means a dirty connector or a transceiver outside its specified range. A rising transmit error count on a half-duplex link with collisions means the link is saturated, which is a capacity problem rather than a fault.

Some adapters fold several distinct hardware conditions into one error counter. The number of frames dropped because their length was outside the permitted range, frames with an alignment fault, and frames that failed a checksum the adapter verified may all increment the same counter or may each have their own, depending on the driver. Where the driver exposes them separately, the separate counters are worth reading, because a run of oversize frames suggests a maximum transmission unit mismatch on the path rather than a damaged medium. That mismatch has its own failure signature, visible as a size threshold below which traffic succeeds, and it is one of the cases where a capture at both ends settles the question faster than any counter.

Counters that are correct and still misleading

A link can lose traffic with every counter at zero. Three mechanisms produce this.

The first is a drop above the interface: a firewall rule, a policy route, or a socket-level limit discards the packet after it has been counted as received and before it is delivered. The interface counters record a successful receive, and the loss appears only in the filtering subsystem’s own counters, which are often not sampled at all.

The second is a drop below the interface: a switch port, a policer, or an upstream queue discards the frame before it reaches the host. The host’s receive counter simply does not increment, which is indistinguishable from the traffic never having been sent unless something else is counted. Correlating the sender’s transmit count with the receiver’s receive count is the only way to localise this, which is why a capture at both ends is often the fastest confirmation.

The third is a counter that does not exist. Some discard points in the stack are counted only in aggregate, some only when a debug option is enabled, and some not at all. A negative result from a counter set is therefore weak evidence, and the strength of the conclusion should match the coverage of the instrumentation rather than the confidence with which the number zero is reported.

Reading a counter set as a sequence

The practical method is to order the checks by where they sit in the path, from the medium inward, and to stop at the first counter that moves. That ordering is the same one used for interface state generally, where link flags and address validity are settled before anything is counted; interface configuration order lays out that sequence.

# per-queue and per-interface totals
ip -s link show dev eth0
ethtool -S eth0 | grep -Ei 'drop|err|miss|overrun'
# socket-level drops, which the interface counters never see
ss -lntu
nstat -az | grep -i drop

The interface totals give receive, transmit, drop, and error figures at the device level. The driver’s own statistics, where exposed, add the hardware’s view, including conditions the kernel does not model. The socket statistics expose the per-socket receive queue, which is where a software discard is usually attributable to a specific listener, and the kernel’s internal counters expose discard points that are not per interface at all.

Two rules make the sequence reliable. First, always read counters twice with an interval between them: a single sample of a monotonic counter says nothing about current behaviour. Second, record which counters are absent as well as which are zero, because the absence of an instrumented discard point is a different fact from the presence of an unincremented one, and only the second is evidence.

Why the counters cannot settle the question alone

Every counter is local. It records that this host discarded a packet, never that a packet was lost between two hosts. Attributing a loss to a specific device requires an observation on both sides of that device, which is what a capture provides, and captures bring their own artifacts that can imitate loss. The interaction between the two techniques is covered in packet capture placement.

Where counters do earn their place is in ranking hypotheses. A host with a rising overrun count and a stable error count is almost certainly under-provisioned in interrupt handling, not fighting a bad cable, and that ranking directs the next check to the right subsystem. The counters do not prove the cause; they exclude most of the alternatives cheaply, which is the useful property.

References

  1. Linux kernel documentation — network interface statistics — The Linux kernel documentation
  2. RFC 2863: The Interfaces Group MIB — IETF
  3. ip-link(8) — Linux manual page — The Linux man-pages project