Serialization Delay and the Cost of Small Packets

Serialization delay is the simplest term in the latency budget and the only one a sender can compute in advance with no measurement at all: bits divided by link rate. It is also the term that responds most directly to packet size, which makes packet size look like a free latency lever. It is not free. Reducing packet size reduces the time a packet occupies a link, but it increases the number of packets needed to carry the same bytes, and every device on the path pays a per-packet cost that does not shrink with the packet.

The trade shows up in three places: header overhead that consumes capacity, packet-rate limits in forwarding hardware, and interrupt or descriptor processing on end hosts. Each has a different crossover point, and on a given path usually one of them binds first.

The arithmetic of one packet on one link

A link of rate $C$ bits per second places a packet of $L$ bytes on the medium over $8L/C$ seconds. For a full-size Ethernet payload of 1,500 bytes, that is 12,000 bits. At 10 Mbit/s the packet occupies the link for 1.2 milliseconds, at 100 Mbit/s for 120 microseconds, at 1 Gbit/s for 12 microseconds, and at 10 Gbit/s for 1.2 microseconds.

The link is unavailable to any other packet for that entire interval. A second packet arriving one microsecond after the first must wait for the remainder of the first packet’s transmission before its own begins. That is the mechanism behind head-of-line blocking in a single FIFO queue: a large packet in service delays everything behind it by its own serialization time, regardless of how small those packets are.

On a slow link this effect dominates. A 64-byte packet is 512 bits, so at 1 Mbit/s it takes 512 microseconds, and at 128 kbit/s it takes 4 milliseconds. Voice over a low-rate link is the classic case: the codec produces a frame every 20 milliseconds, and a frame of a few hundred bytes must be transmitted within that interval to keep the stream continuous. The serialization time is a large fraction of the packetization interval, which is why low-rate links need header compression and why they are sensitive to any other traffic sharing the queue.

What shrinking packets costs in capacity

Every packet carries headers that are pure overhead from the perspective of the payload. An Ethernet frame with an IPv4 and TCP header carries 14 bytes of Ethernet header, 20 of IPv4, and 20 of TCP, plus 4 bytes of frame check sequence, so 58 bytes of overhead around a 1,460-byte payload. At the maximum segment size those headers are about 4 percent of the frame. With a 100-byte payload they are 37 percent of it. With a 40-byte payload the headers are larger than the data they carry.

The capacity consequence is direct: to move a megabyte of payload with 1,460-byte segments takes about 719 packets, and with 100-byte segments about 10,500 packets. If the link is the bottleneck, the second case consumes roughly 50 percent more link time for the same payload, so the effective throughput available to the application drops by a third. This is why maximum segment size matters for bulk transfer and why tunnels that add encapsulation headers reduce throughput on paths where the extra header pushes the payload down a size class.

The transport layer has a second reason to prefer large segments. A TCP sender’s throughput is bounded by window size divided by round-trip time. To reach a given rate over a long path it needs many bytes in flight, and those bytes are counted in segments only insofar as the window is expressed in bytes; the segment count matters because each segment requires an acknowledgement and each acknowledgement is a packet on the reverse path. Halving segment size doubles the acknowledgement rate for the same data rate, which is a real cost on the reverse path of an asymmetric access link.

Packet rate as the binding constraint

Forwarding hardware has a maximum packet rate that is independent of packet size in the regime where headers dominate. A switch that can process one packet per clock cycle at a given pipeline rate has a fixed packets-per-second ceiling; at the maximum frame size it reaches its advertised bit rate, and at small frame sizes it reaches the packet ceiling long before the bit ceiling.

This is why switch datasheets quote throughput in millions of packets per second alongside gigabits per second, and why the gigabit figure is qualified as being measured at a particular frame size, usually the maximum. Filling a 10 Gbit/s port with minimum-size frames takes about 14.88 million packets per second, because each 64-byte frame occupies 672 bits on the wire once the 8-byte preamble and start delimiter and the 12-byte interframe gap are counted. Filling the same port with 1,518-byte frames takes only about 813,000 packets per second, since each of those occupies 12,304 bits. A forwarding pipeline that tops out at 5 million packets per second therefore fills the link with large frames and reaches only about 3.4 Gbit/s with small ones, and the payload rate is lower still because the headers travel too.

Software forwarding hits the same wall differently. Each packet costs a descriptor, a buffer allocation, a header parse, a route lookup, and often an interrupt or a poll iteration. Those costs are per packet and do not halve when the packet halves. A userspace forwarder that handles a million packets per second spends the same per-packet work whether those packets carry 64 bytes or 1,500, so its bit rate varies by a factor of twenty-three while its packet rate stays flat.

Small packets as a deliberate choice

Despite the costs, small packets are sometimes exactly right, and the cases are specific.

Interactive applications send small requests and receive small responses. A request of 200 bytes does not benefit from a 1,460-byte segment; sending it in one small packet is the minimum possible serialization delay and the minimum possible round trip. Batching requests to fill a large segment adds delay equal to the time spent waiting for more requests, which for an interactive workload is the dominant cost.

Some link types have their own reasons to keep frames small. A voice stream’s packetization interval sets the frame size, and increasing it increases the codec’s own delay. Real-time control traffic, where a stale measurement is worthless, prefers frequent small updates to infrequent large ones.

The decision is therefore about what the packet is for. Bulk transfer wants the largest segment the path allows, which is what path MTU discovery exists to determine. Interactive exchange wants the smallest number of round trips, which usually means small packets and no batching. A path carrying both has to choose per flow, and the queueing arrangement that keeps the two from interfering is the subject of priority and fair queueing for mixed traffic.

Middle ground and the mechanisms that shift the crossover

Between the extremes there are mechanisms that reduce the cost of small packets without giving up their latency benefit.

Header compression removes the redundancy in the per-packet headers of a flow, so a 40-byte payload does not need 40 bytes of IPv4 and TCP header repeated on every packet. On low-rate links this changes the overhead fraction from more than 100 percent to a few percent, and it is the reason voice over low-bandwidth links is feasible at all.

Segmentation offload moves the splitting of large buffers into packets from the CPU to the network adapter, so the host pays one descriptor cost for a large chunk of data rather than one per segment. The wire still carries the same number of packets, so this helps the host and not the network, which is the right division when the host is the bottleneck.

Congestion control that operates on bytes rather than packets reduces the incentive to use tiny segments as a rate-limiting device. A sender that paces its bytes evenly can use large segments and still avoid the burst that would otherwise fill a queue.

None of these removes the fundamental trade. A link has a bit rate and a packet rate, and the two constrain different packet sizes. The engineering question is which constraint a given path actually imposes, and the answer comes from the link rate, the forwarding hardware, and the mix of traffic, not from a general preference for small or large packets.

For the latency budget as a whole, serialization is usually a small term on fast links and a decisive one on slow links. It is the term that a change in application behaviour can move, and it is the term that a change in application behaviour can also make worse. Its interaction with the round trips that a protocol needs is treated in counting round trips in connection setup, because a protocol that sends many small packets pays both costs at once.