The TCP/IP Split and the End-to-End Argument

A file transfer that crosses four networks passes through equipment owned by four organisations, none of which can be assumed to have the same failure model. One link may be a satellite channel with a round trip measured in hundreds of milliseconds; another may be a local cable that loses a packet only when a buffer overflows; a third may be a radio channel where loss is continuous and expected. A network that tried to guarantee reliable delivery across that path would have to hold data at every hop until the next hop acknowledged it, which means a copy of the data in memory at each of the four organisations and a delay at each of them equal to that hop’s round trip.

The end-to-end argument says that this is the wrong place to put the function. Reliability, ordering, duplicate suppression, and integrity checking can be implemented completely only at the endpoints, because only the endpoints know whether the data actually arrived and whether it is correct. Any implementation inside the network is an optimisation that can reduce the work the endpoints do, but it cannot replace it. If the endpoints must check anyway, the interior check is redundant except insofar as it saves retransmission over a long path.

That argument, published by Saltzer, Reed, and Clark in 1984 but circulating in the ARPANET community well before, is the design constraint behind the split of the original Transmission Control Program into TCP and IP. It is also the most frequently misquoted idea in network design, usually in the form “the network should be dumb”, which is a stronger and less defensible claim than the one actually made.

The argument is about placement, not about dumbness

The paper’s formulation is conditional. A function placed in the network can be justified when it is a performance optimisation for a function the endpoints must implement anyway, and when the optimisation does not compromise other requirements. A checksum in a link layer is justified on exactly this basis: the endpoints still verify the data, but a link-level check catches corruption early and avoids a retransmission across the whole path. The check is redundant and useful.

What the argument rules out is placing a function only in the network, so that endpoints are entitled to assume it. That is the case the ARPANET’s hop-by-hop reliability created. NCP did not retransmit because the IMPs did, so a host written against NCP was correct only on a subnet whose every hop acknowledged. Attach such a host to a network without that property and it fails, and no amount of correct behaviour in the new network can repair it, because the host is not prepared to detect the failure.

The practical test for a design is therefore not “is this function in the network” but “if this function is removed or fails, do the endpoints still behave correctly”. A network that caches, compresses, or aggregates passes the test. A network that terminates a transport connection, acknowledges data on the receiver’s behalf, or rewrites application payloads does not.

What IP promised and what TCP had to add

IP’s contract is deliberately thin. It delivers a datagram to a destination address, with a header checksum covering the header only and a total length field, and it may fragment the datagram if a link cannot carry it whole. It does not promise delivery, ordering, duplicate suppression, or that the payload is intact. The header checksum covering only the header is a consequence of the same reasoning: verifying the payload is the endpoint’s job, and doing it at every router would repeat work that the transport layer must do anyway.

#include <stdint.h>

/* The IPv4 header fields that make the thin contract possible. */
struct ipv4_header {
    uint8_t  version_ihl;      /* version in high nibble, header words in low */
    uint8_t  tos;
    uint16_t total_length;     /* header plus payload, in bytes */
    uint16_t id;               /* used only when fragmentation occurs */
    uint16_t frag_off;         /* flags in the top 3 bits, offset in 8-byte units */
    uint8_t  ttl;              /* decremented per hop; zero means discard */
    uint8_t  protocol;         /* 6 = TCP, 17 = UDP, 1 = ICMP */
    uint16_t checksum;         /* covers the header only, not the payload */
    uint32_t src, dst;
};

Two fields carry the failure reporting. The time-to-live is a hop counter, decremented at each router, and a datagram whose counter reaches zero is discarded with an ICMP Time Exceeded message returned to the source. This bounds the damage from a routing loop: without it, a packet caught in a cycle would circulate until a queue overflowed somewhere, consuming capacity indefinitely. The identifier and fragment offset exist because a link may have a smaller maximum transmission unit than the datagram, and the reassembly is done at the destination rather than at each hop, which is again the end-to-end placement.

What IP does not provide is any signal that a specific datagram was lost. ICMP reports conditions about the path, not about individual data. A router that drops a packet because its queue is full may send an ICMP Source Quench, and the specification is explicit that a host must not treat that message as authoritative, because the router is reporting a local condition at one moment and the endpoint cannot verify it. TCP’s response to loss is therefore inferred from the absence of an acknowledgement rather than from a notification, which is why the retransmission timer and its estimation are load-bearing.

The cost the endpoints pay

Moving reliability to the endpoints means every endpoint implements it. A TCP connection maintains send and receive sequence spaces, a retransmission queue, a reassembly queue for out-of-order segments, a timer, and a window. The sender cannot discard a byte until the peer has acknowledged it, so the memory held per connection is at least the bandwidth-delay product of the path: a connection over a path with 100 ms round trip and 10 Mbit/s capacity has roughly 125 kilobytes in flight, and the sender must keep that much buffered to keep the path full.

That memory is the price of the placement decision, and it is paid per connection rather than per path. Ten thousand connections through one router hold their buffers in ten thousand endpoints, while the router holds only a queue per interface. The aggregation is what makes the internet scale: the interior of the network holds state proportional to the number of links, and the endpoints hold state proportional to the number of conversations.

The duplication is real. A file transferred over TCP is checksummed by the link layer, by IP’s header checksum, by TCP’s checksum, and often by the application’s own hash. Each layer’s check catches a different class of fault: the link checksum catches a bad bit on the wire, TCP’s checksum catches corruption introduced inside a router’s memory, and the application’s hash catches a bug in the sender. Removing any one of them leaves a fault class undetected.

Where the argument is now contested

The argument assumes endpoints are trustworthy and that the network’s interior is not. Both assumptions have been eroded. Middleboxes that terminate TCP, inspect payloads, and re-originate connections violate the placement rule directly, and they exist because endpoints are frequently not trustworthy and because the network operator wants control the endpoints will not give. A proxy that acknowledges data before the receiver has it makes the sender’s view of the path wrong, and the resulting behaviour depends on buffers the sender cannot see.

The other erosion is measurement. A network operator who wants to know whether a path is delivering data can observe the interior only if some function is instrumented there; the end-to-end argument pushes all the interesting state to endpoints owned by someone else. This tension is why explicit congestion notification exists as a compromise: the router marks a packet rather than dropping it, and the endpoint still makes the decision, so the function stays at the endpoint while the information comes from the interior.

The argument also does not say that endpoints must be identical. It says that a function required for correctness must be implemented at the endpoints, and that the endpoints are the only parties who can know whether it succeeded. Two endpoints may implement it very differently, and the interoperability requirement is only that they agree on the wire format. That is why TCP’s congestion control can differ between implementations while the segment format is fixed, a separation explored in AIMD Congestion Control and the Collapse It Prevented.

What the split made possible

Separating IP from TCP allowed protocols that wanted less than TCP to exist. UDP carries a datagram with ports and a length, no connection, no retransmission, and an optional checksum that covers a pseudo-header including the addresses, so a datagram delivered to the wrong address fails the check. Applications that implement their own reliability on top of UDP, as DNS resolvers do with a retry timer, choose a different point on the same curve.

It also made the network’s forwarding decision independent of the transport’s state, which is what allows a router to be built from a table and a queue rather than from per-connection logic. The address in the packet is enough to forward it, and the address’s structure is the only thing the router must understand. How that structure evolved from fixed classes to arbitrary prefixes, and why the change was forced, is the subject of Classful Addressing, Subnets, and the Move to CIDR. The property that any endpoint could be addressed by any other held only while addresses were globally unique and reachable, and the translator described in NAT, Port Overloading, and the Address Exhaustion Boundary is what gave that property up in exchange for keeping the address space usable.