Internet Gateway, NAT, and Source Address Translation

Two functions are commonly conflated because both sit between a private subnet and the internet, and both are configured with a single route table entry. They do different things. An internet gateway attaches a VPC to the provider’s public network and performs one-to-one translation between an instance’s private address and a public address assigned to it. A NAT gateway performs many-to-one translation, letting a whole subnet share one public address for outbound connections. The first preserves reachability in both directions; the second creates a connection-oriented path that exists only as long as a flow does.

The gateway is a translation, not a router hop

An internet gateway is described as attached to a VPC, and the route table entry that uses it has the gateway as its target. This suggests a device on a path, which is misleading. The gateway has no address on the VPC side, no interface, and no queue that a packet waits in. It is a function the fabric applies to packets whose route resolves to it.

Outbound, a packet from an instance carries the instance’s private source address. The fabric rewrites the source to the elastic address associated with the instance, recomputes the checksums, and emits the packet on the provider’s network. Inbound, a packet addressed to the public address is rewritten to the private address and delivered. Both directions are stateless in the gateway itself: the association between the two addresses is configuration, held for as long as the address is assigned, not per flow. This is what makes an instance with a public address reachable by anyone who can route to that address, with no connection having been opened first. The filtering that limits who may reach it is the interface group, evaluated after the rewrite.

The rewrite has a consequence that is easy to miss. Because the source address seen by the remote server is the public one, and the private address never appears on the wire, a server that logs connections records the public address and nothing else. Two instances behind different public addresses are indistinguishable from two separate hosts, and two instances sharing a public address through a NAT are indistinguishable from each other unless the source port differs. Attribution of a connection to an instance requires the provider’s translation record, which is kept for a limited time and is not part of the packet.

Many-to-one and the port as the discriminator

A NAT gateway exists because public addresses are a finite resource and because most subnets contain many hosts that need outbound access but no inbound reachability. It translates the source address of every outbound packet to its own address, and also rewrites the source port so that replies can be demultiplexed.

The port rewrite is the mechanism. If two instances both open a connection to the same destination on port 443, both packets have the same destination tuple after the address rewrite, and the only field that distinguishes them is the source port. The gateway allocates a distinct port for each, records the mapping, and on the return path looks up the destination port of the arriving packet to recover the original source address and port. The mapping table is the state that makes the design work, and its size is the design’s limit.

RFC 3022 describes this as traditional NAT with port translation, and the constraint it imposes is a hard one: the number of simultaneous outbound flows through one public address is bounded by the number of ports available for translation, which is at most 65,535 per protocol, and in practice far fewer because the low range is reserved and the allocation is not dense. A subnet that opens a very large number of concurrent connections through one NAT gateway reaches that limit, and the symptom is new connections failing to establish while existing ones continue.

The allocation policy also matters for determinism. Some implementations preserve the source port when it is free and remap only on collision, which makes the mapping predictable and helps protocols that embed addresses in payloads. Others randomise, which is better for privacy and for resisting port prediction. RFC 4787 discusses both behaviours for UDP and the requirements differ by category, which is why two NAT implementations can behave differently on the same traffic without either being wrong.

One direction only, by construction

A NAT gateway accepts connections initiated from inside and refuses those initiated from outside. There is no configuration that changes this, because the mechanism has no way to express the alternative: an inbound packet that matches no existing mapping has no destination to be translated to. It could be dropped, or it could be sent to a default host, and neither is a useful general behaviour.

This is why a service that must accept inbound connections cannot sit behind a NAT gateway. It needs either a public address on the interface, which uses the internet gateway path, or a load balancer in front of it, which has its own public address and forwards to private targets. Load balancer listener rules and target selection covers how the balancer terminates the client connection and opens a separate one to the target, which is a different mechanism from translation and behaves differently under failure.

The one-way property also affects protocols that try to establish a return path. An FTP client in active mode asks the server to connect back to a port on the client; behind a NAT, that connection has no mapping and fails. Protocols that carry addresses in their payloads fail for a related reason: the payload names the private address, which the remote peer cannot route to. Application-layer gateways that inspect and rewrite such payloads exist for this reason, and they work only for protocols they understand.

The translation table and its timeouts

Each mapping is created by an outbound packet and removed after a period of inactivity. The timeout differs by protocol because the protocols signal completion differently: a TCP mapping can be released when the connection closes cleanly, and must be held for a while after an abrupt close in case a retransmitted segment arrives; a UDP mapping has no close at all and is released purely on idle time.

struct nat_mapping {
    uint32_t priv_addr;
    uint16_t priv_port;
    uint16_t pub_port;      /* allocated from the gateway's pool */
    uint8_t  proto;
    uint8_t  state;         /* SYN_SENT, ESTABLISHED, FIN_WAIT, ... */
    uint64_t last_seen_ms;
};

/* Inbound: the destination port is the only discriminator. */
struct nat_mapping *nat_inbound(struct nat *n, uint16_t dport, uint8_t proto)
{
    struct nat_mapping *m = table_find(n, dport, proto);
    if (!m)
        return NULL;                    /* no mapping: packet is dropped */
    if (now_ms() - m->last_seen_ms > n->idle_ms[proto]) {
        table_remove(n, m);             /* timed out; nothing to deliver to */
        return NULL;
    }
    m->last_seen_ms = now_ms();
    return m;
}

The important line is the return NULL. An inbound packet with no mapping is not forwarded anywhere; it is discarded. Nothing tells the sender why, and nothing appears in the tenant’s logs unless flow logging is enabled. A client that expects a callback and does not receive one sees only a timeout.

The timeout interacts with the connection tracker described in security groups and the stateful connection tracker, and the two are not the same table. A mapping at the gateway and a flow entry at the interface can expire at different times, and a connection can therefore be alive from the gateway’s point of view and dead from the interface’s. When both expire, the connection is gone even though neither endpoint knows; when only one expires, packets pass one direction and not the other, and the connection hangs in a half-open state that neither side can resolve without a timeout of its own.

Path MTU and the quoted header problem

A NAT rewrites addresses in the IP header and the transport checksum. It must also handle ICMP error messages, which quote the header of the packet that caused them. A Packet Too Big message generated in response to a packet the gateway translated quotes the translated addresses, so the gateway must reverse the translation to deliver the message to the original sender. A gateway that does not do this drops the message, and the sender never learns that its packets are too large.

The result is the classic black hole: small packets pass, large ones vanish, and a TCP connection that completed its handshake stalls when it first sends a full-size segment. RFC 1191 defines the discovery procedure and the dependency on ICMP is explicit in it. In a cloud network the problem is compounded because the MTU inside the VPC may differ from the MTU on the provider’s network outside it, and encapsulation for overlay networks removes further bytes. The arithmetic is in MTU and fragmentation across a VPC boundary.

The practical consequence is that a design relying on path MTU discovery needs ICMP to survive every translation and every filter on the path. Where it cannot be guaranteed, the alternative is to clamp the maximum segment size at the boundary, which requires the gateway to rewrite the MSS option in the SYN packets it forwards. Not every gateway offers that, and the ones that do apply it only to TCP.

Choosing between the two paths

The decision is usually framed as public versus private, but the sharper question is whether the workload needs to accept connections that it did not initiate. If it does, it needs an address that is reachable, which means either a public address on the interface or a load balancer in front. If it does not, a NAT gateway provides outbound access without exposing anything, and its one-way property becomes an advantage rather than a limitation.

The cost of the NAT path is the extra hop and the state. Every packet crosses the gateway, and every flow consumes a mapping, so throughput and connection count are bounded by the gateway’s capacity rather than by the instance’s. A NAT gateway is also a failure domain of its own: it is created in a zone, and a subnet that routes through a gateway in another zone loses outbound access if that zone fails. The standard remedy is one gateway per zone with each subnet routed to the one in its own zone, which costs one gateway per zone but removes the cross-zone dependency. That redundancy is not optional in a design that expects to survive a zone loss, and it is the reason a single-gateway architecture is a single point of failure for outbound traffic even when every instance is healthy.