Header Folding and the Line Limits That Shape MIME Encoding

SMTP transports lines, not bytes. Every constraint in the message format descends from that single decision, made when mail moved between machines with different character sets and line terminators. The result is a wire format with three separate encoding layers, each solving a problem the one below it created.

Lines are the transport unit

RFC 5321 fixes the shape of what crosses the connection. A command line is at most 512 octets including the terminating CRLF. A line of message text is at most 1000 octets including CRLF, which leaves 998 octets of content. Text lines end in CRLF and nothing else; a bare LF or a bare CR is not a line terminator in the protocol, though servers commonly tolerate bare LF on input because too much software emits it.

The 998-octet limit is the root cause of most of what follows. A message body that is a single line of 5000 octets cannot be transmitted as-is. Neither can a binary file, which contains octets with no line structure at all and may contain a byte that happens to be a period at the start of a line.

The period is the second constraint. The end of message is signalled by a line containing a single period, so any body line beginning with a period must be transmitted with an extra period prepended and removed by the receiver. This is dot-stuffing. It is a transparent, reversible transform applied by the sender and undone by the receiver, and it is why a message body of ..foo arrives as ..foo and not .foo: the sender transmitted ...foo.

A naive encoder that emits a body containing CRLF sequences breaks the framing entirely. A body containing the three octets . CR LF at a line start would terminate the transaction early. That failure mode is the reason for the strictest rule in the whole format: message content may not be trusted to be line-safe, so it must be encoded.

Folding is not wrapping

Header fields can be long. A References field on a long thread can exceed a thousand octets, and a DKIM-Signature field routinely does. The format allows a field to continue onto following lines by inserting a CRLF before any whitespace, which turns that whitespace into folding whitespace. Unfolding is the reverse: replace CRLF followed by whitespace with a single space.

Folding is semantically meaningful only as whitespace. The grammar treats the CRLF-plus-whitespace pair as equivalent to a single space, so a folder that breaks a token in half changes the field’s meaning, and an unfolder that removes the whitespace entirely does too. Implementations that re-fold a header after parsing must preserve the semantic value, which means the fold points are constrained by the grammar: they can go before whitespace that already exists, not inside an atom, a quoted string, or a base64 blob that is not designed for it.

The 998-octet limit applies to each physical line, so folding is mandatory rather than cosmetic for long fields. A DKIM-Signature over a message with many signed headers can approach the limit, and the signature covers the folded form of the headers it signs, which is why canonicalization has to specify exactly what happens to whitespace, the subject of DKIM Canonicalization and What It Actually Signs.

There is a hard floor below the fold: a single token longer than 998 octets cannot be folded at all, because folding requires whitespace and the token has none. Base64 in headers is generated in chunks small enough to avoid this, which is one reason the encoded-word limit exists.

Three encodings for three different problems

The layering is worth separating because the three encodings are often confused.

Body transfer encoding handles the 998-octet and binary-safety problem. RFC 2045 defines a Content-Transfer-Encoding field per MIME part. 7bit asserts that the part is already safe: lines under 1000 octets, no octet above 127, no NUL. 8bit relaxes the octet range but not the line length, and requires the server to have advertised 8BITMIME. binary relaxes both and is almost never used over SMTP because the transport cannot carry it without a BINARYMIME extension that few servers implement. The two encodings that always work are base64 and quoted-printable.

Base64 costs a fixed 33 percent expansion: three octets become four characters, and RFC 2045 requires lines of at most 76 characters. The cost is predictable and the transform is trivially reversible, which makes it the right choice for anything already compressed, already binary, or already encrypted, since the expansion is a constant factor on data that does not compress anyway.

Quoted-printable costs between zero and 200 percent depending on content. Printable ASCII passes through unchanged, = becomes =3D, and any octet outside the safe set becomes = followed by two hex digits. Lines are soft-broken with a trailing = at or before 76 characters. For mostly-ASCII text with occasional accented characters, quoted-printable expands by a few percent and leaves the text human-readable in the raw message. For CJK text, nearly every octet needs escaping, and the expansion approaches the base64 worst case while producing a body nobody can read. That crossover is why the choice is content-dependent rather than a matter of taste.

Header encoding is a separate mechanism because the body encodings cannot be applied to headers. A header field is parsed by the receiving server to make routing and filtering decisions, so it cannot be an opaque base64 blob. RFC 2047 solves this with encoded words: a =?charset?encoding?encoded-text?= token that may appear wherever the grammar permits a word. Encoding B is base64 and Q is a quoted-printable variant where _ stands for space.

The encoded word has a hard bound: 75 characters total, counting the charset name, the encoding letter, the delimiters, and the encoded text. A long subject line therefore becomes a sequence of adjacent encoded words separated by folding whitespace, and the receiver must concatenate them without inserting the whitespace between them. That rule is the source of a persistent class of bug: a decoder that unfolds first and decodes second produces a subject with spurious spaces at every chunk boundary.

import base64, quopri, re

MAX_RAW = 45  # 60 base64 chars + "=?utf-8?b?" + "?=" stays under 75

def encode_word(text, charset="utf-8"):
    words, chunk = [], b""
    for ch in text:
        b = ch.encode(charset)
        if len(chunk) + len(b) > MAX_RAW:      # never split a character
            words.append(chunk)
            chunk = b""
        chunk += b
    if chunk:
        words.append(chunk)
    return " ".join(
        "=?%s?b?%s?=" % (charset, base64.b64encode(c).decode("ascii"))
        for c in words
    )

def decode_words(field):
    # Split on folding whitespace; adjacent encoded words join with no separator.
    out = []
    for p in re.split(r"\s+", field):
        m = re.fullmatch(r"=\?([^?]+)\?([bBqQ])\?([^?]*)\?=", p)
        if not m:
            out.append((" " if out else "") + p)
            continue
        charset, enc, body = m.groups()
        if enc.lower() == "b":
            raw = base64.b64decode(body)
        else:
            raw = quopri.decodestring(body.replace("_", " ").encode("ascii"))
        out.append(raw.decode(charset))
    return "".join(out)

The 45-octet ceiling is chosen so the encoded word stays under 75 characters: 45 octets become exactly 60 base64 characters, and =?utf-8?b? plus ?= adds 12. A fixed-offset split at 45 octets would cut through a multi-byte UTF-8 sequence and produce two chunks that each decode to replacement characters, so the encoder above accumulates whole characters and flushes when the next one would overflow.

Where the layers interact badly

The interactions are where implementations fail.

A base64 body containing a line that begins with a period is possible only if the encoder emits a period at a line start, which base64’s alphabet includes. Dot-stuffing handles it at the transport layer, so a receiver that strips stuffing before decoding is correct and one that strips it after decoding is not.

A Content-Transfer-Encoding of quoted-printable with a soft line break inside a multi-byte character is legal but produces a body that decodes to valid text only if the decoder buffers across the break. Decoders that treat each line independently corrupt exactly the characters the encoding was meant to protect.

Headers with encoded words inside a quoted string, or inside a comment, are not decodable: RFC 2047 permits encoded words only in specific positions, and a From field of "=?utf-8?b?...?=" <a@b> is technically malformed. In practice decoders are lenient, and the leniency differs between implementations, which means the same message can display differently in two clients.

The 998-octet limit also interacts with 8BITMIME. A server that advertises it accepts octets above 127 but still requires the line length bound. A client that sends 8-bit text with long lines has satisfied the extension and violated the base protocol, and the failure appears as a truncated message rather than an error, because the receiving server has no way to distinguish a long line from a protocol violation mid-stream.

The overall design is a stack of workarounds that became load-bearing. The line limit exists because early transports could not handle arbitrary binary; the encodings exist to satisfy the line limit; the encoded-word limit exists because folding cannot break a token; and the folding rules exist because the transport is line-oriented. Each layer is now permanent, and every new format feature, whether internationalized headers, signed headers, or structured fields, has to be expressed within them. The submission path that carries these messages into the queue is described in The Split Between Submission and Relay Ports.