Encrypt-and-MAC, MAC-then-Encrypt, Encrypt-then-MAC

Combining an encryption scheme with a MAC looks like it should be a matter of taste. There are three ways to arrange the two operations, and for years implementations chose among them by convenience. The security difference between the arrangements is not a matter of degree: two of the three fail against chosen-ciphertext attacks, and the third is provably secure under standard assumptions.

The three arrangements

Write $E$ for a randomized symmetric encryption scheme, $D$ for its decryption, and $T$ for a MAC over a shared key. The message is $M$, the ciphertext is $C = E_K(M)$.

Encrypt-and-MAC computes $T_{K_2}(M)$ over the plaintext and transmits $C ,|, T_{K_2}(M)$. Decryption recovers $M’ = D_K(C)$ and checks the tag against $M’$. This is the arrangement in SSH, where the packet length and payload are encrypted and the MAC is computed over the plaintext.

MAC-then-Encrypt computes $T_{K_2}(M)$ and encrypts the concatenation: the ciphertext is $E_K(M ,|, T_{K_2}(M))$. Decryption decrypts, splits, and checks. This was TLS up to version 1.2 in its block-cipher suites.

Encrypt-then-MAC computes $T_{K_2}(C)$ over the ciphertext and transmits $C ,|, T_{K_2}(C)$. Decryption checks the tag before decrypting anything. This is the arrangement in IPsec ESP and in TLS 1.3 after the AEAD migration.

The encryption half of each arrangement is a block cipher in a chaining mode, and the MAC half is usually a construction of the kind covered in the CMAC analysis. The three names describe the order of the calls, not the primitives, which is why the same attacks appear with different ciphers and different MACs.

What each ordering authenticates

The distinguishing question is whether the receiver ever decrypts an unauthenticated ciphertext.

In encrypt-then-MAC, the MAC covers the ciphertext, so a modified ciphertext fails the tag check and the decryption function is never invoked on attacker-chosen data. The receiver’s behaviour on a bad tag is a single branch that discards the record. No plaintext, no padding validity, no error detail from the decryption layer reaches the attacker.

In MAC-then-encrypt, the tag is inside the ciphertext, so the receiver must decrypt before it can check anything. The decryption is on attacker-chosen input, and any distinguishing behaviour between “padding invalid” and “padding valid but tag invalid” is an oracle. That distinction is exactly what padding-oracle attacks use: the attacker flips bytes in the final block, observes whether the receiver rejects with a padding error or a MAC error, and learns one byte of intermediate plaintext per query by exploiting the fact that the padding check happens before the tag check. A receiver that emits distinct errors, or takes distinguishable time, hands over the plaintext one byte at a time without breaking the cipher.

In encrypt-and-MAC, the tag is over the plaintext, so the receiver also decrypts before checking. The same oracle applies, plus a second weakness: the tag is computed on the plaintext, so it may leak information about the plaintext, and for a deterministic MAC over a message space that the adversary can partially guess, the tag itself is a confirmation oracle. The construction also provides no protection against an attacker who can produce a ciphertext whose decryption is a message the attacker already knows the tag for, because the tag travels separately and can be paired with a different ciphertext.

The security definitions and where they separate

The formal treatment is Bellare and Namprempre’s analysis of generic composition. The relevant notions are:

Privacy under chosen plaintext (IND-CPA): the adversary sees encryptions of two messages of its choice and must distinguish which one was encrypted.

Privacy under chosen ciphertext (IND-CCA): additionally, the adversary submits ciphertexts and receives decryptions, except for the challenge ciphertext. This is the realistic model for a network protocol, where an attacker can inject traffic and observe the response.

Integrity of ciphertexts (INT-CTXT): the adversary cannot produce a ciphertext that decrypts successfully and was not produced by the legitimate sender, even with access to an encryption oracle.

The result that matters is that encrypt-then-MAC with a secure encryption scheme and a secure MAC achieves both IND-CCA and INT-CTXT. MAC-then-encrypt does not: there exist encryption schemes that are IND-CPA secure and MACs that are unforgeable, whose composition is not IND-CCA, because the MAC check happens after the decryption oracle has already revealed a distinction. Encrypt-and-MAC fails even more broadly, since the MAC over the plaintext can destroy privacy for a deterministic MAC, and it provides no ciphertext integrity at all in the sense above: an attacker can modify the ciphertext and keep the tag, and the receiver learns of the modification only after decrypting.

The gap is not a matter of constructing contrived primitives. Naor and Yung showed that chosen-ciphertext security requires authentication of the ciphertext as part of the construction, and the intuition is simple: an adversary with a decryption oracle must be prevented from using it on anything but properly formed ciphertexts, and the only way to know a ciphertext is properly formed without decrypting it is to authenticate it.

Why TLS 1.2’s MAC-then-Encrypt survived so long

The record layer of TLS 1.2 with a CBC-mode block cipher does MAC-then-encrypt. The specification is careful about the order of checks and recommends that implementations not distinguish padding errors from MAC errors, but the recommendation is advisory and the timing difference remains observable even when the error codes are identical, because the padding check and the MAC check take different amounts of time and touch different amounts of data. The result is a family of padding-oracle attacks that have been demonstrated against implementations that follow the letter of the specification.

The failure is structural rather than a bug in a single library. In MAC-then-encrypt the receiver must decrypt first, and the decryption of CBC is where the padding is validated. Any decision made after decryption but before the MAC check is visible through timing, error codes, or connection teardown behaviour. Implementations can mitigate this by decrypting in constant time and deferring all decisions until the MAC is checked, which is possible but requires the decryption function to run in time independent of the padding content, and a single non-constant branch anywhere in the path reopens the oracle.

TLS 1.3 removed the problem by restricting the record layer to AEAD schemes, which are encrypt-then-MAC by construction: the ciphertext and associated data are processed by a single primitive that produces a tag over both, and the receiver verifies before decrypting. This is the same move as RFC 5116’s AEAD interface, which presents encryption and authentication as one operation so that the ordering is not a choice the caller can get wrong.

Nonce and length binding

Whichever ordering is chosen, the MAC must cover more than the ciphertext. It must cover the sequence number, or a nonce, and it must cover the length. Without a sequence number, a record can be replayed or reordered and still verify, because the bytes are unchanged. Without a length, a ciphertext that decrypts to $M$ and one that decrypts to $M ,|, X$ may not both be valid, but the absence of an explicit length makes the parsing depend on the decryption output, which reintroduces a decision before the tag check.

Encrypt-then-MAC with the sequence number and length outside the ciphertext, both under the tag, is the standard shape. The record format of TLS 1.3 and the ESP format of IPsec both do this: the sequence number is implicit and incremented per packet, the length is in the clear but authenticated, and the tag covers the header fields that the receiver depends on. An implementation that authenticates only the payload has authenticated the plaintext but not its position, which is enough for many applications and not enough for a transport.

/* Encrypt-then-MAC record. The associated data is rebuilt by the receiver,
   never transmitted: sequence number, type, version, and ciphertext length. */
static void put_be64(uint8_t *p, uint64_t v)
{
    for (int i = 7; i >= 0; i--) { *p++ = (uint8_t)v; v >>= 8; }
}

uint8_t ad[13];
put_be64(ad, seq);
ad[8] = type; ad[9] = version_major; ad[10] = version_minor;
ad[11] = (uint8_t)(ct_len >> 8); ad[12] = (uint8_t)ct_len;

/* OpenSSL low-level HMAC API; OpenSSL 3.0 deprecates these in favour of EVP_MAC.
   The tag is checked before any decryption. */
unsigned int tag_len;
HMAC_CTX *h = HMAC_CTX_new();
HMAC_Init_ex(h, mac_key, key_len, EVP_sha256(), NULL);
HMAC_Update(h, ad, sizeof ad);
HMAC_Update(h, ct, ct_len);
HMAC_Final(h, tag, &tag_len);
HMAC_CTX_free(h);

The associated data is not transmitted as a separate field; it is reconstructed by the receiver from the header and its own sequence counter, and the tag is computed over the reconstruction. If the reconstruction differs from what the sender used, the tag fails. This is why the sequence number must be implicit and synchronized: it is authenticated by construction, not by transmission.

The cost of the ordering

Encrypt-then-MAC requires two keyed passes over the data, one for encryption and one for the MAC over the ciphertext, plus a pass to buffer the ciphertext before tagging. The buffering is the practical cost: a streaming implementation must either hold the whole record or tag incrementally as it encrypts, and the latter is possible for many MACs but not for all. Decryption is cheaper in the reverse direction, since a bad tag discards the record without touching the cipher.

MAC-then-encrypt lets the sender compute the MAC while streaming and encrypt afterwards, which is convenient for a layered implementation where the MAC sits in one module and the cipher in another. That convenience is what the construction trades against chosen-ciphertext security.

Encrypt-and-MAC allows the receiver to process the decryption and the MAC in parallel, since neither depends on the other’s output, which suits hardware datapaths that overlap the two units. It pays for that with the oracle and the plaintext-tag leak.

A dedicated AEAD primitive avoids the question altogether by computing both from one key schedule and one pass, in the manner described in the Poly1305 construction: the MAC key is derived from the same keystream as the encryption key, the associated data and ciphertext are authenticated together, and the interface exposes only a combined operation. When a combined primitive is available, composing separate ones is a source of exactly the errors described here, and the correct choice is to use it.