Migrating Guests Whose Memory the Hypervisor Cannot Read

A confidential virtual machine is one whose memory is encrypted with a key that the hypervisor does not possess. The CPU decrypts pages as it accesses them, and the encryption is bound to the guest’s identity through a per-VM key derived at launch. The hypervisor still schedules the vCPUs, maps the memory, and handles exits — it simply cannot read the contents.

Every technique in this cluster so far has depended on the hypervisor being able to read guest memory. Zero-page detection compares bytes; compression finds redundancy in bytes; delta encoding diffs bytes against a previous version. Under encryption, all three see ciphertext, which is indistinguishable from random. The migration path has to be rebuilt around that blindness, and the rebuild changes both the cost model and the trust model.

What encryption removes from the pipeline

Encrypted guest memory, as the CPU sees it, is ciphertext with per-page tweaks. Two properties follow immediately:

No content-based optimization is possible. A page of zeros encrypts to something that looks exactly like a page of random data. Detecting sparse pages would require decrypting them, which the hypervisor cannot do. Compression over ciphertext expands it slightly, because ciphertext has no redundancy for the compressor to exploit. The only encoding left is framing and, where the architecture permits, a page-integrity metadata side channel that says whether a page is valid rather than what it contains.

No inspection is possible. A migration that validated guest state — checking for a corrupted page table, verifying a device descriptor — cannot do so. This is the point of the technology, and it means migration cannot be a debugging tool for a confidential guest in the way it can for an ordinary one.

The dirty tracking from Hardware Dirty-Bit Tracking for Guest Memory survives, because the nested page table’s dirty bit is set by the CPU on a write regardless of encryption. The CPU knows a write happened and where; it does not need to know what was written. The bitmap is therefore as accurate as ever, and the pre-copy convergence arithmetic of Dirty Page Rate and the Convergence Condition still holds — with a worse constant, because the bytes on the wire are the full ciphertext of every dirtied page.

Keys, and why the destination must be trusted

The guest’s memory is encrypted under a key that the guest’s own secure environment controls, derived from a secret injected at launch and never visible to the hypervisor. Moving the memory to another host does not move the key, and re-encrypting for transport requires a key the destination can use.

The architecture’s answer is a second layer: memory is re-encrypted for transport under a transport key that both endpoints hold, and the destination re-encrypts it under its own host key. The transport key is not the guest’s key; it protects the stream in transit. The guest’s key never leaves the guest’s security context, which means the destination’s CPU must be able to resume the guest under the same key material.

That requirement forces an attestation step. Before the destination can accept the guest’s state, it must prove to the source — or to the guest’s owner — that it is running a genuine, unmodified platform with the expected firmware and configuration. The proof is a signed report from the destination’s security processor, bound to a nonce chosen by the party doing the verification. Without that report, moving the guest to a destination that has been compromised would hand the guest’s memory to the attacker, encrypted or not, because a compromised destination can extract the key through the same interfaces that legitimately use it.

The verification party is usually not the hypervisor. In the common deployment, a management service holds the policy — which platforms are acceptable, which firmware versions, which measurements — and performs the attestation on the guest’s behalf. The source hypervisor only learns whether the check passed.

The two-ended key transfer protocol

The mechanics are usually delegated to a small component that runs inside the guest trust boundary and is authorised by the guest itself. Its job is to package the encrypted state in a form the destination can accept, including the guest’s key material wrapped for the destination’s security context.

The sequence has more steps than an ordinary migration because each side must prove itself:

The source initiates a migration and the guest (or its policy) authorises it, producing a wrapped export of the key material bound to the destination’s identity.
The destination produces an attestation report; the verifier checks it against policy.
The destination’s security context imports the key material, establishing the guest’s encryption context on the new host.
Memory is transferred as ciphertext, page by page, under the transport protection.
The destination validates integrity metadata and resumes the guest.

Steps 1 and 3 are the ones with no analogue in ordinary migration. The key material is wrapped for a specific destination, so a migration cannot be retargeted mid-flight: if the chosen destination fails attestation, the whole procedure restarts with a new target and new wrapping. This makes migration slower to start and impossible to fail over automatically to an arbitrary spare host, which is a significant operational change from non-confidential migration.

Integrity metadata is state that must travel

Confidential architectures maintain per-page integrity information: a version or counter that detects replay, and a check value that detects modification by the hypervisor. This metadata is not part of the guest’s address space but it is part of the guest’s correctness, and it must move with the memory.

The consequence is that the migration stream is larger than the guest’s RAM. Each 4 KiB page carries its ciphertext plus whatever integrity record the architecture defines, and the record must be authenticated along with the page. A destination that reconstructs the metadata incorrectly — or that accepts stale metadata from a previous migration of the same guest — will either reject valid pages or accept a replay.

The replay problem is the interesting one. An attacker controlling a destination that has previously hosted the guest could present old metadata for a page, rolling the guest back to a state it has already passed. Defences bind the metadata to a migration epoch and to the destination’s identity, which means the epoch counter is itself part of the state that must be tracked and never reused.

Failure modes specific to this path

The failure surface is wider than for ordinary migration, and the failures are less recoverable:

Attestation failure on the destination aborts before any state moves. Cheap, and the guest is unaffected. This is the good case.
Key import failure after the destination has accepted the stream leaves the destination holding ciphertext it cannot decrypt. The guest cannot resume on either side unless the source still has its key context, which it does only until it destroys it. The ordering must keep the source’s context alive until the destination confirms a successful resume.
Transport key compromise exposes the guest’s memory in transit. The transport protection is only as strong as the key exchange that established it, which is why the attestation covers the key exchange and not just the platform.
Guest refusal: a confidential guest may simply not permit migration. Many deployments configure guests to disable it, because migration requires the guest to trust a policy decision it cannot verify itself. The hypervisor cannot override that refusal, which is the intended behaviour.

The absence of content inspection also removes a diagnostic that operators rely on. When an ordinary migration stalls, an operator can examine the stream, count zero pages, check compression ratios, and identify whether the guest is dirtying too fast. For a confidential guest, the stream is opaque and the only signals are the dirty bitmap’s statistics and the byte counts. Diagnosis moves from inspection to inference.

What remains cheap and what becomes expensive

The parts of the pipeline that operate on addresses rather than contents are unaffected: dirty tracking, pass scheduling, the convergence test, device state serialization, and the stop-and-copy ordering. The parts that operate on contents are gone: sparse detection, compression, and delta encoding. In practice the loss is large for guests whose memory is mostly idle, because the zero-page skip is the single biggest saving in ordinary migration.

The compensation is that the encryption itself costs bandwidth, not CPU on the migration path: the CPU that encrypts and decrypts is doing so anyway for the guest’s execution, and the migration stream carries ciphertext that requires no additional transformation on the source. The destination must re-encrypt under its own key, which is a per-page cost on the receive path and can become the bottleneck when the link is faster than the destination’s crypto throughput.

The overall result is that confidential guests migrate with a bandwidth requirement close to their full dirty rate with no compression multiplier, a mandatory attestation delay at the start, and a much narrower set of acceptable destinations. Those constraints are the direct price of removing the hypervisor from the trust boundary, and they are the reason confidential computing and routine live migration are in tension even when the mechanisms exist.