The Anatomy of a Virtual Instance: vCPU Threads, Guest Memory, and Device Models

An instance is sold as a shape: a number of vCPUs, a quantity of memory, an optional accelerator, and a network attachment. Beneath that shape the provider runs a virtual machine monitor that has to turn those abstract counts into real host resources, and the mapping is neither one-to-one nor transparent. A vCPU is a schedulable host thread. Guest physical memory is a set of host pages reached through an extra level of address translation. The disk and network are not hardware at all but a protocol spoken between the guest driver and a process on the host. Each of these three substitutions has a cost that shows up as observable behaviour inside the guest.

A vCPU is a host thread, not a core

When the guest executes an instruction stream, the hypervisor has a thread of its own that the host scheduler can preempt, migrate, and deschedule. Entering the guest is a transition from host context to guest context, and leaving it is the reverse. On x86 the transition is managed by a hardware structure that saves host state and loads guest state; the processor supports a defined set of entry and exit reasons, and every exit is a switch of address space, privilege, and register file.

The cost of the transition is the reason the design of the guest matters. A guest that executes a privileged instruction without a hardware-provided escape causes an exit, and each exit costs on the order of a microsecond in the best case, considerably more when it invalidates translation caches or branch-prediction state that must be rebuilt. Instructions that the hardware can execute natively — ordinary arithmetic, loads and stores to guest memory, most branches — do not exit at all. The Popek and Goldberg conditions describe exactly which instructions must trap for a machine to be virtualizable: privileged instructions that touch machine state must trap, and they must be trapped only when executed in the guest. Architectures that violate the conditions require binary translation or paravirtualized replacements for the offending instructions, which is why the historical x86 virtualization problem was about popf, sgdt, and ring compression rather than about raw speed.

Two consequences follow for anyone sizing an instance. First, the guest sees the vCPU count but not the host’s physical topology; the scheduler decides which physical core, hyperthread sibling, or NUMA node the thread runs on, and it may change that decision between two guest instructions. Second, an idle vCPU is not a reserved core. The host oversubscribes, and the guest’s own scheduler has no visibility into whether the thread it is running on shares a physical core with another tenant. Both facts are the subject of steal time and oversubscription accounting, and they are why a guest’s measured instructions-per-cycle can drop without any change to its own workload.

Guest physical memory is not host physical memory

The guest operates on guest physical addresses, which are contiguous from its point of view. The host maps those to machine frames that are scattered, and it can change the mapping without the guest noticing. Two mechanisms do the work. Shadow page tables, the software approach, maintain a per-guest page table that maps guest virtual addresses directly to machine frames, and the hypervisor must intercept every guest write to a page table so the shadow stays coherent. Nested paging, the hardware approach, adds a second translation: the guest’s page tables translate virtual to guest-physical, and a host-owned table translates guest-physical to machine. A single guest memory access can therefore require a two-dimensional walk, each level of which may miss in the translation caches.

Nested paging removes the need to write-protect guest page tables, which eliminates a large class of exits, but it makes each translation cache miss more expensive because the hardware may have to walk both tables. The mitigation is that the processor caches combined translations, so a guest with good locality pays the two-dimensional cost rarely. A guest with a large random working set pays it often. The practical symptom is that memory-latency-sensitive code on a virtualized instance sees higher and more variable latency than the same code on bare metal, and that the penalty grows with the size of the guest’s page tables.

Memory is also the resource most often adjusted while the instance runs. A balloon driver inside the guest can be asked to pin and release pages, which returns them to the host without the guest’s kernel ever seeing a hot-unplug event. The balloon’s size is the difference between what the guest believes it has and what the host has committed to it, so an over-ballooned guest starts swapping inside its own page cache while reporting free memory. This is a common cause of the confusing state where a guest shows available memory but its application latency has risen sharply.

/* A guest page-table walk under nested paging: two dependent lookups per level.
 * The host page-table root is supplied by the hypervisor, not the guest. */
static uint64_t translate(uint64_t gva, uint64_t guest_cr3, uint64_t host_root)
{
    uint64_t gpa = gva;
    for (int level = 3; level >= 0; level--) {
        uint64_t entry_gpa = gpa + (((gva >> (12 + 9 * level)) & 0x1ff) << 3);
        uint64_t pte = read_guest_phys(entry_gpa, guest_cr3, host_root);
        if (!(pte & PTE_PRESENT))
            return FAULT;
        gpa = (pte & PTE_FRAME_MASK) | (gva & ((1ULL << (12 + 9 * level)) - 1));
    }
    return gpa;
}

The loop above is not how hardware implements the walk — the walker is pipelined and caches intermediate results — but it shows the dependency structure: each guest-physical read must itself be translated, so the cost of one guest page walk is not four memory reads but up to twenty.

Devices are a protocol, not a chip

A guest expecting a SCSI controller or an Intel-branded network card would need the hypervisor to emulate that device’s register interface faithfully, which means trapping every programmed I/O access and every memory-mapped register read. That path is slow enough that a single emulated register write can cost more than a thousand guest instructions. The alternative is a paravirtualized device: the guest loads a driver that knows it is talking to a host process, and the two sides exchange descriptors through a shared ring in memory.

The virtio family defines that arrangement. A device exposes a set of queues; each queue is a ring of descriptors that point into buffers the guest has made available. The guest writes a descriptor, advances an index, and notifies the device by writing to a transport-specific location. The host reads the index, performs the I/O, writes results back into the buffers, and signals completion by updating a used ring. The number of exits per request is reduced from one per register access to one per batch, and the data itself moves without the hypervisor ever interpreting it.

The transport matters as much as the queue format. Early virtio transports used programmed I/O; later ones use memory-mapped registers, and modern ones use a PCIe capability structure that the guest can discover by walking configuration space. Interrupt delivery has moved from emulated interrupt controllers to a message-signalled path where the host writes directly into guest memory to raise a vector, avoiding a trip through the virtual interrupt controller. Each of these changes removes exits, and the aggregate effect is that a well-configured paravirtualized NIC can approach the throughput of a physical one while still being subject to the host’s scheduler. The same exit-avoidance logic drives the design of the isolation boundary in serverless cold starts, where a minimal device model is what keeps environment creation short.

Where the abstraction leaks

Three behaviours inside a guest have no bare-metal analogue. The first is time: the guest’s notion of wall-clock time comes from a paravirtualized clock source, and the guest may observe that time jumps when it is descheduled. Code that measures elapsed time to make decisions, rather than using a monotonic counter with a known relationship to the host, will see the jumps. The second is CPU identity: the guest may see a feature set that is the intersection of the host’s capabilities and the hypervisor’s exposed mask, so an instruction that works on one instance type faults on another even when the advertised processor family is the same. The third is device latency: a network round trip inside a virtualized guest includes the host’s scheduling delay, so tail latency is dominated by whether the vCPU thread was runnable when the interrupt arrived.

None of these are defects. They are the price of multiplexing a machine among tenants, and each one has a mitigation — a paravirtualized clock, a stable CPU model, interrupt affinity and polling. What matters for capacity planning is that the mitigations are configuration choices, and an instance type is partly a promise about which ones the provider has already made.