Entering Long Mode From a Boot Loader

A 64-bit kernel is compiled to run at addresses that no 32-bit mode can express, and its entry code assumes it is already executing in 64-bit mode with a working page table. Nothing in the kernel’s first instruction establishes that. The transition happens in the boot loader, and it is a sequence of maybe a dozen instructions whose order is fixed by the architecture and whose failure mode is a triple fault with no output.

The reason it is not simply a flag is that long mode is not a mode the processor can enter directly. It is a mode that becomes active when paging is enabled while a specific enable bit is already set. Both conditions must be satisfied at the same moment, and the page tables that paging will use must be correct before that moment arrives.

The three prerequisites

Long mode requires that physical address extension be enabled, that the extended feature enable register have its long-mode bit set, and that a valid page-table hierarchy be installed in CR3. The order among these is mostly free except for the last two, which interact.

CR4.PAE must be set before paging is enabled. With PAE clear, the page-table format is the 32-bit two-level format, which cannot describe a 64-bit address space, and setting the long-mode enable bit while PAE is clear is an invalid combination that the processor refuses. Setting CR4.PAE alone is harmless: a 32-bit kernel can run with PAE on, using the same three-level table format that long mode uses, just with smaller entries and fewer levels.

The long-mode enable bit lives in the IA32_EFER model-specific register at bit 8, and it is reachable only through RDMSR and WRMSR with the register number 0xC0000080. Writing that bit does not enter long mode; it declares the intent. The processor will enter long mode at the moment paging is enabled, provided the other conditions hold. Until then it continues executing in protected mode with 32-bit semantics, which is what makes the sequence possible at all: the code that sets up the page tables is still 32-bit code.

Why the page tables must be identity maps

The moment CR0.PG is set with long-mode enable already set, the next instruction fetch goes through the new page tables. The instruction that follows the mov to CR0 is fetched using the mapping that CR3 describes. If the loader’s code is at physical address 0x100000 and the page tables map virtual 0x100000 to physical 0x100000, the fetch succeeds. If they map it anywhere else, or do not map it at all, the fetch faults, and because the loader has no IDT installed the fault escalates to a triple fault and a reset.

This is why the standard sequence builds an identity map covering at least the loader’s own code, its stack, the page tables themselves, and the region where the kernel image has been placed. A common shortcut maps the first gigabyte with 2 MiB pages, which requires a single page-directory-pointer table entry, a single page directory, and 512 page-directory entries, each pointing at a 2 MiB physical region with the page-size bit set. That is 4 KiB for the PDPT plus 4 KiB for the PD, and it covers everything a kernel’s early code needs.

With 4-level paging, the hierarchy is: one PML4 entry, one PDPT entry, one PD entry with the page-size bit set, and the result is a 2 MiB page. The address bits are split 9/9/9/9/12, and in long mode the top bits of a canonical address must all equal bit 47. An identity map of the low gigabyte is comfortably canonical.

/* 32-bit code building a 2 MiB identity map before enabling long mode */
static uint64_t pml4[512] __attribute__((aligned(4096)));
static uint64_t pdpt[512] __attribute__((aligned(4096)));
static uint64_t pd[512]   __attribute__((aligned(4096)));

void build_identity_map(void)
{
    pml4[0] = (uint64_t)(uintptr_t)pdpt | 0x3;  /* present, writable */
    pdpt[0] = (uint64_t)(uintptr_t)pd   | 0x3;

    for (unsigned i = 0; i < 512; i++) {
        /* 2 MiB pages: bit 7 selects the large-page format */
        pd[i] = (uint64_t)i * 0x200000ULL | 0x83;
    }
}

The 0x83 in each directory entry is present, writable, and page-size. Without the page-size bit the entry would be interpreted as a pointer to a page table, and the processor would read whatever those 4 KiB contain as further entries.

The register sequence

The canonical sequence, in the order the architecture requires:

    mov     eax, cr4
    or      eax, 1 << 5             ; CR4.PAE
    mov     cr4, eax

    mov     ecx, 0xC0000080         ; IA32_EFER
    rdmsr
    or      eax, 1 << 8             ; EFER.LME
    wrmsr

    mov     eax, offset pml4        ; physical address of the PML4
    mov     cr3, eax                ; 32-bit write: upper 32 bits are zero

    mov     eax, cr0
    or      eax, 0x80000001         ; CR0.PG | CR0.PE
    mov     cr0, eax                ; long mode becomes active here
    /* still executing 32-bit code, but in compatibility mode now */
    ljmp    $0x08, $long_entry      ; far jump to a 64-bit code segment

Two details are easy to get wrong. First, CR3 must be loaded before CR0.PG is set, and the value must be a physical address, because paging is off at that point and the loader is writing a register the memory unit will interpret directly. Second, the mov to CR0 does not switch the code segment. The processor enters long mode’s compatibility sub-mode, in which the existing 32-bit code segment continues to execute with 32-bit semantics. The code after that instruction is still 32-bit code, and it is only after a far jump to a descriptor whose long-mode bit is set that the processor begins fetching 64-bit instructions.

That far jump needs a GDT whose target descriptor has the long-mode attribute set: bit 21 of the descriptor’s high doubleword, with the descriptor’s default-operation-size bit clear, since 64-bit code segments have no meaningful default size. A far jump to a descriptor without that bit is legal but does not produce 64-bit execution: the processor stays in compatibility mode and the target code runs with 32-bit semantics, which is not what a kernel compiled for a 64-bit address space expects. The failure then appears much later, as a fault on the first instruction that assumes a 64-bit register width.

What goes wrong

The most common failure is a page table built with the wrong entry format. A 32-bit PAE page directory entry and a long-mode page directory entry share the low bits, so a table copied from a 32-bit PAE setup looks correct, but the levels above it do not: a page-size bit in a PML4 or PDPT entry is reserved in both formats, and a loader that promotes a page directory to a higher level carries the bit with it. The processor faults on the first access that walks through the bad entry, and since the loader has no handler installed, the fault escalates to a reset.

The second is the CR3 value itself. CR3 holds a physical address, and a loader that has already enabled paging for its own use is running at virtual addresses that its current tables translate. If it computes the address of its PML4 as an ordinary pointer, that value is a virtual address, and writing it to CR3 installs a table base that points wherever that virtual address happens to land in physical memory. The next instruction fetch after paging is re-enabled then reads an entry from a location that is not a page table at all, and the machine resets.

The third is the stack. A 32-bit loader cannot place its stack above the 4 GiB boundary, because the only register it can write is ESP, and the value it writes is 32 bits wide. If the new page tables map the loader’s stack region, the push instructions after the far jump work; if they do not, the first push faults, and the fault handler does not exist yet. Loaders therefore either identity-map the region holding their stack or switch RSP to a mapped address with a 64-bit mov immediately after the jump.

A fourth is the GDT. The far jump reads a descriptor from the table whose base and limit are in GDTR, and that read happens after the new page tables are active. A GDT that sits in a region the new tables do not map produces a general-protection fault on the jump itself, which is indistinguishable from a bad descriptor unless the loader has arranged some way to report it. The same applies to the descriptor’s contents: the target code segment must have the long-mode attribute set and the default-operation-size bit clear, and a descriptor copied from a 32-bit flat-model setup has neither.

Why the kernel does not do this itself

The transition could in principle be done by the kernel’s first instruction, if the kernel were loaded as 32-bit code. Some kernels did exactly that historically, carrying a 32-bit entry stub that performed the mode switch. The reason the work moved into the loader is that the loader already has to build a memory map, read a configuration, and place the kernel image, and doing the mode switch there means the kernel’s entry code can be compiled as ordinary 64-bit code with no mode-specific assembly.

The cost is a loader that is architecture-aware in a way a simple chainloader is not. An EFI application runs in long mode already, so on that path the firmware has done the transition and the loader inherits it; a legacy BIOS loader has to do it itself. The two paths converge on the same kernel entry contract, which is why the kernel image’s loader-facing flags, described in why boot loaders are split into stages, include one asserting that the kernel supports being entered directly in 64-bit mode, so a loader can tell which entry it may use.

What the kernel does immediately after entry is a separate problem: it must discard the identity map, install its own higher-half mapping, and keep executing across the change. That relocation is covered in kernel decompression and physical relocation.