Software Breakpoints: Replacing an Instruction with a Trap Byte

A breakpoint set on a source line is, underneath, a single byte written into the process’s executable text. The debugger resolves the line to an address, saves whatever byte is stored there, writes a trap instruction in its place, and lets the program run. The next time control reaches that address the processor raises an exception, the kernel stops the process, and the debugger is told. The impression that the program paused at a chosen statement is maintained by bookkeeping around that byte: saving it, restoring it, adjusting the program counter, and putting it back before the program continues.

The instruction that exists to be overwritten

On x86 the trap is int3, encoded as the single byte 0xCC. The two-byte form int 3 (CD 03) exists as well, but a debugger cannot use it. x86 instructions have variable length, and replacing a one-byte instruction with a two-byte one would shift every following instruction by a byte, invalidating every address the debugger has recorded. Since the trap must fit inside any instruction slot, the encoding is exactly one byte, and the breakpoint address is exactly the address of the replaced instruction.

Fixed-width architectures do not have this constraint and use a whole instruction. AArch64 uses BRK with a 16-bit immediate carried in the low bits, ARM uses BKPT, RISC-V uses EBREAK. In each case the replacement occupies one instruction word, so the breakpoint address is again the address of the replaced instruction and the debugger’s job reduces to the same save-patch-restore cycle.

int3 is a trap rather than a fault: the processor finishes the instruction and saves the address of the next instruction as the return address. That single architectural detail is the reason for the most characteristic piece of debugger arithmetic, described below. The kernel’s own instrumentation uses the same byte on x86, where kprobes plants 0xCC and handles the resulting exception, which is why a breakpoint placed inside kernel text requires the kernel’s cooperation rather than a raw memory write.

The choice of a trap rather than a dedicated breakpoint register at every address is a space argument. A program can carry thousands of breakpoints in a debugger session, and one byte per breakpoint in the text costs nothing that the process was not already paying for code. The alternative, dedicating hardware to each breakpoint, runs out at four on x86, which is the subject of hardware breakpoints and debug registers.

Patching a word to change a byte

Linux exposes the inferior’s memory through ptrace, but not a byte at a time. PTRACE_PEEKTEXT and PTRACE_POKETEXT transfer one word, which is sizeof(long) bytes and therefore eight on x86-64, and the address must be word-aligned. A debugger that wants to change one byte therefore reads the containing word, masks the new byte into the right bit offset, and writes the word back. The read-modify-write is not atomic with respect to the tracee: if another thread writes the same word between the read and the write, that write is lost. This is one reason a debugger stops all threads before inserting a breakpoint in a multi-threaded process, and why insertion happens while the tracee is in a stop.

#include <errno.h>
#include <stdint.h>
#include <sys/ptrace.h>

/* Write one byte into a tracee's memory, preserving its neighbours. */
static int poke_byte(pid_t pid, uintptr_t addr, uint8_t value)
{
    uintptr_t aligned = addr & ~(uintptr_t)(sizeof(long) - 1);
    unsigned shift = (unsigned)(addr - aligned) * 8;
    unsigned long word;

    errno = 0;
    word = (unsigned long)ptrace(PTRACE_PEEKTEXT, pid, (void *)aligned, 0);
    if (word == (unsigned long)-1 && errno != 0)
        return -1;

    word = (word & ~(0xffUL << shift)) | ((unsigned long)value << shift);
    return ptrace(PTRACE_POKETEXT, pid, (void *)aligned, (void *)word) == -1 ? -1 : 0;
}

Writing through ptrace bypasses the page protection of the tracee. Text is mapped read-only, but the kernel performs the store on the tracee’s behalf after checking that the tracer is attached. For a private file mapping the write breaks copy-on-write, so the modified page becomes private to that process. The file on disk is unchanged, and a second process running the same executable or library does not observe the breakpoint. That is what makes it safe for a debugger to set breakpoints in a shared C library while other processes on the machine continue to use it.

The cost is that the breakpoint lives in one process’s address space. A forked child, or a second run of the same binary, needs the byte inserted again. Debuggers therefore track a breakpoint as a set of locations, each a specific address in a specific process or shared object, and re-insert at every new inferior.

Why the counter moves back by one

When the processor executes 0xCC it reports the exception with the saved instruction pointer pointing at the byte after the trap. The debugger must subtract one before the address makes sense: what the kernel reports is not the breakpoint but the breakpoint plus the length of the trap instruction. GDB calls this adjustment the breakpoint’s kind, and it is architecture-specific. On fixed-width machines the exception is a fault and the saved address is the breakpoint itself, so the adjustment is zero.

Resuming is a three-step operation, and skipping any step produces a distinctive failure. The original byte is written back first, because a single-step with 0xCC still in place would trap at the same address and make no progress. The thread is then stepped exactly one instruction. Only when that step completes is 0xCC written again and the thread continued. If the step itself faults, or the thread receives a signal, the debugger has to decide whether the breakpoint should be reinserted before the signal is delivered, since the handler may return to the same address.

The reinsertion step is a genuine race in a multi-threaded program. Between removing the byte and putting it back, the instruction at that address is the original one, and any other thread reaching it runs straight through. All-stop debugging avoids the race by keeping every thread stopped except the one being stepped. Non-stop debugging either accepts the window or uses a hardware breakpoint for the step, which is one of the reasons hardware support exists at all.

A related edge case is a breakpoint on an instruction that does not return normally: a call that never comes back, a jump into another function, or a syscall that blocks. The step completes when the next instruction is about to execute, which may be somewhere else entirely. The debugger reinserts at the original address regardless, because a breakpoint belongs to an address rather than to a thread’s path through it.

Code that is shared, read-only, or generated

Three kinds of text defeat the simple model.

The first is memory mapped MAP_SHARED. A write to a shared mapping is visible to every process that maps the same object, because there is only one copy of the page. A breakpoint there stops every process that reaches the address, including processes the debugger has never heard of. Shared memory used for JIT code or for a shared library mapped without privacy has this property, and debuggers generally warn rather than silently patching.

The second is self-modifying code. A just-in-time compiler writes new instructions over an address that may hold a breakpoint, discarding the 0xCC or invalidating the debugger’s saved byte. A debugger that restores a cached byte later writes the wrong value, and the program executes a stale instruction. Runtimes that want generated code to be debuggable register the generated regions with the debugger, and GDB has a JIT interface for exactly this; the debugger then re-reads the original bytes instead of trusting a cache.

The third is code that does not exist. A breakpoint on a line inside a function that the optimiser inlined has no address of its own; the line table may map it into the middle of the caller, or not at all. This is a property of the debug information rather than of the byte patch, and it is covered in location lists and inlined frames.

What remains is the useful core. The patch is one byte, reversible, and general enough to work on any instruction boundary the architecture allows. The parts that are not reversible, namely the stepping window, the per-process scope, and the dependence on a byte that stays put, mark where software breakpoints stop being the right tool and where the processor’s debug registers or a trap instruction executed by the target itself take over.