What a Sysroot Actually Restricts in a Cross Build

--sysroot=/path/to/rootfs reads like a confinement mechanism. It is not. It is a prefix-rewriting rule applied to a specific list of built-in search paths, and everything outside that list — -I flags, -L flags, environment variables, the contents of a pkg-config file — passes through to the compiler and linker untouched. A cross build with a perfectly populated sysroot still compiles against the host’s headers if one -I/usr/include reaches the command line, and the resulting object fails later in a way that points at the wrong library.

Understanding the option means knowing which searches it prefixes, which it ignores, and what a target root filesystem has to contain for the prefixed searches to find anything.

The layout the compiler expects under the prefix

When a toolchain is configured with --with-sysroot, its built-in directory lists are written with an = prefix, which the driver expands to the sysroot. The conventional expansion of those entries is:

Entry in the driver’s list	Resolves to
=/usr/include	$SYSROOT/usr/include
=/usr/lib	$SYSROOT/usr/lib
=/lib	$SYSROOT/lib
=/usr/lib/<triple>	$SYSROOT/usr/lib/<triple>

The last row is the multiarch arrangement used by Debian-derived distributions, where each ABI gets its own library directory and the triple in the path is the same string the toolchain was configured with. A sysroot assembled for a different triple has the libraries in a directory the driver will never look in, and the first symptom is cannot find -lc rather than a version mismatch.

gcc -print-sysroot reports the configured root, and prints an empty line when the toolchain has none. That empty line is a usable diagnostic: a cross compiler built without --with-sysroot will accept --sysroot on the command line and use it for some searches, but its built-in paths are absolute and stay absolute, so headers come from the host even though the flag was passed.

The root does not need to be a complete filesystem. It needs the headers the target’s libc installs, the libraries the linker will resolve against, the startup objects crt1.o, crti.o, crtn.o and their position-independent variants, and — for linking against the dynamic loader — the loader itself. Binaries and configuration files under the root are never read by the compiler.

Header search, and where the sysroot sits in it

The include search order for #include <stdio.h> on a GNU toolchain is roughly: directories given with -I, then the compiler’s own private include directory holding stddef.h, stdarg.h, and the fixed-width variants of limits.h, then the sysroot-prefixed system directories, then anything added with -idirafter. Three consequences follow.

First, the compiler’s private headers are not sysrooted in the same way as the libc headers. They ship with the compiler, describe the compiler’s view of the target, and are consistent with it. Mixing a GCC 12 include directory with the libc headers from a root built for GCC 8 produces occasional disagreements about types such as size_t or wchar_t that are hard to trace, because both files are valid C.

Second, -I beats the sysroot unconditionally. A build system that appends -I/usr/include — often because a dependency’s .pc file said so — puts the host’s headers ahead of the target’s. On a host whose libc is the same family, many headers are close enough that compilation succeeds; structures whose layout depends on the target word size will then be wrong, and the program misbehaves at run time rather than failing at build time.

Third, environment variables bypass the whole scheme. CPATH and C_INCLUDE_PATH add directories to the search path as if they had been passed with -I, and they are inherited from the shell that invokes the build. A sysroot that is correct in a Makefile can still be defeated by an exported variable, and nothing in the build output records that it happened.

Library search and the linker script indirection

The linker’s -l search follows its own order: -L directories in the order given, then the sysroot-prefixed built-in directories, then the default directories. GNU ld applies the sysroot to search-path entries that begin with = and to absolute paths named inside linker scripts, which is how the scripts installed by the C library express themselves. A glibc libc.so on an AArch64 root is not an ELF shared object at all; it is a small text file:

/* /usr/lib/aarch64-linux-gnu/libc.so */
GROUP ( /lib/aarch64-linux-gnu/libc.so.6
        /usr/lib/aarch64-linux-gnu/libc_nonshared.a
        AS_NEEDED ( /lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 ) )

Those paths are absolute, and they are resolved against the sysroot when ld has one. Pass the same script to a linker without a sysroot and it reads the host’s /lib/aarch64-linux-gnu/libc.so.6, which either does not exist or, on a distribution that installs foreign-architecture libraries under /usr/lib, exists and may be a different version from the one the sysroot was built with. The failure mode of a missing sysroot is therefore not always a clean error; it is sometimes a link against the host’s copy of a library the root also contains.

libc_nonshared.a in that script is worth noting for a different reason: it holds the parts of glibc that must be linked statically into every program, such as the atexit registration thunks and __libc_csu_init variants. A sysroot that contains only libc.so.6 links until the first program that needs one of those objects, then fails with an undefined reference that has nothing to do with the user’s code.

Explicit -L paths are taken literally. This is why a .pc file containing -L/usr/lib/x86_64-linux-gnu poisons a cross link even when --sysroot is set: the linker searches the host directory. PKG_CONFIG_SYSROOT_DIR exists to rewrite such paths before they reach the compiler, and using it is the difference between a sysroot that governs the build and one that merely participates in it. The wider problem of build systems that assume they can execute their test programs is covered in configure tests that execute.

Absolute paths that survive into the output

A sysroot is a build-time concept. Three kinds of absolute path can still end up in the binary, and they mean different things.

DT_NEEDED entries normally hold a soname such as libc.so.6 with no directory component, so they resolve against whatever root the loader runs in. This is the design that makes a root filesystem movable.

PT_INTERP holds the absolute path of the dynamic loader, for example /lib/ld-linux-aarch64.so.1. The path is absolute with respect to the runtime root, not with respect to the build machine, so a binary built against a sysroot at /opt/roots/bookworm still records the interpreter path the target will use. A sysroot whose loader lives at a different path produces a binary that cannot start, and the error names a file the build machine does have.

Debug information records the build directory through DW_AT_comp_dir and through the directory tables of the line program. Building inside /build/package-1.2.3 leaves that string in every object unless the compiler is told to rewrite it, and the path is often a container path that exists nowhere else. -fdebug-prefix-map rewrites the prefix at compile time; the same problem and the same remedy apply to __FILE__, which expands to the path as given on the command line.

What belongs in a root, and what copying /usr/include gets wrong

The most direct way to build a sysroot is to unpack the target’s development packages into a directory: the libc development package, the kernel headers, and the development packages of every library the build links against. Distributions package these as archives that can be extracted without executing anything, which is why this is the usual approach on a build machine that has no emulator installed.

Copying a running target’s /usr/include is a different operation and a worse one. It follows no dependency information, so it captures headers for libraries whose development files were never installed and misses the ones that were. It copies absolute symbolic links that point outside the copied tree. And it captures the target’s current library versions but not the linker scripts or the .a archives that the link step actually needs, because those live in the development package rather than in the runtime one.

The test of a sysroot is not whether it contains the right directories but whether a link succeeds with no path outside the root in the command line. gcc -print-search-dirs, gcc -E -v on a trivial file, and ld --verbose all print the search lists actually in use, and comparing those lists against the intended root is more informative than checking that the files exist.

A sysroot also does not isolate the compiler’s own target-independent defaults. A toolchain configured for the wrong --target still searches its own prefix first, and a sysroot pointed at the right root does not correct a driver whose built-in linker emulation is wrong. The triple and the root have to agree, and the triple is fixed at toolchain build time — the subject of reading a target triple. Both are inputs to the same contract, and neither substitutes for the other.