Target Triples, Sysroots, and What Actually Changes When Cross-Compiling

A cross-compiler is a compiler whose output runs somewhere other than the machine it runs on. The name of the place is encoded in a target triple, a short hyphenated string such as x86_64-linux-gnu or aarch64-unknown-linux-musl, and it is tempting to think that specifying the triple is the whole of cross-compilation. It is the smallest part. Changing the target changes the instruction set and the object file format, which the compiler handles internally, and it also changes every assumption the compiled program makes about the operating system it will meet: the size of a pointer, the layout of a struct, the names and semantics of the C library entry points, the startup code that runs before main, and the paths from which headers are read. Those latter facts live outside the compiler binary, in a directory tree, and getting that tree right is where cross-compilation actually fails.

The anatomy of a triple and why it has four parts

The canonical form has four fields: architecture, vendor, system, and ABI. The architecture and system fields are the ones read first. The vendor field exists for historical reasons and is usually unknown or a hardware vendor’s name; toolchains differ on whether it affects anything. The ABI field is the one that does the most silent work.

Consider arm-linux-gnueabihf. The architecture is arm, the system is linux, and the ABI suffix says hard-float: floating-point arguments are passed in the hardware floating-point registers rather than in general-purpose registers or on the stack. A toolchain built for arm-linux-gnueabi, without the hf, uses the soft-float convention. Both produce ARM Linux executables. Neither can call into the other’s libraries, because the calling convention for a function that takes a double differs at the register level. Symbol resolution alone will not catch it: the symbol name is identical and the relocation resolves, so the mistake surfaces as a wrong value or a crash at run time. Toolchains that record the ABI in per-object attributes can catch the mismatch at link time, but only for the attributes they check.

The system field encodes both the kernel interface and, often, the C library. linux-gnu and linux-musl denote the same kernel interface with different libc implementations, which differ in the set of symbols they export, in their handling of NSS and dynamic loading, and in their startup paths. windows-gnu and windows-msvc denote the same kernel with two entirely different ABI families: different name mangling, incompatible C++ runtime ABIs, and no C++ standard library shared between them.

Longer forms exist. Some append a version to the system field, as in freebsd12, and some accept a further component that separates the environment from the ABI. They matter because a program built against a newer kernel’s system call interface may reference symbols that an older one does not provide, and because the C library’s own symbol versions are tied to the version it was built for. Toolchains that accept these fields use them to select the right set of compatibility symbols at link time.

What the triple actually selects inside the compiler

Inside the front end, the target description supplies a set of values that determine the layout of every type in the program: the size and alignment of the fundamental integer types, the size of a pointer, the endianness, the width of the long double format, and the calling convention used to pass and return aggregates. Predefined macros follow from those values. A program that computes sizeof(void*) at compile time and branches on it compiles to one branch or the other; the other branch is discarded, but the compiler still has to parse it, and a program that assumes 64-bit pointers in a way the parser cannot ignore will fail even in the dead branch.

The backend then selects instruction encodings, register allocation constraints, and the relocation types it emits. An object file for one architecture cannot be linked into an executable for another, so the assembler and linker must be built with matching target support. A single toolchain binary can support many targets; whether it does is a build-time decision, and a compiler that was configured for one target will reject another with a message about an unsupported target rather than producing wrong code.

Everything above is contained within the compiler and its companion programs. Everything below is not.

The sysroot and the three trees it stands for

A sysroot is a directory that presents the target’s filesystem from the root downward: headers under usr/include, libraries under usr/lib, and the startup objects that the linker needs to build an executable. Pointing the toolchain at a sysroot redirects every path lookup that would otherwise search the host.

Three separate searches go through it. The preprocessor’s include path is the first: angle-bracket includes must resolve against the sysroot’s headers rather than the host’s. The linker’s search path is the second: libraries named on the command line, and the C library itself, must come from the sysroot. The third is implicit and the one most often missed: the compiler’s own private include directory contains headers it generates for the target, such as those defining the standard integer types in terms of the target’s type sizes, and those must be the target’s versions rather than the host’s.

The startup objects are where the sysroot requirement becomes concrete. Before main runs, a program needs a stack established by the kernel, the C library initialised, and constructors for static objects called. On a GNU/Linux target that work is in files conventionally named for the entry point and the initialisation sequence, and the linker pulls them in automatically. They are target-specific binary objects, not source. A cross-link without them fails with an undefined reference to the entry symbol, and the failure appears at the link step after every compilation succeeded.

# A cross-link names the sysroot once; every search derives from it
aarch64-linux-gnu-gcc -c util.c -o util.o
aarch64-linux-gnu-gcc --sysroot=/opt/sysroots/aarch64-root \
    util.o -o util

The second invocation’s failure modes are informative. If the sysroot path is wrong, the linker reports a missing crt object by name. If the sysroot contains a libc built for a different ABI, the link may succeed and the program fail at startup with a dynamic loader error. If the sysroot’s headers came from a different libc version than its libraries, a compilation can succeed against a declaration that no longer matches the implementation, which is the worst case because nothing reports it until run time.

Building the sysroot versus borrowing it

There are two ways to obtain a sysroot, and they differ in what they guarantee.

Borrowing copies the relevant subtrees from a machine of the target architecture, or from a distribution’s development package for that architecture. It is fast and usually correct for a single distribution version, and it is what a container image of the target platform provides almost for free. Its weakness is that the copy is a snapshot of one particular installation, with the headers and libraries that installation happened to have, and reproducing the exact snapshot later requires reproducing the installation.

Building a sysroot means compiling a C library for the target and installing it into a staging directory, then adding the kernel headers for the target. This is what a distribution’s cross-toolchain package does, and what projects that need reproducibility do themselves. It takes far longer and requires the C library’s build to be configured for the target, but the result is a directory whose contents are determined by pinned source versions and build flags. The distinction matters for the same reason that pinning any other dependency matters: a sysroot built from unpinned sources is a hidden variable in the build.

A hybrid is common: use a borrowed sysroot for the C library and kernel headers, and add project-specific dependencies into the same tree so that every library the link needs comes from one place. The hybrid has a specific hazard, which is that the project’s own libraries must be built for the target too, and a library built for the host that happens to be found in the sysroot’s library directory will link without complaint and fail at run time. Versioned library filenames make this less likely but do not eliminate it, because a host build of the same version produces the same filename.

The parts the triple does not describe

Two classes of mismatch survive a correct triple and a correct sysroot.

The first is the kernel interface. A triple names the kernel family, not its version. If the target runs an older kernel than the one whose headers were used, a program may call a system call that the kernel does not implement, or use a structure whose layout changed. Some toolchains allow the minimum kernel version to be stated so that the C library’s compatibility wrappers are selected; without that, the effective minimum is whatever the C library was built for. Pinning the sysroot as a hashed input, rather than as a path, is what makes this class of drift visible; the reasoning is worked through in Hermetic Build Actions.

The second is the runtime environment around the program. Dynamic loading, thread-local storage, and the location of shared libraries depend on the loader and on configuration files in the target’s filesystem, none of which are part of the sysroot in a way that the linker checks. A binary that links cleanly against a sysroot can still fail to start on the target because the interpreter path recorded in its headers does not exist there. That path is a property of the link, not of the program, and it is one of the few things about the target that can be changed after the fact without rebuilding.

The practical conclusion is that a cross-toolchain is a compiler plus a description of a filesystem, and the description is the part that needs maintenance. A build that pins its compiler version and leaves its sysroot floating has pinned the smaller half of the problem. Making the sysroot an explicit, declared input is the same discipline that Hermetic Build Actions applies to every other file the action reads. The next layer, how a build system keeps the compiler and sysroot consistent while producing identical artifacts on repeated runs, is where cross-compilation meets reproducibility.