Reading a Preprocessor Diagnostic Through Macro Expansion

A compiler diagnostic is a triple: a location, a severity, and a message. For code that came from a macro, the location is the hard part. The preprocessor expands macros before the parser sees anything, so by the time a syntax error is detected the tokens exist only in the expansion, and their original positions are metadata that the front end must carry along. Front ends do carry it, and the diagnostic machinery uses it, but the presentation is frequently misread because the file and line printed at the top of the message are not always the file and line the reader expects.

Where the location in the message comes from

Every token in the front end carries a source location. The location is not a single line number but a small structure: a pointer into a line table, plus a macro expansion context when the token did not come directly from a file. The line table records, for each file the front end has read, its name, its size, and the offsets at which each line begins.

When the preprocessor expands a macro, the resulting tokens inherit a location that names the macro invocation site, with a chain back to the macro definition. The chain is what allows the front end to print two locations for one diagnostic: the place the offending token appears after expansion, and the place the macro that produced it was defined. This is why a diagnostic for a one-line macro invocation can print a note pointing at a completely different file, sometimes a header several levels deep.

The distinction that trips people up is between the diagnostic’s primary location and its note locations. The primary location is chosen by the diagnostic’s own logic, and different diagnostics choose differently. A type error in an expression produced by a macro usually reports the invocation site, because that is the code the programmer wrote. A warning about an unused parameter reports the parameter’s declaration. A diagnostic about a redefinition reports the second definition and notes the first. Reading the message as “the error is on this line” is wrong in the cases where the primary location is a deliberate choice rather than a consequence.

The expansion stack and how to read it

Front ends typically print an expansion trail as a sequence of notes, each one naming a macro and a location. Read from the outermost to the innermost, the trail says: this token came from an expansion of macro A at line N, which itself came from an expansion of macro B at line M, and so on, until the token that was literally written in a file.

A worked shape, with invented names to keep the structure visible:

#define WRAP(x) (x)
#define CHECK(cond) WRAP(cond)
int f(void) { return CHECK(a + b); }

If a is undeclared, the diagnostic’s primary location is likely the a in the last line, and the trail contains notes for the WRAP expansion and the CHECK expansion. The trail is not decoration: without it, a reader looking at the macro definition sees x, which is not the token the compiler complained about. With it, the reader can reconstruct exactly which argument text was substituted.

Two properties of the trail are worth knowing because they affect how it reads. First, the trail is generated from the macro expansion context stored on the token, not from re-running the preprocessor, so it reflects the expansion path that actually produced the token. Second, macros that expand to nothing, or that expand to tokens which are then discarded by a conditional, leave no trail for tokens that no longer exist. A missing note is not evidence that a macro was not involved.

Why the line number can point into a generated file

Build systems generate source. A parser generator emits a .c file with a line directive at the top saying which grammar file and which line the following code corresponds to. The preprocessor honours those directives, and every diagnostic inside the generated code reports the grammar file, not the generated file. Which generator ran, and with what inputs, is a build-graph edge that no compiler can observe, which is why dependency discovery stops at headers.

/* emitted by the parser generator, not written by hand */
#line 42 "parser.y"
static int yyparse(void) { /* ... */ }

This is a benefit until it is not. A diagnostic that points at line 42 of a grammar file is actionable when the generated code corresponds one-to-one with the grammar rule, and misleading when the generator’s output has been post-processed or when the line directive is stale. When a diagnostic points into a file that does not exist in the tree, the line directive is the first thing to check: either the generator was not re-run, or the path it recorded is relative to a directory that is no longer the working directory.

The same mechanism explains diagnostics that point at a header’s line number but a source file’s name, or vice versa. The line table is a list of files, each with its own base offset, and a location is an offset that resolves through the table. When a file is included twice, the front end may enter it once and reuse the entry, so two textual inclusions map to one table entry. Offsets still resolve correctly because they are computed per inclusion, but a reader comparing a line number across two inclusions can be confused by a file that appears to be reported at a line beyond its length.

Suppressing and promoting diagnostics around macros

Because macros produce most of the noise, front ends provide a way to suppress warnings inside macro expansions. The mechanism is a diagnostic pragma that names the warning and the region, applied around the macro definition or around the invocation. It is the correct tool for a macro whose expansion legitimately produces a warning that cannot be avoided in the definition itself.

The tool is also easy to misuse. A suppression applied to a whole header silences the warning for every user of the header, including the cases where the warning was real. A suppression applied to a single macro definition is narrower and is the usual recommendation. The distinction is visible in the diagnostic machinery: suppression state is pushed and popped with the source region, so a pragma inside a macro definition affects only expansions of that macro, while a pragma at file scope affects everything after it.

The opposite direction is treating warnings as errors. A build that enables that mode has to decide whether a warning from a system header counts. Front ends generally treat system headers differently, suppressing most warnings from them by default, which is why a build that turns warnings into errors usually still compiles against a standard library. The classification is by search path, not by content, so a header moved from a system directory into the project becomes subject to the full warning set and can break a build that previously passed.

From diagnostic to source edit

The useful output of all this machinery is a minimal reproduction. Three steps produce one reliably.

Compile only the preprocessed output, with the preprocessing stage’s output captured to a file and then compiled directly. The diagnostic then cites the expanded text, which contains no macros, and the error is at the cited line by construction. This confirms the mechanism even though the expanded file is unreadable.

Then reduce the macro trail: take the innermost macro in the trail, substitute its arguments textually, and check whether the diagnostic persists. If it does, the outer macros are irrelevant and the inner one is the culprit. If it does not, an outer expansion is doing something the substitution missed, usually token pasting or stringification, which changes the tokens rather than merely moving them.

Then check the definition site for the constructs that behave differently from function calls. A macro that evaluates an argument twice turns a side effect into two side effects; a macro that does not parenthesise its expansion changes the parse of the surrounding expression; a macro whose parameter is used in a stringification context is not evaluated at all. These are not diagnostics problems, but they produce diagnostics that read as if the compiler misunderstood the code, and the trail is what shows that the compiler understood it exactly.

The last resort, when a diagnostic is genuinely opaque, is to ask the front end for its internal view of the source location: most support a flag that dumps the token stream or the location table. That output is verbose and intended for compiler developers, and it settles the question of which file and offset a location refers to when the human-readable message is ambiguous. When the confusion is about which program produced the message at all, the fix is one level up: the driver’s stage dispatch determines whether the front end, the assembler, or the linker was speaking.