Template Rendering Under Variable Input: Determinism, Escaping, and Secret Leakage

Most configuration files on a machine are generated, and the generator is a template: a file with placeholders that is filled from data. The template is the surface where typed configuration data becomes text, and every problem in that transformation is silent at the point of generation and loud at the point of use, when the consuming program parses the file and finds something other than what the operator intended.

Rendering is a pure function only if its inputs are

A template renders deterministically when the same template and the same data produce the same bytes. That sounds trivial and fails routinely, for four distinct reasons.

Iteration order. A template that loops over a hash renders keys in whatever order the language’s hash iteration provides. Some languages randomise it per process. Some preserve insertion order. Some sort. A template that emits a list of key=value lines from a hash therefore renders differently across runs on the same machine with the same data, which defeats the checksum that the file resource uses to decide whether to rewrite the file. The observable symptom is a file reported as changed on every apply, with no diff to show.

Values that are not in the data. Templates can read the clock, the hostname, the process ID, or the output of a command. Each of these makes the output a function of something outside the declared inputs, so the file changes whenever that something changes. Timestamps are the common case, and they are nearly always avoidable: the version or hash of the input is a stable substitute.

Ordering across templates. A file assembled from several fragments concatenated in a loop over a directory listing inherits the filesystem’s iteration order, which is stable in practice on most filesystems when the directory is unchanged and is not guaranteed by any of them.

Encoding and newline handling. A template rendered in text mode may translate line endings. A value containing a carriage return, a tab, or a non-ASCII character arrives in the output in whatever form the renderer’s encoding produces. Text-mode rendering on Windows and on POSIX systems disagree about the bytes, and a checksum comparison between a file rendered there and consumed here reports a difference that is real but not the difference anyone intended.

Escaping is per-format, not per-language

A template is almost always rendering into a structured format: INI, YAML, JSON, a shell script, a systemd unit, an nginx configuration. Each of these has its own quoting rules, and the template’s expression syntax does not know which one it is emitting into.

The failure is consistent in shape. A configuration value that contains a colon, a hash, a quote, or a newline is valid as a data value and invalid as a bare scalar in YAML. YAML 1.2 resolves a plain scalar containing : followed by a space as a mapping key, and a # preceded by whitespace starts a comment. A value that arrives from a password generator with a : in it therefore renders a YAML file whose structure differs from the one the template author wrote, and the whole file fails to parse or, worse, parses into a different shape.

# Rendering with string interpolation produces invalid YAML
database:
  password: {{ db_password }}      # renders: p@ss:word -> a second map key
  url: "postgres://{{ db_user }}:{{ db_password }}@db/app"

The correct approach is to have the renderer serialise the structure rather than concatenate text: build a data structure in the template language and emit it with a YAML or JSON emitter that owns the quoting rules. Where the template must emit text, the value goes through an explicit escape for the target format, and the escape function is chosen by the format, not by the value. JSON requires escaping for ", \, and control characters below 0x20, and permits everything else as UTF-8; RFC 8259 states this precisely, and it is the format with the smallest escape surface. YAML has more ways to be wrong: plain, single-quoted, double-quoted, and block scalars each quote differently, and only the double-quoted form uses backslash escapes.

Shell scripts are the worst case, because they have four quoting contexts (unquoted, single-quoted, double-quoted, and heredoc) and a value’s correct quoting depends on where it appears. A password with a $ renders correctly inside single quotes and expands as a variable inside double quotes. A value with ! inside double quotes in an interactive shell triggers history expansion; the same value in a non-interactive script does not. The safe shape for shell output is to emit values as single-quoted strings with embedded single quotes handled by the standard '\'' sequence, and to never interpolate a value into a position where it will be word-split.

Rendered files and the secret they contain

A rendered configuration file is a plaintext copy of every value it contains. For non-secret values this is unremarkable. For secrets it is the central problem of the whole rendering step, and it has three parts.

The first is the file mode. A rendered file inherits the mode the resource declares, and the resource’s default is very often the mode that the tool uses for ordinary configuration: world-readable. A password written to a world-readable file is disclosed to every local account, which on a shared host includes every service. The mode has to be declared explicitly for any file whose content derives from a secret, and the declaration has to be applied before the content, or there is a window in which the file exists with the wrong mode. Tools that create the file with the final mode and then write into it close the window; tools that create it with a default mode and then chmod leave it open.

The second is the intermediate file. Writes are usually staged: the tool writes a temporary file and renames it into place, because a partial write on a crash would leave a corrupt configuration. The temporary file is a second copy of the secret, in the same directory, with the umask-derived mode unless the tool sets it. It is removed by the rename, so the copy is short-lived, and it is visible to anything that can list the directory during the window. Writing secrets on a filesystem shared with untrusted processes is therefore a different risk than writing them on a single-tenant host.

The third is the diff. When a file resource rewrites a file that contains a secret, the tool’s change report may include the old and new content, and the change report goes to a log, a console, and often a central collector. Every render of a changed secret then becomes a disclosure in the log pipeline. Redaction is the tool’s responsibility, and the reliable form of it is to mark the value as secret at the data layer so that the renderer knows not to print it, because classifying files as sensitive after the fact requires knowing which files those are.

What a rendered file cannot express

The rendering step loses type information permanently. A boolean renders to true, True, or yes depending on the emitter, and the consuming program may accept one and reject the others. An integer that came from a string renders with no marker of its kind, so a value that is 0644 in the data becomes 644 in the file if any step in the pipeline treated it as a number, which changes a permissions declaration into a different mode. A floating-point value renders in the shortest form that round-trips in that language, which may be 1e+06 where the consumer expects 1000000.

None of these can be fixed in the template, because the template only sees text. The fix is at the boundary: the file format should be one whose consumer parses types unambiguously, or the rendering should go through a serialiser that knows the consumer’s format. Where the consumer is an INI parser with no type system, the template has to render exactly the string the consumer’s parser will map to the intended value, and that mapping is a property of the consumer, not of the configuration system.

The last limitation is that rendered output is a snapshot. A file rendered at apply time reflects the data at apply time; a program that reads the file later sees that snapshot and nothing else. Configuration that must change without a redeploy has to be read from a live source with its own change mechanism, which is what externalising configuration from the artifact is about. Rendering is the right tool for the configuration that belongs to the deployment, and the wrong tool for the configuration that belongs to the running process’s decisions.

The coupling runs the other way as well: a rendered file’s checksum is computed over bytes, so any nondeterminism in the render appears to the apply loop as a change. Reviewing a template for iterating over unordered collections is therefore not a style matter; it is what keeps a file resource’s diff meaningful.