Externalised Configuration and Immutable Artifacts: Where the Boundary Sits

A build artifact that differs per environment defeats the point of testing it. An artifact that carries no environment-specific configuration at all still has to learn where its database is. The resolution that most systems converge on is a strict separation: the artifact is built once and is a function of the source and its pinned dependencies; the environment supplies a small set of values at run time; and the process binds the two at startup. The interesting material is in where that boundary is drawn, because both directions of error are common and neither is obvious from a configuration file.

What must not be inside the artifact

The configuration that has to be outside is the set of values that legitimately differ between environments. The defining property is not secrecy, although secrets are a subset. A database endpoint differs between staging and production and is not a secret. A feature flag’s default differs and is not a secret. A log level differs and is neither.

Twelve-factor formulates the test as: could the codebase be made open source at any moment without compromising any credentials? That test identifies the subset that is also sensitive, and it is a useful lower bound. The stronger test for the artifact boundary is: if this value were baked in, would the artifact have to be rebuilt to change it? A value that must change without a rebuild belongs outside. A value that is a property of the release, such as a version string or a build timestamp, belongs inside, because it is part of the identity of the artifact and should be discoverable from it.

Values that are derived from the environment deserve a mention because they straddle the line. A connection pool size computed from the available memory, a worker count derived from the CPU limit, a cache size derived from the disk. These are not supplied; they are computed from the substrate at startup. Putting the computation in the artifact is correct and is where the substrate-layer divergence described in environment parity as a measurable claim enters: the code reads the environment and takes a different path, so parity at the configuration layer does not imply parity of behaviour.

Binding at startup versus reading late

Once the values live outside the artifact, the process must obtain them. Two shapes exist, and they differ in when the read happens and what a change costs.

Bind at startup. The process reads its entire configuration during initialisation, validates it, and either starts or exits. A change requires a restart, which the deployment machinery performs by replacing the process. The advantages are that the configuration is a single consistent snapshot for the lifetime of the process, that invalid configuration is caught before the process accepts traffic, and that no component needs to watch for changes. The disadvantage is that any config change is a deployment, with whatever coordination that implies.

Read continuously. The process or a sidecar watches the configuration source and reloads on change, either wholesale or for selected values. This buys change without restart and costs consistency: a value read at time T and another at T+1 may come from different generations of the configuration, so a component must either version the whole snapshot or accept torn reads. Reload also has to handle the difference between values that can be applied in place (a log level, a timeout) and values that require reinitialising a resource (a listen address, a TLS certificate, a connection string), and the second class is nearly always handled by the same restart that the first class was supposed to avoid.

The mixed shape is the common one and the most confusing: some values are read at startup, some are watched, and nothing in the configuration marks which is which. A change to a watched value takes effect immediately and a change to a startup value silently does not, and the operator who made both in one edit sees partial application with no error. The discipline that removes the ambiguity is to declare the reload behaviour per key, and to default to “requires restart” so that an unmarked key does not quietly become live-editable.

Secrets: injection paths and their residues

Secrets are configuration values whose handling has extra constraints: they must not appear in the artifact, must not be committed to the configuration repository, and must be readable only by the process that needs them. The paths differ in what they leave behind.

Environment variables. The process reads a variable from its environment. The value is in the process’s memory and in the environment block, which is readable by the same user via /proc/<pid>/environ on Linux and inherited by every child process the program spawns. A value that reaches a subprocess’ environment has been distributed to code that did not ask for it, and a crash dump may contain the environment.

Files mounted into the filesystem. A secrets manager or an orchestrator writes each secret to a file, typically under /run/secrets by the convention that several container runtimes follow, and the process reads it. The file’s mode determines who can read it, and the file’s lifetime is the mount’s lifetime. The residue is that the value is on disk inside the container or the host, and is readable by anything with permission for as long as the file exists.

Retrieval over an API at startup. The process authenticates to a secrets service and fetches the value, so the value exists only in memory. The cost is an additional dependency at startup, an authentication mechanism that must itself be bootstrapped without a secret, and a failure mode in which a restart during an outage of the secrets service leaves the process unable to start even though its configuration is otherwise complete.

None of these is universally correct. The selection criterion is the threat model: against accidental disclosure in logs and error messages, all three work if the value is marked sensitive; against a local attacker with the process’s UID, none of them work, because the value is readable by the process and therefore by anything running as the process. The differences that matter in practice are the residue (environment block, file on disk, memory only) and the bootstrap failure mode.

The configuration drift that the boundary creates

Externalising configuration moves the source of truth outside the artifact and outside the configuration repository, and that creates a state that no longer has a declared owner. The artifact’s identity is pinned by a digest; the configuration values live in whatever store the platform provides, which may be a parameter store, a config map, a set of environment variables in a deployment manifest, or a console.

If those values are edited in the store and not in the source that produces the store, the running system diverges from every record of it. The artifact’s digest does not change, so the deployment record says nothing changed. A restart then produces a process with different behaviour from the one it replaced, with no artifact change to explain it. This is the same divergence problem as within a host, moved to a layer the configuration tool may not manage.

The remedies are the same in kind. The store must be produced from a source under version control, so that the effective values are a function of a commit. The deployment must record the configuration version alongside the artifact digest, so that a process’s behaviour can be attributed to a pair rather than to an artifact alone. And the store must be compared against its source, so that an edit made directly to the store is detected, which is a drift check whose target is a configuration store instead of a host.

What the boundary does not fix

Three claims made for immutable artifacts do not hold as stated.

“Deploy the same artifact and the environments are equivalent.” They are equivalent at the artifact layer only. The substrate still differs, and the configuration store still differs by design.

“Immutable means no configuration management.” The artifact is immutable, and the runtime environment it is placed into is not. Something must place it, mount the secrets, set the limits, and configure the network, and that something is configuration management with a different resource model.

“Rolling back is redeploying the previous artifact.” Only if the previous configuration version is also restored. An artifact rolled back against a newer configuration can run with values it was never tested with, which is a combination that exists in no environment except the one just created.

The boundary is worth drawing carefully precisely because it is a boundary between two systems with different change rates: the artifact changes on release, the configuration changes on operational need. Each layer should have a source, a version, and a record, and the deployment should reference both. That record is also what makes an emergency change auditable, which is the concern of emergency overrides and break-glass changes.