Provider Versions, Schema Changes, and State Migration

A provider is a separately versioned program that translates configuration and state into API calls. It is not part of the configuration language, and it is not part of the tool’s release cycle. That separation is what makes the ecosystem possible (hundreds of providers evolving independently) and it is also the source of the most subtle class of failure in declarative provisioning: the configuration has not changed, the state has not changed, and the plan is different because the program reading them changed.

Three things can move under a configuration at once. The provider binary can be upgraded, changing how attributes are computed and compared. The remote API can change, adding fields or altering responses. The provider’s schema can be revised, renaming an attribute or changing its type. The first two are version-management problems; the third has a dedicated mechanism.

The lock file is the only record of what actually ran

Providers are resolved from a registry at initialization, constrained by version requirements written in the configuration. A constraint such as ~> 5.0 admits every patch and minor release in the fifth major line, which means two people initializing the same configuration a month apart can get different binaries and therefore different plans.

The lock file records the exact version and, more importantly, a cryptographic checksum of the package that was installed. It is generated at initialization and is meant to be committed. Its function is not to pin a preference but to make the resolved set reproducible: a later initialization either matches the recorded checksums or fails, and failure is the correct outcome when the recorded binary is no longer available or has been replaced.

Without the lock file, reproducibility depends on the registry’s immutability guarantees, which are a policy of the registry rather than a property of the system. With it, the checksum is verified locally, and a mismatch is detected before any provider code runs.

The lock file covers the providers a configuration directly requires and their transitive dependencies, but only for the platforms that were resolved. A lock file generated on one operating system contains checksums for that platform’s packages, and initializing on another platform requires adding that platform’s checksums. The result is a lock file that changes when it is used on a different architecture, which is a legitimate change and should be reviewed as one.

Schema version is per resource type, not per provider

Every managed resource type carries a schema version number in state. When a provider changes the shape of its stored data (renaming a field, splitting one attribute into two, changing a string to a list) it increments that number and supplies an upgrade function that transforms the old representation into the new one.

The upgrade runs during refresh, before the diff. The provider reads the raw stored object at its recorded version, applies each successive upgrade until the object reaches the current version, and then maps it into the new schema. If a version has no upgrade function, the tool fails rather than guessing, and the failure names the resource and the versions involved.

This mechanism has hard limits. It can transform data that is present. It cannot recover data that the old schema did not record, and it cannot know what the remote object looked like at the time. An upgrade that needs an API call is outside the contract: the stored object is transformed in isolation, so a field that must be populated from the API cannot be filled in during upgrade. Providers work around this by leaving the field null and letting refresh populate it afterwards, which means the plan immediately following an upgrade may contain an update that exists only to fill in the new field.

There is also an ordering constraint. State written by a newer provider cannot be read by an older one. The upgrade is one-way. Downgrading a provider after an apply has written the new schema version produces a state the old binary cannot parse, and the only recovery is to restore the state from before the upgrade or to upgrade again. This is why provider upgrades are best performed in a branch, with a plan reviewed before the apply, and why the state should be backed up beforehand.

Where upgrades break a clean plan

A provider upgrade that changes a default value produces a plan with updates across every resource of that type, even though nothing in the configuration changed. The new default is applied by the provider during the diff, and the recorded value no longer matches. Whether that is desirable depends on the default: a new timeout or a new tag is harmless, a changed retention period is not.

A provider upgrade that adds an attribute which the API always returns produces a similar wave, because refresh records the value and the configuration does not set it. If the attribute is computed, the plan is clean and the state grows. If it is optional with a default, the plan may propose setting it, and every resource gets an update.

A provider upgrade that changes the comparison rule for an attribute (normalizing a CIDR block, trimming whitespace, treating an empty list and an absent field as equal) changes which differences appear. It can make a previously noisy plan clean, or it can surface a difference that was always there and was being suppressed. The second case is the one that produces an unexpected apply, and it is indistinguishable in the plan from a deliberate change.

None of these are defects. They are the cost of the provider being a program with its own release cycle, and the mitigation is procedural: pin the version, upgrade deliberately, and read the plan that the upgrade produces before applying it. The upgrade itself should be a separate change from any configuration edit, so that a plan full of unexpected updates can be attributed to one cause.

The API can move without the provider

The provider maps the API’s response into its schema. If the API adds a field, the provider ignores it until a new provider version exposes it. If the API changes a field’s format, the provider’s mapping may break in a way that surfaces as a diff on every plan or as a failure to parse.

Some changes are worse because they are silent. An API that begins returning a normalized value for a field the configuration sets in a different form produces a permanent diff. An API that begins returning an empty list where it previously returned null changes the recorded value without changing anything meaningful. An API that begins requiring a new parameter makes every create fail with an error the provider does not translate.

The provider’s release notes are the only reliable signal, and they are not part of the configuration. This is an unavoidable asymmetry: the configuration is versioned and reviewed, the provider is pinned, and the API is neither.

Managing the upgrade as a change

A workable sequence treats the provider as a dependency with a migration cost.

Pin exact versions in the lock file, and require the constraint in the configuration to admit only the versions that have been tested. Initialize with the lock file present and fail if it cannot be satisfied.

Upgrade in a branch that changes only the version constraint and the lock file. Run a plan against a copy of state, not against the live state, when the provider is used by a large configuration; a refresh against a copy reveals the wave of updates without touching anything. Review the plan for three categories: updates caused by new defaults, updates caused by newly exposed attributes, and any replacements.

Perform the upgrade apply as its own change, before any configuration change that depends on the new provider’s behaviour. This keeps the two causes separable in the history.

Back up state before the first apply under a new schema version. The one-way nature of the upgrade makes the backup the only rollback path that does not involve reconstructing objects.

The lock file is also where an inconsistency becomes visible. A configuration that requires one major version while the lock file records another is a configuration that has been initialized in two places with different constraints, and the checksum verification is what stops it from silently resolving to whichever version happens to be available.

What cannot be versioned away

Some provider behaviour is not schema and not versioned: the set of attributes that force replacement, the ordering of API calls inside a create, the retry policy for a throttled request. These are implementation details that a minor release can change, and the plan will reflect the change only in its consequences.

The practical consequence is that a clean plan under one provider version is evidence about that version. Reproducibility requires pinning the binary, not just the configuration, and the lock file is the artifact that makes the pinning real. That is a narrower guarantee than it appears (it fixes the program, not the API it talks to) but it is the strongest one available, and it is the reason the file belongs in version control next to the configuration rather than in a cache directory.