Deriving Least Privilege from Recorded Access Instead of from Intent

Least privilege is stated as a property of a permission set, but the permission set is not the thing anyone observes. What is observed is a sequence of calls, each recorded with a principal, an action, a resource, and an outcome. The temptation is to treat that sequence as the specification: whatever was called is what should be allowed. The gap between the sequence and the specification is the subject worth understanding, because it is large and it does not announce itself.

The trail records attempts, not needs

Every recorded event has an outcome. Some succeeded, some were denied, and some failed for reasons unrelated to authorization, such as a malformed request or a resource that did not exist. A candidate policy derived from the trail has to decide what to do with each class.

Succeeded calls are the easy case and the one naive tools use. They are the set of actions that were permitted and exercised. But a call can succeed under a policy that was broader than necessary, and the record cannot distinguish a call that needed the permission from a call that merely had it. A job that lists every bucket in the account and uses one of them records a successful call against a wildcard resource; the derived policy will name the wildcard unless the tool does something cleverer.

Denied calls are the more informative class, and the more dangerous to automate against. A denied call means a workload wanted a permission it did not have. It might be a workload doing something it should never do, or it might be a workload whose policy was misconfigured. Adding every denied action to the candidate policy is a mechanism for turning an attacker’s probing into a permanent grant, and it is exactly what a naive “fix the errors in the log” loop does.

Failed calls with an authorization-related error are ambiguous, and the error text is the only thing that distinguishes them. A failure due to a missing permission looks similar to a failure due to a resource that does not exist unless the message is parsed, and the parsing depends on the service’s error format. Treating every failure as a missing permission overcounts; treating none as one undercounts.

The observation window is a sample, not the population

A policy derived from thirty days of history covers the calls made in thirty days. It does not cover the calls made during a quarterly close, a failover drill, an incident response, or a deployment of a feature that has not shipped yet. Each of those is a legitimate use of the workload’s permissions that the window missed.

The failure this produces is delayed and specific. The candidate policy is applied, nothing breaks for weeks, and then the annual process runs and fails at the first step that needed a permission nobody had exercised since the policy was written. The failure arrives at the worst possible time, which is the time the process only runs once a year.

The mitigation is to widen the window past every periodic process the workload participates in, and to include the deployment path in the sample. A deployment exercises permissions that the running workload never does: creating resources, attaching policies, writing to artifact stores, invalidating caches. If the deployment principal is derived from the runtime history, the derived policy will be missing all of it.

There is a second dimension to the sampling problem. The trail records the principal that made the call, and the principal is often a role session with a generated name. Grouping events by session name gives a per-instance history that is too small to be a sample; grouping by role gives a history that mixes every workload that assumes the role. The right unit is usually the role plus the session-name prefix, because that is the granularity at which the workload is deployed and at which the policy is attached.

From events to statements

The transformation is a series of reductions, and each one loses information that must be recovered elsewhere.

The first reduction is from events to action names. The action is recorded as a service and an operation, and the policy language spells it as service:Operation. The mapping is mechanical, and it is where the case-insensitivity of action matching matters: the recorded spelling may differ from the documented one, and the policy will match either.

The second reduction is from events to resource identifiers. Each event carries the ARN of the resource, or enough information to construct it. Some events carry no resource at all, because the action is not resource-scoped, and those actions take a wildcard in the resource field by necessity. An action that genuinely operates on everything is not an over-grant when written with *; the over-grant is writing * for an action that is resource-scoped.

The third reduction is the interesting one: collapsing a set of concrete ARNs into a set of patterns. A hundred bucket names in one account with a common prefix reduce to a prefix pattern plus a wildcard, and that pattern matches the hundred and also matches the hundred-and-first. The choice between enumerating and generalizing is a choice about what happens when a new resource appears. Enumerating fails closed and requires a policy edit per resource. Generalizing fails open and requires a policy review per new resource class. Neither is wrong, and the decision belongs to the person who owns the resource, not to the tool that produced the candidate.

The fourth reduction is from many statements to few. Policies have character limits, and a candidate with a thousand statements will not fit. Merging statements with the same action set and overlapping resource sets reduces the count, and the merge is where an over-broad resource pattern can quietly absorb a narrow one. A merge that produces a pattern matching resources outside the observed set has widened the grant, and the widening is invisible unless the tool reports it.

What the trail cannot see

The trail records control-plane and, optionally, data-plane events. Whether object-level reads are recorded at all depends on the trail configuration and on the service, and many trails record only management events by default. A candidate policy for a data-plane-heavy workload derived from a management-only trail will be missing every read the workload performs, which is usually most of them.

The second blind spot is the resource side. A policy that grants access through a resource-based policy does not produce an event attributed to the caller’s identity policies, so the trail shows the call succeeding and says nothing about which document allowed it. Deriving identity policies from that history produces grants that duplicate the resource policy, and removing the resource policy later leaves the identity policy in place as an accidental grant.

The third blind spot is the conditions. An event records the request parameters and the source address, but not which condition clause matched. A policy derived from history cannot infer that a call succeeded because the principal’s tag matched the resource’s tag; it can only see that the call succeeded. Writing the condition back in requires knowing the intent, and intent is not in the log.

The fourth is the denial of service to oneself: a call that was never attempted because the workload’s own code avoids it. A client library that checks a local capability flag before making a call leaves no event, and the permission it would have needed never appears. This is rare and it is the one that produces the most confusing gaps, because there is no error anywhere to explain the missing action.

Applying a candidate without breaking the workload

The safe sequence is to apply the candidate as a second document alongside the existing one, not as a replacement. The intersection of the two is the existing permission set, so nothing breaks. Then the candidate is monitored: calls that the existing policy allows and the candidate does not are logged as would-be denials, and each one is either a gap in the candidate or a permission that should be removed. The list is the real output of the exercise, and it is a list of decisions rather than a policy.

This is the same shadow-mode discipline used for any restriction that can break a running system, and it depends on being able to tell, for a single call, which document allowed it. That question is harder than it sounds, because the evaluation order combines several documents and an allow from any of them produces the same success. The composition rules are in the evaluation order, and the reason a boundary can be tested the same way is in permission boundaries and service control policies: a ceiling only removes, so applying one in shadow mode is safe by construction.

The final step is to remove the original document. Until that happens, the candidate has no effect on the effective permission set, because the union of the two is the original. A least-privilege programme that stops at the shadow phase has produced documentation and no reduction, and the documentation will drift out of date while the permissive policy remains in force.

The residual and why it is acceptable

Even a carefully derived policy is broader than the workload’s true need, because it is derived from what the workload did rather than from what it must do. The residual is the set of permissions that were exercised once, by a person debugging, under a role that also serves the workload. Removing those requires a judgement the trail cannot supply.

The honest position is that least privilege derived from history is an upper bound on need and a lower bound on observed use, and the gap between them is closed by review rather than by tooling. That gap shrinks as the observation window lengthens and as the workloads are separated into roles narrow enough that one role’s history is one workload’s behaviour. Separating roles is the intervention that improves the derivation most, and it is also the intervention that most reduces the blast radius when a credential leaks. Both effects come from the same change, which is why the pattern of one role per workload, assumed through the mechanism described in role assumption and session tokens, is worth the operational cost of managing more roles.