Tenant Isolation in an Internal Platform: Which Boundary Does the Work

A tenant is a team that shares a platform with other teams and must not be able to harm them. “Harm” is the operative word, and it decomposes into four distinct failures: reading another tenant’s data, exhausting a shared resource, addressing another tenant’s services, and consuming another tenant’s identity. Each failure has a different mechanism that prevents it, and each mechanism has a different failure mode of its own. The common mistake is to build one boundary, call it isolation, and assume it covers all four.

Names are not a security boundary by themselves

A namespace in a container orchestrator provides name scoping: two tenants may both create a workload called api without collision, and a tenant’s service discovery resolves api to its own workload. This is useful and it is not isolation. The names are scoped; the objects behind them are not.

What a namespace does buy is a unit of attachment for everything else. Resource quotas attach to a namespace. Network policy selects pods by namespace label. Role bindings grant permissions scoped to a namespace. Service account token projection is per-namespace. The namespace is therefore the anchor, not the wall.

The wall, for data, is the identity attached to the workload. Two workloads in different namespaces that run with the same cluster-wide permission can read each other’s secrets and modify each other’s objects, because the authorization system resolves the identity and the identity is the same. Isolation therefore begins with the rule that no tenant workload runs with an identity that can act outside its own namespace, and that the platform’s own controllers are the only identities with cross-namespace reach. A role binding that grants a tenant’s service account permission on a cluster-scoped resource type, or on a namespace other than its own, defeats every other boundary at once.

Resource quotas and the difference between reserved and enforced

A resource quota limits the aggregate requests and limits a namespace may declare. Its semantics are worth being precise about, because they interact badly with scheduling in a way that produces the classic platform complaint: the quota says there is room and the scheduler says there is not.

A quota constrains declared requests, not consumed capacity. A namespace with a quota of 40 CPU may declare forty pods each requesting one CPU; the scheduler then places them wherever the cluster has capacity, which may be nowhere. The quota has done its job by preventing the tenant from declaring more than its share, and it has not guaranteed that the share exists. Guaranteeing the share requires reserved capacity: a node pool dedicated to the tenant, or a scheduling class that only admits the tenant’s workloads onto nodes the platform has set aside. The two mechanisms are complementary and neither substitutes for the other.

The second interaction is between quota and eviction. When a namespace exceeds its quota, the admission of new objects fails with a quota error, and running workloads are untouched. When a node runs out of a compressible resource such as memory, the eviction order is determined by the quality-of-service class derived from requests and limits, and workloads with no limits are evicted first. A tenant that declares requests but no limits is protected from quota exhaustion and exposed to eviction, which is usually the wrong way around for the tenant’s own expectations.

Limits, where they exist, are enforced by the cgroup hierarchy the container runtime creates. CPU limits are enforced by the scheduler’s bandwidth control: a cgroup that exceeds its quota is throttled until the next period, and the period is typically 100 ms. That number matters because a workload with a tight CPU limit and a bursty profile spends part of every period throttled even when the node is idle, and the latency it observes looks like contention on a machine with nothing else running.

Network policy and the default-deny question

Network policy in a container orchestrator is enforced by the CNI plugin, not by the orchestrator itself. The API objects are declarative and the enforcement is plugin-specific, which means the same policy manifest can be enforced strictly, loosely, or not at all depending on which plugin is installed. A platform that promises network isolation must therefore name the plugin in its contract, because the guarantee lives in the data plane.

Policy is additive in the allow direction. A pod is isolated for ingress once any policy selects it, and then only the traffic explicitly allowed by some policy selecting it is permitted. This is the mechanism behind default-deny: create a policy that selects every pod in the namespace and allows nothing, and the namespace’s pods accept no inbound connections until a more specific policy adds an allowance. The important detail is that egress and ingress are separate: a default-deny ingress policy leaves egress wide open, and a tenant can still reach the internet, the metadata endpoint, and every other tenant’s services.

Egress control is where most isolation gaps live. A workload that can reach another tenant’s cluster-internal service has effective read access to it, regardless of namespaces. Restricting egress to the platform’s own service endpoints plus the tenant’s declared dependencies turns the network into a second authorization system, and it must be maintained as one: a dependency added without a policy change fails at runtime with a connection timeout, which is a legible failure, and a dependency removed without a policy change leaves a stale allowance that nobody revisits.

The failure mode to plan for is the policy that selects nothing. Label selectors are exact matches on key and value, and a typo in a selector produces a policy that applies to zero pods while reporting as created and valid. Nothing in the API surfaces this. The only reliable defense is an assertion in the platform’s own test suite that a known-bad connection is refused between two instantiated tenants.

The metadata endpoint and other ambient authority

Every isolation argument has to account for authority that arrives without being requested. The instance metadata endpoint is the canonical example: a workload that can reach the node’s metadata address can obtain the node’s own identity, and the node’s identity is by construction more privileged than any tenant’s. A network policy that allows all egress for the sake of convenience reopens this path entirely, which is why the egress default matters more than the ingress default in most platform designs.

The same class of problem appears in mounted host paths, in the container runtime’s socket, and in any downward API field that exposes node-level information. None of these is a namespace boundary violation; each is a channel that bypasses the identity model. A platform that enumerates ambient authority channels and closes them by default is doing more for isolation than one that adds a fourth layer of namespace-scoped naming.

Identity, and why shared credentials undo the rest

The fourth failure class is identity confusion, and it is the one that survives all the boundaries above. If a tenant’s workload can obtain a credential that another tenant also holds, the platform has two namespaces and one principal.

The mechanism that prevents it is per-workload identity: a token whose audience is a specific service, issued to a specific workload, with a bounded lifetime, and validated by the receiving service rather than by the network path. A shared secret in a configuration map is the opposite: it is transferable, long-lived, and unauthenticated with respect to the caller. When the platform’s secret store issues credentials per workload, revocation becomes possible, which is the property that makes the isolation boundary meaningful after an incident rather than only before one.

Choosing the boundary deliberately

The practical rule is to state, for each tenant, which of the four failures the platform prevents and by what mechanism, and to test the statement. Namespaces provide scoping and the attachment point. Quotas bound declared demand; reserved capacity bounds actual supply. Network policy bounds reachability, subject to the plugin’s enforcement. Identity bounds authority and is the only boundary that remains effective once a workload is running.

The boundary that is easiest to get wrong is the one that combines a namespace with a cluster-wide role. It looks like isolation from the manifest, it reads like isolation in review, and it grants a tenant the ability to modify another tenant’s objects. The quota arithmetic that decides whether a tenant’s declared share fits the cluster’s real capacity is covered in capacity arithmetic for shared quota.