What an HSM Boundary Actually Guarantees

A hardware security module is defined less by what it computes than by what it refuses. A general-purpose machine with a key in memory can be made to reveal that key by anyone who can read memory; a module is built so that the key cannot leave, and so that the operations available on it are a fixed, auditable list. The value of the device is the constraint, and the engineering work is in designing a system whose needs fit inside that constraint.

The boundary is a set of operations, not a wall

The naive picture is a box that holds keys and will not release them. The useful picture is a box that exposes a small set of operations, each of which takes a key handle and plaintext or ciphertext, and returns a result. The key never crosses the boundary; the data does.

That shape has consequences. An operation that the module does not implement cannot be performed on a key inside it. If the module offers encryption and decryption but not, say, a particular key-derivation construction, then a key that must undergo that derivation cannot live there, or must be exported (which the module may forbid) and derived elsewhere (which defeats the purpose). The feature list of the device therefore constrains the architecture, not just the implementation.

The interface is standardized. PKCS #11, maintained by OASIS, defines the object model, the mechanism list, and the function set that applications use to talk to a token. Keys are objects with attributes: a class, a key type, a permitted usage set, and flags such as whether the key is extractable or sensitive. Sessions establish a logged-in context, and operations run against object handles. The same interface covers a discrete appliance, a plug-in card, and a software token, which is convenient for portability and dangerous for assumptions: a software token implementing PKCS #11 provides no hardware boundary at all, and an application cannot tell the difference from the API.

The attribute that matters most is the extractability flag. A key marked non-extractable cannot be read out in plaintext through the interface, regardless of the caller’s privileges. What the module will still do is use the key: an attacker who obtains the handle can encrypt or decrypt with it. That is the fundamental trade the boundary makes — it converts key disclosure into operation abuse — and it means authorization and audit at the operation level are as important as the non-extractability itself.

What the certification claims and what it does not

FIPS 140-3 validates a cryptographic module against a set of requirements covering the specification, the interfaces, the physical and logical security, the operational environment, key management, electromagnetic interference, self-tests, and design assurance. Validation is performed against a stated security level from 1 to 4, with the levels adding requirements in specific areas.

The level says something precise and something limited. Physical security requirements increase with level: level 1 assumes the module is a production-grade component with no specific physical resistance, level 2 adds tamper evidence such as seals or coatings, level 3 adds tamper resistance and identity-based authentication, and level 4 adds detection and response to environmental attacks. The higher levels therefore say more about what happens when someone has the device in hand.

The certificate says nothing about the surrounding system. A validated module installed in a host whose administrator can invoke any operation, with no logging, is a validated module in an unprotected system. Nor does validation imply that the protocol built on top is sound, that the key management policy is sensible, or that the device firmware has no defects — the validation covers the requirements, and those requirements are a floor.

The other axis is key attestation: a statement, signed by the module, describing a key it holds — its type, its attributes, its origin. Attestation lets a remote party verify that a key was generated inside a genuine module rather than imported, and that it is non-extractable. That claim is what makes a module useful in a distributed system, because it substitutes for physical inspection of every device. Its strength depends on the chain from the module’s identity certificate to a root the verifier trusts.

Moving operations across the boundary

The boundary forces a choice about where each operation runs. Three patterns recur.

The first keeps everything inside: the module performs both the key operation and the payload operation. That works when the payload is small, as with a signature over a digest, a wrap of a data key, or an HMAC over a short message. Throughput is bounded by the module, which is often the reason it is not used for bulk data.

The second uses the module only for key operations. The module holds a master key and unwraps a data key, and the host performs the bulk encryption with the unwrapped data key in its own memory. The data key crosses the boundary in plaintext, which is acceptable if its scope is one object and its lifetime is short. This is the standard shape of envelope encryption with a hardware root, and its security rests on the data key being cheap to replace and narrow in scope.

The third derives a key inside the module and uses it for a bounded operation. A key derivation mechanism takes a base key and a context and returns derived key material, which the module may allow to leave. The derived key’s exposure is then bounded by the derivation, not by the base key.

Choosing among them is a decision about what crosses the boundary and how often. Each crossing is an interface call with a fixed overhead — a session lookup, an attribute check, a command to the device — and for small operations that overhead can dominate the cryptographic work. A design that unwraps a data key per record pays that overhead per record; a design that caches the unwrapped key for a batch pays it once.

Rate, latency, and the queue

A module is a shared, serializing resource. Even devices that process commands in parallel have a finite number of cryptographic cores, and a host with hundreds of threads issuing operations will form a queue. The relevant number is operations per second at the module, not per core of the host, and it becomes the throughput ceiling for anything that requires a key operation.

That ceiling interacts badly with per-request designs. A service that unwraps a data key for every read, with a module that handles a bounded number of unwraps per second, has a maximum read rate independent of the storage backend. The usual fixes are caching unwrapped data keys for a short time, batching operations where the interface allows it, and reserving the module for operations that genuinely require it.

Latency has a similar shape. A local module on a PCIe bus answers in microseconds; a network-attached module adds a round trip; a remote key service adds another. The differences are large enough to change architecture: an operation that costs microseconds can be performed per request, and one that costs milliseconds must be amortized.

Availability is the third constraint. A module is a single point of failure for every operation that requires it. High-availability configurations replicate keys across two devices, or back one device with another that holds the same keys wrapped under a shared root, and the replication itself is a key-management problem: the export and import must preserve non-extractability, which typically requires the keys to be wrapped under a transport key held by both devices.

Where the guarantee ends

The boundary protects keys from the host. It does not protect them from the module’s own firmware, from a supply-chain compromise of the device, or from an operator who can invoke operations. The last of these is the most common practical gap: a module that will sign anything for an authenticated administrator is a module whose keys can be used by that administrator, and the only defence is authorization policy and audit outside the device.

Side channels are a separate concern. A module’s physical protections raise the cost of attacks that measure power, timing, or electromagnetic emissions, but the API itself can leak through timing. An operation whose duration depends on the key, or an error path that differs between a padding failure and a MAC failure, gives an attacker an oracle regardless of where the key lives. Constant-time behaviour is a property of the implementation, not of the boundary.

Finally, the boundary does not survive a design that exports keys routinely. If the operational model requires exporting keys for backup, for migration between devices, or for use by a service that cannot reach the module, the exports become the weakest link, and the module’s non-extractability attribute is being circumvented by policy. Key backup and escrow have legitimate uses, and they belong in the design explicitly rather than as an escape hatch. The mechanisms for retiring keys that have left the boundary, and for proving they are gone, are covered in key destruction and cryptographic erasure.