Engineering & Architecture Brief Published: 2026-11-14
We rejected session affinity between API clients and HSM partitions even though it is the faster design. The gateway now treats every REST or gRPC call as self-contained. It checks a session out of a pool for one HSM, runs a single-shot operation and returns the session, and no client ever owns a handle. This record explains what we gave up, why the failure behaviour outweighed it, and the one class of traffic that would have reversed the decision.
The option we turned down The affinity design is not a strawman, and a careful team could ship it. Each API client, or each tenant connection, is pinned to a long-lived PKCS#11 session on one HSM partition, already logged in, with any ephemeral session objects kept in place between calls. That gives native multipart operations, so a client can stream a large payload through C_EncryptUpdate or C_SignUpdate without the gateway ever buffering it, and it avoids per-request session setup, which is what vendor client libraries quietly assume anyway. Our estimate is that it saves roughly one HSM round trip on most short requests, which matters at signing volumes where the HSM link, not the crypto, is the bottleneck. It also maps cleanly onto key management interface standards such as PKCS#11 and KMIP, which model the conversation as stateful by design.
Session handles are the wrong unit of failure A PKCS#11 session handle belongs to one token on one device, and nothing in the standard lets you move it. With affinity, every client pinned to a partition inherits that partition's health, and an HSM restart turns into a burst of CKR_SESSION_HANDLE_INVALID errors in the middle of multi-step operations that the client then has to unwind on its own. The failure that settled the question was worse than a restart: a half-open partition that still answers C_GetSessionInfo heartbeats while C_Sign calls stall behind an internal lock, so health checks report green while pinned clients pile up waiting. In the chosen design every call carries its full context. The gateway computes digests itself for hash-then-sign mechanisms and gives each HSM call a deadline of 400 ms. When that deadline passes, the gateway marks the slot suspect, drains it from the pool and replays the request on a peer partition holding the same key material. A sick HSM therefore costs the caller one deadline of extra latency instead of a broken operation, and the pool shrinks instead of the client failing.
The retry rule The gateway replays a request on another HSM only when the operation creates no persistent object, so signing, verification, decryption and wrapping retry freely, while key generation and unwrap into a token object go through a single writer that carries an idempotency key and, after a timeout, checks by CKA_ID lookup whether the object exists instead of blindly running the call again.
What would have flipped it The deciding factor is mechanisms that cannot be pre-hashed. Pure EdDSA, which arrived with PKCS#11 v3.0, signs the whole message rather than a digest, and authenticated streaming encryption of large payloads under keys that never leave the HSM has the same property, because the gateway cannot shorten the work into one stateless call without holding the entire payload in its own memory, and that breaks the zero host exposure guarantee the platform is built on. If that traffic had been the dominant workload rather than a niche one, affinity with explicit restart semantics would have been the honest answer. The cost we accept instead is pool sizing: keeping logged-in sessions open on every partition consumes per-device session limits even when nobody is using them, and our estimate is that this caps usable concurrency at around two thirds of what affinity would allow on the same hardware. The open question for the freeze is whether to add a separate streaming endpoint with affinity and a written no-retry contract, so the exception stays visible instead of leaking into the default path.
Technical FAQ: Could the gateway not simply replay a multipart operation on a healthy HSM after a failure? Only by buffering the whole stream until the final call succeeds, which means plaintext or message bodies sit in gateway memory for the length of the operation. At that point the gateway has rebuilt the exposure the HSM was supposed to remove, so we require multipart callers to restart from the beginning rather than pretend the operation is resumable.