Cold Start Anatomy: What Happens Between a Request and the First Instruction

A function invocation that finds an already-initialized execution environment costs a network round trip and a dispatch. One that does not must build the environment first, and the difference between the two cases is one of the few places in cloud computing where the latency penalty is measured in hundreds of milliseconds rather than microseconds. The delay is not a single cost but a sequence of them, and understanding the sequence is what makes it possible to predict which workloads will be affected and which mitigations will help.

The stages in order

An invocation begins at a front end that authenticates the request, resolves which function and version it targets, and selects a placement. The front end does not run code; it produces a decision. That decision has to reach a worker that has, or can obtain, capacity for the function.

If no warm environment exists, the worker must create one. Creation proceeds in stages that differ in cost by orders of magnitude. The first is allocation of a slot on a worker that has the function’s artifact cached or can fetch it. The second is materializing the isolation boundary: on a microVM-based platform this means starting a virtual machine with its own kernel, and on a container-based platform it means creating namespaces and cgroups around a shared kernel. The third is starting the language runtime, which for an interpreted or JIT-compiled language means loading a large runtime image and warming its internal data structures. The fourth is executing the function’s initialization code, which the platform runs once per environment and which may itself open connections, load configuration, or precompute tables. Only then does the handler run.

Each stage has a different cost profile and a different set of mitigations. A microVM boot is dominated by kernel initialization and device setup and is measured in tens to low hundreds of milliseconds. Runtime initialization for a large managed runtime is comparable, and for some runtimes exceeds the boot cost. Handler initialization is under the customer’s control and ranges from nothing to several seconds. Network dispatch and queueing sit on top of all of it.

Why the isolation boundary costs what it does

The isolation boundary is the part of the stack that cannot be skipped, because multi-tenant execution requires it. Its cost depends on what is being isolated.

A container shares the host kernel. Creating one is a set of system calls that build namespaces for process IDs, mounts, networking, and users, plus cgroup filesystem writes to establish resource limits. There is no kernel boot, so the cost is small — typically a few milliseconds to low tens of milliseconds, dominated by filesystem setup and by the cost of exec’ing the runtime. The weakness is that the kernel attack surface is shared: a kernel vulnerability is a cross-tenant vulnerability, and the platform must therefore be conservative about which kernel features and which system calls it allows.

A microVM gives each environment its own kernel. The cost is a boot, but the boot can be made much cheaper than a general-purpose virtual machine’s by removing everything the workload does not need. A minimal device model exposes only a virtio block device for the function’s code, a virtio network device, and a serial console; there is no BIOS, no PCIe enumeration beyond the minimum, and no support for arbitrary hardware. Kernel images are built with a reduced configuration and boot directly into the init process. The saving over a general-purpose virtual machine is substantial because the general-purpose boot path spends much of its time discovering and initializing hardware that the function will never touch.

The engineering that makes this work is mostly about memory. A microVM’s memory is allocated as a file-backed mapping that the host can populate lazily and share across environments running the same function version. The guest kernel and the runtime’s read-only pages can be mapped from the same physical pages in many environments, with copy-on-write handling the pages that differ. Snapshotting a booted microVM — pausing it after initialization and restoring it later — turns the boot cost into a memory restore plus a device-state rehydration, which is faster still because the guest never re-executes its boot path.

invocation arrives
  -> front end: auth, routing, version resolution        (network RTT scale)
  -> placement: choose worker with capacity or cache     (sub-millisecond to ms)
  -> [cold only] slot allocation and artifact mapping    (ms)
  -> [cold only] isolation boundary creation             (ms to ~100 ms)
  -> [cold only] runtime initialization                  (~10 ms to >100 ms)
  -> [cold only] init code, once per environment         (customer-controlled)
  -> handler execution                                   (workload)

The bracketed stages are the ones a warm pool eliminates. A platform keeps a small number of pre-created environments per function, or per runtime and memory configuration, so that the common case is a restore rather than a creation. The pool is a bet on which functions will be invoked next, and its size is a trade between idle memory cost and the probability of a cold start. The same memory arithmetic that governs instance sizing applies here at a smaller granularity: a pool of idle environments is reserved capacity that no tenant is currently using.

The concurrency interaction

Cold starts do not scale linearly with request rate in the way that queueing delays do. They interact with the platform’s concurrency model, and the interaction produces the characteristic latency curve of serverless systems.

When a burst arrives and the platform has no warm environments, it must create many at once. Creation consumes worker CPU, memory, and I/O bandwidth, and those are the same resources the already-running environments need. The result is that a burst’s cold starts slow each other down: the first environments boot quickly, later ones wait for the worker to have capacity to boot them. Meanwhile the requests that are waiting are also waiting in a queue, and the queue grows.

The platform’s response is to limit how fast it will scale a function, and that limit is exposed as a configuration knob. A function configured to add a large number of concurrent executions per unit time will recover from a burst faster but will place more load on the workers doing the booting. A function configured conservatively will protect the workers but will queue for longer. Neither setting changes the per-environment cold-start cost; they change how many cold starts happen simultaneously.

This is where the concurrency model of a function platform becomes a latency parameter rather than a throughput parameter. A single environment handling one request at a time multiplies the number of environments needed by the request rate, and each of those environments is a potential cold start. An environment that handles several requests concurrently reduces the number of environments and therefore the number of cold starts, at the cost of sharing CPU and memory among those requests and losing the isolation that made the single-request model attractive.

What the customer can control

Three levers are under the customer’s control, and their effectiveness differs.

The first is initialization placement. Code that runs at module load happens once per environment and is not billed as handler time on most platforms, but it delays the first request that environment serves. Moving expensive setup out of the handler and into module scope amortizes it across all requests the environment serves, which is a win when environments serve many requests and a loss when they serve one. The platform’s own lifecycle hooks formalize this: an initialization callback runs before the handler, and the platform may run it speculatively before any request arrives.

The second is artifact size and runtime choice. A smaller deployment package reduces the time to fetch and map it, and a runtime with a smaller initialization footprint reduces the runtime stage. The effect is bounded: on a platform where the isolation boundary dominates, halving the package size does not halve the cold start.

The third is provisioned concurrency, which is the customer paying for pre-created environments. It converts a variable latency into a fixed cost, and the cost is the price of an idle environment times the number reserved. The decision is an economic one: reserve enough to cover the burst that matters, accept cold starts beyond that.

The asymmetry that shapes architectures

The reason cold starts drive architecture more than they drive tuning is that they are asymmetric across request types. A cold start on a user-facing request is visible as a slow page load. A cold start on a background job is invisible. A cold start on a request that is already near a timeout is a failure, not a slowdown, because the initialization time is charged against the same deadline.

That asymmetry produces the standard division of labour: latency-critical paths get provisioned concurrency or are kept warm by traffic, while batch and event-driven work absorbs cold starts because its deadlines are generous. It also produces the counterintuitive result that a function invoked steadily is cheaper and faster than one invoked rarely, since steady traffic keeps environments warm and rare traffic guarantees that every invocation pays the full sequence.

The cost of the isolation boundary is also what determines whether a workload belongs on a function platform at all. A workload whose per-request work is a few milliseconds and whose cold start is a hundred milliseconds spends most of its life paying for the boundary. A workload whose per-request work is seconds amortizes the boundary to a rounding error. The break-even is not a fixed number, but it moves with the ratio of initialization to execution, and that ratio is the quantity to estimate before choosing the model.