Serverless Execution and the Per-Event Invocation Model

A serverless function is a deployment unit that exists only while handling one event. The platform creates the execution environment, invokes the handler with a payload, and may reuse that environment for a later event or discard it immediately. Nothing in the programming model offers a place to put state between invocations that the platform promises to preserve, and nothing offers a process that outlives the work.

That single property generates the style’s whole character. Statelessness is not a discipline the team adopts; it is the only thing the runtime supports. Concurrency is not a thread pool whose size is chosen; it is the number of environments the platform creates, bounded by a quota. Failure handling is not a try block around a call; it is the event source’s retry policy acting on an entire invocation. Designing in this style means designing around the invocation boundary, and most of what goes wrong comes from treating it as an ordinary request handler.

The invocation boundary sets the lifecycle

A function’s environment goes through three phases. It is created and initialised, which may involve starting a runtime, loading the code, and running any module-level initialisation. It handles one or more events, with the platform free to reuse the environment for subsequent events of the same function. It is destroyed, at a moment the function does not control and without a guaranteed shutdown hook.

Initialisation is paid once per environment rather than once per event, which creates the incentive to move connection establishment and client construction outside the handler. The incentive has a limit: an environment that is created and then never reused paid the initialisation cost for nothing, and a client that holds a connection open competes with the platform’s own scaling. A relational database is the recurring casualty. Each environment opening its own connection means the connection count tracks the number of concurrent environments rather than the number of application servers, and a pool sized for the latter is exhausted long before the function’s concurrency quota is reached. The usual remedies are an external connection pooler, a database proxy that multiplexes, or an interface that does not require a persistent connection.

Cold start is the latency the first event in a new environment pays, and it is visible to the caller because the invocation is synchronous from the event source’s point of view. The cost depends on runtime, package size, and initialisation work, and it is commonly in the hundreds of milliseconds and can be seconds for a large package on a cold node. It matters most at low traffic, when environments are not warm, and during a burst, when many environments start at once. Provisioned concurrency, where the platform keeps a number of environments initialised, removes the latency and removes the elastic scaling benefit for that capacity — it is dedicated, billed capacity by another name.

The invocation also has a hard ceiling on duration. AWS Lambda caps a single invocation at 15 minutes, and other hosted runtimes impose comparable limits. Any task whose runtime is not bounded by the work size has to be split into steps with state stored between them, which pushes coordination into an external service.

Concurrency is environments, and throttling is backpressure

Each environment handles one event at a time in most function runtimes, so the concurrency of a function equals the number of environments. The platform scales that number up to a configured reserve, then throttles. Throttling is the style’s backpressure mechanism, and its behaviour differs by event source.

For a synchronous caller — an HTTP request through an API gateway — a throttled invocation returns an error to the client, and the client sees a failure that may be transient. There is no queue in front of the function unless one is added explicitly.

For an asynchronous invoker, the platform queues the event and retries the invocation. Events that fail repeatedly go to a configured dead-letter destination or are discarded according to the source’s policy. The queue in front of the function is the invoker’s, and its depth is the measure of how far the system is behind.

For a stream source — a partitioned log — concurrency is bounded by the number of partitions, because each partition is processed in order by one invocation at a time within a shard. Adding function concurrency does not increase throughput beyond the partition count, and a batch size that is too small wastes per-invocation overhead while a batch that is too large makes a single failed batch expensive to retry.

The asymmetry matters for capacity planning. The quantity to manage is the event rate per partition and the duration of the handler, not the request rate per instance, and the failure mode under overload is a growing queue with an unchanged error rate at the function itself.

Retries make idempotency mandatory

An asynchronous event source is at-least-once. A handler that completes but whose acknowledgement is lost will be invoked again with the same event, and a handler that fails partway leaves whatever it wrote in place. The function’s own execution is the only thing the platform can retry; it cannot undo partial writes.

Idempotency therefore has to be built from an external fact: a record of which event identifiers have been processed, with the processed marker written in the same transaction as the effect, or a conditional write that makes the second application a no-op. Both require the event to carry a stable identifier and both add a write to every invocation.

import json

def handler(event, context):
    failed = []
    for record in event["Records"]:
        try:
            apply_once(record["messageId"], json.loads(record["body"]))
        except TransientError:
            # report only the failed records; the source retries those and not the rest
            failed.append({"itemIdentifier": record["messageId"]})
        except PermanentError:
            quarantine(record)  # malformed or rejected: do not retry forever
    return {"batchItemFailures": failed}

Partial batch failure is the detail that makes queue-based handlers usable: returning the failed identifiers lets the source redeliver only those, provided the batch window and visibility timeout are configured so that the retry does not race the original. A handler that throws on the first bad record redelivers the whole batch, and every record before the failure is processed again.

Retries also interact with ordering. A record that fails and is retried after later records have succeeded is processed out of order, and a handler that assumes per-key order from a queue is wrong. Only the ordered stream sources provide per-key ordering, and only within a shard.

The failure that is specific to this style: recursion

A function triggered by a bucket write that writes to the same bucket triggers itself. Nothing in the platform detects the cycle; the invocation rate doubles per generation until the concurrency quota is exhausted or the bucket’s prefix rule stops matching. The same shape appears with a database trigger writing to the table it watches, or a queue consumer republishing to its own queue.

Recursive invocation is cheap to prevent and expensive to diagnose: a distinct output location or prefix, a marker that the handler checks before acting, and a limit on the trigger’s own filtering. Platforms that provide per-function concurrency limits let a runaway function be capped independently of the rest, which contains the damage while the loop is found.

A related failure is the retry storm after a downstream outage. Every queued event fails and is retried; the retries consume the concurrency that the recovering downstream needs in order to recover. Capping the function’s concurrency and setting a maximum retry count with a dead-letter destination converts an outage into a delay and a backlog rather than a sustained flood.

Composition without a process to hold the state

A workflow that spans several functions needs somewhere to keep the intermediate state and the progress. Two shapes are common. Choreography has each function publish the event that triggers the next, and the workflow exists only as the set of subscriptions; it is simple to extend and impossible to see in one place, and a partially completed workflow has no record of what remains. Orchestration places a coordinator outside the functions — a state machine service or a durable workflow engine — that calls each function and records each step.

The orchestration shape reintroduces a central component, which is the thing the style otherwise avoids, but it makes the failure state visible and the retry policy explicit per step. Choreography’s failure state is a set of events that were published and never consumed, discoverable only by inspecting each queue.

What the style does not provide is a boundary for shared code. A library imported by every function is deployed with every function, so a change to it is a change to every deployment, and version skew between functions is normal. Keeping the shared surface small — payload schemas, not business logic — is what keeps a set of functions from becoming a distributed monolith whose parts must be deployed together. The payload schemas are the same published-event contract discussed in event notification, and the ordering assumptions are those of the pipes-and-filters pipeline moved across a network boundary where each stage is billed per invocation.

The style fits spiky, event-shaped work: file processing, webhook handling, scheduled tasks, glue between managed services. It fits poorly where latency must be both low and predictable, where connections are long-lived, or where the work has no natural completion point. Those are the conditions under which the invocation boundary costs more than it saves.