Event Time and Processing Time in a Stream Operator

A stream of events carries two independent notions of time. One is attached to the record: the instant the event happened, recorded by whatever produced it. The other is the operator’s own clock, read at the moment the record is processed. An aggregate that reads the first produces the same result no matter how fast or slow the data arrives. An aggregate that reads the second produces whatever the delivery schedule happens to imply. Everything else in this cluster follows from that distinction, because a stream processor must decide which clock every operator consults, and the choice is not reversible after the fact.

Three timestamps, not two

Production systems usually carry at least three time values, and conflating them is the source of most wrong window totals.

The event timestamp is a field inside the record, written by the producer. For a click it is when the click happened; for a sensor reading it is the sampling instant; for a database change record it is the commit time of the transaction that produced it. It is data, not metadata, and the pipeline has to be told where to find it, because nothing in the record format marks it as special. A Kafka topic full of JSON objects has no schema-level concept of “the time field” unless the consumer declares one.

The ingestion timestamp is assigned when the record enters the processing system, by the broker or the source connector. It is monotonic per partition only if the producer appends in order, and it says nothing about when the underlying event occurred.

The processing timestamp is read from the operator’s clock when the record reaches a particular operator. It advances whether or not any data arrives, it differs between two operators handling the same record, and it differs between two runs of the same job.

Only the first is stable across replays. That is the whole argument for event time.

What an arrival-clock aggregate actually computes

Consider a count of events per minute, where the minute is taken from the operator’s clock at the moment the record is handled. If the source stalls for thirty seconds and then delivers a burst, the entire burst lands in one or two minutes of output, and the counts for the minutes during the stall are zero or missing. Replay the same records through the same job on an idle machine and the boundaries fall elsewhere. The output is a function of the transport, not of the events.

This is not a subtle statistical distortion. It is a change in what the query means. A processing-time window answers “how many events did the system handle during each wall-clock minute”, which is a legitimate operational metric — throughput, load, latency — and is exactly what a monitoring dashboard wants. An event-time window answers “how many events happened during each minute of real time”, which is what a business metric wants. A pipeline that uses the wrong one produces a number that is internally consistent and answers a different question than the one asked.

The distinction becomes unavoidable during backfill. Reprocessing a month of history through a processing-time window assigns all of it to the minutes in which the reprocessing ran, so the historical output is meaningless. An event-time window reassigns each record to the minute it belongs to, and the reprocessed output matches the original output for the same input, which is the property that makes a pipeline testable at all.

Reading the timestamp correctly

Extraction looks trivial and is where unit and timezone errors enter. The common failure is a timestamp in seconds where the arithmetic expects milliseconds, which puts every event in January 1970, or a naive local-time string converted with the process’s default timezone, which shifts windows by the offset.

from datetime import datetime, timezone

def event_millis(record: dict) -> int:
    raw = record["ts"]                 # ISO 8601 with an explicit offset
    dt = datetime.fromisoformat(raw)
    if dt.tzinfo is None:
        raise ValueError("timestamp has no offset")   # refuse rather than guess
    return int(dt.timestamp() * 1000)

The explicit rejection matters more than the conversion. A record whose timestamp lacks an offset cannot be placed on an absolute timeline, and silently assuming UTC or local time produces a pipeline that is correct for one producer and wrong for another. Formatting rules for the offset form are specified in RFC 3339; the practical rule is that every producer must emit an offset, and the extractor must reject records that do not.

The second decision is resolution. Millisecond resolution is the common default because a signed 64-bit count of milliseconds covers hundreds of millions of years, and because sub-millisecond ordering is rarely meaningful for business events. Microsecond and nanosecond timestamps appear in tracing data and in some hardware captures; they force a choice about the width of the timestamp type that propagates through every operator’s state.

The price of event time: buffering and waiting

Using event time requires the operator to hold records until it can be sure no earlier record is still coming. That certainty is never available from the data alone — a record with an old timestamp may arrive at any moment, and nothing in the stream bounds how old. The processor therefore needs an external signal that says “nothing before time T will follow”, and that signal is a watermark. The window cannot close, and the aggregate cannot be emitted, until the watermark passes the window’s end.

The cost is latency proportional to how far the watermark trails the data. A watermark that trails by ten seconds gives at most ten seconds of additional output delay; a watermark that trails by an hour gives an hour. The delay is not a tuning artifact that a faster machine removes, because it is bounded by the disorder in the input, not by the processor’s speed.

The second cost is state. Records held pending a window must live somewhere: in memory for small windows, spilled to disk for large ones. A one-hour window over a high-rate stream retains everything the stream produced in an hour, which for a busy source is a large multiple of the per-record size. Keyed State Layout and the Cost of Keeping It covers where that memory goes and what it costs.

Where the arrival clock is still correct

Processing time is not a fallback for when event time is inconvenient. It is the right clock for questions about the system rather than about the events.

Timeouts are the clearest case. “Fire if no matching record has arrived in five seconds” is a statement about arrival, and an event-time version of it would fire based on the timestamps in the data, which may all be hours old. Session expiry driven by inactivity is the same shape: the gap that ends a session can be measured in event time, which groups a user’s activity by real idle periods, or in processing time, which groups it by gaps in delivery. Both are meaningful and they differ, so the choice has to be deliberate.

Latency measurement is another. The interval between a record’s event timestamp and the moment it reaches an operator is a processing-time quantity by construction, and it is the standard way to detect a lagging source.

A practical pipeline often uses both clocks in different places, and the rule that keeps it coherent is that no single output should mix them without saying so. A window that aggregates on event time but emits on a processing-time timer produces results whose contents are deterministic and whose emission instants are not; that is a legitimate design, but the emitted record should carry both the window boundaries and the emission time so a consumer can tell which is which.

The invariant worth preserving

The property that makes a stream pipeline debuggable is that its output is a function of the event-time content of the input. Reordering the delivery of the same records, replaying them at a different rate, or splitting them across a different number of partitions should not change the result, provided the watermark eventually advances past them and lateness is handled by a stated policy. An operator that reads the arrival clock gives up that property permanently, because no amount of care in the rest of the pipeline restores information that was never recorded.

Watermarks are the mechanism that makes the invariant affordable, and they are the subject of Watermarks as Completeness Signals.