Tag
watermarks
- Jan 13, 2026 Watermarks as Completeness Signals How a watermark asserts that no earlier event will arrive, how it is propagated through operators, and what happens when the assertion is wrong. data engineering
- Oct 2, 2025 Measuring Disorder to Size a Lateness Budget How to derive the watermark delay and allowed lateness from the arrival-delay distribution, and why a percentile is the only honest input to that choice. data engineering
- Jul 4, 2025 Reconciling a Load: Counting Rows Is the Weakest Possible Check A row-count comparison agrees on many wrong tables, so reconciliation has to compare keys, then values, and needs a position to compare at. data engineering
- Mar 19, 2025 Resumable Backfills: Watermarks, Chunk Boundaries, and Gap-Free Progress How a long-running backfill records where it stopped, why the watermark must advance over a total order rather than a clock, and what idempotent chunks cost. distributed systems
- Oct 15, 2024 Interval Joins Over Event Time Why a stream-stream join must buffer both sides, how the watermark of the slower input gates the match, and where the retained state actually goes. data engineering