Tag
columnar-layout
- Dec 11, 2025 Columnar Layout and the Cost of Reading One Field How storing each field in its own contiguous run changes the bytes a scan touches, why homogeneous columns compress better, and what the layout gives up on point reads and updates. data engineering
- Jun 24, 2025 Dictionary Encoding and the Cardinality Cliff How replacing repeated values with ordinals shrinks a column, why the win collapses as distinct values grow, and what happens to a column chunk when the dictionary outgrows its budget. data engineering
- Feb 10, 2025 Block Compression and Columnar Layout for Log Storage Why compressing a block of records beats compressing each record, how column-wise encoding exposes value locality, and what the block size trades away. observability
- Jul 16, 2023 Repetition and Definition Levels in Nested Columns How two small integers per value encode the full structure of nested and repeated fields, what an empty list looks like on disk, and why the encoding stays independent of the record shape. data engineering
- Mar 9, 2022 Null Representation: Validity Bitmaps Against Sentinel Values How a columnar format records the absence of a value, why a separate bitmap costs one bit per value, and when an in-band sentinel breaks the type system. data engineering