Observability · Cluster
Logging
Structure, levels, and retention.
- 01 Record Framing: Why a Log Line Needs a Delimiter Contract How a log stream is cut back into records, why newline delimiting breaks on embedded newlines, and what octet counting and escaping cost in exchange.
- 02 Structured Fields Versus Formatted Message Strings What is lost when a log record is a printf template, how field-value encoding preserves types, and why templates still matter for cardinality.
- 03 Severity Levels as a Filtering Contract Why syslog orders severities from 7 downward, how numeric thresholds let a producer discard records before they are built, and what a level change does to a stream.
- 04 Retention Policy as a Storage Budget How per-stream retention windows turn into bytes on disk, why compression and index size dominate the estimate, and where deletion actually costs money.
- 05 Inverted Index Construction and Query Planning for Log Search How tokenized log records become delta-encoded posting lists, how a time-partitioned index prunes work, and when a scan beats an index lookup.
- 06 Repeat Suppression and Template Fingerprints How collapsing identical messages into a count preserves signal, why a fingerprint must be computed over the template rather than the text, and where collapsing loses information.
- 07 Backpressure and Loss at the Collection Boundary Why a log shipper must choose between blocking the application and dropping records, and how buffer sizing, disk queues, and drop accounting change that choice.
- 08 Block Compression and Columnar Layout for Log Storage Why compressing a block of records beats compressing each record, how column-wise encoding exposes value locality, and what the block size trades away.
- 09 Counting Distinct Values Without Storing Every Record How log-frequency analysis uses register-based sketches, what the relative error formula implies for sizing, and when an exact set is the cheaper answer.
- 10 Durable Append: Group Commit, Torn Tails, and Recovery What fsync actually orders, why a log file needs a per-record checksum, and how batching many appends into one flush changes the latency of every record.