Topic domain
DevOps
Delivery pipelines and the operational feedback loop.
- Articles
- 80
- Clusters
- 8
- Foundational
- 24
- Intermediate
- 32
- Advanced
- 24
Artifacts, caching, and reproducibility.
- Content-Addressed Artifact Identity and the Cost of Mutable Tags
- What a Build Cache Key Must Cover
- Sources of Nondeterminism in Build Output
- Layer Cache Invalidation in Container Image Builds
- Passing Build State Between Jobs Without a Shared Workspace
- Trust Boundaries in Shared Build Caches
Blue-green, canary, and progressive rollout.
- Blue-Green Cutover Mechanics at the Routing Layer
- Connection Draining and In-Flight Request Handling at Cutover
- Weighted Canary Routing at the Edge and What the Weight Measures
- Health Signals for Gating a Progressive Rollout
- Expand and Contract: Schema Changes That Survive an Overlap
- Session Affinity, Sticky Routing, and Version Overlap
Desired state and environment parity.
- Convergent State Application and Why a Run That Changes Nothing Still Costs Money
- Idempotency Keys and Command Resources That Cannot Be Described Declaratively
- The Dependency Graph Behind an Apply: Ordering Edges, Cycles, and Partial Failure
- Environment Parity as a Byte-Level Claim: What Divergence Between Stages Actually Costs
- How a Configuration Value Resolves: Precedence, Merge Modes, and Missing Keys
- Template Rendering Under Variable Input: Determinism, Escaping, and Secret Leakage
Versioning, changelogs, and branching.
- Semantic Versioning as a Compatibility Contract
- Deriving a Changelog from Commit Messages
- Lockfiles and How a Version Range Becomes a Tree
- Release Trains, Cut Points, and Hotfix Branches
- Pre-release Channels and Promotion in Package Registries
- Fixed and Independent Versioning in a Monorepo
Patching, backups, and disaster recovery.
- Conffile Conflicts During Package Upgrades
- Dependency Resolution and Partial Upgrades
- Reboot-Required and Live Kernel Patching
- Crash-Consistent and Application-Consistent Backups
- Incremental Backup Chains and Restore Depth
- Checksum Verification and Restore Drills
On-Call
10Escalation, runbooks, and fatigue.
- Escalation Policies as Timeout-Driven State Machines
- Alert Fingerprints, Grouping, and Inhibition
- Rotation Coverage Math and the Handoff Contract
- What a Page Must Carry to Be Actionable
- Anatomy of a Runbook Step
- Severity Classification as a Routing Input
Fault injection and game days.
- Steady-State Hypotheses and the Abort Condition in Fault Injection
- Bounding Blast Radius: How a Fault Injection Chooses Its Targets
- What a Kill Fault Looks Like to the Kernel: Signals, Freezes, and Orphans
- Packet-Level Fault Injection: Latency, Loss, and Partitioning
- Clock Skew Injection and the Failures That Depend on Wall Time
- Injecting Failures at the RPC Boundary: Codes, Timeouts, and Resets
Self-service and policy as code.
- Level-Triggered Reconciliation and the Desired-State Loop
- What an Admission Webhook Can and Cannot Decide
- Evaluation Semantics of a Policy Query Language
- Template Ownership When Scaffolding a New Service
- Saved Plans, State Serial, and Why Apply Must Be Pinned
- Delegating Provisioning Without Delegating the Account
All DevOps articles
- Oct 19, 2026 Progressive Delivery Controllers as Reconciliation Loops How a rollout becomes a cluster object with a status field, how the controller moves traffic through a provider abstraction, and where the loop conflicts with autoscalers and GitOps sync.
- Oct 18, 2026 Release Gates: Verifying the Cut Before It Ships What a release gate can mechanically check, why reproducibility is the only claim about a build that a second machine can settle, and how a gate fails without becoming a step people bypass.
- Sep 26, 2026 Fixed and Independent Versioning in a Monorepo How a repository containing many packages decides whether they share one version or several, what the coupling does to upgrades, and how change detection assigns a bump to each package.
- Sep 2, 2026 Structuring a Game Day: Scenario, Roles, and Safety Nets What separates a drill from an experiment, how a scenario document is written so that it constrains the fault and not the response, and the safety nets that must exist before the exercise starts.
- Aug 26, 2026 Resource Exhaustion Faults: Memory, File Descriptors, and Inodes How memory pressure differs from allocation failure, why descriptor and inode exhaustion produce distinct error codes, and how control groups let a fault be applied without touching the host.
- Aug 23, 2026 Template Rendering Under Variable Input: Determinism, Escaping, and Secret Leakage Why a configuration template can produce a different file from identical inputs, how quoting inside structured formats fails, and where rendered secrets end up on disk.
- Jul 29, 2026 Rotation Coverage Math and the Handoff Contract How 168 hours of weekly coverage divide across a team, what shift length costs, and what has to cross the boundary at every handoff.
- Jul 27, 2026 Shared Build Cache Design: Action Keys, Content Store, and Eviction Why a build cache is two stores with different lifetimes, how write-once content addressing enables deduplication, and which eviction and failure policies keep a cache outage harmless.
- Jul 19, 2026 Pre-release Channels and Promotion in Package Registries How candidate builds are published under pre-release identifiers, how dist-tags and channels make the same version resolve differently for different consumers, and why promotion should never rebuild.
- Jul 19, 2026 The Dependency Graph Behind an Apply: Ordering Edges, Cycles, and Partial Failure How a configuration tool derives execution order from resource references, why the graph is not the file, and what a failed node leaves behind for the next run.
- Jul 14, 2026 Crash-Consistent and Application-Consistent Backups What a point-in-time image really contains, why journal replay only repairs the filesystem, and how to quiesce a running service without stopping it.
- Jul 13, 2026 Anatomy of a Runbook Step Why a runbook step needs a precondition, a bounded action, and a verification that proves the system changed, and how steps fail when any of the three is missing.
- Jul 10, 2026 Sources of Nondeterminism in Build Output Timestamps, absolute paths, directory iteration order, and parallelism leak into object files and archives. How each is detected by repeated builds and removed at the source.
- Jun 21, 2026 Conffile Conflicts During Package Upgrades How dpkg and rpm decide whether a configuration file has been edited locally, why neither will merge, and how unattended upgrades end up keeping stale configuration.
- May 12, 2026 Injecting Failures at the RPC Boundary: Codes, Timeouts, and Resets Why the shape of an injected failure decides the caller behaviour it provokes, how deadlines and retries multiply load, and what circuit breaker states actually reveal under injection.
- May 9, 2026 What an Admission Webhook Can and Cannot Decide The position of admission control in the write path, the fields a webhook receives, and why cross-object invariants and external state cannot be enforced there.
- Mar 26, 2026 Rollback Boundaries: Which Layers a Reversal Can Restore Code, configuration, schema, data, messages, and external effects each reverse differently; how to place the irreversible step last and keep a rollback window that is real.
- Mar 21, 2026 Fencing and Quorum in Automated Failover Why a new primary must be able to prove the old one cannot write, how leases fail when a machine is paused rather than dead, and where quorum buys safety more cheaply.
- Mar 6, 2026 Log Shipping and the Recovery Point Objective How the commit acknowledgement ladder sets the true data-loss window, why a WAL archive lags by a whole segment, and how to measure the gap rather than assume it.
- Feb 27, 2026 Delegating Provisioning Without Delegating the Account How a self-service pipeline obtains cloud credentials through workload identity federation, what the trust policy must pin, and how the blast radius stays bounded.
- Feb 17, 2026 Checksum Verification and Restore Drills Distinguishing repository integrity from data verification and from a real restore, how much sampling is statistically worth doing, and what no checksum can detect.
- Jan 8, 2026 Release Trains, Cut Points, and Hotfix Branches How a release branch is cut from a moving mainline, what the cut point guarantees, and how a hotfix reaches both the released line and the mainline without duplicating the change.
- Jan 7, 2026 Follow-the-Sun Coverage and the Cost of Every Handoff The arithmetic of covering a full day with regional shifts, the overlap a real transfer needs, and why an incident that crosses three shifts loses more than time.
- Jan 2, 2026 Recovery Ordering for Multi-Service Estates Why a restore is a topological problem, how legitimate designs create dependency cycles, and why the credentials needed to restore usually live inside the thing being restored.
- Dec 29, 2025 Fault Injection as a Continuous Regression Test How an experiment becomes a repeatable test case with a pass predicate, why resilience tests fail for environmental reasons far more often than for real regressions, and what a green suite does not prove.
- Dec 5, 2025 Bounding Blast Radius: How a Fault Injection Chooses Its Targets The three axes that bound an injected fault, how deterministic partitioning selects a stable subset of targets, and why aggregates that span treated and untreated hosts hide the result.
- Nov 18, 2025 Drift Detection Without Reading Every Host: Sampling, Fingerprints, and Reconciliation How to find hosts whose state no longer matches their declaration, what a fingerprint over a state subtree can and cannot prove, and how detection frequency trades against read cost.
- Oct 9, 2025 Decision Rules for Aborting or Promoting a Rollout Why repeatedly checking a canary against a fixed significance threshold inflates false aborts, how sequential methods and error-budget burn rates avoid that, and what sample sizes the arithmetic demands.
- Sep 27, 2025 Feature Flags Versus Traffic Routing as Rollout Controls Why a flag and a routing weight expose different populations, what it costs to keep both code paths in one binary, and how evaluation consistency and stale defaults cause failures.
- Sep 25, 2025 Passing Build State Between Jobs Without a Shared Workspace Artifacts move through object storage, not a shared directory. How upload, transfer, and extraction fail, and how a digest manifest makes a consumer verify what it received.
- Sep 9, 2025 Semantic Versioning as a Compatibility Contract What each of the three version numbers is allowed to promise, how the ordering rules work, and why the compatible-change rule only holds for the parts of a surface a project actually constrains.
- Aug 29, 2025 Incident Roles and the Status Update Cadence Why the person with the most context should not be the one coordinating, how the commander, operations, and communications roles divide the work, and what a status update owes its readers.
- Aug 15, 2025 Layer Cache Invalidation in Container Image Builds Why an instruction invalidates every layer after it, how the builder decides an instruction changed, and the ordering tricks that keep the expensive steps cached.
- Jul 29, 2025 Lockfiles and How a Version Range Becomes a Tree Why a range in a manifest is not a pin, what a lockfile records that the manifest cannot, and how resolvers settle or fail on conflicting requirements.
- Jul 23, 2025 Weighted Canary Routing at the Edge and What the Weight Measures Whether a traffic weight applies to connections, requests, or users, how hashing turns percentages into fixed partitions, and why small canaries cannot resolve small error differences.
- Jul 22, 2025 How a Configuration Value Resolves: Precedence, Merge Modes, and Missing Keys The lookup rules that turn many layers of data into one value, why first-found and deep-merge hierarchies disagree, and why undefined and empty are different states.
- Jun 13, 2025 Testing Policy Before It Gates the Cluster Unit tests over policy rules, fixtures captured from the position where the policy runs, and the differential check that makes enforcement a measurable step.
- May 29, 2025 Convergent State Application and Why a Run That Changes Nothing Still Costs Money How a desired-state tool decides whether a resource already matches, why that decision is a diff and never a comparison of files, and what idempotence actually guarantees.
- May 26, 2025 Health Signals for Gating a Progressive Rollout Why readiness probes cannot decide whether a new version is good, which signals can, and how probe timing and baseline choice set the number of bad requests before an abort.
- Apr 27, 2025 Session Affinity, Sticky Routing, and Version Overlap How affinity keeps a client on one instance, why it biases a canary sample, and what has to be true of the session payload and its keys for a pinned session to survive promotion.
- Apr 4, 2025 Alert Fingerprints, Grouping, and Inhibition How an alerting system decides that two firing alerts are the same problem, how grouping batches notifications, and why unstable labels cause page storms.
- Feb 28, 2025 Break-Glass Changes: Overrides That Are Auditable and Expire How to let an operator change a host during an incident without permanently losing the declaration, why an override needs an owner and a deadline, and how the loop treats it.
- Feb 26, 2025 Evaluation Semantics of a Policy Query Language Why a policy language evaluates rules as queries over a document, how undefined differs from false, and where the cost of a rule actually goes.
- Feb 22, 2025 Artifact Provenance from Commit to Deployed Digest Signed attestations bind an artifact digest to the materials and builder that produced it. What the statement must contain, how it is verified, and which attacks it does not stop.
- Feb 21, 2025 Connection Draining and In-Flight Request Handling at Cutover How an instance retires without dropping requests: deregistration order, keep-alive reuse, TCP and TLS teardown, and the timers that decide whether clients see errors.
- Feb 7, 2025 Trust Boundaries in Shared Build Caches A cache entry written by an untrusted build is an input to a trusted one. How key collisions become code execution, why content addressing does not help, and how namespaces separate the two.
- Jan 16, 2025 What a Kill Fault Looks Like to the Kernel: Signals, Freezes, and Orphans The difference between SIGTERM, SIGKILL, and freezing a process, what each leaves behind in sockets and locks, and why reaping behaviour decides whether a container restarts cleanly.
- Jan 7, 2025 Separating Injected Faults from Organic Failures in Telemetry How to mark injected traffic so it survives every hop, why filtering at query time beats deleting at ingestion, and how to keep an exercise out of the alert path without blinding it.
- Dec 19, 2024 Blue-Green Cutover Mechanics at the Routing Layer Where the traffic switch actually happens, why DNS record changes make poor cutover levers, and what the standby fleet must be able to carry before and after the switch.
- Dec 4, 2024 Externalised Configuration and Immutable Artifacts: Where the Boundary Sits Why the same artifact should run in every environment, which configuration must move outside it, how the binding happens at startup, and what fails when the boundary is drawn wrong.
- Sep 26, 2024 Dependency Resolution and Partial Upgrades Why installing one package can remove another, how versioned dependencies and library sonames constrain an upgrade, and what differs between upgrade commands.
- Sep 20, 2024 Versioning the Platform Interface Under Existing Tenants How a self-service interface changes without breaking its consumers: explicit versions, compatibility guarantees, and why the parts nobody uses break first.
- Sep 11, 2024 Packet-Level Fault Injection: Latency, Loss, and Partitioning How delay, loss, and reordering are emulated in the kernel queueing discipline, why DROP and REJECT produce different failures in callers, and how TCP converts a small packet loss into a long stall.
- Aug 26, 2024 Severity Classification as a Routing Input How a severity level selects the channel, the responders, and the update cadence, and why definitions anchored in observable impact outperform definitions anchored in alarm.
- Aug 23, 2024 Measuring Alert Fatigue Instead of Describing It Which quantities make page volume and actionability comparable across rotations, how to collect them without rewarding the wrong behaviour, and how the numbers drive deletion.
- Jul 5, 2024 Deprecation Windows and the Machinery Behind Compatibility Shims How an old interface is kept working while its replacement is adopted, what a shim must translate, and how to decide when removal is safe rather than merely scheduled.
- Jun 28, 2024 Template Ownership When Scaffolding a New Service A generator produces a fork of the template the moment it runs. What that implies for update paths, three-way merges, and the split between generated and referenced configuration.
- May 29, 2024 Rebuilding a Pinned Artifact Without Its Original Environment Keeping the recipe instead of the bytes only works if the toolchain, dependencies, and clock can be reconstructed. What a pinned environment has to name and what cannot be rebuilt at all.
- Apr 22, 2024 Environment Parity as a Byte-Level Claim: What Divergence Between Stages Actually Costs Why staging and production diverge even when both are built from the same configuration, which differences change behaviour, and how to make parity a measurable property.
- Mar 1, 2024 Steady-State Hypotheses and the Abort Condition in Fault Injection How a fault-injection exercise defines normal behaviour as a falsifiable claim about metrics, how tolerance windows set its resolution, and why the abort condition is a separate signal.
- Feb 1, 2024 Idempotency Keys and Command Resources That Cannot Be Described Declaratively Why some operations cannot be expressed as desired state, how guard and sentinel patterns approximate idempotence, and why the approximation leaks under concurrency.
- Jan 16, 2024 Level-Triggered Reconciliation and the Desired-State Loop How a platform controller converges observed state toward declared state, why it re-reads everything on every pass, and what that costs in API traffic and correctness.
- Oct 26, 2023 Scheduling a Job Graph Around Its Critical Path The longest path through the dependency graph bounds pipeline latency regardless of machine count, and the cost of fan-out decides whether widening the graph helps at all.
- Oct 20, 2023 Deriving a Changelog from Commit Messages How a structured commit header turns release notes into a mechanical transformation, which parts of the convention are load-bearing, and where the encoder and the reader disagree.
- Aug 12, 2023 Reboot-Required and Live Kernel Patching What the reboot flag actually tracks, why a replaced shared library keeps running inside existing processes, and which kernel changes a live patch cannot make.
- Mar 13, 2023 Quota Accounting Lags Admission, and What That Permits Quota is enforced against a recomputed usage figure rather than a reservation. How the check-then-act window opens, what falls into it, and where a second guardrail is needed.
- Feb 2, 2023 Escalation Policies as Timeout-Driven State Machines How an alert moves from first notification to a human who acknowledges it, what each timer actually measures, and where the routing logic breaks down.
- Jan 8, 2023 Describing Breaking Changes So Consumers Can Act on Them Why a one-line note that an API changed forces every consumer to read the diff, and the shape of an entry that names the affected surface, the failure it produces, and its replacement.
- Dec 28, 2022 Immutable Artifact Coordinates and Why Digests Matter How a version-plus-repository name identifies a byte string, what breaks when the mapping can change, and how attestations bind a build input set to the digest that was published.
- Nov 26, 2022 What a Build Cache Key Must Cover Under-keying produces wrong artifacts that look valid; over-keying destroys the hit rate. How to enumerate the true inputs of a build action, and where the environment leaks in.
- Oct 10, 2022 Sync Ordering and Pruning in a Git-Driven Control Loop A reconciler that applies a directory of manifests has to decide what order to apply in and what to delete. Both decisions have failure modes that a diff never shows.
- Sep 24, 2022 Incremental Backup Chains and Restore Depth How changed-block tracking works, why deduplicated storage makes deletion expensive, and why restore time grows with the number of links in the chain.
- Jul 21, 2022 Clock Skew Injection and the Failures That Depend on Wall Time The difference between clock offset, skew, and drift, the two ways to inject a time fault without breaking the host, and which classes of software fail when wall time jumps.
- Jun 27, 2022 What a Page Must Carry to Be Actionable The fields a notification needs so a responder can decide whether to act, where to look first, and who else is affected, within the length limits of a phone screen.
- Jun 17, 2022 Key Escrow for Encrypted Backups How envelope encryption separates the key that protects data from the key that protects keys, why rotation is cheap until it is not, and how escrow can outlive the system it protects.
- Jun 10, 2022 Expand and Contract: Schema Changes That Survive an Overlap The three-phase pattern that keeps two application versions reading one schema, which DDL operations take heavy locks, and how to bound the lock window with batched backfills.
- Mar 19, 2022 Plan and Apply Across a Fleet: Partial Convergence and the Limits of Ordering Why a fleet-wide apply cannot be ordered like a single host, what the plan file captures and where it is stale, and how batches expose half-migrated intermediate states.
- Feb 16, 2022 From Postmortem to Alert Change: Closing the Detection Loop How a reconstructed timeline becomes a specific detection gap, how action items are classified so they change alerts and runbooks, and how to prove the change took effect.
- Feb 9, 2022 Content-Addressed Artifact Identity and the Cost of Mutable Tags Why a digest derived from bytes is the only stable name for a build output, how Merkle chunking makes large artifacts hashable, and what a mutable tag hides between resolution and pull.
- Jan 25, 2022 Saved Plans, State Serial, and Why Apply Must Be Pinned A saved plan is a reviewable artifact tied to one state snapshot. What it records, why apply rejects a stale one, and where the guarantee stops.