Topic domain
Backend Systems
Server-side request handling, state, and messaging.
- Articles
- 80
- Clusters
- 8
- Foundational
- 24
- Intermediate
- 32
- Advanced
- 24
Servers, middleware, and connection lifecycle.
- The Accept Loop: From listen() to an Accepted Connection
- HTTP/1.1 Message Framing and the Cost of Connection Reuse
- Route Matching Precedence and the Cost of Ambiguity
- Middleware Composition and Why Order Changes Behaviour
- Thread-per-Connection, Event Loops, and the Cost of Blocking
- Reading Request Bodies Without Holding Them in Memory
API Styles
10REST, RPC, and GraphQL tradeoffs.
- Resource Identity and URI Design in REST
- Safety and Idempotence in HTTP Method Semantics
- RPC Interface Definitions and Wire Framing
- Content Negotiation and Representation Formats
- Cursor Pagination and the Cost of Deep Offsets
- GraphQL Execution: Resolvers, Fan-Out, and Query Cost
Caching
10Invalidation, tiers, and stampedes.
- Cache-Aside and Read-Through: Who Owns the Miss
- TTL Expiry and the Two Ways an Entry Dies
- Why LRU Eviction Needs a Hash Map and a Linked List
- Invalidation by Deletion and by Versioned Key
- Request Coalescing and Probabilistic Early Expiration
- Caching Absence with Negative Entries and Bloom Filters
Delivery semantics and consumer groups.
- Append-Only Logs and the Queue Abstraction
- Delivery Semantics and Where Duplicates Come From
- Acknowledgements, Offsets, and Visibility Timeouts
- Consumer Groups and Partition Assignment
- Producer Batching, Compression, and Acknowledgment Pipelining
- Credit-Based Flow Control and Consumer Prefetch Limits
Scheduling, idempotency, and retries.
- The Job Record State Machine: Enqueue, Claim, and Acknowledge
- Visibility Timeouts and Lease Heartbeats in Job Claims
- Idempotency at the Effect Boundary, Not the Handler Entry
- Retry Ladders and Dead-Letter Promotion for Failed Jobs
- Cron Triggers, Misfire Policy, and the Runs That Never Happen
- Holding Delayed Jobs: Buckets, Timers, and the Cost of Time-Ordered Scans
Statelessness, stickiness, and stores.
- How a Session Identifier Binds Requests to Server State
- Cookie Scope, Attributes, and Transport Rules
- Signed Cookie Tokens and the Limits of Statelessness
- Load Balancer Session Affinity and Its Failure Modes
- Key Design and Eviction in External Session Stores
- Session Fixation, Rotation, and Revocation
Timeouts, backpressure, and circuit breaking.
- Deadline Propagation and the Shared Request Time Budget
- Retry Budgets and the Load Multiplication They Prevent
- Queue Time, Service Time, and What a Timeout Measures
- Bounded Queues and Load Shedding at the Admission Boundary
- Circuit Breaker State Machines and Half-Open Probing
- Concurrency Limits Derived from Little's Law
Throughput, tail latency, and pools.
- Little's Law and the Size of a Request Pool
- Head-of-Line Blocking in a Shared Worker Pool
- Fan-Out and the Amplification of Tail Latency
- Connection Pool Checkout and the Exhaustion Cliff
- Queueing Delay and the Cost of Service-Time Variance
- Batching and the Amortization of Per-Request Cost
All Backend Systems articles
- Oct 23, 2026 Refresh Token Rotation and Replay Detection How rotating refresh tokens limit the value of a stolen credential, why a used token presented twice means compromise, and how a grace window keeps honest clients from tripping the alarm.
- Sep 21, 2026 Sharding Session Data and the Size Ceiling How session keys are partitioned across nodes as the store grows, why resharding is safer at a partition layer, and what a large cookie costs on every request.
- Sep 20, 2026 RPC Interface Definitions and Wire Framing How an interface definition becomes bytes on a connection, from protobuf field encoding to gRPC length-prefixed messages and JSON-RPC envelopes.
- Sep 8, 2026 Thread-per-Connection, Event Loops, and the Cost of Blocking What a blocked read consumes in each concurrency model, why readiness notification changes the shape of the code, and where each approach fails under load.
- Aug 26, 2026 Consistent Hashing and What Resharding Costs a Cache Tier Why modular hashing turns a single added node into a near-total cache flush, how ring and rendezvous schemes bound the keys that move, and what slot-based clusters still charge for rebalancing.
- Aug 14, 2026 Metastable Failure and the Load That Keeps a System Down Why some outages persist after the trigger is gone, how retries and cold caches hold a system in a low-capacity state, and what it takes to break the loop.
- Aug 11, 2026 Priority Classes and Tenant Fairness in Shared Job Queues Why a single ordered queue lets one tenant or one class consume the whole worker pool, how weighted dispatch and per-tenant caps differ, and what to measure when fairness is the goal.
- Jul 29, 2026 In-Process Caches: Retained Heap and the Cost of a Hit What a per-process cache actually retains beyond its values, how object headers and hash-table slots multiply the footprint, and why the garbage collector makes an unbounded local cache a latency risk.
- Jul 10, 2026 Concurrent Mutations and Versioned Session Records Why whole-record writes lose updates when a browser issues requests in parallel, and how compare-and-set, fencing tokens, and delta operations keep the record coherent.
- Jul 4, 2026 Chained Jobs and Fan-In Completion Without a Coordinator How a job that spawns children can know when they are all finished, why the counter must count terminal outcomes rather than successes, and how the successor is triggered exactly once.
- Jun 26, 2026 API Versioning and the Mechanics of Schema Evolution Which schema changes are actually breaking, where a version identifier belongs, and how deprecation windows and data migrations interact with old clients.
- Jun 22, 2026 Graceful Shutdown and the Ordering of Connection Draining The sequence a server must follow to stop without dropping requests, why a closed listening socket resets queued connections, and how drain deadlines are sized.
- Jun 13, 2026 Middleware Composition and Why Order Changes Behaviour What a middleware chain actually is at runtime, which orderings the mechanism forces, and how buffering middleware breaks streaming responses.
- Jun 11, 2026 Coordinated Omission in Latency Measurement Why measuring response time only for completed requests hides the worst delays, how the bias arises, and which measurement schemes avoid it without inventing numbers.
- Jun 11, 2026 Credit-Based Flow Control and Consumer Prefetch Limits A broker that pushes messages needs permission to send, and the counter that grants it decides both memory consumption and how evenly work lands across consumers.
- Jun 10, 2026 HTTP/2 Stream Multiplexing and Flow Control on the Server How independent streams share one TCP connection, why flow control is owed at two levels, and what per-connection compression state costs a server.
- Jun 4, 2026 Hit Ratio Arithmetic and How to Size a Cache Tier How to convert a hit ratio into source load and latency, why the miss-rate curve flattens before the working set is covered, and which measurement choices make a cache look better than it is.
- Jun 3, 2026 Exactly-Once Delivery Across System Boundaries The dual-write problem, the outbox that removes it, and the narrow sense in which a message can be applied exactly once when the database and the broker are separate systems.
- Mar 13, 2026 Idempotency at the Effect Boundary, Not the Handler Entry Deduplicating a job at the point where it changes the world: unique constraints, request keys, delta-to-fact conversion, and the two-phase effect that survives a crash mid-call.
- Feb 6, 2026 Delivery Semantics and Where Duplicates Come From At-most-once, at-least-once, and exactly-once describe the disposition of a single message when an acknowledgement is lost, not a property of the network.
- Feb 6, 2026 Streaming APIs: Framing, Backpressure, and Resumption How server-sent events, WebSocket, and gRPC streams delimit messages, what happens when a consumer reads slowly, and how a broken stream resumes.
- Jan 28, 2026 Concurrency Limits Derived from Little's Law How the relation between in-flight requests, arrival rate, and latency produces a concurrency limit, and why an adaptive limit tracks a dependency that changes speed.
- Jan 28, 2026 Read-Your-Writes Across Replicated Session Stores Which consistency guarantees a session actually needs, why a replica read can undo a rotation, and how version tokens and home-region ownership keep a user from seeing their own past.
- Jan 12, 2026 Draining a Worker Pool Without Losing or Doubling the Work The shutdown sequence for a job worker: stop claiming, cancel at a deadline, release leases instead of waiting for expiry, and keep heartbeats alive until the handlers stop.
- Jan 8, 2026 Hedged Requests and the Economics of Tail Tolerance How issuing a duplicate request after a delay caps the tail, what the duplicate costs in load, and how to choose the delay so the trade stays favourable.
- Dec 27, 2025 Fan-Out and the Amplification of Tail Latency How issuing a request to many backends converts a rare slow response into a common one, the arithmetic behind that amplification, and the structural ways to cap it.
- Dec 19, 2025 Cron Triggers, Misfire Policy, and the Runs That Never Happen Turning a recurrence expression into at-most-one job per period, choosing what to do with slots missed during downtime, and why a schedule that never fires is invisible without an external expectation.
- Dec 8, 2025 Power of Two Choices and the Cost of a Random Probe Why sampling two backends and picking the less loaded one beats both random assignment and global state, and how stale or misleading load signals degrade the guarantee.
- Nov 21, 2025 Batching and the Amortization of Per-Request Cost How grouping work into larger units trades latency for throughput, which fixed costs actually amortize, and where a batch becomes slower than the sum of its parts.
- Nov 10, 2025 Queue Time, Service Time, and What a Timeout Measures A client-side timeout covers every wait in the path, including queues the client never sees, and that decomposition explains why a server can look fast while its callers time out.
- Nov 6, 2025 Safety and Idempotence in HTTP Method Semantics What each method promises about side effects, how caches and retrying intermediaries use those promises, and where real endpoints quietly break them.
- Oct 22, 2025 Invalidation by Deletion and by Versioned Key Why deleting a cache entry after a write is a race rather than a guarantee, how versioned keys turn invalidation into a change of address, and what each approach costs when the cache and the source disagree.
- Oct 14, 2025 GraphQL Execution: Resolvers, Fan-Out, and Query Cost How a GraphQL document becomes a tree of resolver calls, why list fields produce N+1 queries, and which limits keep a single endpoint from being a denial-of-service vector.
- Oct 3, 2025 Holding Delayed Jobs: Buckets, Timers, and the Cost of Time-Ordered Scans Why millions of future-dated jobs concentrate contention on one index page, how time bucketing and a separate promotion step spread it, and where in-memory timer wheels fit.
- Sep 19, 2025 Ordering Guarantees and Head-of-Line Blocking A log guarantees order only within a partition, and that guarantee survives retries, consumer parallelism, and rebalances only under specific conditions.
- Sep 12, 2025 Key Design and Eviction in External Session Stores How session records are keyed, sized, and expired in Redis and Memcached, and why an eviction policy can turn a memory shortage into a login outage.
- Sep 12, 2025 Retry Ladders and Dead-Letter Promotion for Failed Jobs How job retries differ from request retries: escalating delays held as queue state, caps in wall-clock time, crash loops, and what promotion to a dead-letter state must preserve.
- Aug 17, 2025 Consumer Groups and Partition Assignment How a group coordinator keeps one owner per partition, what a generation change does to in-flight work, and why the assignor choice changes rebalance cost.
- Jul 24, 2025 Log Compaction, Retention, and Replay Retention deletes from the start of a log and compaction keeps only the newest value per key, which turns the log into a replayable store of current state.
- Jul 17, 2025 TTL Expiry and the Two Ways an Entry Dies How deadline-based caching separates logical expiry from physical reclamation, why lazy and active expiry put the cost in different places, and how clocks and replicas make a TTL a bound rather than a promise.
- Jul 9, 2025 Cursor Pagination and the Cost of Deep Offsets Why OFFSET grows linear in page depth, how keyset predicates keep page cost flat, and what an opaque cursor has to carry to stay correct.
- Jul 9, 2025 Dead Letter Queues and Poison Message Quarantine Which failures deserve another attempt, how a broker counts attempts without understanding the error, and why the quarantine queue needs an owner and a bound.
- Jul 3, 2025 Header Limits, Deadlines, and Slow-Connection Defence Why an attacker who sends one byte a second ties up more than CPU, how absolute deadlines differ from per-read timeouts, and what limits a server has to count.
- Jun 24, 2025 Cookie Scope, Attributes, and Transport Rules The exact matching rules that decide whether a browser attaches a session cookie to a request, and how Domain, Path, Secure, HttpOnly, and SameSite narrow that decision.
- Jun 17, 2025 Cache-Aside and Read-Through: Who Owns the Miss How responsibility for a cache miss is divided between application code and the cache itself, what each arrangement costs in round trips, and how they fail differently when a write races a read.
- Jun 7, 2025 Bounded Queues and Load Shedding at the Admission Boundary An unbounded queue converts an overload into a latency problem and then into a memory problem, while a bounded one forces the choice of which requests to drop and when.
- May 9, 2025 Append-Only Logs and the Queue Abstraction Destructive reads and append-only retention produce different broker state, different fan-out costs, and different limits on what a consumer can ask for.
- Apr 26, 2025 Contention Ceilings and the Limits of Parallel Scaling Why adding workers to a shared counter, lock, or cache line stops helping, how the serial fraction bounds speedup, and which designs remove the ceiling instead of raising it.
- Apr 23, 2025 Bulkhead Isolation and the Design of Partial Degradation Partitioning capacity so one dependency cannot consume all of it, and deciding in advance which features are worth serving when the partition is full.
- Apr 22, 2025 Idempotency Keys and the Unknown-Outcome Problem How a client-supplied key turns a retried mutation into at-most-once execution, where the key has to be stored, and which failure windows remain open.
- Apr 10, 2025 Why LRU Eviction Needs a Hash Map and a Linked List The data structure behind least-recently-used eviction, why exact recency ordering costs two pointers per entry, and what CLOCK and frequency-based policies give up to avoid that cost.
- Apr 9, 2025 Cancellation Propagation and the Work That Outlives Its Caller Cancellation is a cooperative signal that reaches only the code that checks it, so a cancelled request can still commit a write; scoping the non-cancellable part is the design work.
- Feb 22, 2025 Acknowledgements, Offsets, and Visibility Timeouts A broker can learn about completion only through an explicit signal or an expiring lease, and the timing of both decides how much work is repeated after a crash.
- Jan 26, 2025 Producer Batching, Compression, and Acknowledgment Pipelining A produce request has a fixed cost per round trip and per replica, and batching, compression, and in-flight limits each trade latency for a different part of that cost.
- Jan 14, 2025 Head-of-Line Blocking in a Shared Worker Pool Why one slow request class poisons the latency of every class behind it in a FIFO queue, and which partitioning or scheduling schemes actually remove the convoy.
- Dec 16, 2024 Retry Budgets and the Load Multiplication They Prevent How per-client retry counts turn a partial dependency failure into a full outage, and why a budget denominated in successful requests bounds the amplification instead.
- Dec 15, 2024 Deadline Propagation and the Shared Request Time Budget Why independent per-hop timeouts cannot bound end-to-end latency, and how an absolute deadline carried in request metadata turns the budget into arithmetic.
- Nov 23, 2024 How a Session Identifier Binds Requests to Server State What the opaque string in a session cookie actually denotes, how much entropy it needs, and why the identifier and the record it points at have separate lifetimes.
- Nov 22, 2024 Circuit Breaker State Machines and Half-Open Probing The three states, the counters that drive transitions, and why the half-open probe count is the parameter that decides whether a recovering dependency survives its own recovery.
- Nov 19, 2024 Load Balancer Session Affinity and Its Failure Modes How sticky routing is implemented at layer 4 and layer 7, why consistent hashing limits the damage of a node loss, and which failures affinity cannot prevent.
- Sep 20, 2024 Dual Writes, Replication Lag, and Read-Your-Writes Through a Cache Why a cache turns one write into two writes with no shared transaction, which orderings lose updates, and the session-level guarantees that make a client see its own changes without giving up the cache.
- Sep 8, 2024 Reading Request Bodies Without Holding Them in Memory How to consume a request body as a stream, what backpressure means at the socket, and where size limits, decompression, and multipart parsing go wrong.
- Aug 19, 2024 Hedged Requests and the Arithmetic of Tail Latency Why the slowest of several parallel calls sets the latency of a fan-out, and how a delayed second attempt trades a fixed slice of extra load for a large reduction in the tail.
- Jul 26, 2024 Queueing Delay and the Cost of Service-Time Variance Why waiting time depends on variance rather than only on mean load, how the M/M/1 and M/G/1 results express that dependence, and what reducing variance buys.
- Jul 10, 2024 Error Models Across REST, gRPC, and GraphQL How each style encodes failure, why status-code categories are not enough, and how a client decides whether an error is safe to retry.
- May 23, 2024 Session Fixation, Rotation, and Revocation Why authenticating in place hands an attacker a valid session, how rotation must be sequenced to survive concurrent requests, and what revocation can and cannot reach.
- Apr 30, 2024 HTTP/1.1 Message Framing and the Cost of Connection Reuse How a server decides where one HTTP message ends and the next begins, why chunked encoding exists, and what persistent connections change about failure handling.
- Apr 8, 2024 Caching Absence with Negative Entries and Bloom Filters How a cache represents the answer "this does not exist", why missed negative lookups are more dangerous than missed positive ones, and what a Bloom filter costs in bits per key and false positives.
- Mar 19, 2024 Resource Identity and URI Design in REST How REST separates the name of a thing from the bytes that represent it, and which URI choices survive changes to the underlying data model.
- Mar 12, 2024 Visibility Timeouts and Lease Heartbeats in Job Claims Why a claimed job stays hidden from other workers only until a deadline, how heartbeats extend that deadline safely, and what happens when the worker is alive but cannot renew.
- Dec 29, 2023 Proxy Boundaries, Forwarded Headers, and the Hop-by-Hop Contract Which headers stop at an intermediary, how the client address is reconstructed without trusting the wrong hop, and why framing disagreement produces smuggling.
- Dec 7, 2023 Connection Pool Checkout and the Exhaustion Cliff What a checkout actually costs, why exhaustion degrades a service sharply rather than gradually, and how wait queues, validation, and recycling interact under load.
- Nov 11, 2023 The Accept Loop: From listen() to an Accepted Connection What the kernel does between the TCP handshake and a readable socket, why listen() takes a backlog argument, and how accept wakeups distribute work across processes.
- Oct 16, 2023 Request Coalescing and Probabilistic Early Expiration What happens when a hot key expires under load, how single-flight and leases reduce the burst to one load, and how early recomputation spreads the remaining work before the deadline arrives.
- Sep 28, 2023 Little's Law and the Size of a Request Pool Why the number of requests in flight equals arrival rate times service time, how that identity fixes a pool size, and why adding workers past that point buys no throughput.
- May 13, 2023 Route Matching Precedence and the Cost of Ambiguity How a dispatcher chooses one handler among overlapping patterns, why specificity beats registration order, and where percent-decoding changes which route wins.
- Mar 7, 2023 The Job Record State Machine: Enqueue, Claim, and Acknowledge How a durable job queue represents work as a record with an explicit state machine, why claiming must be a single conditional update, and which guarantees the record cannot provide.
- Feb 8, 2023 Signed Cookie Tokens and the Limits of Statelessness How a signed token encodes session facts, what the signature actually guarantees, and which operations stop being possible once the server keeps no record.
- Dec 26, 2022 Content Negotiation and Representation Formats How Accept headers, quality values, and the Vary header decide which representation a cache may reuse, and why most APIs settle on a single format.
- Apr 14, 2022 Fencing Tokens and the Worker That Did Not Notice It Was Replaced How a monotonic lease epoch stored beside the protected data stops a stale worker from committing, why a timestamp cannot serve as the token, and what the guarantee excludes.