Performance Engineering · Cluster
Throughput Scaling
Contention, batching, and parallelism.
- 01 Throughput, Latency, and Why Concurrency Is Not Parallelism How the relation between completion rate, response time, and in-flight work constrains every scaling decision, and why overlapping waits and splitting computation buy throughput in different ways.
- 02 Amdahl's Law and the Serial Fraction That Caps Speedup Why a fixed serial portion of a workload bounds speedup no matter how many processors are added, how to estimate that fraction, and where the law stops describing real scaling.
- 03 Critical Sections, Futexes, and What a Mutex Really Costs The path a mutex takes from uncontended atomic to sleeping thread, how Linux futexes avoid a syscall in the common case, and why uncontended cost predictions fail under contention.
- 04 Batching and the Amortization of Per-Item Cost How grouping work converts a fixed per-operation cost into a per-batch cost, where the delay comes from, and why batch size is a throughput constraint rather than a tuning knob.
- 05 Sharded Counters, Padding, and False Sharing Why two independent counters placed next to each other can cost more than one shared counter, how cache-line granularity forces padding, and what sharding gives up in exchange.
- 06 Bounded Queues and Backpressure as Throughput Control Why an unbounded queue converts overload into latency and then into collapse, how a bound pushes the refusal upstream, and which drop policy makes refusal cheap.
- 07 Work Stealing and the Cost of Task Granularity How per-worker deques with stealing keep locality for producers and absorb imbalance, why the victim is chosen at the tail, and where the stealing overhead overtakes the gain.
- 08 Compare-and-Swap Retry Loops Under Contention Why a lock-free read-modify-write loop that is faster than a mutex with two threads is slower with twenty, and how to bound, batch, or replace the retry.
- 09 NUMA Placement and the Cost of Touching Another Socket's Memory How first-touch page placement decides which socket holds a page, what a remote access costs relative to a local one, and why thread pinning without memory placement leaves the cost in place.
- 10 The Universal Scalability Law: Why Throughput Peaks and Then Falls How contention and coherency terms extend Amdahl's bound into a curve that predicts a maximum, what retrograde scaling looks like, and how to fit the two parameters from measurements.