Removing Work Before Making It Faster

There are two ways to reduce the cost of an operation: perform it more cheaply, or stop performing it. The second is bounded by a different arithmetic. Accelerating a function by a factor $s$ approaches a limit set by the irreducible work inside it; eliminating a call site removes its cost entirely, and if the call site was redundant — computing a value that was already known, fetching data that will not be read — the result is not a faster program but a smaller one.

The reason this deserves separate treatment from ordinary optimization is that the two are searched for by different methods. Accelerating work is found by reading a profile’s self-time and asking why the code is slow. Removing work is found by asking why the code runs at all, which is a question a profile cannot answer, because a profile records that the work happened, not whether it was needed.

What redundancy looks like in ordinary code

Redundant work rarely appears as a duplicate function. It appears as a value recomputed inside a loop that could have been hoisted, an accessor that revalidates state on every call, a serialization pass that converts a structure to bytes twice on one path, a check repeated in an inner loop whose invariant was established by the outer loop. None of these are visible as a single hot entry in a profile if the surrounding code is also busy; they show up as a diffuse spread of self-time across a small set of functions that account for far more total time than their logic suggests.

A second class is over-fetching. A query selects every column of a wide row to use two of them, and the transfer and decode cost of the unused columns is charged to the query’s time. A request fetches a full object graph to check a single field. A page loads a dataset whose length is known from a count query issued moments earlier. The cost is real and is attributed to the operation that fetched, which makes it look like an unavoidable property of the operation rather than a consequence of the request’s shape.

A third class is work performed to satisfy a guarantee nobody uses. Sorting output that will be consumed as a set, validating input already validated at the boundary, deep-copying a structure that is never mutated, maintaining an index over a collection that is always scanned. Each of these is defensible in isolation; together they can be a substantial fraction of a program’s cost, and each is removable only by establishing that no consumer depends on it.

The evidence that a call site is redundant

Removal requires more than the absence of an obvious use. The question is whether the work is observable — whether any correct execution can distinguish a program that does it from one that does not. For pure computation the criterion is simple: if the result is already available and the computation has no side effects, removing it is safe, subject to the cost of establishing that the result is available (which is the memoization question treated in cache hit-rate break-even).

For I/O the criterion is harder, because I/O is observable through timing, ordering, and side effects. Removing a read is safe when the data is provably unused; removing a write is safe only when the write is idempotent and redundant with another, or when no reader can observe the intermediate state. Removing a lock acquisition is safe only if the invariant it protected holds at that point by another argument. These are correctness arguments, and the reason removal is often deferred in favour of acceleration is that acceleration does not require them.

For logging and metrics the criterion is a policy question rather than a semantic one: the call is not redundant in the sense of computing a value, but its output may be sampled, batched, or dropped when it is not consumed. The cost model has to account for the fact that the cost of an unused log statement is paid on the write path and only occasionally on the read path.

The cost of the removal itself

Eliminating work is not free, and the cost has three components that are easy to underestimate because they do not appear in the profile.

The first is the branch or check that guards the removal. Hoisting an invariant out of a loop adds a variable and possibly a bound check; skipping a query when the cache holds the data adds a lookup. The marginal saving is the eliminated cost minus the cost of the check, and for cheap operations the check can dominate — the reason a general-purpose “only do this if needed” wrapper around a five-nanosecond operation is often a net loss.

The second is the invariant that must be maintained. A value computed once and reused is correct only as long as nothing mutates its inputs; every mutation site becomes a place where an invalidation must be added, and every missed invalidation is a correctness bug that appears as a stale read rather than as a crash. This is the classic trade-off that makes caching a hazard, and it applies equally to hoisted computation and to deduplicated I/O.

The third is the loss of a check. Removing a validation removes its diagnostic value. If a malformed input previously failed loudly at the point of a redundant check, removing that check relocates the failure to a later point, frequently to a place where the error message no longer names the cause. Cheaper failure diagnosis is a real cost, paid later and by someone else.

// Before: resolves the same path for every entry in the listing.
fn describe_all(entries: &[Entry]) -> Vec<String> {
    entries
        .iter()
        .map(|e| format!("{} in {}", e.name, resolve_base(&e.base)))
        .collect()
}

// After: resolve each distinct base once. The map is worth its cost only
// because resolve_base is expensive relative to a hash lookup.
fn describe_all(entries: &[Entry]) -> Vec<String> {
    let mut resolved: HashMap<&str, String> = HashMap::new();
    entries
        .iter()
        .map(|e| {
            let base = resolved
                .entry(e.base.as_str())
                .or_insert_with(|| resolve_base(&e.base));
            format!("{} in {}", e.name, base)
        })
        .collect()
}

The second version is not always the faster one. If resolve_base is a pointer chase through already-resident memory, the hash lookup, the allocation of the map, and the borrow machinery cost more than the repeated resolution, and the change is a regression that a profile would show as growth in the map’s own self-time. The decision requires knowing the ratio, which is what the ablation in measuring component cost by ablation supplies.

Why removal changes the shape of the remaining problem

A program whose cost has been reduced by acceleration retains its structure: the same operations, the same order, the same number of syscalls, each cheaper. A program whose cost has been reduced by removal has a different structure, and that is both its advantage and its risk.

The advantage is that removal is not subject to the diminishing returns of a local factor. The first doubling of a function’s speed captures a third to a half of the available gain in that region; eliminating it captures all of it, and eliminates the region from the profile so that the next candidate’s fraction is computed on a budget that no longer includes it. Removal also removes the irreducible part, which acceleration cannot touch: a sort’s comparison count can be reduced by a better algorithm, but the scan that produces the output is gone only if the output is not produced.

The risk is that removal changes observable behaviour in ways that are not covered by tests. Reduced I/O changes timing and therefore concurrency; eliminated logging removes the evidence that a later investigation needs; a query that selects fewer columns changes the schema coupling in a way that is discovered when a migration adds a column to the projection. The mitigation is not to avoid removal but to remove deliberately, with the observability question asked in advance: what will no longer be visible if this is not done?

Choosing between the two

A useful ordering for a candidate list is to ask, for each entry, whether the work can be removed before asking how fast it can be made. The question costs a minute and occasionally removes the need for the week. The asymmetry in return is decisive when the removal is safe: no local speedup factor of a redundant operation can beat not performing it, and the same reasoning applies at a coarser grain to whole stages of a pipeline, as in the critical path discussion where a stage with slack is best removed rather than tuned.

Where removal is unsafe or unaffordable, acceleration resumes its normal role, and the remaining question becomes the one the rest of this cluster addresses: what is the realistic factor, and does the saved time survive everything downstream of the change.