Forecasting Completion Dates from Historical Throughput

A date can be produced without any estimate of effort. The input is a record of how many items the system finished in each recent period, and the operation is resampling: draw past periods at random, accumulate their completions, and count how many periods are needed to retire the remaining work. Repeated a few thousand times, this yields a distribution of completion periods, and a percentile of that distribution is a defensible date.

The method is unappealing to anyone who believes a date must be derived from an analysis of the work, because it contains no analysis of the work at all. Its virtue is that it does not require the arithmetic that breaks when ranges are added; it only requires that the recent past resembles the near future closely enough for its variation to be informative.

Resampling the observed rate

The data are counts per period. Twelve weeks of history might read four, six, three, seven, five, five, two, eight, six, four, five, seven completed items. Nothing else is needed: no per-item estimates, no sizes, no assignments.

The forecast loop keeps a counter of items remaining, draws one period’s count at random, subtracts it, and repeats until the counter reaches zero, recording the number of periods consumed. Drawing with replacement treats the observed counts as an empirical distribution, which is the bootstrap: the sample stands in for the population, and resampling from it approximates the sampling distribution of the statistic of interest, here the number of periods to finish a fixed quantity of work.

import random

random.seed(11)
weekly = [4, 6, 3, 7, 5, 5, 2, 8, 6, 4, 5, 7]   # items completed per week
remaining, trials = 40, 20_000

def weeks_to_finish(remaining):
    weeks = 0
    while remaining > 0:
        remaining -= random.choice(weekly)       # a week drawn from history
        weeks += 1
    return weeks

results = sorted(weeks_to_finish(remaining) for _ in range(trials))
for p in (0.50, 0.85, 0.95):
    print(p, results[int(p * trials) - 1])

The median of the output sits near the ratio of remaining items to mean weekly throughput, which is Little’s law applied to the whole backlog rather than to a stage. The upper percentiles are set by the slow stretches inside the record: a history containing a two-item week makes a run of slow weeks possible in the simulation, and that run is what the eighty-fifth percentile represents.

What the percentile means

The output is a statement about the counting unit, not about any particular item. The eight-fifth percentile says that in simulations consistent with the recorded periods, four out of five runs finished within that many weeks. It says nothing about which items finish first, and nothing about whether the work is the right work.

Two properties follow. The date improves as the count of completed periods grows, because more history means a better characterised distribution; and the date is insensitive to optimism about individual items, because no individual item enters the calculation. The second property is the point. A system can be bad at estimating tasks and still produce a well-calibrated date, provided its throughput is measured consistently and does not trend.

The choice of unit is a decision with consequences. Counting items treats a one-line configuration change and a multi-week migration as equal, so splitting large items raises measured throughput without changing the amount of value delivered. Counting nothing but completions of a uniform size class avoids that, at the cost of a longer history per class. Whatever the unit, it belongs in the record next to the numbers, because a throughput series that silently changes units cannot be pooled with its own past.

How much history the answer can use

The simulation cannot produce variation the record does not contain. With twelve observed periods, the resampled counts take twelve distinct values, so the resulting distribution is coarse, and the extreme percentiles are close to the fastest and slowest weeks present. The ninety-fifth percentile of a twelve-period record is essentially the answer implied by an unusually fast week followed by a run of typical ones. It cannot represent a stall longer than any stall observed, which is a known weakness: tails are estimated from tails.

There is also arithmetic on the estimate itself. The mean of k period counts has standard error proportional to the standard deviation of the counts divided by sqrt(k), so the effective sample size for the average is k and not the number of items completed. A system completing twenty items a week for twelve weeks has a much better characterised rate than a system completing two, because its weekly counts are less dominated by integer rounding at small numbers.

For a stable process, a history of ten to twelve periods gives a usable central estimate and a rough upper percentile. Longer records mix regimes: a reorganisation, a change of unit, or a new intake policy makes the older half of the record describe a system that no longer exists. The usual practice is to use a window short enough to be current and long enough to include variation, and to state the window alongside the date.

When periods are not independent draws

Resampling with replacement discards the order of the periods, keeping only their marginal distribution. This is deliberate, and it is also an assumption: that a slow week carries no information about the next week.

Throughput series usually violate it. Holidays cluster, a release cycle produces a burst of completions followed by a quiet period, and a single blocked dependency can depress several consecutive periods. Under positive autocorrelation, the variance of a k-period total exceeds the variance implied by independent draws, so the simulated distribution is too narrow at the upper end, exactly where commitments are read.

The diagnostic is straightforward: compute the variance of the sums of consecutive blocks of length b from the record and compare it with b times the single-period variance. A ratio above one indicates positive correlation at that lag. Where it appears, a block bootstrap, which resamples contiguous blocks rather than single periods, preserves the local structure and widens the tail accordingly. A simpler alternative is to resample over a longer base period, for example a month rather than a week, so that the intra-period correlation is inside the unit being drawn.

Scope has to be pinned down

The simulation retires a fixed quantity. It has no mechanism for new work arriving, and none for work being abandoned. Both happen: intake continues while the backlog is being burned down, and some items are dropped.

Three treatments are defensible, and mixing them without saying so is not. The first pins scope at forecast time and treats new items as a separate commitment, so the date describes the items currently in the queue. The second forecasts net flow, subtracting arrivals from completions, which requires a second series and produces a negative-flow condition that terminates the simulation in failure rather than in a date. The third stops intake for the duration, which is a policy decision and not a forecast at all.

Abandoned work is subtler than new work, because abandonment raises measured throughput without retiring anything. Counting a cancelled item as complete inflates the rate permanently, and the inflation is invisible in aggregate numbers.

Reading a date out of the distribution

The output should be reported as at least two percentiles, typically the fiftieth as the working expectation and the eighty-fifth or ninety-fifth as the commitment, with the window, the unit, and the scope stated. Reporting only the median invites the reader to treat it as a promise, which fails half the time by construction.

The forecasts are also measurable in the sense described in calibrating interval estimates: each published eighty-fifth percentile is a coverage claim, and a record of published dates against actual completions shows whether the method is honest for this system. A forecast that is repeatedly beaten is not a good sign; it means the published percentile is too conservative, and the extra buffer is being paid for in every planning cycle. Recomputing the distribution as the record grows, then, is not a failure of confidence but the intended behaviour: the numbers move when the evidence moves, and the movement is the mechanism working.