Deriving Test Inputs from Equivalence Partitions and Boundaries

A function that accepts a 32-bit integer and a string of at most 255 bytes has on the order of 2^32 × 2^2040 distinct inputs. No suite enumerates that space, and no suite comes close. Every test suite is therefore a sampling strategy, and the only interesting question is what the sample is supposed to represent. Partition analysis makes the claim explicit: the input space divides into classes within which the program behaves identically, so one representative per class suffices, with extra attention paid to the lines between classes.

That claim is a hypothesis about the implementation, not a theorem about programs. Most of the technique’s value and most of its blind spots follow from the fact that the hypothesis is usually true and occasionally, expensively, false.

What a partition actually asserts

A partition is a set of inputs that the specification says will be handled the same way: same validation outcome, same branch, same output formula. A quantity discount with bands at 10 and 100 units has three such classes, and one value from each of 5, 50, and 500 tests the hypothesis that behaviour is constant inside a class.

The hypothesis is what makes the sample small. It is also what makes the sample weak. A partition boundary the specification does not mention cannot be sampled, because the analysis never looks there. If the implementation quietly caps a counter at 127 because the field is an int8_t, spec- derived partitions will not find it. Partition analysis detects disagreements between specification and implementation about class membership, which is the most common shape of a defect but not the only shape.

Two structural requirements follow. Partitions must be disjoint, since overlapping classes waste cases on inputs already covered. They must also cover the space, or the gap is an untested region that no representative stands for. Enumerating classes by hand is easy for one parameter and becomes arithmetic for four.

Why the interesting cases sit on the edges

Inside a class, behaviour is assumed constant, so a mid-class sample can only confirm the assumption. At the boundary the predicate changes value, and the implementation’s comparison operator is what decides. Off-by-one errors, < written where <= was specified, an inclusive upper limit that should have been exclusive, a length check against 255 rather than 256: every one of those lives at an edge.

Boundary value analysis therefore samples each edge three times: immediately below, at, and immediately above. For a specified range of 1 to 100 that yields 0, 1, 2, 99, 100, 101. For a string of length 0 to 255 it yields the empty string, one byte, 254 bytes, 255, and 256. The empty string deserves its own mention: it is a boundary that crashes code whose loop assumes at least one element, and it is the value a generator built from “random alphanumeric” almost never produces.

Type limits are boundaries the specification usually omits and the implementation always has. A signed 8-bit counter reaches its maximum at 127 and wraps to -128 on the next increment; an unsigned 32-bit counter wraps at 4,294,967,296. A total computed as quantity times unit price overflows silently in languages that do not trap, and a test at quantity 100 never notices. Three cases per type limit, at 127, at 128, and at the maximum, cover the wraparound class.

Floating point is the hard case, because the boundary is not a decimal. One tenth has no exact binary representation, so a comparison against a literal 0.1 fails for the value a caller would naturally write. Monetary rounding points, the magnitude at which a sum loses its low bits, and the smallest normal and subnormal magnitudes are all real boundaries. A mid-class sample such as 0.5 tests none of them.

Building the set from the specification

The discipline that separates partition analysis from guessing is the source of the classes. They come from the written requirement, not from reading the code. Classes read out of the implementation merely re-encode whatever the implementation does, including its bugs: a branch that should not exist becomes a partition that is dutifully tested and passes.

A worked shape, for a shipping cost function that takes a weight in kilograms and a zone number:

Parameter	Class	Representative	Edge cases
weight	0 to 1 kg	0.5	0, 0.001, 1.0, 1.001
weight	1 to 30 kg	15	30.0, 30.001
weight	above 30 kg	—	rejected
zone	1 to 3	2	0, 4
weight × zone	any valid pair	(0.5, 2)	—

Each row is a claim, and each edge triple is a separate claim about where the claim changes. Six rows produce roughly a dozen cases, which is small enough to read and specific enough to fail for a reason.

Two rules keep the table honest. The first is that every class needs an observable difference from its neighbours; a partition whose expected output is identical to the adjacent partition is not a partition but a duplicate. The second is that rejection paths are classes too. A validator with three acceptance bands and one rejection branch has four partitions, and the rejection branch is the one that tends to be written last and tested least.

The combination explosion, and pairwise as a compromise

Four parameters with three, four, five, and six classes have 360 combinations. Exhaustive combination testing is affordable for a two-parameter form and unaffordable for a configuration object. The standard reduction is n-way combinatorial coverage: choose a set of cases such that every combination of n parameter values appears in at least one case. Pairwise coverage, with n = 2, is the usual choice.

The arithmetic is favourable. Take the two parameters with the largest class counts, p and q. Every case fixes one value for each parameter, so a case covers exactly one of the p·q value pairs for that parameter pair, which forces at least p·q cases; and a grid over those two parameters, with the remaining columns filled in cyclically, stays at or near that count. Four parameters with three, four, five, and six classes therefore need on the order of 30 cases rather than 360.

Pairwise is a heuristic, not a guarantee, and the reason matters. Most defects involve one parameter or a relationship between two; a defect that requires three specific values simultaneously will be missed by a 2-way set. When three parameters interact, as when a currency, a payment method, and a region that together select a tax rule, 3-way coverage is the defensible choice, at a cost that grows with the product of the three largest class counts.

The other limit is that combinatorial coverage treats parameter values as independent, which they are not. A start date after an end date is not a pair worth covering; it is an invalid pair that must be excluded from the generator, and excluding it correctly is part of the specification work.

Where the method misleads

Three failure patterns account for most of the technique’s bad reputation.

Partitions that encode implementation structure. Reading classes off a switch statement produces a suite that pins the current branch layout. When the branch layout is refactored, every test changes, and the suite has been measuring its own shape.

Continuous inputs with unknown boundaries. A latency budget, an angle, a probability: the specification gives a range and no internal edges, but the implementation has them anyway: a lookup table with a discontinuity, a fast path for small values. Sampling three points in a continuous range tests almost nothing. The remedy is to find the internal edges from the specification’s intent, or to add randomized sampling, which is the territory of property-based testing.

Unicode and locale boundaries. String length in bytes, in code points, and in grapheme clusters differ. A 255-byte limit admits 255 ASCII characters and 63 four-byte code points; a test that fills a buffer with 'a' never exercises the truncation path that splits a multi-byte sequence. The boundary is real, and it is invisible to a partition derived from the number 255 alone.

The cases as data

Boundary cases are easiest to audit when they are table entries rather than separate functions. Each row states an input and the expected outcome, and the failure message names the row.

import pytest

@pytest.mark.parametrize(
    "weight_kg, zone, expected",
    [
        (0.0, 1, 500),        # empty shipment, lower edge of the first band
        (0.001, 1, 500),      # just inside the first band
        (1.0, 1, 500),        # band edge, inclusive by specification
        (1.001, 1, 800),      # just past the edge
        (30.0, 1, 800),       # upper edge of the heaviest band
        (30.001, 1, None),    # over the limit, must be rejected
        (1.0, 0, None),       # zone below the valid range
        (1.0, 4, None),       # zone above the valid range
    ],
)
def test_shipping_cost(weight_kg, zone, expected):
    if expected is None:
        with pytest.raises(ValueError):
            shipping_cost(weight_kg, zone)
    else:
        assert shipping_cost(weight_kg, zone) == expected

The table is the artefact worth reviewing. The code around it is mechanical, and the interesting review question is which edges are missing: the boundary at which a fractional kilogram rounds, the zone transition, the currency-dependent threshold.

What the technique leaves to other methods

Partition and boundary analysis answer which inputs to try. They say nothing about whether the observed output is correct, which is the oracle problem, and nothing about which statements the cases execute, which is what coverage criteria measure. The three techniques are complementary rather than ranked: partitions choose the inputs, coverage reports which code they reached, and the oracle decides whether what happened was right. A suite with strong coverage and a weak oracle is a suite that runs a great deal of code and checks very little of it.