Transposed Convolution and the Checkerboard Artifact

A layered network that reduces spatial resolution must eventually increase it again: segmentation needs a full-resolution mask, generation needs a full-resolution image, and super-resolution needs more pixels than it was given. The operator that performs the inverse is conventionally named transposed convolution, and the name is precise. It is not an inverse in the algebraic sense, since the forward convolution loses information and nothing recovers it, but it is exactly the transpose of the linear map that convolution defines, which is to say it is the operator that a backward pass through a convolution computes.

That lineage explains most of its behaviour, including the artefact it is best known for. A strided convolution’s transpose places unequal amounts of kernel on different output positions, and the inequality is periodic with the stride. Nothing in the operator hides that period: it is a fixed spatial pattern that the network has to learn to counteract, and a network that does not fully counteract it emits a grid.

The operator is the gradient of the forward pass

Write the forward convolution as a matrix $A$ acting on the flattened input: $y = Ax$, with $A$ of shape $(H_{out}W_{out}C_{out}) \times (HW C_{in})$, sparse, with one entry per (output position, input position, kernel tap) triple. Its transpose $A^\top$ maps a vector in output space back to input space. Backpropagation through the forward layer multiplies the incoming gradient by $A^\top$; a layer implementing $A^\top$ as a forward operation is therefore literally the transposed convolution.

Concretely, $A^\top$ scatters each input value to the positions its kernel touched, accumulating overlaps. With stride 1 the input grid and output grid coincide in extent, so each tap lands at a distinct shifted copy and the result is a full convolution with the flipped kernel. With stride $S > 1$ the output grid is $S$ times wider, so the input value at position $i$ scatters to positions $iS + m$ for each tap $m$. That is an insertion of zeros between input values followed by a normal convolution, which is the formulation most implementations use.

import numpy as np

def transposed_conv1d(x, w, stride):
    """x: length n input, w: length k kernel. Computes A^T x for a stride-s conv."""
    out_len = (len(x) - 1) * stride + len(w)
    y = np.zeros(out_len)
    for i, v in enumerate(x):
        y[i * stride:i * stride + len(w)] += v * w   # scatter and accumulate
    return y

x = np.array([1.0, 2.0, 3.0])
w = np.array([1.0, 2.0, 3.0])
print(transposed_conv1d(x, w, stride=2))

Extending to two dimensions happens independently per axis, and extending across channels is a matrix product: the layer has $C_{in}$ input channels, $C_{out}$ output channels, and its backward pass uses the same weights transposed in the channel index as well, so the parameter tensor of the transposed operator is the same tensor in the same layout, applied with the roles of input and output swapped.

The formula for output extent follows from the scatter: $(H_{in} - 1)S + K - 2P$, where $P$ is the cropping applied to the output. For an input of extent 8, kernel 3, stride 2, and no crop, the output is 17, not 16, the extra position being the edge of the last kernel’s footprint. A decoder that mirrors an encoder needs the output to be exactly $2H_{in}$, which requires $(H-1)S + K - 2P = 2H$; for $S=2$ this reduces to $K - 2P = 2$. Kernel 4 with a one-pixel crop on each side satisfies it exactly, which is why decoders are full of 4×4 stride-2 kernels. Kernel 3 does not, since it would need a crop of half a pixel, and the standard configuration handles the remainder differently: kernel 3 with stride 2, padding 1, and one extra output position appended at the far edge to absorb the missing half pixel.

Why the overlap is not uniform

The checkerboard arises from the number of kernel applications that land on each output position. For stride $S$ and kernel $K$, position $p$ in the output receives contributions from each input index $i$ and tap $m$ with $iS + m = p$, that is, from all taps whose index has residue $p \bmod S$. The count is roughly $K/S$ for taps concentrated in the middle and lower for the two boundaries of the window. When $K$ is not a multiple of $S$, the counts differ across residues.

With $K=3, S=2$, taps at residues 0 and 1 alternate: even output positions receive two taps and odd positions receive one (at the edges, fewer). The learned kernel therefore has to arrange for the sum of two taps at even positions and the single tap at odd positions to produce matching statistics, and if it does not, which is easy since the constraint is not in the parameterisation, the output carries a period-2 modulation in both axes, a checkerboard at the stride’s scale.

The artefact is worse in the parts of the output that are least constrained. In a generative model the high-frequency content is largely unconstrained by the loss, so the modulation persists; in a segmentation decoder the artifact appears along boundaries. It is also multiplicative with depth: a stack of transposed convolutions compounds the patterns from each stage, and the coarsest stage’s grid is the most visible because it sits at the largest scale.

# overlap count per output position for 1-D transposed convolution
def overlap_counts(n_in, k, stride):
    counts = [0] * ((n_in - 1) * stride + k)
    for i in range(n_in):
        for m in range(k):
            counts[i * stride + m] += 1
    return counts

print(overlap_counts(4, 3, 2))   # 1,1,2,1,2,1,2,1,1 -- uneven in the interior
print(overlap_counts(4, 4, 2))   # 1,1,2,2,2,2,2,2,1,1 -- uniform away from the ends

The second line is the fix in its simplest form: kernel size divisible by stride gives uniform overlap, so no position is systematically favoured. That is why 4×4 kernels with stride 2 are the conventional choice, and why 2×2 stride 2 is even safer. It is not a complete fix, since the learned kernel can still produce a modulated response, but it removes the structural bias that makes the artifact nearly unavoidable.

A second structural fix is to initialise the kernel so that the upsampling is an interpolation rather than an arbitrary mixture: a bilinear kernel of size $K \propto S$ distributes weight smoothly across overlapping taps, and a network that starts from an interpolation needs to learn only the residual detail. Empirically this converges at least as fast as random initialisation and removes the early-training checkerboard. It is worth being precise about what it does and does not do: it changes the starting point of optimisation, not the family of functions the layer can represent.

A resize followed by a convolution

The cleaner construction separates the two operations. Nearest-neighbour or bilinear resizing by an integer factor is a fixed, non-overlapping operation with exactly uniform overlap, so it cannot produce a positional bias. A subsequent convolution with stride 1 then mixes channels and refines the interpolated values without any stride interaction. The cost is the extra pass over the larger tensor: the resize writes an $S^2$ times larger buffer that the convolution immediately reads.

For nearest-neighbour resize the cost is the same as transposed convolution with a kernel that is a shifted identity, which is why “resize plus convolution” and “transposed convolution” are the same operation in that special case. For bilinear resize the effective kernel is separable and its weights are fixed rather than learned, which is the substantive difference: the layer gives up the ability to learn a non-interpolating upsampling but gains a guaranteed smooth basis.

The third option is to avoid upsampling at the last step entirely by predicting at low resolution and applying a parameter-free resize to the final output. For segmentation the usual form is to predict logits at the stride-8 or stride-16 level and bilinearly upsample the mask to full resolution; the lost detail is recovered not by the upsampler but by concatenating the encoder’s high-resolution features, which is the skip connection at the centre of semantic segmentation and skip connections.

Sub-pixel and its equivalence

Pixel shuffle, the depth-to-space operation, takes a tensor with $S^2$ times the channels, at full spatial extent divided by $S$ in each axis, and rearranges elements so that the channel axis becomes the sub-pixel phase. It performs an exact $S \times S$ upsampling with no overlap at all: each output position comes from exactly one input position, so the overlap count is uniformly one and the artifact is structurally impossible.

Its equivalence to transposed convolution is exact for the right weight arrangement. A transposed convolution with stride $S$ and kernel $K = S$ that is applied to $S^2$ channel groups is the same computation as a 1×1 convolution producing $S^2 C$ channels followed by a pixel shuffle. The difference is bookkeeping and efficiency: pixel shuffle uses one contiguous reshape plus a transpose, which most libraries implement as a copy, while the transposed convolution’s scatter has poorer locality. The reshape form also makes the uniform-overlap property obvious from the shapes, which is an argument for expressing upsampling this way when the model’s arithmetic permits.

The reconstruction mirrors the resolution reduction described in pooling as a downsampling operator, and the catch is the transposition cost: an $S^2 C$-channel tensor at reduced extent rearranged to $C$ channels at $S$-times extent moves the same number of bytes as the output, and the rearrangement is a strided copy whose efficiency depends on the layout convention (channels-first versus channels-last). In channels-last layouts the operation is a near-contiguous copy; in channels-first layouts it is a gather. The cost is bandwidth, not arithmetic, and on hardware where the tensor is already large that is the difference between an upsampling stage that is free relative to the convolution it follows and one that is not.

The assumption that upsampling is local

All of these operators interpolate from a neighbourhood. Bilinear interpolation reads four neighbours, transposed convolution reads at most $K^2$, and pixel shuffle reads exactly one. None of them can invent structure that the reduced tensor does not imply, so the coarse features must already encode the shape. This is the constraint that makes decoder design converge on the pattern of upsampling plus skip connections rather than on ever more elaborate upsamplers: the information that the resolution increase needs comes from the encoder, and an upsampler that reads only the coarsest features is being asked to hallucinate detail that a network trained by pixel-wise loss will render as blur or as the periodic pattern above.