UV Chart Packing and Why Gutters Are Measured in Texels

Unwrapping a mesh produces a set of charts: flat pieces of the surface, each with its own local parameterization, each with a boundary. To be sampled, every chart must be placed somewhere in one image, and the placement is not free. Rotating a chart changes how much of the atlas it occupies; scaling it changes how many texels represent a given area of surface; and moving it changes which texels its boundary filter taps reach. Atlas packing is the problem of choosing all three for every chart at once, and the objective is not area utilization alone.

The parameterization is already the constraint

The shape of a chart is determined by the unwrapper, not the packer, and the unwrapper has already made the trade that dominates everything after. Flattening a curved surface necessarily distorts something. Conformal unwraps preserve angles and distort area; equiareal unwraps preserve area and distort angles; most practical unwrappers minimize a weighted combination and leave a residual stretch that varies across the chart.

Stretch matters to the packer because texel density is what the artist is trying to equalize. If two charts represent equal surface area but one is stored twice as large in UV space, the surface it covers receives twice the texel density and shows more detail, and at the same viewing distance sharper features. A packer that maximizes area utilization will happily scale a chart down to fit a gap, producing a region of the model with visibly lower texture resolution than its neighbours. The seam between two charts with different densities is a visible line where detail level changes.

The quantity that should be equalized is texels per unit of surface area, so the packer’s scaling term is not free. In practice this is expressed as a uniform scale applied to all charts, with the relative sizes fixed by the unwrapper’s stretch field, and the uniform scale chosen so the total fits the atlas. That makes the packing efficiency directly proportional to the resulting resolution: a packing that fills 60% of the atlas leaves every chart scaled down by $\sqrt{0.6}$, losing about 22% of linear resolution compared with a packing that fills 100%.

The packing problem is a variant of bin packing with rotation

Each chart has an axis-aligned bounding box in UV space after any rotation, and the packing problem becomes: place a set of rectangles into a square, minimizing wasted area, with each rectangle optionally rotated by 90-degree increments.

This is NP-hard in general. Practical packers use heuristics, and the heuristics differ mainly in what order they place rectangles and how they represent the free space.

The simplest is shelf packing: place rectangles in rows, each row as tall as its tallest member. It is fast, deterministic, and wastes space whenever the rectangles’ aspect ratios differ: a tall thin chart forces the whole shelf to be tall, and the space above the short charts in that shelf is lost. Utilization in the 60–70% range is typical.

MaxRects keeps a list of maximal free rectangles and, for each new chart, tries every free rectangle and both orientations, scoring by the best fit. When a rectangle is placed, every free rectangle that intersects it is split. It achieves noticeably higher utilization at a cost of memory proportional to the number of free rectangles and time proportional to their number per placement.

A guillotine packer restricts splits to full cuts across a free region, which produces an arrangement that can be described as a binary tree of cuts. That restriction loses some utilization but makes the result cheap to encode and to modify incrementally, since a new chart can be inserted by walking the tree.

The ordering rule matters as much as the free-space structure. Placing large charts first is the usual choice, since a large chart that arrives late may not fit anywhere. A refinement sorts by height or by maximum dimension, which tends to keep shelves even. The alternative is a “best area fit” ordering recomputed after each placement, which costs more time for a modest gain.

Two properties are worth demanding of any packer regardless of heuristic: determinism and stability. Determinism means the same input yields the same atlas, so a build is reproducible. Stability means adding one chart changes few others, which guillotine packing gives partially and MaxRects gives poorly, because an early insertion reshuffles every later split.

Where the padding comes from

The gap between charts is not decorative. It exists because a texture fetch is never a point sample: the sampler reconstructs a value from a neighbourhood, and that neighbourhood extends past the chart’s boundary.

For bilinear filtering, the tap at a UV coordinate reads the four texels surrounding it. A fragment whose UV lies within half a texel of the chart’s edge therefore reads a texel outside the chart, which belongs to a different chart or to unused space. That produces a thin line of colour bled from a neighbouring chart along the seam. With mip mapping the problem grows: level $n$ of a mip chain is a box filter over $2^n \times 2^n$ texels of level 0, so a fragment sampling level $n$ reads an effective footprint of $2^n$ texels in each direction, and near a chart boundary that footprint crosses into the neighbour even when the level-0 UV is well inside.

The padding must therefore be sized for the deepest mip level at which the chart is still sampled, and it must be filled with the chart’s own content extended outward. The operation is called a dilation, and without it the gutter contains whatever the atlas background is, which bleeds as a wrong colour rather than as a neighbouring chart’s colour.

The required width is a function of the filter support and the mip depth. A bilinear tap at level $n$ spans two texels of that level, so a fragment whose coordinate sits on the chart’s boundary reads up to one texel of level $n$ beyond it. One texel of level $n$ is $2^n$ texels of level 0, so a gutter of $g$ base-level texels covers every level for which

$$2^{,n} \le g$$

Above that level, the footprint of a single tap crosses into the neighbouring chart. A full chain from a 1024×1024 base runs to $n = 10$, which demands a gutter of 1024 base-level texels, the entire width of the atlas, so a full chain is never compatible with a gutter at all. Real pipelines therefore cut the chain at a level the gutter can cover and clamp the sampler’s maximum LOD so that no fragment reaches the levels above it.

The practical compromise: choose a gutter of a small fixed number of texels at level 0, commonly 4 to 16, dilate the chart into it, and limit the mip chain to the level that gutter still covers. A 4-texel gutter holds to level 2, so a 1024 atlas stops at 256×256; a 16-texel gutter reaches level 4, stopping at 64×64. The surface then loses the far-distance mips, and at a distance the atlas aliases. A larger gutter buys deeper mips at the cost of smaller charts and therefore lower base resolution. That is the axis: gutter width against usable area against maximum mip depth, and all three cannot be maximized.

Dilation, and the difference between copying and extrapolating

Dilation takes each texel in the gutter and assigns it the value of the nearest chart texel, repeated outward. Implemented naively, that is a distance transform over the atlas: for each gutter texel, find the nearest filled texel and copy it.

A simple iterative form is adequate and is what most tools use:

/* Grow chart content outward by exactly one texel.
   src_mask: 1 where the pixel belongs to a chart.
   Copies from src into dst, then reports the newly filled pixels. */
static void dilate_pass(const unsigned char *src_rgba,
                        const unsigned char *src_mask,
                        unsigned char *dst_rgba,
                        unsigned char *dst_mask,
                        int w, int h)
{
    memcpy(dst_rgba, src_rgba, (size_t)w * h * 4);
    memcpy(dst_mask, src_mask, (size_t)w * h);

    for (int y = 0; y < h; y++) {
        for (int x = 0; x < w; x++) {
            size_t i = (size_t)y * w + x;
            if (src_mask[i]) continue;              /* already filled */
            for (int dy = -1; dy <= 1; dy++) {
                for (int dx = -1; dx <= 1; dx++) {
                    int nx = x + dx, ny = y + dy;
                    if (nx < 0 || ny < 0 || nx >= w || ny >= h) continue;
                    size_t j = (size_t)ny * w + nx;
                    if (!src_mask[j]) continue;
                    memcpy(dst_rgba + i * 4, src_rgba + j * 4, 4);
                    dst_mask[i] = 1;
                    goto next;                      /* one source is enough */
                }
            }
        next:;
        }
    }
}

Two properties of that structure deserve attention. Reading only from src and writing only to dst is what makes one pass grow the content by exactly one texel in the eight-connected sense; writing in place would let a texel filled early in the scan feed a neighbour later in the same pass, producing a directional smear along the scan order. And because growth is one texel per pass, a gutter of width $g$ needs $g$ passes with two buffers swapped between them, each pass costing a full image traversal. For a 2048×2048 atlas and a 16-texel gutter that is 67 million texel visits and two full-image copies per pass, which is why production tools use a jump-flood or a two-pass chamfer distance transform instead of iterating.

Dilation by copying the nearest texel is not the same as extrapolating the chart’s gradient, and for some content the difference is visible. A chart whose content ends in a strong gradient, such as a normal map where the encoded vector varies continuously, copies the boundary value outward as a constant, so the mip-averaged value at the seam is pulled toward the edge value rather than continuing the trend. The standard fix is to encode the chart with a small border of extrapolated content in the source, or to generate mips before dilation so that the boundary of each level is averaged from real content. The second is what most pipelines do, and it requires that mip generation run on the un-dilated atlas, which means the atlas must be laid out so that each chart’s mip footprint does not overlap its neighbours — the same gutter calculation, applied at each level.

The transform from chart space to atlas space

Placing a chart means applying a similarity transform to its UVs: uniform scale, rotation by a multiple of 90 degrees, translation. Recording it as a per-vertex UV rewrite is the simple option, and it makes the atlas coordinates indistinguishable from authored ones, which is convenient for the runtime and inconvenient for anyone who later wants to change the atlas.

The alternative is to store the transform separately, as a per-chart scale, rotation, and offset, and apply it in the vertex shader or in the loader. This costs one extra transform per vertex and makes repacking cheap, because the charts keep their local UVs and the atlas is rebuilt by changing a table. For a mesh with a few hundred charts the table is small, and the benefit is that a re-pack at a different resolution or with a different gutter does not require touching the mesh data.

The rotation deserves a specific note. Rotating a chart by 90 degrees is not a similarity transform on the parameterization in the sense of preserving orientation: it reverses the winding of the triangles in UV space. For most content that is harmless, but any operation that relies on the UV winding flips with it, including a signed-area test for back-facing detection in a decal projection and a tangent basis derived from UV derivatives. The tangent basis in particular must be rebuilt after rotation, because the tangent and bitangent are defined by the direction of increasing $u$ and $v$, and a chart rotated 90 degrees has those directions swapped and one negated.

Failure modes that survive validation

Atlas packing fails in ways that no validator catches, and knowing the symptoms shortens the search.

A chart that touches the atlas border has no gutter on that side. Bilinear taps at the border read the clamp or border colour, and under a repeat wrap mode they read the opposite edge of the atlas, which is a completely unrelated chart. Border-adjacent charts must either be inset by the full gutter or be paired with a border wrap mode, and the choice must match what the sampler state says.

Overlapping charts are geometrically legal in UV space and produce content from one chart bleeding into another only where the bounding boxes are placed adjacently, so the error looks like localized contamination rather than a global failure. A packer that works with rotated bounding boxes must test the rotated shapes, not the axis-aligned boxes, or charts whose boxes overlap in one orientation slip through.

Texel density mismatch across charts produces a visible change in apparent sharpness at the seams, and it is caused upstream in the unwrap, not in the packer. Measuring the per-chart scale factor relative to surface area and reporting the spread is the diagnostic; a spread of more than a few tens of percent is usually visible.

And the atlas dimension interacts with the texture format. A non-power-of-two atlas cannot use the full mip chain without rounding at each level, and a block-compressed atlas needs its dimensions to be multiples of the block size or the edge blocks are padded, which changes the accounting described in block-compressed texture bit rates. The channel semantics of the packed result are a further constraint on what the atlas can hold, treated in channel packing conventions.