Channel Packing: Which Data Belongs in Which Channel

A physically based material needs several spatially varying scalars: a roughness value, a metallic mask, sometimes an ambient occlusion term, sometimes a height. Each is a single channel of data. Storing each in its own grayscale texture means four texture fetches, four cache lines, four mip chains, and four sampler units for what is often four sets of numbers describing the same surface at the same texel. Packing them into the four channels of one RGBA texture collapses that to one fetch. The catch is that not every permutation of assignments is equivalent, because the channel decides what compression does to the values and what filtering does at the boundaries.

The channel decisions that matter

Four constraints drive the layout, and they pull in different directions.

Compression treats luminance and chroma differently. When a GPU format stores a colour texture, it allocates more bits to perceptual brightness than to the two chroma channels, on the assumption that the eye is less sensitive to colour detail. When the same format stores data, that asymmetry becomes noise applied to whatever is in each channel. A packed texture holding roughness in the channel that receives fewer bits loses precision where a grayscale format would not have.

Filters act on channels independently. Bilinear reconstruction averages each channel separately, which is what a scalar field wants. But some values are not meant to be averaged. A mask with values exactly 0 or 1 that gets averaged at an island boundary yields intermediate values, and a shader that branches on “greater than 0.5” sees a band of arbitrary width along the seam. The band’s width in texels is a property of the filtering, not of the authored data.

The green channel has a pre-existing convention. Normal maps store a vector in tangent space, and there are two widespread conventions for the $y$ component’s sign. Whichever one an authoring tool emits, the importer must be able to flip it. If green is occupied by roughness, the flip cannot be expressed as a channel permutation and must be a per-texture decode step, which is more state to carry. This is why the packed layout that leaves green for the normal and uses red and alpha for other scalars appears often enough to have become a de facto shape.

Alpha may not be free. In a texture whose alpha is unused, the fourth channel is available for packing. In a texture that is also the base-colour map with alpha blending, it is not. The two uses conflict, and a pipeline that assumes alpha is always spare will eventually pack a mask into a transparency channel and produce surfaces that dither.

The layouts that are actually used

Three arrangements recur. Making the reasoning explicit shows why each exists.

The first is the ORM shape: red occlusion, green roughness, blue metallic. It matches the channel meanings of a common shader convention, keeps all three inside the RGB triple where sRGB and linear decode flags apply uniformly, and leaves alpha for something else or for nothing. Its weakness is that all three scalars now live in a colour-filtered texture, and if the format is a block-compressed colour format, the chroma channels carry less resolution — roughness in green is at the mercy of the chroma allocation, and for masks that is visible as blotching across smooth gradients.

The second is roughness in alpha. Here red and green hold something else, frequently a normal map’s $x$ and $y$ tangent components, and roughness rides in alpha where it is often stored at full precision even in block formats that subsample colour. This shape also lets a shader read roughness from a texture it was already sampling, so the material needs one fewer binding. The cost is dependency: the roughness is no longer available independently of the normal, so a surface that needs roughness but not normal detail — a decal, a glass pane — pays for a normal fetch it does not use.

The third is a single-channel texture per scalar, which is the layout every packing scheme is trying to avoid. It has one real advantage: a single-channel format stores that channel at full resolution, so a value that must survive filtering without contamination from neighbours in other channels keeps its precision. For a height field used for parallax or displacement, this often wins outright, because the height is read with derivatives and any chroma subsampling becomes a visible terraced surface.

Layout	Channels used	Strength	Failure mode
ORM	R, G, B	One fetch, uniform decode	Chroma subsampling dulls roughness
Roughness in alpha	A (plus RG)	Full-precision roughness	Coupled to normal fetch
Separate masks	One texture each	No cross-channel damage	Four fetches per shading point

Why a scalar field does not want sRGB decode

A packed texture must be tagged with the colour-space convention that says how to decode its samples, and the tag applies to the whole texture, not per channel. Base colour is authored in sRGB and must be linearized before lighting math, because the values were chosen perceptually. Roughness, metallic, occlusion, and height are not perceptual encodings; they are physical fractions that happen to be stored in an eight-bit channel. Passing them through an sRGB-to-linear conversion maps $0.5$ to about $0.214$, which shifts every material’s roughness by a large factor and is invisible on a matte surface but obvious on a polished one.

The inverse mistake is equally common: a base-colour texture tagged linear renders as washed out and undersaturated, because the values are being treated as if they were already linear light.

The tag is therefore per texture and applies to all four channels, which is exactly why base colour and roughness cannot share a texture in a PBR pipeline without one of them being wrong. A pipeline that wants one fetch for both must decode in the shader — sample, convert the colour channels and not the data channels. That works with a texture tagged linear, costs one extra set of multiply-and-power operations per fetch, and is the only arrangement that preserves correctness for both kinds of data in one binding. The alternative used by many engines is to keep the base colour separate and accept the second fetch.

Eight bits per channel is the other constraint. A roughness value quantized to 256 steps is fine for most surfaces, since the perceptual difference between adjacent steps is small at typical viewing angles. It is not fine for a value that gets multiplied after interpolation, or for one used as a divisor. Metallic and occlusion are effectively masks and tolerate coarse quantization; a height field used for steep parallax does not.

Building the packed texture without breaking the seams

The assembly step is where packing goes wrong in practice. Compositing four grayscale images into RGBA requires that all four have identical dimensions and identical UV alignment. A pipeline that resamples one channel to match another invents filtered values for data that was authored as masks, and any slight offset between the sources becomes a visible misregistration in the shading rather than in the image.

A more reproducible approach builds the packed texture from the shader-side definition rather than from the images. If the shader reads roughness from texture(orm, uv).g, the packing step is a function that takes the four source fields and emits the four output channels, and the parameters of that function are recorded as part of the material definition. Then a change to the convention is a code change in one place and re-running the packer, rather than an asset edit repeated across a library.

The packing operation itself, in terms of the inputs, is a channel permutation with an optional invert:

/* Compose four single-channel sources into one RGBA texture.
   src[k] holds width*height bytes; invert is a per-output flag
   because occlusion is sometimes authored inverted. */
static void pack(const unsigned char *const src[4], const int invert[4],
                 unsigned char *dst, size_t npixels)
{
    for (size_t i = 0; i < npixels; i++) {
        for (int c = 0; c < 4; c++)
            dst[i * 4 + c] = invert[c] ? (unsigned char)(255 - src[c][i])
                                       : src[c][i];
    }
}

Two properties of that loop matter downstream. It does no resampling, so a caller must reject inputs whose dimensions differ rather than scaling them. And it is a pure per-pixel map, so it is safe to run before or after mip generation only if the mip filter is also per-channel — which it is, which means mip levels of a packed texture are exactly the pack of the mip levels of its sources only when the pack function is linear. An invert is linear; a gamma conversion is not, and packing an sRGB-encoded scalar into a mip chain produces levels that differ from re-packing a separately downsampled image.

Mip generation for packed data has one more wrinkle. A block-compressed colour format’s chroma channels are computed from an ellipsoid fit over the block’s colour values, which assumes the data behaves like colour. Applying that to a roughness field in one channel and metallic in another works arithmetically but spends bits on a model of perceptual error that does not apply. The alternative is a format that treats all channels alike — the block formats designed for normals and other data, discussed in block-compressed texture bit rates.

The normal map interaction, and why the flip matters

A tangent-space normal map encodes $(x, y)$ into two channels and reconstructs $z$ from the unit-length constraint, $z = \sqrt{1 - x^2 - y^2}$. Which channel holds which is a convention, and the $y$ sign convention differs between authoring tools. The two cases are commonly called OpenGL-style, where $+y$ points up in texture space, and Direct3D-style, where it points down. Neither is better; they differ by whether the image’s row order was treated as bottom-up or top-down when the encoding tool captured the surface.

Wrong flip produces lighting that is inverted along the vertical axis only: a bump that should catch light from above is lit as though it were a dent. Horizontal features are unaffected, which is why the error can survive inspection on the wrong model.

Because the reconstruction depends on the sign, a pipeline that packs the normal’s $y$ into green and uses red and alpha for other scalars cannot fix the flip by permuting channels at load; the flip must be applied to the fetched value before reconstruction, or the source texture must be rewritten. A pipeline that packs the normal’s $xy$ into red and green can apply the flip as a simple negate of one sampled component with no repacking.

The reconstruction itself has a numerical edge: near the silhouette of a strong bump, $x^2 + y^2$ approaches one and the square root’s derivative goes to infinity, so quantization in $x$ and $y$ produces large errors in $z$. Clamping the sum slightly below one avoids a negative radicand and the resulting NaN, at the cost of flattening the steepest normals. Combined with the renormalization that any quantized normal needs, this is the practical limit on how steep a normal map can encode.

Where the decision belongs

Channel assignment is a contract between an asset and the shader that reads it, and the contract has to live somewhere machine-readable. The two options are in the texture metadata — which channel means what is not expressible in the container formats, which carry channel counts but not semantics — or in the material definition that references the texture. Material definitions in glTF express it by naming a dedicated property per map; formats without that vocabulary express it by convention plus documentation, which is the fragile case. A packed texture delivered without its convention is not portable: the values load, the shader runs, and the material is wrong in a way that no validation catches.

The related conditioning problem — deciding which surfaces share a texture and how much empty space the atlas wastes — changes the texel density available to every channel and therefore how much quantization the packed values can tolerate; that layout question is worked through in UV chart packing and gutter space. And the container that a packed texture is ultimately delivered in decides whether the per-channel compression asymmetry described here applies at all, which is where KTX2 containers and Basis transcoding takes over.