Exact Area Coverage and the A-Buffer

Sampling a pixel at $n$ points approximates the covered fraction as $k/n$, a rational with denominator $n$. For a pixel straddling an edge, that fraction is the only thing the image needs: the correct colour is a mix of the two sides weighted by the area. Computing the area directly removes the quantization entirely, and for the common case of a triangle edge crossing a rectangle, the computation is not hard. The difficulty is not in the area formula but in knowing which surfaces cover which parts of the pixel, which is where the A-buffer enters.

Area under a single edge

Take a pixel as the unit square $[0,1]^2$ and a single line crossing it, defined by the same edge equation used for coverage: $E(x,y) = ax + by + c$. The portion of the square where $E \ge 0$ is a polygon whose area can be computed in closed form. The integral of the coverage over the square is

$$A = \int_0^1 \int_0^1 H(E(x,y)), dx, dy$$

where $H$ is the step function. Because $E$ is affine, the region $E \ge 0$ inside a convex polygon is a convex polygon, and its area follows from clipping the square against the line and applying the shoelace formula to the result. For a single edge this is at most five vertices and a handful of multiplies, with no iteration and no sampling error.

/* Covered area of a unit square by the half-plane a*x + b*y + c >= 0.
   Clips the four corners against the line, then sums cross products. */
function coverageArea(a, b, c) {
  const pts = [[0,0],[1,0],[1,1],[0,1]];
  const out = [];
  for (let i = 0; i < 4; i++) {
    const p = pts[i], q = pts[(i + 1) % 4];
    const dp = a * p[0] + b * p[1] + c;
    const dq = a * q[0] + b * q[1] + c;
    if (dp >= 0) out.push(p);
    if ((dp >= 0) !== (dq >= 0)) {
      const t = dp / (dp - dq);
      out.push([p[0] + t * (q[0] - p[0]), p[1] + t * (q[1] - p[1])]);
    }
  }
  let area = 0;
  for (let i = 0; i < out.length; i++) {
    const p = out[i], q = out[(i + 1) % out.length];
    area += p[0] * q[1] - q[0] * p[1];
  }
  return Math.abs(area) / 2;
}

For a triangle, coverage is the intersection of three such half-planes, and the area is computed by clipping the square against all three in turn. The result is exact up to floating-point rounding of the intersection parameters, and it is exact in the sense that no sampling grid intervenes. The cost is a fixed amount of work per pixel touched by the triangle’s boundary plus a trivially full or empty result for pixels entirely inside or outside, and the difficult part is deciding which case a pixel is in without doing the clipping work. Testing the four corners against all three edges classifies a pixel as fully inside, fully outside, or mixed; only the mixed case needs the area computation.

This is the mechanism behind analytic antialiasing. Its output is a fractional coverage value in $[0,1]$ per pixel, and the blend that follows is a single lerp between the two sides. A line rendered this way has a smooth, continuous edge whose position is represented with the precision of a float rather than the precision of a sample grid, which is why the technique produces visibly better thin-line and near-horizontal-edge rendering than any modest sample count. The point-sampled coverage test is the degenerate case where only the corners of the classification matter.

Where the single-edge model breaks

Fractional coverage assumes that the pixel’s colour is a mix of exactly two values: the fragment and whatever was behind it. That holds when one triangle crosses the pixel edge and the rest of the pixel is covered by background or by a single surface. It fails when two different surfaces both partially cover the same pixel. Two edges crossing the pixel divide it into regions with potentially three or four different colours, and a single coverage fraction cannot represent that: the pixel has a colour for each region, and the correct output is the area-weighted average of all of them, not a two-way mix.

The failure is not dramatic on a smooth silhouette, where the neighbouring regions are often the same object. It is severe where geometry overlaps: foliage, wire fences, text rendered from outlines, and any scene with many thin, overlapping surfaces. A pixel at such a junction may contain pieces of five surfaces, and a two-value blend picks one of them and blends it with the background, producing an edge that jumps between candidates as the camera moves.

Solving it requires storing, per pixel, a list of fragments with their coverage and depth, so that the resolve can composite them in depth order, each contributing its own area. That is the A-buffer.

The A-buffer’s structure

Carpenter’s A-buffer accumulates per pixel an unordered list of fragments, each carrying colour, depth, and a coverage mask or geometry description. Because the list is built by however many triangles happen to touch the pixel, in whatever order they arrive, the resolve step must sort them by depth and composite front to back, accumulating each fragment’s contribution scaled by the fraction of the pixel not yet covered.

/* Resolve an A-buffer pixel: fragments sorted front to back, each with an
   area fraction in [0,1]. Remaining transparency starts at 1. */
typedef struct { float r, g, b, depth, area; } frag_t;

void resolve_pixel(const frag_t *f, int n, float out[3])
{
    float remain = 1.0f, acc[3] = {0, 0, 0};
    for (int i = 0; i < n && remain > 0.0f; i++) {
        float w = f[i].area;                 /* how much of the pixel this covers */
        if (w > remain) w = remain;          /* cannot exceed what is left */
        acc[0] += f[i].r * w;
        acc[1] += f[i].g * w;
        acc[2] += f[i].b * w;
        remain -= w;
    }
    /* Whatever is left is background; assume it contributes the clear colour. */
    out[0] = acc[0];
    out[1] = acc[1];
    out[2] = acc[2];
}

The per-fragment coverage is stored either as an explicit geometric description or, in the modern compressed variant, as a small fixed number of sample bits with the fragment’s area inferred from them. A 4-bit mask gives 16 coverage levels, recovering much of MSAA’s behaviour while keeping multiple fragments. A fully geometric representation keeps exact area and arbitrary edge counts, at the cost of variable-length records and the sort.

Depth ordering is the hard part. If fragments arrive in arbitrary order, the list must be sorted before compositing, which is a per-pixel sort of a variable-length list, and that is not a shape amenable to wide GPU execution. If the rasterizer guarantees front-to-back order via a binning pass, the list need not be sorted, and the resolve can composite as it goes. The tiled rasterizer is what makes that guarantee affordable, and it is why the modern implementations of the idea are described in terms of tile-local sorting rather than per-pixel sorting.

Memory, and why the GPU version is bounded

Carpenter’s original A-buffer allowed linked lists of fragments in a software rasterizer, with a scheme for coalescing adjacent fragments that share a surface. The memory is unbounded in the worst case: a pixel crossed by a hundred thin triangles gets a hundred fragments, and the storage grows with scene complexity rather than with resolution. On a GPU, where the framebuffer must be a fixed allocation sized at resource creation, unbounded per-pixel lists are impossible without a fallback.

Modern hardware therefore fixes a maximum per pixel, typically a small number such as four or eight fragments, and handles overflow by either dropping the farthest fragment or by merging. Dropping produces the wrong answer whenever the pixel genuinely needs more fragments than the cap, and the artifact is a missing surface in a dense overlap. Merging produces an approximation. Both are acceptable in practice because the case is rare and the alternatives are worse, but the cap is a real limit and it is why hardware A-buffer schemes are described as approximate antialiasing rather than exact.

The storage per pixel is the product of the fragment cap, the per-fragment payload, and the pixel count. A cap of eight with 16 bytes per fragment is 128 bytes per pixel, which for a 1920×1080 target is over 250 MB before the resolve output. This is the arithmetic that keeps the exact approach confined to offline renderers, where the frame is rendered once and the budget is measured in hours rather than milliseconds, and to hardware in a compressed form that trades the cap for residency.

What the exact approach buys and what it cannot do

The visible benefit of area coverage is a silhouette whose edge position is correct to a fraction of a pixel and whose slope is not quantized to the sample pattern. A second benefit is that thin geometry behaves correctly: a line of one pixel width stays one pixel wide and does not shimmer between one and two pixels as it moves, because its coverage is computed rather than sampled. Thin-feature stability is where the difference between sampling and exact coverage is most obvious, and it is the reason text and vector graphics pipelines generally avoid MSAA and implement analytic coverage themselves.

The technique cannot resolve shading aliasing any more than MSAA can. Its coverage is geometry coverage. A high-frequency texture or a sharp specular highlight still aliases within the fragment, and the fix for that remains supersampling or a filtered lookup. Likewise, opacity is handled only if the fragment’s coverage and its alpha are distinguished: a semi-transparent surface’s contribution is area times alpha, and conflating the two produces the same doubled edge that multisampling exhibits.

The last limitation is assembly cost. The area computation needs the triangle’s edge equations in a normalized form that supports intersection parameters at a reasonable precision, and the clip-against-square operation must be done per edge rather than per sample. In a hardware pipeline this is a fixed-function unit of moderate complexity; in a software rasterizer it is straightforward but the per-pixel branching between the full, empty, and mixed cases is what limits throughput. The classification test is the part that keeps it fast, and it is the same corner test used to reject a triangle’s bounding box one level up.