Rasterization Rules: Pixel Centers and the Top-Left Fill Convention

A rasterizer converts a triangle described by three floating-point vertices into a set of fragments, each carrying a pixel coordinate and a set of interpolated values. The obvious implementation — decide which pixels the triangle’s area overlaps — is the wrong one. Area overlap is ambiguous on every shared edge: two triangles that meet along a diagonal both touch the pixels on that diagonal, and a coverage test based on overlap will either include those pixels twice or, if the test is asymmetric, exclude them from both. The standard solution is to stop thinking about areas and test a single point per pixel instead.

One sample point, not one pixel square

Every pixel owns exactly one sample location for the purpose of single-sample rasterization: the pixel center, at coordinates (x + 0.5, y + 0.5) for the pixel whose lower-left corner is at integer (x, y). A fragment exists for a pixel if and only if that point lies inside the triangle. The rest of the pixel’s area is irrelevant.

This choice is what makes the pipeline deterministic. Coverage of a triangle no longer depends on its shape at sub-pixel scale, only on which sample points fall inside it. A triangle whose edge passes a thousandth of a pixel from a sample point either includes that point or does not, and the answer is stable across frames and across implementations that follow the same rules.

The cost is geometric: a triangle smaller than a pixel may cover no sample point at all and disappear, and a triangle covering a single pixel produces one fragment regardless of how much of the pixel it fills. Both behaviours are required, not defects. The second is why the derivative calculations used for texture filtering are defined per 2×2 group of fragments rather than per fragment, as quad shading and helper invocations describes.

Sample positions are not required to be at the center in multisampled configurations. With four samples the rasterizer tests four distinct points per pixel, and the count of covered points becomes the coverage mask. The fill rule described below applies identically to each sample position, which is why a multisampled edge produces a partially covered pixel rather than a binary in-or-out verdict; multisample coverage and resolve takes that up.

Edge functions and the half-space test

The inside test is expressed with edge functions. For an edge from vertex a to vertex b, and a point p:

E(p) = (b.x - a.x) * (p.y - a.y) - (b.y - a.y) * (p.x - a.x)

This is the signed area of the parallelogram spanned by the edge vector and the vector from a to p, doubled. It is zero exactly on the line through a and b, positive on one side and negative on the other. A point is inside a triangle when all three edge functions agree in sign.

The sign convention depends on the coordinate system and the winding order, and getting it wrong is the single most common rasterizer bug: the triangle appears with its inside and outside exchanged, which for a convex triangle looks like nothing at all until culling is enabled. A reliable approach is to compute the signed area of the triangle once, flip the sign of all three edge functions if the area is negative, and then treat non-negative as inside. That normalisation makes the rest of the code independent of vertex ordering.

/* Coverage test for one sample point, with the top-left rule applied.
   Screen space: y grows downward, interior is where every edge
   function is >= 0 after normalisation by the triangle's signed area. */
#include <stdbool.h>
#include <stdint.h>

typedef struct { int32_t x, y; } point;   /* fixed point, 1/256 pixel units */

typedef struct { point a, b; } edge;

static int64_t edge_fn(edge e, point p)
{
    int64_t dx = (int64_t)e.b.x - e.a.x;
    int64_t dy = (int64_t)e.b.y - e.a.y;
    return dx * ((int64_t)p.y - e.a.y) - dy * ((int64_t)p.x - e.a.x);
}

/* An edge is top-left when the interior lies below it (a horizontal edge
   running right in x) or to its right (a non-horizontal edge running
   upward, which in a y-down screen space means dy < 0). */
static bool is_top_left(edge e)
{
    int64_t dx = (int64_t)e.b.x - e.a.x;
    int64_t dy = (int64_t)e.b.y - e.a.y;
    return dy < 0 || (dy == 0 && dx > 0);
}

bool covers(const edge e[3], point p)
{
    for (int i = 0; i < 3; i++) {
        int64_t v = edge_fn(e[i], p);
        if (v < 0) return false;
        if (v == 0 && !is_top_left(e[i])) return false;  /* on a non-owned edge */
    }
    return true;
}

The zero case is where the fill convention lives. Without it, a sample exactly on a shared edge is included by both triangles, and with alpha blending the seam darkens or lightens depending on the blend factors. With an asymmetric rule that assigns each edge to exactly one of the two triangles that share it, the sample is included exactly once.

The top-left rule

The rule is stated in terms of the edge’s orientation as seen by the triangle’s interior. An edge is a top edge if it is horizontal and the interior is below it; an edge is a left edge if it is not horizontal and the interior is to its right. A sample that falls exactly on a top or left edge is inside the triangle; a sample on any other edge is outside.

The asymmetry is what makes it work. Every shared edge is a top or left edge for exactly one of the two triangles that own it, because the two triangles see the edge from opposite sides. A horizontal edge shared by a triangle above and a triangle below is a top edge for the lower triangle and a bottom edge for the upper one. A slanted shared edge is a left edge for the triangle on its right and a right edge for the triangle on its left. So exactly one triangle claims each boundary sample, no matter how the mesh is triangulated.

Two consequences follow. First, the rule is only correct if both triangles are wound consistently — the interior-side determination must agree between them. A mesh with mixed winding produces seams even though each triangle is individually covered correctly, which is why winding order matters for reasons beyond backface culling, as winding order and backface culling discusses.

Second, the rule must be evaluated on the same arithmetic the coverage test uses. If the edge function is computed in floating point and compared against zero, two triangles sharing an edge may compute slightly different values for the same sample point, because the two computations traverse the edge in opposite directions and round differently. The sample then lands on the wrong side of zero for one triangle and is dropped by both, leaving a one-pixel crack along the seam. Integer or fixed-point edge functions eliminate this: the same edge evaluated in either direction produces exactly negated values, so an exact zero is exact for both.

Subpixel precision and its limits

Vertices arrive as floating-point screen coordinates but are not tested at that precision. Implementations snap them to a fixed subpixel grid before rasterizing. OpenGL requires at least four bits of subpixel precision, exposed as the implementation’s subpixel bits; Direct3D’s rasterization rules specify a fixed grid with several fractional bits. Snapping is not an optimisation detail — it is part of the contract, because the fill convention’s guarantees only hold on a grid where exact zeros are representable.

The grid has a visible cost at the small end. A vertex at x = 10.0001 and one at x = 10.0002 may snap to the same grid position, collapsing a thin triangle to zero area. A triangle whose three snapped vertices are collinear covers no samples and produces no fragments; it is not an error, and implementations are permitted to discard it early. Geometry that is thin relative to the subpixel grid therefore flickers in and out of existence as it moves, which is the problem conservative rasterization for thin geometry addresses by changing the test from “is the sample inside” to “does the triangle touch the pixel at all”.

Snapping also interacts with clipping. Vertices outside the view volume must be clipped before rasterization, because a vertex at a large negative w projects to coordinates far outside any representable grid range, and the edge functions for a triangle with one such vertex overflow fixed-point arithmetic. The standard remedy is a guard band: a rectangle larger than the viewport, inside which vertices are left unclipped and rasterization proceeds normally, with fragments outside the viewport discarded by the scissor test. Homogeneous clipping and the guard band works through where the guard band’s edges can be placed.

What the rules do not promise

The rules fix coverage, not order. Fragments produced by one triangle are ordered relative to each other in an implementation-defined way, and fragments from different triangles have no ordering guarantee at all beyond what the depth test and blending impose. A rasterizer that walks pixels in a different order produces identical results only because the per-fragment operations are order-independent for opaque geometry, which stops being true the moment blending is enabled.

The rules also do not promise that a fragment’s interpolated attributes are exact at the sample point. Interpolation is done in floating point from the triangle’s vertex attributes, and two triangles sharing an edge interpolate that edge from different vertex pairs, so a value evaluated on the seam can differ in the last bits. That is the same discontinuity that motivates watertight rasterization in offline renderers, and on the GPU it surfaces as hairline shading differences along shared edges rather than as missing pixels.

Finally, the fill convention says nothing about which side of a nearly edge-on triangle is visible. A triangle whose projected area is a few grid units wide may cover a sample for one frame and not the next as it rotates, and no amount of care in the coverage test removes that aliasing. It is a consequence of point sampling, and the only fixes are more samples per pixel or a coverage-based test that asks a different question.