From Joint Hierarchy to Skinning Palette

A skinned mesh carries vertices in a neutral shape and a set of joints that move them. The joints form a tree, each joint stores a transform relative to its parent, and the renderer needs one matrix per joint that maps a vertex from the neutral shape into its animated position. Producing those matrices is a traversal of the tree combined with a per-joint constant, and the constant is the part that confuses people: it is an inverse of the joint’s bind transform, stored in the file rather than derived at runtime, and its presence explains why a skeleton can be re-bound and why a mesh with wrong bind data collapses toward a point.

The bind pose is a coordinate system, not a rest animation

A vertex is authored in mesh space. A joint has its own space: the space in which the joint sits at the origin with its axes aligned to whatever convention the rigger chose. Skinning requires moving a vertex from mesh space into joint space, applying the joint’s current world transform, and moving back out. The middle step is what animates; the two outer steps are the bind transformation. Composing them, the matrix that a renderer actually needs is

$$M_j = W_j \cdot B_j^{-1},$$

where $W_j$ is the joint’s current world transform and $B_j$ is its world transform in the bind pose. At the bind pose $W_j = B_j$, so $M_j$ is the identity and the mesh renders in its authored shape. Away from the bind pose, $M_j$ is the deformation.

The reason the file stores $B_j^{-1}$ explicitly rather than reconstructing it is that the bind pose of a joint is not recoverable from the joint’s rest transform. A rig can be bound with joints placed at the origin and offset by geometry, or bound with joints at the mesh’s surface; the same rest positions with different bind matrices produce different deformations. glTF makes this explicit: a skin carries an inverseBindMatrices accessor holding one 4×4 matrix per joint, and the specification states that the joint’s world transform is multiplied by that inverse. A reader that ignores the accessor and uses the joint’s rest transform instead gets a mesh that renders correctly at rest only if the two happen to coincide, and shears everywhere else.

/* Palette entry for joint j: world transform times inverse bind matrix.
   Matrices are column-major 4x4, the convention glTF accessors use. */
static void build_palette(const mat4 *world, const mat4 *inv_bind,
                          int joint_count, mat4 *palette) {
    for (int j = 0; j < joint_count; j++)
        palette[j] = mat4_mul(world[j], inv_bind[j]);
}

/* World transforms by a single pre-order pass; parents precede children
   in the joint array, which glTF requires for the skin's joint list. */
static void compute_world(const int *parent, const mat4 *local,
                          int joint_count, mat4 *world) {
    for (int j = 0; j < joint_count; j++) {
        if (parent[j] < 0) world[j] = local[j];
        else               world[j] = mat4_mul(world[parent[j]], local[j]);
    }
}

That second loop is the whole skeleton evaluation, and it is where the topology requirement comes from: world[parent[j]] must already be computed, so the joint array must be ordered such that a parent appears before its children. Depth-first order satisfies this and is what most exporters emit. An arbitrary order needs a topological sort once at load, or a recursion, and the iterative version is preferred because a deep chain of joints can exceed a small stack budget when the poses are evaluated per frame.

What composition order means in practice

The local transform of a joint is a rotation, a translation, and usually a scale, combined as translate-then-rotate-then-scale in the joint’s own frame. The choice of order is a convention, and glTF fixes it: a node’s matrix is built as $T \cdot R \cdot S$, applied to a column vector, so the scale acts first, then the rotation, then the translation. Reversing translate and rotate puts a joint’s offset in the rotated frame, which rotates the entire sub-hierarchy about the joint’s origin instead of around the parent’s. The symptom is unmistakable and common in imported data: a limb that orbits its parent rather than bending at the joint.

Non-uniform scale along a chain is the harder case, and it does not have a clean answer. A parent’s non-uniform scale propagates into every child’s world matrix, so a child that is uniformly scaled in its own local space becomes sheared in world space. Skinning a vertex with such a matrix is valid linear algebra and yields a sheared deformation, which is sometimes what a squash-and-stretch rig wants and is usually an authoring error. Some systems compensate by decomposing the world matrix and discarding the shear, which produces a different wrong answer: the joint’s orientation no longer matches the hierarchy, and the mesh separates at the seams between joints. The practical guidance is to keep scale uniform on joints that lie on a deformation chain, and to let the inverse bind matrix absorb whatever scale is present at binding time.

Rotations in the hierarchy are quaternions in glTF, and composing two quaternions is cheaper than composing rotations as matrices only when the composition is done once per joint. In a skeleton of sixty joints, per-frame world evaluation is sixty quaternion multiplies and sixty matrix multiplies, roughly a few thousand floating point operations. That is negligible next to the vertex work it enables, which is why skeleton evaluation is never the bottleneck and why the interesting costs sit in the palette’s consumers.

The palette is a texture, and its size is a render state

A palette is an array of matrices, and the shader indexes it by joint number. On an API that permits uniform arrays of matrices, the palette lives in a uniform buffer; on an API where uniform space per stage is limited, the standard alternative is to pack the matrices into a texture and sample it in the vertex shader with an index derived from the joint attribute. The texture path is not a workaround for convenience; it exists because a uniform buffer must be sized and bound per draw, and a large palette consumes a fixed slice of a resource that is shared with everything else in the shader.

The arithmetic of size is straightforward. A 4×4 matrix in 32-bit floats is 64 bytes; as three rows of four floats with the fourth row implied, it is 48 bytes; as three quaternion-plus-translation pairs, it is 28 bytes before padding and typically 32 with alignment. A skin with 100 joints therefore costs 6400 bytes as full matrices or 3200 as compact form. Those numbers are small in isolation and become a bandwidth question when the palette is re-uploaded every frame for every animated character, and when several hundred characters are drawn. This is the axis where keyframe reduction and palette compression trade against fidelity, and where the decision to shade with 1, 2, or 4 influences per vertex decides how much of the palette a vertex can touch.

Influences are the other half of the budget. A vertex with four joint indices and four weights costs 16 bytes in the common layout of four bytes of indices and four floats of weights, against 12 bytes for position and 12 for normal. That is more attribute data than the geometry itself, and the fourth influence is often unused: many meshes bind with one or two. Hardware and format limits vary widely, so a format that permits an arbitrary count and a runtime that clamps to a fixed maximum is the usual arrangement, with the clamped influences renormalized so the vertex does not shrink.

Palettes are cached, which makes staleness a real failure

A skeleton that does not move does not need a new palette. The obvious optimization is to skip evaluation and upload when nothing changed, and it is correct only if the invalidation is complete. Setting a joint’s local transform dirties that joint and every descendant; setting the skin’s inverse bind matrices dirties the whole palette; changing the mesh’s association with a different skin dirties it too. A cache keyed on the root node’s transform misses the descendant case, because a child can be animated while the root is untouched.

The failure mode when invalidation is wrong is not a crash but a pose that lags or freezes. A joint whose palette entry was computed for the previous frame applies the old deformation while its neighbours use the new one, and the mesh tears at exactly the boundary between them. The tearing is often mistaken for a skinning weight problem, which is why the symptom is expensive to diagnose.

A subtler staleness comes from double transformation. The palette already maps mesh space to world space; a vertex shader that applies the palette and then the mesh node’s world matrix transforms the vertex twice. In bind-pose terms the vertex ends up at $W_{\text{node}} \cdot W_j \cdot B_j^{-1}$, which is correct only if $W_{\text{node}}$ is identity or if $B_j$ was captured in the node’s frame. glTF keeps these separate by defining the skinned mesh node’s own transform as not applying to skinned vertices, and engines that ignore that rule produce meshes that shift by the node’s position whenever the node is moved.

Re-binding and the cases where it goes wrong

Binding a mesh to a skeleton is a correspondence problem: for each joint, the vertex positions must be expressed in that joint’s frame. When a mesh is bound at a different pose from the one the animation expects, the inverse bind matrices are wrong, and the compensation is per-joint: at the bind pose, a joint whose inverse is off by a translation moves its influenced vertices by that translation, and a mesh bound at the wrong pose therefore appears displaced in pieces rather than deformed smoothly. Re-binding recomputes the inverses from the joint’s current world transforms, which fixes the rest shape and invalidates any animation authored against the previous bind. That is why binding is a modelling step and not a runtime operation.

Degenerate cases are worth enumerating because they all produce the same visual: geometry pulled to a point. A zero joint weight leaves a vertex unaffected by animation until the mesh node moves, and if all weights are zero the vertex stays at its authored position while its neighbours move. A joint index out of range either reads garbage from beyond the palette or clips to zero depending on the shader’s clamp, and the safest runtime validation is at load time, not in the vertex shader. An inverse bind matrix that is singular cannot be inverted and often arrives as NaNs from an exporter that divided by zero scale; a NaN in one palette entry spreads to every vertex that joint influences, and then to the whole mesh once those vertices are drawn.

The palette is the junction where the curve evaluators on one side and the rasterizer on the other meet. Linear blend skinning is the consumer that turns palette entries and weights into vertex positions and introduces its own artifacts, and Inverse kinematics is the producer that writes local transforms into the hierarchy before evaluation runs.