Skinning, animation and morph targets on a Model

CNA snapshot 009d40f5  ·  Deep Dives › Models, glTF & 3D  ·  source links pinned to 009d40f5

A skinned or animated Model carries three independent kinds of motion: a joint palette driven by SkinningData and AnimationPlayer, rigid scene-node clips applied to the model's own bones, and CPU-blended morph targets on individual mesh parts. This page states the exact semantics of each at this snapshot — the three index spaces a glTF skin involves, the skeleton-root prefix that fixed the D8 coordinate-space defect, how clips are resampled at import, what the player validates, how morph weights are blended and evaluated — and the order in which a frame should apply them. It is for game code that animates imported models and for maintainers of the importer and the animation types.

Two skeletal systems, deliberately separate

CNA contains two skeletal-animation systems that do not share a runtime graph. SkinningData plus AnimationPlayer (AnimationPlayer.hpp) animate an ordinary Model; they mirror the classes of Microsoft's XNA Skinned Model Sample, which real XNA never shipped as framework code, and are CNAEXT. SkinnedModelEXT is the Avatar-oriented model, skeleton, clip and draw type, explicitly not built from ModelBone/ModelMesh/ModelMeshPart and loaded only from .skinnedmodel.json; its contract is on the avatars deep dive.

The two share their keyframe vocabulary on purpose: Keyframe and AnimationClip are using aliases of KeyframeEXT and AnimationClipEXT, so both paths use the same track, clip and content-loading code. What they do not share is sampling: AnimationPlayer.cpp's SampleTrack is a documented local copy of SkinnedModelEXT's, not a hidden conversion, because forcing the two public types together would couple two distinct asset contracts. Morph targets are a third, unrelated mechanism attached one level down, to mesh parts (see morph targets).

SkinningData on a Model

A skinned model carries its skeleton on Model::Tag, the Skinned Model Sample's convention (XNA's Model has no skinning property). SkinningData holds BoneCount, SkeletonHierarchy (a flat parent-index array in topological order, -1 for a root — the representation glTF and most exporters use, not a pointer tree), BindPose (bone-local), InverseBindPose (inverse of each bone's bind-pose world transform), SkeletonRootPrefix (below), AnimationClips keyed by name, and two CNAEXT fields, SkeletonRootNodeIndexEXT and SkeletonRootNameEXT, which record the rig root the file's skin.skeleton declares so an application can find it. Those two fields have no effect on any transform CNA computes, by design.

A glTF file may contain several skins. Direct loading keeps every one: Model::getSkinsEXTProperty() lists each skin's SkinningData and exactly the meshes its palette drives, and Tag aliases the first. Animate each skin with its own player and install its palette only on its own meshes; the loader's effect cache never lets two skins share one effect, so one palette cannot overwrite another. A model with rigid node animation and no skin carries ModelAnimationsEXT on Tag instead (see clips).

AnimationPlayer, precisely

The player holds a pointer to a SkinningData that must outlive it; it owns only the playback position and three matrix arrays. Its semantics, from AnimationPlayer.cpp:

• Construction copies the bind pose into the local array, sizes the world and skin arrays from BoneCount, and computes the bind pose immediately, so a freshly constructed player already returns a drawable palette.
• StartClip(clip) stores a pointer to the clip (which must also stay alive) and rewinds to zero.
• Update(time, relativeToCurrentTime, loop = true) adds time to the position or seeks to it. With a clip of positive duration, looping reduces the position modulo the duration on raw ticks and shifts a negative remainder into range (floor-modulo, so reverse playback wraps cleanly and a long session costs the same as a fresh one); without looping it clamps to [0, Duration]. With no clip, or a clip of zero duration, the position is held at zero.
• Recompute first requires SkeletonHierarchy, BindPose and InverseBindPose to have exactly BoneCount entries, else System::ArgumentException. It starts every bone at its bind pose, then samples each track whose key list is non-empty and whose bone index is in range; any other track is silently ignored. Sampling holds the first key before it and the last after it, and between keys interpolates translation and scale linearly and rotation spherically, with a linear search for the bracketing key. Composition runs in index order: a parent index greater than or equal to the bone's own index throws ArgumentException (which also rejects self-parenting and cycles); any negative parent means "root", not only -1.
• Outputs: GetBoneTransforms() is each bone's local transform, GetWorldTransforms() its model-space transform (what a weapon attached to a hand bone needs), and GetSkinTransforms() is InverseBindPose[i] * world[i] — the bone's delta from bind pose, which is what GPU skinning needs and what SetBoneTransforms expects.

The size check is not a complete validator for hand-built data. The constructor casts BoneCount to size_t to size its working vectors before the check runs, so a negative count requests an enormous allocation and typically fails with a standard allocation or length exception rather than the later ArgumentException. Content readers reject such counts (the .cnj skeleton reader accepts 0 to 100,000; glTF skins are capped at 72 joints), so validate a hand-built SkinningData as non-negative before constructing a player.

A worked two-bone example

The fixture in AnimationPlayerTests.cpp is the smallest rig that exercises hierarchy composition: bone 1 is a child of bone 0 offset by (0, 1, 0) in bind pose, and bone 0's only track moves it from the origin to (2, 0, 0) over one second.

SkinningData data;
data.BoneCount = 2;
data.SkeletonHierarchy = {-1, 0};                    // bone 0 is a root, bone 1's parent is 0
data.BindPose = {Matrix::getIdentityProperty(),
                 Matrix::CreateTranslation(Vector3(0, 1, 0))};
data.InverseBindPose = {Matrix::getIdentityProperty(), Matrix::getIdentityProperty()};

BoneTrackEXT track;
track.BoneIndex = 0;
track.Keys.push_back(Keyframe{System::TimeSpan::FromSeconds(0.0), Vector3(0, 0, 0)});
track.Keys.push_back(Keyframe{System::TimeSpan::FromSeconds(1.0), Vector3(2, 0, 0)});

AnimationClip clip;
clip.Duration = System::TimeSpan::FromSeconds(1.0);
clip.Tracks.push_back(track);
data.AnimationClips["Move"] = clip;

AnimationPlayer player(data);                        // data must outlive player
player.StartClip(data.AnimationClips["Move"]);

// Every frame:
player.Update(gameTime.getElapsedGameTimeProperty(), /*relativeToCurrentTime=*/true);
skinnedEffect.SetBoneTransforms(player.GetSkinTransforms());

A Keyframe defaults its rotation to identity and its scale to one, so these keys carry translation only. Half a second in, bone 0's world translation is (1, 0, 0) and bone 1's is (1, 1, 0): the child inherits the parent's motion through composition. The suite's other cases pin clip start, middle and end, clamping and wrapping, relative accumulation, the bind pose with no clip, the inverse-bind multiplication and both refusal rules; AnimationPlayerClampAndLoopTests.cpp adds negative-time wrapping, the zero-duration hold and end-equals-start for a looping clip.

Who installs the palette

Model::Draw sets only the ordinary world, view and projection matrices; it never installs a bone palette, and it rejects a mesh effect without IEffectMatrices. The application (or a helper) calls SetBoneTransforms on every SkinnedEffect or SkinnedPbrEffect the skin's meshes use before drawing. Model::CopyAbsoluteBoneTransformsTo is not a palette: its array is indexed by scene node, not by joint, and contains no inverse bind matrices.

One default matters. A skinned effect's palette starts as MaxBones identity matrices, which means "every joint matrix is the identity", not "no skinning": drawn that way, a glTF mesh is posed in joint space and its mesh-node cancellation never applies, so it renders visibly wrong rather than merely still. The glTF, .cnj and .cnb readers therefore call ApplyBindPoseBoneTransformsEXT(model, skinningData) at load, which computes the bind-pose palette with a clip-less AnimationPlayer and pushes it onto every matching skinned effect — only onto the meshes mapped to that skin when the model has a SkinsEXT mapping, and onto all skinned effects for older paths that have none. It returns the number of effects posed and holds no state; any later SetBoneTransforms simply overwrites it. Because loaded effects are shared by every copy and every cached load of the model, two characters drawn from one loaded model must each install their own palette immediately before their own draw.

Scene indices are not palette indices

The glTF scene is flattened into ModelBone indices, while a skin's joints array defines a separate list whose order need not be parent-before-child. BuildSkeleton (GltfImportCore.cpp) reorders the joints breadth-first into topological order, records an old-to-new map, and rewrites every packed JOINTS_0 value through it (refusing a result that would not fit a byte). A joint whose parent chain does not resolve within the joint set, or a joint listed twice, is refused.

  glTF node  --flatten-->  scene-node index  ==  ModelBone index (Model::Bones)  <--+
                                                                                    |  sceneNodeIndexToPaletteIndex
  skin.joints[k]  = source-joint ordinal k  (the value JOINTS_0 stores)             |    (per skin; -1 = not a joint of it)
        |                                                                           |
        |  BuildSkeleton: breadth-first, parent-before-child reorder -> oldToNew[k] |  paletteIndexToSceneNodeIndex
        v                                                                           |    (-1 = joint outside the imported scene)
  palette index p  -->  BindPose[p]  InverseBindPose[p]  SkeletonRootPrefix[p]  <---+
        ^                                 |
        |  packed JOINTS_0 bytes          v
        |  rewritten via oldToNew    AnimationPlayer: world[p], skin[p] = IBM[p] * world[p]
                                          |
                                          v
                          SkinnedEffect / SkinnedPbrEffect palette (at most 72)

That gives three indices worth naming: the scene-node index (placement in the selected scene, what rigid clips and cameras use), the source-joint ordinal (what the file's JOINTS_0 contains), and the palette index (what SkinningData arrays, skeletal clip tracks and the effect palette use). SkeletonResult also carries sceneNodeIndexToPaletteIndex and paletteIndexToSceneNodeIndex, because the file-local joint order is a third space and the scene graph must never be reordered to make the others coincide. Treating any two as interchangeable works only on conveniently authored fixtures; the reversed-joint-order fixtures (for example RuntimeGltfModelTest.LoadsSkinnedAnimatedModelDirectlyFromGltfWithReversedJointOrder) exist to rule that accident out.

The coordinate-space failure that produced D8

A skin's root joint may have ancestors that are not themselves joints — an armature node, say. The original BuildSkeleton walked parents only within the joint set, so it dropped the armature's world transform from the reconstructed bind hierarchy while keeping the file's inverse-bind matrices, which already encoded that ancestry. Every skinned vertex was then multiplied by the inverse of what was lost; for a uniformly scaled armature that is a division by the scale, and the character collapsed toward the origin. That was forensic defect D8.

The corrected root term, computed per root joint in the import core, is

P_root = W_ancestor * inverse(W_meshNode)

The first factor restores the scene ancestry above the joint set. The second cancels the transform of the node that instances the skinned mesh, which glTF requires because the mesh is placed by its joints; CNA parents a skinned mesh to the model's synthetic root, so without the cancellation that node's transform would be missing, and with a node bone on top it would be applied twice. The walk does not stop at glTF's optional skin.skeleton hint — the import core's own comments record that truncating the ancestry there reproduces D8 in a new disguise — and a joint hanging off a node outside the selected scene still gets its ancestry through cgltf's world-transform calculation rather than losing it.

Why the prefix must stay separate

It is tempting to multiply P_root into the root bone's bind-pose local matrix. That works until an animation channel replaces that local transform: at the first sampled frame the recovered ancestry disappears again. CNA therefore stores one prefix per root in SkinningData::SkeletonRootPrefix and the player composes

W_root(t) = L_root(t) * P_root

so an animated root substitutes only its own local transform, exactly like any other bone. The prefix is applied only when the array has exactly BoneCount entries; an empty (or otherwise mismatched) array reads as identity, which keeps older .cnj skeleton sidecars without the third block loading unchanged. The separation is both the fix and the compatibility mechanism.

The evidence is designed to discriminate. The skin-mesh-node-transform fixture gives its only joint an identity bind pose and inverse bind matrix, so the whole joint matrix is the cancellation of the mesh node's T(0, 0, 50). GltfSkinSpaces.MeshNodeTransformIsCancelledExactlyOnce (GltfSceneGraphBonesTests.cpp) expects a translation of exactly −50: 0 would mean the cancellation is missing, and −100 that it was applied twice. It also checks that the mesh node's bone still exists with its +50 transform and that the mesh is parented to bone 0. RootJointCarriesTheSceneAncestryAboveTheJointSet covers the ancestry half on skin-armature-ancestor.

Clip extraction

For a skeletal clip, ExtractClips forms, per animated bone, the union of that bone's translation, rotation and scale key times, deduplicated at a 1e-9 tolerance, and evaluates each channel at every union time with its own interpolation mode; a channel absent for that bone falls back to the decomposed bind pose. LINEAR is evaluated directly. STEP holds the value at the start of the half-open interval, so at exactly the next key time the next value is already in force (an earlier version returned the previous key there, which, because union resampling puts every interior key on an interval boundary, made a three-key STEP channel play 0, 0, 20 instead of 0, 10, 20). CUBICSPLINE uses glTF's Hermite basis with tangents scaled by the interval length, and a cubic rotation is renormalised after component-wise evaluation. A clip's duration is its largest sampler input time.

The resulting KeyframeEXT records do not retain interpolation modes or tangents: the player interpolates linearly (and spherically) between them. A cubic channel is therefore exact at every union time and generally only a piecewise approximation between them; export denser keys if a curve's shape between keys matters. Morph weight tracks are different — they keep their tangents (below).

Only channels whose target resolves through the skin's joint map enter a skeletal clip. Rigid node TRS animation goes through the same resampling code (ExtractSceneNodeClips), with scene-node indices instead of palette indices, and every clip records which space it is in through AnimationClipEXT::TargetSpace (ClipTargetSpaceEXT::JointPalette or SceneNode). On an unskinned model Model::Tag carries ModelAnimationsEXT, whose clips are applied with ApplyClipToBonesEXT(model, clip, time): it clamps the time to the clip, writes each tracked bone's interpolated local transform and leaves untracked bones alone, and it throws std::invalid_argument for a palette clip, since applying palette indices to Model::Bones would pose the wrong bones silently. A skinned model's Tag is already occupied, so rigid tracks not carried by any retained skin are counted and reported (rigid-animation-dropped-on-skinned-model) rather than hidden. That is D6 fixed for the ordinary rigid-model route, without pretending that one legacy pointer can hold two unrelated carriers. Channels whose target is outside the selected scene, and paths the importer does not handle, are counted and reported too.

Morph targets

MorphTargetDataEXT (MorphTargetEXT.hpp — the header is named after the feature, the type is MorphTargetDataEXT) rides on a mesh part's Tag, the same attachment convention one level down. It keeps the part's zero-weight BaseVertexBytes, the Stride, per-target PositionDeltas, NormalDeltas and TangentDeltas (an empty inner vector means that target has none of that kind), the current Weights, an optional WeightTrack, and the flat-normal recomputation switch RecomputeFlatNormalsEXT with its TriangleIndicesEXT. The tutorial-level usage is on Tutorial 113; the exact blend is:

• BlendMorphTargetsEXT(morph, weights) throws std::runtime_error unless weights has exactly as many entries as there are targets — no padding, no truncation (WrongWeightCountThrows).
• Positions are the base plus the weighted sum of position deltas. Normals get the weighted sum of normal deltas and are renormalised, because a weighted sum of unit vectors is not a unit vector. Tangent directions (xyz) are blended and renormalised the same way; the handedness w is never blended, since interpolating +1 and −1 passes through 0, which is not a handedness.
• Normal and tangent offsets come from the canonical stride table, so every layout with a normal slot — including the PBR strides 48, 60, 68, 76 and 80 — morphs its normals. (An older version listed only strides 32, 52 and 56, so PBR morph targets silently kept their rest-pose normals; ThePbrStridesAreNotExcludedFromNormalBlending pins the fix.) Texture coordinates, joints, weights and colours are copied from the base unchanged.
• With RecomputeFlatNormalsEXT set — a primitive that authored no normals, which the importer fully splits — the blend recomputes each face's normal from the morphed positions and re-orthogonalises any tangent against it, because such a primitive may not legally carry normal deltas.
• SetMorphWeightsEXT(part, weights) finds the carrier by dynamic_cast on the part's tag (throwing if there is none), blends, stores the weights and re-uploads the whole vertex buffer with SetDataRaw. This is CPU morphing by design: it works with every effect on every renderer, and costs one full buffer rebuild and upload per weight change.

A hand-built carrier must keep its arrays consistent: the blend indexes NormalDeltas[t] for every target with a non-zero weight without checking that the outer vector is long enough (only TangentDeltas is guarded), and it indexes each non-empty inner delta vector by vertex without checking its length. Push an empty inner vector for a target without normals, as the test fixture does.

A two-target worked example

MorphTargetEXTTests.cpp builds every case on one stride-32 triangle (position, normal, UV; base normals +Z) with two targets: target 0 pushes every vertex +1 in Z with no normal change, target 1 moves only vertex 0 by +2 in X and tilts its normal toward +X.

MorphTargetDataEXT morph;
morph.BaseVertexBytes = BuildBaseTriangleBytes();   // 3 vertices, stride 32
morph.Stride = 32;
morph.PositionDeltas.push_back({Vector3(0, 0, 1), Vector3(0, 0, 1), Vector3(0, 0, 1)}); // target 0
morph.NormalDeltas.emplace_back();                                                       // none
morph.PositionDeltas.push_back({Vector3(2, 0, 0), Vector3(0, 0, 0), Vector3(0, 0, 0)}); // target 1
morph.NormalDeltas.push_back({Vector3(1, 0, 0), Vector3(0, 0, 0), Vector3(0, 0, 0)});
morph.Weights = {0.0f, 0.0f};                        // base pose
part.setTagProperty(&morph);                         // ModelMeshPart::Tag

SetMorphWeightsEXT(part, {1.0f, 1.0f});              // blend on the CPU, re-upload the buffer
// vertex 0: (0,0,0) + (0,0,1) + (2,0,0) = (2,0,1)
// vertex 1: (1,0,0) + (0,0,1)           = (1,0,1)

Driving both targets to full weight is the case worth reasoning through: blending is additive, so vertex 0, the only one both targets touch, receives both deltas, while vertices 1 and 2 receive only target 0's. Its normal is (0,0,1) + (1,0,0) = (1,0,1), of length √2, and comes back as the unit vector (0.707, 0, 0.707) — the renormalisation BlendedNormalIsRenormalizedToUnitLength pins. The UV is untouched.

Animated weights

glTF's weights channel targets the node that instances a mesh, not a joint, so it has no bone index and lives on its own track type, MorphWeightTrackEXT, imported onto MorphTargetDataEXT::WeightTrack independently of any skin. ExtractMorphWeightTrack takes the first node in the file's node array that instances the mesh — so every placement of a mesh instanced by several nodes plays that node's track — and keeps cubic in and out tangents unbaked. EvaluateMorphWeightsEXT(track, seconds) returns an empty vector for an empty track, the only key's weights for a one-key track, clamps at both ends, holds with the same half-open STEP rule as the bone channels, evaluates a real Hermite curve for a cubic track whose bracketing keys carry tangents, and falls back to linear when they do not. A mesh's initial weights are the instancing node's own weights when present, else the mesh's; non-zero defaults are blended and uploaded at load, and the mesh bounding sphere describes that loaded pose.

Scale and capacity

The converter's unitScale applies to every translation-bearing quantity, including animation translation values and their cubic tangents and morph position deltas, never to rotation or scale channels; runtime import fixes it at 1.0. Both stock skinned effects accept at most 72 palette matrices (MaxBones), and a glTF skin with more joints is refused at import with its joint count named (the skin-73-joints fixture) — truncation would collapse the vertices bound to the missing joints toward the origin. Each packed vertex carries four one-byte joint indices and four weights. A legal glTF rig larger than those limits must be split or reduced before import.

A frame, in order

1. Choose the clip and advance each player (Update).
2. Compose absolute skeleton transforms, including root prefixes (done inside the player).
3. Form skin matrices from worlds and inverse binds (GetSkinTransforms()).
4. Install each skin's palette on every skinned effect its meshes use.
5. Apply rigid scene-node clips with ApplyClipToBonesEXT, if the model has them.
6. Evaluate morph weights and call SetMorphWeightsEXT only when they changed.
7. Call Model::Draw (or the manual loop) on the graphics thread.

Model::Draw's scratch vector is thread-local, which removed a cross-thread scratch race, but graphics resources and the shared model graph are not thereby thread-safe: keep sampling, palette installation, morph uploads and drawing under the application's graphics-thread ownership (see model threading).

Evidence

Checked by reading the TARGET sources and tests at 009d40f5; not built or executed. Import-side behaviour is pinned by the skin, animation, morph and clip suites under the glTF import tests and cross-route parity by the converter suite described on the CNJ toolchain page; pixel evidence for skinned and morphed fixtures is part of the L7 campaigns on glTF conformance. User-level walkthroughs: Tutorial 112, Tutorial 57.