Avatars: the inert XNA API and the skinned extension

CNA snapshot 009d40f5  ·  Deep Dives › Input, audio, media & services  ·  source links pinned to 009d40f5

CNA ships two avatar systems that meet in one class and share nothing else. The XNA-compatible AvatarRenderer, AvatarDescription and AvatarAnimation reproduce what the XNA 4.0 reference assembly observably does when no Xbox LIVE avatar service exists: they validate, they report Unavailable, and they draw nothing. A separate, opt-in CNAEXT path built on SkinnedModelEXT loads CNA's own avatar content and draws it with GPU skinning. This page gives the exact contract of both, the palette mathematics, the content format, and what the pixel evidence does and does not prove; it is for anyone porting an avatar-using XNA title or building on the extension.

Two independent systems

The faithful API and the extension are deliberately decoupled. Calling the extension never changes what the XNA members report: State stays Unavailable, BindPose still throws, and the XNA Draw overloads stay no-ops, whether or not real rendering was enabled on the same object. The two skeletons are unrelated too: the XNA surface speaks in a fixed 71-bone array, while a SkinnedModelEXT has whatever skeleton its content defines.

“Faithful” is scoped to translated behaviour that tests pin. It does not mean that the C++ code is binary-identical to the managed assembly, and it does not mean that any Xbox avatar service or asset is available. The whole avatar family lives in the GamerServices module, so it exists only when CNA_ENABLE_NET is on (the default); see gamer services internals.

The XNA avatar API: inert by fidelity

The real avatar body mesh, textures, animation data and description format were streamed at run time from Xbox LIVE servers that no longer exist, and the XNA 4.0 reference assembly never bundled any of it. Off an Xbox, the reference implementation therefore draws nothing. FNA has no avatar classes at all, so CNA's source was translated from the XNA 4.0 reference assembly rather than inferred from FNA, and the design record avatar-real-rendering-ext.md states the rule: the faithful behaviour is the unconditional default, preserved rather than “fixed”.

AvatarRenderer

• Constructors ignore their arguments. Both AvatarRenderer(AvatarDescription*) and AvatarRenderer(AvatarDescription*, bool useLoadingEffect) discard the description and the flag; every instance starts identical (AvatarRenderer.cpp).
• State is forced on every read. getStateProperty() assigns AvatarRendererState::Unavailable and returns it each time it is called; it is not merely an initial value, and no code path ever produces Ready or Loading.
• BindPose always throws. It checks the raw state field for Ready, which nothing sets, and throws InvalidOperationException (“The avatar's bind pose is not available.”); after Dispose it throws ObjectDisposedException first.
• ParentBones is real data. The 71-entry parent table (root -1) was decoded from the reference assembly, not derived; AvatarRenderer::BoneCount is 71.
• Both Draw overloads validate and stop. Draw(IAvatarAnimation*) throws ArgumentNullException for a null animation (a CNA addition), copies the animation's transforms and expression, and forwards. Draw(const std::vector<Matrix>&, AvatarExpression) throws ObjectDisposedException after disposal and ArgumentException unless given exactly 71 matrices, then returns without drawing.
• Lighting properties are stored. LightDirection, LightColor and AmbientLightColor are plain values; only the extension's draw reads them (defaults: direction (0, 0, -1), light 0.65 grey, ambient 0.35 grey).
• Disposal. Dispose() calls the protected Dispose(bool), which marks the object disposed and releases the extension's effect, model and device pointer.

AvatarDescription

• The byte constructor requires exactly 1,021 bytes (ArgumentException, “Resource data must be exactly 1021 bytes.”) and copies them. IsValid is true when the length is right and byte 0 is non-zero; the other 1,020 bytes are opaque, because the proprietary format was never public.
• getDescriptionProperty() returns a copy of the bytes. Height lazily becomes 0.0f and BodyType lazily becomes AvatarBodyType::Female, whatever the bytes say.
• Both CreateRandom() and CreateRandom(AvatarBodyType) return the same all-zero, therefore invalid, description. The second overload validates that the enum is 0 or 1 (ArgumentOutOfRangeException otherwise) and then ignores it, exactly as the reference does.
• BeginGetFromGamer throws for a null or disposed gamer, completes before it returns (the callback runs inline) and hands back a heap-allocated IAsyncResult the caller must delete. EndGetFromGamer throws ArgumentException for a result it did not produce and otherwise returns the all-zero invalid description.
• Changed is a per-instance event, matching Microsoft's declaration (one cached description per signed-in player). Nothing in CNA raises it; a copied description carries the subscribers it had when copied (AvatarDescription.hpp).

AvatarAnimation and the presets

Every AvatarAnimation, whatever AvatarAnimationPreset it is constructed with, holds 71 default (all-zero) matrices, a Length of TimeSpan::Zero and a default AvatarExpression. Update(elapsed, loop) implements real clamp and wrap arithmetic, but with a zero length the position is always pinned to zero. The preset is used for exactly one thing: it seeds a CNAEXT clip name, readable with GetRealClipNameEXT() and replaceable with SetRealClipNameEXT(). There are 31 presets (Stand0 to Stand7, Clap, Wave, Celebrate, ten Female* and ten Male*); AvatarAnimationPresetToClipNameEXT maps each to its literal enumerator name and throws ArgumentException for an unrecognised value (AvatarAnimationPresetNamesEXT.cpp).

The SkinnedModelEXT contract

SkinnedModelEXT (SkinnedModelEXT.hpp) lives in the graphics module and is deliberately not built on Model/ModelBone/ModelMesh, whose parent-bone hierarchy describes rigid per-mesh animation rather than per-vertex GPU skinning. It is move-only (copy construction and assignment are deleted) because it owns GPU buffers. Its public data is plain:

• BoneCount, ParentBoneIndices (root is -1), BindPoseLocal (each bone relative to its parent) and InverseBindPoseGlobal (inverse of each bone's bind-pose world transform).
• Parts: a vector of PartEXT descriptors, each a Name, a non-owning ModelMeshPart* and an optional Texture2D*. The model owns the vertex buffers, index buffers, mesh parts and textures in private vectors; there is no per-part material object beyond the texture.
• Clips: an std::unordered_map from name to AnimationClipEXT, which holds a Duration, a vector of BoneTrackEXT (a BoneIndex and ascending KeyframeEXT keys) and a TargetSpace (JointPalette by default; SceneNode is for rigid node animation elsewhere). A KeyframeEXT defaults its rotation to identity and its scale to one, so a key may carry translation only.

The skeleton has no relation to the XNA 71-entry arrays: no attempt is made to align counts, indices or semantics. The bundled bodies (avatar/male/avatar and avatar/female/avatar, each a 19-bone skeleton with 21 clips: the eleven shared presets plus that body's ten gender-specific ones) sit far below SkinnedEffect's 72-bone limit; other content may use any count up to that limit.

Opting in: EnableRealRenderingEXT and DrawRealEXT

EnableRealRenderingEXT(GraphicsDevice&, std::shared_ptr<SkinnedModelEXT>) throws ObjectDisposedException after disposal (it used to succeed and effectively “undispose” the object) and ArgumentNullException for a null model, then stores the device and model and creates a SkinnedEffect. IsRealRenderingEnabledEXT() reports whether a model is held. DrawRealEXT(clipName, position, loop) throws InvalidOperationException (“call EnableRealRenderingEXT first”) when no model is held, samples the clip, and draws every part. AvatarBodyTypeToContentNameEXT is the one place a body type is mapped to a content name; it is not derived from AvatarDescription::getBodyTypeProperty(), which faithfully never carries real data.

// Inside a Game subclass; renderer_ is a std::unique_ptr<AvatarRenderer> member,
// description_ an AvatarDescription member, clipTime_ a System::TimeSpan member.
void LoadContent() override {
    description_ = AvatarDescription::CreateRandom();          // all-zero, invalid
    renderer_ = std::make_unique<AvatarRenderer>(&description_); // State stays Unavailable

    auto model = getContentProperty().Load<std::shared_ptr<SkinnedModelEXT>>(
        AvatarBodyTypeToContentNameEXT(AvatarBodyType::Male));   // "avatar/male/avatar"
    renderer_->EnableRealRenderingEXT(getGraphicsDeviceProperty(), model);

    AvatarAppearanceEXT look;
    look.setShirtColorProperty(Color(40, 90, 200, 255));
    renderer_->SetAppearanceEXT(look);
}

void Draw(const GameTime& gameTime) override {
    renderer_->setWorldProperty(Matrix::getIdentityProperty());
    renderer_->setViewProperty(view_);
    renderer_->setProjectionProperty(projection_);
    clipTime_ = clipTime_ + gameTime.getElapsedGameTimeProperty();
    renderer_->DrawRealEXT("Stand0", clipTime_, /*loop=*/true);
    Game::Draw(gameTime);
}

The fragment was syntax-checked with g++ -std=c++23 -fsyntax-only against the snapshot's public headers (with a sibling sharp-runtime checkout for the System headers and CNA_RENDERER_EASYGL defined); it was not built or run. It needs a 3D-capable renderer: SDL_RENDERER and the other 2D-only renderers refuse the vertex buffers and the effect.

Animation sampling and the bone palette

SkinnedModelEXT::ComputeBoneTransformsEXT(clipName, position, loop, outWorldBones) (SkinnedModelEXT.cpp) turns one clip time into the matrices SkinnedEffect::SetBoneTransforms expects. The steps, in order:

1. Look up the clip. An unknown name throws ArgumentException (“Unknown animation clip”).
2. Check the skeleton arrays. ParentBoneIndices, BindPoseLocal and InverseBindPoseGlobal must each have exactly BoneCount entries, or ArgumentException is thrown before any indexing.
3. Normalise the time. A clip with a zero (or negative) duration is sampled at zero. Otherwise, with loop the position is reduced by tick modulo and a negative remainder is shifted into [0, Duration), so negative time wraps as floor-mod; without loop it is clamped to [0, Duration].
4. Start from the bind pose. Every local transform is initialised from BindPoseLocal, so a bone with no track holds its bind pose.
5. Sample the tracks. A track with no keys, or whose BoneIndex is negative or not below the bone count, is skipped without touching memory. Before the first key the first key is used and after the last key the last; between keys, translation and scale use Vector3::Lerp and rotation uses Quaternion::Slerp, with the interpolation amount computed in seconds. The local matrix is CreateScale(s) * CreateFromQuaternion(r) * CreateTranslation(t).
6. Compose in topological order. For bone i the parent index must be less than i; otherwise ArgumentException is thrown. That single rule also rejects a bone that is its own parent and every cycle, because the highest-index member of a cycle would need a parent with a higher index. A root's world transform is its local transform (any negative parent index means root, not only -1, exactly as in AnimationPlayer); a child's is local * world[parent] (row-vector convention).
7. Produce the palette. Each output entry is InverseBindPoseGlobal[i] * world[i], ready for SetBoneTransforms; the output vector is resized to BoneCount.

A two-bone worked example

SkinnedModelEXT model;
model.BoneCount = 2;
model.ParentBoneIndices = {-1, 0};
model.BindPoseLocal = {Matrix::getIdentityProperty(),
                       Matrix::CreateTranslation(Vector3(0.0f, 1.0f, 0.0f))};
model.InverseBindPoseGlobal = {Matrix::getIdentityProperty(), Matrix::getIdentityProperty()};

BoneTrackEXT track;                     // drives bone 0 only
track.BoneIndex = 0;
track.Keys.push_back(KeyframeEXT{TimeSpan::FromSeconds(0.0), Vector3(0.0f, 0.0f, 0.0f)});
track.Keys.push_back(KeyframeEXT{TimeSpan::FromSeconds(1.0), Vector3(2.0f, 0.0f, 0.0f)});

AnimationClipEXT clip;
clip.Duration = TimeSpan::FromSeconds(1.0);
clip.Tracks.push_back(track);
model.Clips["Move"] = clip;

std::vector<Matrix> palette;
model.ComputeBoneTransformsEXT("Move", TimeSpan::FromSeconds(0.5), /*loop=*/false, palette);
// palette[0].getTranslationProperty().X == 1.0f  (halfway from 0 to 2)
// palette[1] translates by (1, 1, 0): its bind offset composed with the moved parent

The example was syntax-checked the same way (not executed). The inverse bind poses are identity here, so the palette equals the world transforms; with real content the inverse bind pose cancels the bind pose, so a clip sampled exactly at rest reduces every palette entry to identity. That identity check is how the bundled content was debugged.

Four guards and the failures they replaced

All of these are in SkinnedModelEXTTests.cpp (present; not executed for this page). The generic skinning pipeline and SkinnedEffect's palette upload are explained in Tutorial 57: SkinnedEffect and skeletal animation.

Lighting, appearance and wardrobe

The draw and its light rig

XNA's avatar renderer exposes one directional light and an ambient term, not BasicEffect's three-light rig, and DrawRealEXT configures its SkinnedEffect to match on every call: DirectionalLight0 is enabled with the renderer's LightDirection and LightColor and a zero specular colour; DirectionalLight1 and DirectionalLight2 are disabled; the effect's specular and emissive colours are zeroed; and AmbientLightColor is set last, so nothing can overwrite it. The lit result is therefore (ambient + key light × N·L) × tint. The defaults (key light 0.65, ambient 0.35) keep an avatar visible without any configuration. Then, for each part, the diffuse colour is set to the part's tint, the part's texture is bound, the effect is applied, the part's vertex and index buffers are set and one indexed triangle list is drawn.

AvatarAppearanceEXT: a tint model

AvatarAppearanceEXT is a CNA-invented set of five colours (skin, hair, shirt, pants, shoes), not a reconstruction of the proprietary 1,021-byte description. The private PartTintEXT routes a part by case-sensitive substring of its name, testing in order "Hair", "Shirt", "Pants", "Shoes"; any other part (including the body) receives the skin colour. Part names come straight from the content (for example CNAAvatarShirt), which is why substring rather than equality is used. The appearance changes colours only: it adds no geometry and swaps no per-part texture, and each bundled part texture is a plain white placeholder so that the tint is the colour.

Attaching and replacing parts

AttachPartEXT(SkinnedModelEXT&& other) moves every part of an independently loaded model onto this one. Its only compatibility check is other.BoneCount == BoneCount (ArgumentException otherwise); a different skeleton with the same count is not detected, so attach only pieces built from the same canonical skeleton, whose joint indices are then already correct. Before appending, it calls RemovePartEXT for every incoming part name, so re-attaching a same-named piece replaces the old one instead of drawing both. RemovePartEXT(name) removes every part with that name and frees its owned vertex buffer, index buffer, mesh part and texture; erasing directly from the public Parts vector would leak them, because Parts holds only non-owning descriptors. Tests: AttachingSameNamedPartReplacesTheOldOne, RemovePartFreesOwnedResources, RemovePartRemovesAllMatchingNamesNotJustFirst.

Two lighting defects, historically

Both are fixed at this snapshot and are kept here because they teach something. First, an earlier draw path called EnableDefaultLighting() and then overrode only the first light: XNA's generic fill and back lights leaked into every avatar, and the ambient value could be clobbered; tests missed it because the extra lights happened to back-face the test quads. The current explicit rig above replaced it. Second, the GL stock shaders used by EasyGL for the skinned, vertex-lit and environment-mapped variants once multiplied the emissive colour (into which SkinnedEffect folds ambient) by the diffuse colour a second time, so ambient landed as ambient × diffuse², crushing dark materials. Three rounds of ambient tuning failed before the formula itself was compared with FNA's; GlStockShaderSources.hpp now computes lightSum * diffuse + emissive and records the fix. CNA's own record reports that the avatar's near-black pixel fraction, measured by its visual regression script at 4.1–6.0 % before the fix, fell to 0 % afterwards. Why no earlier test caught it is the second lesson: the existing skinned and environment-map tests used diffuse components of exactly 0 or 1, where x * x equals x, so the double multiply was invisible. The discriminating test added with the fix, easygl_emissive_ambient_composition_test.cpp (registered as EasyGL_EmissiveAmbientComposition, and as Vulkan_EmissiveAmbientComposition for Vulkan), pins the diffuse colour to (0.25, 0.5, 0.75) with all directional lights off, so the correct composition gives (64, 128, 191) and the old one (16, 64, 143). The general rules: when repeated tuning of a parameter does not move a defect, suspect the formula, not the value; and test with operands for which the wrong formula and the right one disagree.

The avatar content format

SkinnedModelTypeReader in ContentManager.cpp loads Load<std::shared_ptr<SkinnedModelEXT>> from a small JSON manifest and binary sidecars:

The reader validates what it can: a bone count outside 0–100,000 is a ContentLoadException (checked before any allocation); a vertex file whose size is not a multiple of the stride, an index file that is not whole 16-bit indices, or an index that references a missing vertex is rejected; a part entry without a vertices or indices field, or with a non-positive stride, is skipped (a named file that does not exist is an error); an animation entry's clip may also name a .cnj AnimationClip asset so several models can share clips.

Three traps that generalise to other importers

• Resolve sidecars from the manifest's own directory. Every path in the manifest is resolved relative to the manifest, not to the content root, so a bundle such as Content/avatar/male/ is self-contained and relocatable. The resolution goes through ResolveContainedPathRelativeToFile, so a sidecar path cannot escape the content root (or, for a bundle loaded explicitly from outside the root, its own directory). The original reader resolved against the root and failed as soon as a manifest lived in a subdirectory.
• Never pass several side-effecting reads as arguments of one call. Keyframe components were once built as Vector3(reader.Read<float>(), reader.Read<float>(), …); C++ does not specify the evaluation order of function arguments, the compiler reordered the reads, and components were scrambled. The current reader (CnjCanonicalRead.cpp, ReadCnjAnimationClipSidecar) reads each float into a named local first.
• Reordering a hierarchy means remapping everything indexed by it. ComputeBoneTransformsEXT needs parents before children, so the asset converter (convert_avatar.py) sorts bones topologically; glTF gives inverse bind matrices and each vertex's JOINTS_0 in the skin's original joint order, and both must be remapped along with the parent indices, or bones skin with the wrong bind pose and weights.

Skinned glTF and .cnj models do not use this type: they load as an ordinary Model with SkinningData, as Tutorial 57 explains.

The bundled avatar is not visually defect-free

The bundled bodies and garments are procedurally generated capsule-and-sphere shells, and CNA's own record at the snapshot (NEXTnet.md, item 16 of its remediation list) keeps the avatar visual quality “partially fixed”. What it reports, measured with the repository's own tooling and not re-measured for this page:

• Shirt and skin fragments show around the raised shoulder and chest in the Wave clip, but the garment-versus-body crossing counts are essentially the same at rest and in Wave (shirt 64 of 228 vertices in both). The defect is therefore in the rest-pose shell geometry, rotated into view by the animation, not in clip sampling or skinning.
• The visible offenders are the collar and the cuffs. The collar is the top cap of the shirt's Spine1 shell (radius about 0.176) burrowing into the head sphere (radius 0.15), measured at up to 0.131 m; the waist overlap is mutually hidden because each garment buries inside the other.
• Four remedies were implemented, measured and reverted: narrowing the garment blend radius (no clear improvement), trimming boundary caps (pants burial rose from 21 to 39 vertices), flat cylinders with joint spheres (more speckle, new groin crossings, lower brightness), and 32 instead of 12 segments (smoother edges, same fragments, 4.2× the vertex data).

The record's proposed direction is to size each garment shell against the largest body radius it must clear along its span, or to author the torso garment as one shell. The third open choice is to accept the tessellation cost: raising the segment count from 12 to 32 is a one-line change to the segments value in both meshcraft generators (tools/avatar_builder/generate_body_meshcraft.py and generate_clothes_meshcraft.py), at 4.2 times the vertex data, and the record leaves that trade to the project owner rather than counting it as a fix. Until then the extension is renderer-functional with a known cosmetic limitation of the bundled content; content you author yourself is not affected.

Renderer coverage and evidence

Three renderer-agnostic integration programs in the gamer-services examples exercise the extension through public API only: avatar_real_render_integration_test.cpp, avatar_attach_part_integration_test.cpp and avatar_tint_routing_integration_test.cpp. The EasyGL examples CMake file registers them as EasyGL_AvatarRenderer_RealRender, EasyGL_AvatarRenderer_AttachPart and EasyGL_AvatarRenderer_TintRouting, and the Vulkan examples CMake file registers the same three sources as Vulkan_AvatarRenderer_*, and the generated EasyGLParityCorpus.cmake rebuilds them unchanged for OpenGL4 as OpenGL4_EasyGLParity_AvatarRenderer_* (skipped when the CNA_GamerServices target does not exist); all need CNA_ENABLE_NET and a display. The design document's renderer table still says Vulkan was “not yet smoke-tested”; the Vulkan registrations were added later, so the CMake files are the current statement.

Read the pixel evidence narrowly. The real-render program builds a synthetic one-bone quad (no asset pipeline involved), clears to green, translates the bone by half a screen, sets RasterizerState::CullNone because the quad is back-facing under the default state, and reads three single pixels with GetBackBufferData: green, red, green. That proves palette, effect and draw wiring moved geometry on that renderer; it does not validate the asset converter, the 19-bone content or the look of a full avatar. Full humanoids, clip changes, appearance, attachment, wardrobe swaps, the faithful-versus-extension boundary and a two-process network sync are exercised by eight demos (demo_avatar, demo_avatar_animation_gallery, demo_avatar_appearance_tint_studio, demo_avatar_bone_state_boundary, demo_avatar_dual_compare, demo_avatar_multi_attach_stress, demo_avatar_wardrobe_hotswap and, in the net module, demo_net_avatar_sync), which are built only for the five EasyGL identities or VULKAN, with networking on and not for Emscripten; they are demonstrations, not registered tests.

No other renderer family registers an avatar program at the snapshot: EasyGL, Vulkan and OpenGL4 are the three that do, and the OpenGL4 entries are the EasyGL programs rebuilt unchanged, registered but not executed for this page. Most other 3D-capable renderers implement SkinnedEffect in general (Tutorial 57 lists which, and notes the one that refuses it), but nothing about avatar rendering on them should be inferred from EasyGL or Vulkan pixels, and the 2D-only renderers such as SDL_RENDERER throw on the first 3D resource the extension creates. The faithful API's behaviour is pinned by the unit test files AvatarRendererTests.cpp, AvatarDescriptionTests.cpp, AvatarAnimationTests.cpp, AvatarExpressionTests.cpp, AvatarAppearanceEXTTests.cpp and the two name-mapping test files in the gamer-services tests. Nothing on this page was built or executed; test names are source facts, not pass results.

The avatar visual regression floor

The "visual regression script" that the records above cite is avatar_visual_regression_check.py. It is a structural floor, not an image comparison. It runs the real cna_demo_avatar (real content, the real AvatarRenderer and the GPU-skinned draw) under xvfb-run with --smoke 30 --screenshot for three pinned views: a male T-pose, a female T-pose, and a male Wave clip at 25° yaw. Each view's camera is fully determined by those arguments, so fixed pixel boxes stay comparable between runs.

It then measures what an interpenetration seam looks like rather than overall darkness. On the 0–765 R+G+B scale a pixel is dark below 150 and bright above 420, and speckle counts the dark pixels that touch a bright one. A shoe that is legitimately dark is contiguous and scores low; a jagged crossing of two low-poly shells scores high.

The checks, with the values measured when they were recorded (2026-07-18), are:

• a global average brightness of at least 330 (measured 348–364);
• a global very-dark fraction of at most 0.5 % (measured 0);
• per-region speckle ceilings: 20 for the groin (measured 0 and 2), 75 for each foot (measured 46–55) and 150 for the torso and shoulder in Wave (measured 120).

The two feet are separate boxes on purpose: the avatar is mirror-symmetric, so a combined box could average away a defect on one side.

The script needs a built demo, Pillow and xvfb-run. --report prints the metrics and never fails, which is how the ceilings are re-recorded after a reviewed improvement. No CTest registration or workflow runs it. Its own header says twice that passing does not mean the avatar looks correct; it means only that the avatar has not fallen below the recorded state. Read at 009d40f5; not executed.