Verification & Known Issues

How CNA's XNA compatibility is actually checked — and what is verifiably not checked at all

How CNA is verified

An XNA reimplementation can pass its own test suite perfectly and still be wrong, because the tests encode the author's belief about what XNA does. CNA works around that with five layers, three of which compare against something outside the project.

What the test counts actually count

Alpha.1 deliberately needs scoped test facts. Static source inventory, instantiated GoogleTest cases, CTest registrations and executed results answer different questions.

Renderer coverage is configuration-scoped

CNA exposes 50 renderer identities across 46 implementation families. Single-renderer builds remain the default, while CNA_GRAPHICS_RENDERERS can link a compatible set for runtime selection. Both shapes affect verification:

• A green single-renderer run describes the selected family, not the other 45 families.
• A multi-renderer run verifies the compiled set and runtime selection/fallback paths; it does not make every renderer share one capability contract.
• Only the default gets the project-wide CNA_RENDERER_* definition. Renderer-present test definitions identify every family compiled into the test target.
• 50 identities are not 50 equally complete renderers. Some are intentionally 2D-only or produce no pixels.

The XNA 4.0 oracle corpus

This is the strongest verification claim CNA can make, so it is worth being exact about what it is — and about which renderer it applies to.

Under tools/xna-oracle/ sit 39 scenes together with 39 reference PNGs captured from real XNA 4.0 running under Wine + DXVK. CNA renders the same scenes and the images are diffed byte for byte.

The corpus covers 39 scenes, not the API. It is ground truth for what it covers and silent about everything else.

Eight of the thirty-nine

Every image below was produced by the real Microsoft XNA 4.0 runtime, not by CNA and not by hand. CNA's DIRECTX9 renderer draws the same eight scenes and the diff must come back empty at --tolerance 0. They are reproduced throughout the tutorials where they illustrate the technique being taught.

Vertex-coloured triangle — see Tutorial 31.

Triangle strip — see Tutorial 40.

Unlit textured quad — see Tutorial 32.

The same quad with lighting on — see Tutorial 32.

Alpha test — all eight comparisons are in Tutorial 54.

Wrap texture addressing — see Graphics State.

Fresnel-weighted reflection — see Tutorial 56.

Four-bone skinning, per-pixel lit — see Tutorial 57.

These are deliberately austere scenes. A corpus like this is useful precisely because each frame isolates one piece of behaviour, so a diff that fails points at something specific. It is not a demonstration of what CNA can render — for that, see the demos.

What CI covers (and what it doesn't)

The alpha.1 tag contains 21 workflow files. They are not interchangeable: some run focused suites, some are source/coverage gates, some only build or link, and the native Windows graphics workflows require manual dispatch. File count is not a pass count: the intended unfiltered general job and two Input matrix rows still select the removed EASYGL identity and fail at configure time.

Three distinctions prevent overclaiming:

• A declared workflow is not evidence when its configuration is rejected; EASYGL is an implementation family, not one of the 50 selectable identities.
• A configure or final-link lane does not prove runtime behavior.
• A simulator or headless-browser result does not prove physical-device or every-GPU behavior.
• Android has source and NDK support but no workflow in this tag.

Where coverage is thin or absent

These are not "lightly tested", and it is better that you hear it here.

• Storage. The whole storage module carries five test macros, and StorageContainer — the type your save code actually touches — has none. The implementation is real std::filesystem IO; nothing asserts on it.
• Renderer evidence remains per family. The 21 workflows include focused renderer gates, but they do not execute every capability of all 46 implementation families on every supported target.
• Optional layers need their own configurations. CNA_CNAEXT, CNA_DEVICES and CNA_BUILD_C_API default to OFF; a default build says nothing about them. The alpha.1 C API cannot reach runtime verification because its final implementation fails its renderer-count assertion.
• The XNA oracle diff. Covered above, and worth repeating: the project's most rigorous check is not automated.

Current limitations

What follows is the current list of user-facing limitations. Every entry is a declared boundary — a capability CNA does not claim, or a documented deviation — rather than an open defect report. Where an item once appeared here as a bug and has since been fixed or narrowed into a boundary, it has been removed rather than left standing as a warning.

Graphics and renderers

• Compiled effects are renderer-qualified. XNA/FNA D3D9 Effect Framework bytecode works on FNA3D and on opt-in EasyGL, SDL_GPU and Vulkan builds. HLSL .fx source and MGFX remain unsupported; other renderer families report CompiledEffects false.
• Runtime renderer selection is opt-in. A normal build still contains one renderer. Multi-renderer builds must use compatible sets, fallback is disabled by default, and selection latches after the first device.
• Cube faces inside a multiple-render-target set are unimplemented on eight otherwise-capable 3D renderers: DIRECTX9, DIRECTX11, DIRECTX12, BGFX, the EasyGL family, SDL_GPU, OPENGL2 and OPENGL4.
• DIRECTX1 is a declared stub — its entire 3D pipeline throws.
• LLGL is experimental, Linux/X11-only and OpenGL-only; its Vulkan module cannot be selected. OPENGLES2 has no instancing.
• SupportsCapability() still has a permissive legacy default. Renderers must narrow unsupported entries. Multi-stream input and compiled effects are false-by-default because both require explicit implementation.

Platforms

• Web has no save persistence at all. Under Emscripten, SDL_GetPrefPath lands in volatile MEMFS; CNA mounts no IDBFS and never calls FS.syncfs. Every save is silently discarded on page reload.
• On Web, Game must be heap-allocated. A stack-allocated Game is silently corrupted — there is no diagnostic, so this one is worth reading twice.
• Video is absent entirely on Windows, Emscripten and Android. The video translation units are excluded from those builds, so the headers still compile and the link then fails on missing Video/VideoPlayer symbols.
• FFmpeg is a hard requirement on Linux and macOS. No option disables it; configure fails without libavcodec, libavformat, libavutil and libswresample development packages.
• Network discovery is SystemLink only, and nothing at all on Emscripten.

API members that always throw or no-op

• 15 Guide::Show* overlay entry points are documented no-ops. (The Begin/End message-box and keyboard-input pairs are not among them — those are real implementations.)
• A few methods always throw: Achievement::GetPicture(), PropertyDictionary::CopyTo(), NetworkMachine::RemoveFromSession(), ResourceContentManager::OpenStream, and BoundingFrustum::Intersects(Ray) in part.
• 3D audio supports exactly one AudioListener; adding a second throws.

Content and decoding

• .xnb files using Intel E8 preprocessing are not decoded.
• DDS cube maps are limited to DXT1, DXT3 and DXT5.
• .m4a and .aac are unplayable — SDL3_mixer ships no AAC decoder.
• An XACT wave bank containing XMA or WMA data does not throw: it logs to stderr and returns nullptr, so the sound is simply missing. The XNB SoundEffectReader path does throw.
• glTF retains unskinned rigid clips and factor-only PBR, validates required extensions and exposes structured import diagnostics. A mixed skinned model cannot also use Tag for extra rigid clips, so those tracks are omitted with a named diagnostic.

Build-time switches

• CNA_CNAEXT (the CNAEXT engine layer, including AsciiPostProcessEffect, CRTEffect, DepthEffect, PbrMaterial and the CNA::Graphics surface) defaults to OFF.
• CNA_DEVICES (the CNAEXT device layer) also defaults to OFF.

Deliberate deviations from XNA

These are not bugs and are not on a roadmap to be "fixed" — they are design decisions with real consequences for porting, so plan around them.

Reproducing these numbers yourself

Nothing here requires special access. Run these from the exact tag checkout:

# CNA-owned C++ source files under test paths (vendored code excluded)
rg --files -g '*.cpp' -g '!third_party/**' |
  awk '$0 ~ /(^|\/)(test|tests)(\/|$)/' | wc -l

# Static GoogleTest-family definition inventory (not a pass count)
rg -n '^\s*(TEST|TEST_F|TEST_P|TYPED_TEST|TYPED_TEST_P)\s*\(' \
  -g '*.cpp' -g '!third_party/**' | wc -l

# What this configured build actually registers
ctest --test-dir build -N

Building and running the suite needs the sibling repositories checked out next to cna/. They are separate clones, not submodules:

git clone https://github.com/openeggbert/cna.git
git clone https://github.com/openeggbert/sharp-runtime.git   # required by every build
git clone https://github.com/openeggbert/easy-gl.git         # the GL-profile renderers
git clone https://github.com/openeggbert/meta-gl.git         # needed by easy-gl

# Linux also needs the FFmpeg development packages, or configure fails:
#   libavcodec-dev libavformat-dev libavutil-dev libswresample-dev

cd cna
git submodule update --init          # non-recursive is correct, and much faster

cmake -S . -B build -DCNA_GRAPHICS_RENDERER=OPENGLES3
cmake --build build --target CnaTests

# everything registered with CTest (needs a GL context; use xvfb-run on a headless box)
ctest --test-dir build --output-on-failure

Swap OPENGLES3 for another identity to reproduce a different single-renderer slice, or configure a compatible CNA_GRAPHICS_RENDERERS set to exercise runtime selection. The resulting CTest inventory is the only count that applies to that configured build.