I need to investigate a sample failure

CNA snapshot 009d40f5  ·  Development › Maintainer Handbook  ·  source links pinned to 009d40f5

A sample that crashes, renders wrongly or behaves differently from the original is CNA's most convincing bug report and also its most ambiguous one: the fault can sit in CNA, in the C++ translation, in the compiled content, in the build configuration or in the host. This recipe is the triage flow for deciding which, and for reducing the failure to something a maintainer can fix and test. It uses the sample repositories as evidence for their own pinned revisions, never as statements about the TARGET snapshot 009d40f5. Everything was read from source, plans and pinned repository revisions; nothing was built or run.

The sample ecosystem and what each part proves

The sample counts above are the pinned repository's own; they are not CNA facts and change as the plan is edited. The user-facing summary and its caveats are on the XNA compatibility and verification pages.

Find the owner: sample, content, CNA, configuration or host

The sample repository's working rules (rules.md at the pin) state the ownership principle a maintainer should copy: a sample may not compensate for a CNA defect, so a faithful translation that fails is either a translation or content fault (fix it in the sample) or a framework fault (fix it generally in CNA, with a test, never with a sample-name special case). Use the table to decide which.

Reproduce with the sample's CNA revision

1. Record three revisions. The sample commit, the CNA commit and the sharp-runtime commit. cna-samples builds against sibling checkouts ../cna and ../sharp-runtime, both on the next branch, with CNA_SHARP_RUNTIME_ROOT selecting the runtime and CNA_SAMPLES_CNA_ROOT naming an equivalent explicit CNA checkout; its rules require the configured path to be recorded in the sample's evidence. It does not pin CNA, so the CNA commit has to be recovered from the sample's audit or by date.
2. Use the campaign's reference configuration first. Native builds use CNA_GRAPHICS_RENDERER=OPENGLES3; browser builds use WEBGL2 under Emscripten (OPENGLES3 is refused there). A failure on another renderer is a different question (Renderer selection internals).
3. Build one sample. From the sample repository: cmake -B build -DCMAKE_BUILD_TYPE=Debug -DCNA_SAMPLES_ONLY=PrimitivesSample then cmake --build build --target PrimitivesSample_cna_samples, as its README states at the pin. Do not treat a published gallery bundle as a reproduction: it is a Release WebGL 2 artifact of unrecorded CNA revision.
4. Reproduce at the sample's CNA revision, then at TARGET. If it fails at both, it is not a regression and the cause is in one of the layers below. If it passes at the older revision and fails at TARGET, bisect CNA between them with a fixed sample commit; the C++ change list is the input to Change public XNA behaviour.
5. Write down the configuration that failed: the platform, audio and renderer axes, the profile, the compiled-effects options, whether content is pregenerated .xnb or built by cna-content, and the host.

Profile: Reach versus HiDef

The default project profile is Reach (ProjectGraphicsProfile.cpp initialises it to GraphicsProfile::Reach), and TARGET enforces the profile ceilings in the shared device layer on every renderer. Samples written for XNA 4.0 that use these features set HiDef in the original; a port that does not is a translation fault, and a port that does and still throws is a CNA fault. Messages read at TARGET:

Read the profile a device actually has with getGraphicsProfileProperty(), set it with GraphicsDeviceManager::setGraphicsProfileProperty(GraphicsProfile::HiDef) before device creation (or CNA::SetProjectGraphicsProfileEXT), and read renderer answers with GraphicsDevice::GetRendererCapabilityProfileEXT().GetEnglishReport() (Tutorial 152, Tutorial 133). A device that reports a feature as unsupported after HiDef is requested is then a renderer question. Many alpha.1-era tutorials omit the profile, so an old snippet that throws is not evidence against CNA.

Renderer and platform axes

Platform, audio device, renderer and target OS are independent choices (the four axes). To learn which renderers a build actually contains and which one was selected, run cna_demo_renderer_selection (graphics examples), which reports what the build contains instead of assuming. Selection precedence is the API call, then the CNA_GRAPHICS_RENDERER environment variable (in a browser the page property Module.cnaPreferredRenderer, at the environment variable's precedence), then the compiled default; a name that is not compiled in throws rather than being ignored (GraphicsRendererSelection.cpp, rules). Selection latches on first device creation.

• A window-free control. CNA_GRAPHICS_RENDERER=HEADLESS with CNA_HEADLESS_MODE=Trace validates the sample's calls and records a structured call log without pixels, which separates a state or API-misuse fault from a rendering fault (Headless renderer internals); it cannot show shader output.
• Fault injection. CNA_DEBUG_UNAVAILABLE_RENDERERS and CNA_DEBUG_FAIL_RENDERER_INIT make listed renderers fail their probe or initialisation, to exercise fallback chains.
• Browser. Only the Emscripten renderer identities are selectable; saves do not persist across a reload (below); Video throws NotSupportedException; a blocking Game::Run() needs Asyncify in the final link (Tutorial 124).
• Sample-defined host differences. A gallery bundle is non-threaded by construction; a sample that genuinely uses System.Threading needs the threaded configuration and COOP/COEP headers that Pages cannot send.

Content pipeline and effects

The samples use official pregenerated .xnb files where the original pipeline output is available, keeping the original Content.Load<T>() contract; otherwise content is built by cna-content. Runtime CNA loads XNB, CNB or loose files (ContentManager); it does not compile .fx source. Effects are compiled-bytecode only: an XNA/FNA Direct3D 9 Effect Framework binary is accepted on the renderers that enable it (FNA3D always; nine default-OFF build options), MonoGame MGFX is rejected by name, and cna-content compiles .fx at build time through an external legacy fxc discovered from the CNA_FXC environment variable, the baked CNA_FXC_EXECUTABLE or PATH (EffectCompilerService.hpp). Its equivalence to Microsoft's compiler is unverified.

The error guide in fx-compiled-effects.md distinguishes the kinds of fault a maintainer needs to separate: ArgumentException means the bytes are wrong; NotSupportedException means the format, renderer or profile is wrong; a parser diagnostic as std::runtime_error means plausible but corrupt content; ContentLoadException wraps any of them with the asset name. A worked example of a sample finding a real defect: PerPixelLightingSample rendered with red and green matching the original and only blue different, where blue would exceed 1, which traced to Direct3D 9 clamping vertex colours before interpolation where GL ES clamps after it; CNA restores the clamp at the end of the generated shader for the ES profiles with a MojoShader patch in cmake/patches. The tell was a one-channel mismatch that appeared only above 1.

• Is the asset the same bytes as the reference? The in-tree sweep (sweep.py, then classify.py and taxonomy.py) rebuilds each mapped project through a frozen cna-content and compares by bytes; its classes separate identical, semantically identical (for example the same payload under two conforming LZX encoders), an accepted difference (a rasteriser, a block compressor, a compiler version string), a custom pipeline component defined by the sample that CNA cannot load, and environment gaps (a font this host lacks). CNA's plan records the sweep's results at its own runs; the collection it reads is not in the repository, so treat those figures as CNA's records.
• Does a sample define its own processor? A processor no XNA assembly defines is the sample's own and cannot be loaded into CNA; that is a documented gap, not a defect in either side.
• Is the target platform right? XNB readers and headers are platform-specific; an Xbox 360 or Windows Phone target in the header is a content question before it is a reader question (Content pipeline internals).

Storage, persistence and audio

• Storage. The root is chosen from the first applicable of Android's private directory, XDG_DATA_HOME, LOCALAPPDATA, HOME (with Apple's Library/Application Support) and finally the working directory, below <app>, which is the literal game unless StorageDevice::SetAppNameEXT is called. Begin* completes before it returns, so start at End*. In a browser the root lives in Emscripten's in-memory file system and CNA mounts no persistent one, so a save is written and reported successful but is gone after a reload. A failed root resolution latches an empty root (source-read, not exercised), which turns later saves into working-directory writes (Storage internals, Tutorial 141).
• Audio. A build with no usable audio device throws NoAudioHardwareException on effect construction under SDL3; SDL_AUDIODRIVER=dummy and the NULL audio implementation give a device-free control, and CNA_AUDIO_DEVICE steers the ALSA device. High-level playback exists only for the SDL3 and ALSA audio selections (Audio engine internals, environment variables).

Separating a CNA defect from a sample, content or host problem

Run each control below to change one variable, and stop at the first that flips the result.

1. Same sample, HEADLESS renderer with CNA_HEADLESS_MODE=Trace. A HeadlessValidationException naming a rule points at the API sequence (sample or CNA state handling); a clean trace points at rendering.
2. Same sample, HiDef if the original was HiDef. Flips: the port omitted the profile.
3. Same content, built by cna-content versus the pregenerated .xnb. Flips: content generation.
4. Same sample, a second renderer that supports what it uses. Flips: a renderer translation layer, not shared logic.
5. Same CNA revision, a second host or driver. Flips: host.
6. Older CNA revision, same sample. Flips: regression, so bisect.
7. Original C# sample under real XNA. See below.

Extract a minimal repro

1. Remove content first: replace loaded assets with a procedural Texture2D or a colour clear. If the failure survives, it is not content.
2. Strip the game to one Game, one GraphicsDeviceManager and the single call sequence that fails, with the same profile, renderer and platform.
3. Prefer an existing declarative or standalone form: an oracle scene (tools/xna-oracle/README.md lists the keys) for stock effects, SpriteBatch and primitives, or a module example in the style of modules/graphics/examples. Register a new example in that directory's CMakeLists.txt (a target-existence gate tests equality with CNA_GRAPHICS_RENDERER, a resource-availability gate may test membership) and, if it needs a window, give its add_test the private-display environment.
4. Turn the repro into a regression test with the narrowest assertion that fails (Add a regression test); a passing sample is not a test.

When to compare against real XNA

Compare when the question is what XNA does, not what CNA does: a pixel difference on a stock effect, SpriteBatch or primitive draw, a default value, an exception type, or a byte-level content difference. The oracle corpus covers only its 39 scenes; the DIRECTX9 identity is recorded pixel-exact on all 39 through Wine and DXVK, while other renderers are measured against it and mostly not gated, and no workflow runs the whole corpus (Using the XNA oracle as evidence, Tutorial 161). For a whole sample the sample repository's own order of authority is the original C# source, then a real XNA 4.0 build of that unchanged source, then FNA, then Microsoft documentation, and other ports only as supplemental evidence. For content, the byte comparison above is the oracle. A CNA result that matches FNA but not the XNA reference is worth recording rather than resolving silently.

Review checklist

• The three revisions (sample, CNA, sharp-runtime), the axes and the profile are recorded, and a reproduction exists at more than one CNA revision or the reason it does not is stated.
• The owner was decided by a control that flipped the result, not by inspection of the crash site.
• The fix is in the owning layer with a regression test that does not need the sample; no sample-specific branch was added to CNA.
• External counts and results are cited with their pins and are not restated as TARGET facts; anything that was not run is said so.

Related: Debugging cookbook, Tutorial 79, How to understand code you did not write, Modify the Content Pipeline and Change build configuration.