Compatibility levels and the evidence vector

CNA snapshot 009d40f5  ·  Deep Dives › Foundations & compatibility  ·  source links pinned to 009d40f5

"Compatible with XNA" and "tested" each hide several different claims, and CNA's documentation keeps them apart on purpose. This page defines the levels of compatibility, the evidence vocabulary the site uses, what each class of renderer can and cannot prove, and the questions to answer before relying on a capability. It is the reading key for every verification statement on libcna.com, written for porters deciding what to trust and for contributors deciding how to word a result.

Levels of compatibility

There is no honest single percentage for "how compatible is CNA". Earlier project documents and earlier versions of this site quoted type-presence and functional-coverage percentages; they could not be recomputed from the source tree and were withdrawn. What replaces them is a set of distinct questions, each with its own instrument and its own blind spot.

The semantic column of the census is mostly language, not behaviour: 1,039 of the 1,040 documented properties are "semantic" only because C++ spells them getXProperty()/setXProperty() (see what 331/331 and 3,627/3,627 mean). Keeping these rows apart is what lets one sentence be true: every documented member exists, and behaviour is verified only for named subsets.

A claim is a vector, not a score

A source tree can contain a test definition that the selected build never registers; CTest can register a command the host cannot execute; a renderer can return success without consuming effect state; and a pixel can be right on one translated runtime without proving another host. A verification claim is therefore recorded as a vector of coordinates, and a claim that omits one of them is weaker than it sounds.

Four examples show why the coordinates occupy genuinely different tiers:

• The 25-identity, 21-family registry in cmake/RendererIdentities.cmake and its generated descriptor table are strong structural evidence (names, fixed C ABI values, one descriptor per family) and no rendering evidence at all.
• The DIRECTX9 oracle compares 39 scenes with output of the genuine XNA runtime, but both sides ran through Wine and DXVK on Linux; it says nothing yet about native Windows drivers or period hardware.
• HTML_DOM has an executed suite in headless Chromium (htmldom-ci.yml), whereas the Emscripten multi-renderer lane only proves that WEBGL2, CANVAS, HTML_DOM and SVG_DOM configure, build and link into one bundle. The same word, "web", covers both.
• A recorded pass fraction, such as a full ctest of a VULKAN configuration on 2026-09-06, belongs to that campaign's configured population; it is not a current suite total and cannot be added to another campaign's figure (see what the test counts actually count).

An evidence vocabulary

Every page on this site states its evidence basis, and CNA's own records use the same distinctions. The terms are coordinates, not rungs of one ladder: a compile probe can answer an API question a screenshot cannot (the strict leak check proves that a marked call fails to compile), and a hostile-input test and a pixel oracle exercise different contracts.

The evidence box at the top of each deep dive uses a subset of these (source-verified, test-present, build-verified, runtime-observed, oracle-compared, hardware-observed), and the last four appear only when CNA's own records say so and are attributed to CNA. The site never claims to have run anything.

What each kind of renderer can prove

The 25 identities fall into five classes whose evidence ceilings differ by construction. The build default (WEBGL2 under Emscripten, OPENGLES3 on Linux, SDL_RENDERER elsewhere) is a platform convenience, not a maturity ranking.

The capability matrix gives the per-identity answer for each of the 19 GraphicsCapability members, derived from each renderer's code, and pixel evidence by family lists what evidence exists for each family.

Before relying on a capability

A capability answer is a hint, not a contract. The base IGraphicsRenderer::SupportsCapability still returns true for the original members unless a renderer narrows them, and DIRECTX9 inherits that default wholesale; the finer RendererCapabilityProfile (32 features, 22 limits, per-format usage) answers more precisely but still reports what the code declares, not what ran. Before depending on an operation, pin down the operation, not the feature name:

• texture dimension and format, mip and sub-rectangle transfers, and whether the shader actually samples it;
• 2D or cube target, depth format, sample count, resolve, mip generation and later sampling (a cube face inside a multiple-render-target set is refused by DIRECTX9, DIRECTX11, DIRECTX12 and SDL_GPU although each supports both features separately);
• vertex and index formats, stream frequency, base and start offsets, primitive topology;
• blend, depth, stencil, rasterizer and sampler state, including reset behaviour and state caching;
• which stock-effect variant runs, the custom shader dialect or binary form, and parameter lifetime;
• query and readback timing, the logical versus physical presentation rectangle, and device-loss recovery;
• the exact platform and native-handle pair, and the descriptor that was actually selected at run time.

When a result differs between renderers, localise it in the same order. Identify the public selector, the implementation family, the native or translation API underneath, and whether the intended route engaged at all (GraphicsDevice::GetGraphicsRendererType() and GetGraphicsRendererName() report the live one in a multi-renderer build). Then reduce the case to a state transition, a readback or a pixel oracle. A successful process exit alone cannot localise a renderer-specific failure: SOFTWARE accepts shader source it never runs, and a no-pixel renderer can pass a fixture by refusing. Choose the graphics oracle by the failure you could have caused applies the same rule to a maintainer's change.

Counts are snapshots of a method

Static source definitions, CTest registrations, executed cases and oracle scenes are four different populations, and CNA's documents and this site never add them together. The site's test figures (904 files, 12,610 GoogleTest-family definitions) are source counts made with a stated command; ctest -N in a configured tree is the only registration count that applies to that tree; the corpus has 39 scenes in its denominator although 46 scenes exist. A task's closing count in CNA's plan files is a snapshot of one configuration on one date and stays attached to that campaign.

Even a simple count depends on its method. Counting the CNAEXT marker as a code token, with comments stripped, finds 1,843 uses in 330 public headers under modules/*/include; counting every whole-word occurrence, comments included, finds 2,279 in 353 headers. Both are correct and neither counts extension members. Hand-maintained figures also lag generated ones: CNA's top-level README still prints the pre-closure 3,467/3,627 member figure and "0/31" oracle scenes, while the generated census and the corpus directory say 3,627/3,627 and 39.

Rules for stating a claim

1. Source-proven code and a declared capability are prerequisites for a claim, not runtime results.
2. A renderer-engaged claim names the family and the native or CPU operation that was observed.
3. A pixel claim uses an image that could have failed; the oracle's own history records exact matches that proved nothing because the scene did not discriminate (see how a comparison is judged).
4. An oracle claim names the authority and the tolerance.
5. Unsupported, skipped and environment-blocked stay three different outcomes.
6. A recorded result carries its date, host and configuration, and is attributed to whoever recorded it.

A sentence that follows these rules reads: "CNA's records report that DIRECTX9, through Wine and DXVK on an AMD Radeon 780M, matched all 39 scenes at tolerance 0; the last dated report covers 31; not repeated on native Windows; not in CI." A sentence that breaks them reads: "CNA is pixel-perfect." How to word a claim collects more examples.