C API evidence, coverage inventory and release gate

CNA snapshot 009d40f5  ·  Deep Dives › Reference matrices  ·  source links pinned to 009d40f5

CNA's experimental C ABI carries more self-checking machinery than any other part of the repository: a recorded layout baseline, a toolchain matrix, a generated coverage inventory, a limitations document and a release gate that measures itself. This page explains what each of those instruments actually measures, what the gate reports when it is run against snapshot 009d40f5 (which differs from the report committed in the tree), how the coverage inventory decides what is in scope, and how far the evidence for the C library reaches. It is for anyone who has to quote a C API status, maintain the gates, or decide how much to trust the ABI.

What is established, rung by rung

The site's evidence vocabulary (source exists, configures, builds, links, unit tested, integration tested, runtime observed) applies to the C library as it does to everything else. For the C API at 009d40f5 the rungs stand as follows; nothing in this table was built or run for these pages.

"Source-level, not build-verified" is therefore the accurate summary of the library itself. The gate machinery below is different: most of it is build-free by design and can be run by anyone, which is exactly why its measured result at this snapshot can be stated.

Toolchain cells and run configurations

compatibility_matrix.json declares two things, rendered into COMPATIBILITY.md: the compilers and language modes the headers must support, and the renderer configurations the C tests are meant to run in. They have very different evidence behind them.

Header cells: 23 across 7 toolchains

For every cell, generate_compatibility_matrix.py --run compiles each public header on its own and the umbrella header; an installed toolchain that rejects a header fails, an absent optional toolchain is reported as skipped with its reason, and an absent required one fails. The c-api-compat-matrix.yml workflow installs Clang, GCC 14 and the MinGW cross-compiler on Ubuntu 24.04 so that the optional cells can run there too, and runs both --check and --run. Together with the header half of the ABI baseline, this is what CI establishes about the C API: the headers are self-contained and parse in the declared C and C++ modes, and their layouts match the record. The floor is C99, not C17, because the headers must stay consumable by the oldest declared mode; CNA's own C test programs are compiled at C17 with extensions off.

Four run configurations, declared rather than enforced

The same pure-C sources are compiled unchanged into whichever tree is configured, and they are written to accept the renderer's answer: 35 of the programs under tests/pure_c handle CNA_RESULT_NOT_SUPPORTED, and the teardown test treats a renderer that cannot create a 2D texture as covered by its game-only modes. The registrations in modules/c-api/CMakeLists.txt choose the display per test: in an SDL_RENDERER tree most smoke tests get SDL_VIDEODRIVER=dummy, while tests that need an OpenGL context or a real window (CApi_Draw3DSmoke, CApi_OwnedGraphicsDeviceSmoke, CApi_GameSecondaryGraphicsDeviceContext and the seven CApi_TeardownLifetime_* modes) get SDL_VIDEODRIVER=x11 and DISPLAY=${CNA_TEST_DISPLAY} under the display policy of TestDisplayPolicy.cmake.

The four configurations are a declaration. No tool reads them except the renderer of the document, and no preset or workflow at 009d40f5 configures any of those four trees with CNA_BUILD_C_API=ON. The sentence in COMPATIBILITY.md that every smoke program "runs in all four configurations" is CNA's description of its campaign practice; the only dated run record in the tree is the HEADLESS one cited above. The other 21 renderer identities share the same C surface but are not in the matrix at all.

The release gate: a record and a measurement

check_release_gate.py reads release_gate.json, where each criterion carries a recorded state (met, not met or an owner-decision state that measures as blocked), and runs one check function per criterion to measure the current state. Its three modes:

• --run prints each criterion's measured state and "READY" or "NOT READY"; it always exits 0.
• --write regenerates RELEASE_GATE.md from the measurement; the "Release" line is filled from abi.h, so the version cannot go stale in prose.
• --check (the CApiReleaseGate CTest and the c-api-release-gate.yml workflow) fails when any criterion's measurement disagrees with its record in either direction, a met criterion that regressed or a blocked one that quietly became met, and otherwise fails if the committed document differs from what --write would produce.

A "Not ready" verdict is therefore not a failing test: --check passes while the record and the measurement agree that a criterion is unmet. What the check functions measure is less uniform than the criteria's wording suggests.

What each criterion's check actually inspects

Six of the ten criteria are satisfied by the presence of a file, a registration or a string; they say that the machinery exists, and the real evidence for them arrives only when a developer builds the C API and runs ctest -R '^CApi'. That is a reasonable design for a build-free gate, but it means the gate's green rows are not results of running the library. The record's own history in release_gate.json shows why the gate measures instead of remembering: the coverage criterion was closed and reopened by repeated merges, once after a rule that matched a whole header with .* had let 121 new symbols inherit "implemented and tested" without any C route existing.

What the gate measures at 009d40f5

Run against the 009d40f5 tree (executed once, in a read-only copy of the snapshot, for the site's Known Issues verification; the reasoning below was also checked by reading the scripts):

1. generate_coverage_inventory.py --check stops before Doxygen starts. Its scope validation finds three module include trees that nothing classifies (modules/design, modules/diagnostics, modules/inspector), raises "The runtime C API scope model does not describe this tree", and the script exits with status 2. The coverage criterion measures not met, which agrees with its record, but for this reason rather than because of the 468 planned rows.
2. generate_limitations.py --check builds the same inventory, so it fails the same way and also exits 2. The limitations criterion measures not met while its record says met.
3. check_release_gate.py --check therefore reports one disagreement and exits 1; the --run verdict is NOT READY with two unmet criteria. The other eight measure as recorded.

Four CTests carry these failures: CApiCoverageMatrix, CApiLimitations, CApiReleaseGate and CApiCoverageScopeModel (whose fixture file test_coverage_scope.py includes test_the_repository_classifies_every_publishing_module, which reads the real tree). They are registered in ModuleProbes.cmake in every native tests build that finds Python, deliberately outside if(CNA_BUILD_C_API), so an ordinary build carries them; the known-failure list of the unfiltered general test workflow (general-tests-ci.yml) names only one EasyGL stencil test. No workflow run was inspected for these pages. The consequence for readers: the committed coverage summary, LIMITATIONS.md and RELEASE_GATE.md were generated from a tree that did not yet contain those three modules, and at this snapshot they can be neither regenerated nor verified; the figures they print (below) are the last generated snapshot, not a current measurement (see Known Issues). The 0.29.0 ABI is a large experimental surface; it is not a release-ready coverage closure.

How the coverage inventory decides scope

generate_coverage_inventory.py derives every public C++ declaration from modules/*/include/{Microsoft,CNA}/**/*.hpp through Doxygen XML, drops any path with an Internal or Detail segment in any capitalization, and maps each remaining symbol through the rules in coverage_mappings.json. Which modules count at all is decided by MODULE_SCOPE, a table that must be total over the modules that publish an include/ tree.

validate_module_scope fails when a publishing module has no classification, when a table entry names a module that no longer publishes headers, when a prefix entry covers nothing, when a nested module is classified as runtime scope (the scanner would attribute its symbols to the parent), when an exclusion has no reason, or when an out-of-scope subtree rule points at a non-runtime module or a missing directory. The table is total on purpose. The comment beside it records the incident that made it so: when the Content Pipeline became its own module, nothing classified it, and 2,554 of its declarations fell through to a completed task and were reported as missing runtime C bindings for about a fortnight. A total table cannot fail that way; a new module stops the gate and asks. At 009d40f5 it is doing exactly that, and the only way forward is an owner decision per module, not a reclassification to make the gate green.

The inventory's four statuses are defined in the generated coverage summary under docs/c-api: implemented means an exact current C mapping with C-only evidence named (not a behaviour oracle), partial an approved subset with its boundary recorded, planned no complete mapping and an unfinished task that owns the work, not applicable a declaration with no callable C++ behaviour or one belonging to a declared substrate boundary. No symbol counts as implemented because a related C operation exists. The last generated snapshot, per module:

425 further headers are excluded by the scope rules (content pipeline 107, phone 9, platform 27, renderers 138, and the Internal/Detail paths of runtime modules). The figures are not comparable with the alpha.1 inventory (6,712 rows from 421 headers) because the scope model changed in between.

The planned backlog and the limitations document

The 468 planned rows are the only declarations in runtime scope that a C caller cannot reach; the C++ API provides them. plan_binding.md assigns them to open tasks, and its row counts sum to the inventory's figure:

LIMITATIONS.md, generated by generate_limitations.py and described in the gate record as the document a consumer reads to decide whether to adopt the ABI, states the 9,355 total and then partitions only 8,887 of them (8,363 fully mapped, 15 partially mapped, 509 with no C form): its generator keeps only partial and not-applicable rows, so the 468 planned declarations, which have no C route either, do not appear in it. The release gate's note still attributes 654 build-time declarations to CBIND-117 although the plan records 134 after the Content Pipeline was scoped out, and it names neither CBIND-125 nor CBIND-127. Read the planned backlog from the coverage summary and the plan, not from the limitations document (see Known Issues). The 15 partial rows are the entries a working program can be surprised by: a route exists under the expected name but does less than the C++ member, and the document says what to call instead.

Quoting C API status correctly

• Say "ABI 0.29.0, experimental, source-level, not build-verified" for the library; say that no CI workflow builds it.
• Quote coverage as "the last generated inventory (556 headers, 9,355 symbols: 8,363 implemented, 15 partial, 468 planned, 509 not applicable)", and add that the generator cannot currently regenerate it.
• Quote the gate as measured: "Not ready, two criteria unmet (coverage closed, limitations matrix) and --check failing", with the committed "one unmet" as the record it disagrees with.
• Treat a binding's result as evidence for that binding's pinned revision and its ABI generation, never for CNA at 009d40f5.
• Keep the alpha.1 compile blocker as history: it is gone in source, and whether the library builds at this commit is still open.

The maintainer procedure for changing routes, mappings and the gate record is I need to update the C API; the contract a consumer programs against is on Native C API contract.