CNA-BUG-062: MODULE_SCOPE in generate_coverage_inventory.py leaves design, diagnostics and inspector unclassified, so every C API inventory gate stops

CNA snapshot 009d40f5  ·  Known Issues › Current bugs  ·  source links pinned to 009d40f5

The C API coverage generator refuses to run because three public modules are missing from MODULE_SCOPE, so the coverage, limitations, scope-model and release-gate checks fail at this snapshot and the committed coverage numbers cannot be re-derived.

Identifier

CNA-BUG-062

Category

Bug

Subsystem

Documentation & release tooling

Status

Open

Verified against

CNA 009d40f5 (009d40f5dd085c4e674d3479675fac84b12b3e0a)

Severity

Medium (a triage suggestion, not a project priority)

Evidence basis

Reproduced: executed for this entry (the Evidence section names exactly what was run)

Tests touching this area

Yes: see Current tests

Affected contract

tools/c-api/generate_coverage_inventory.py --check (MODULE_SCOPE, validate_module_scope) and the CTests CApiCoverageMatrix, CApiCoverageScopeModel, CApiLimitations and CApiReleaseGate

Expected behaviour

MODULE_SCOPE is documented in the generator as a total function over every directory under modules/ that publishes an include/ tree: a new module must receive an explicit runtime or out-of-scope decision, and until it does the gate stops. Once each module is classified, --check rebuilds the inventory, matches the tracked docs/c-api/COVERAGE.md, and the release gate's measurement agrees with its record.

Actual behaviour at TARGET

At this snapshot modules/design/include (the Microsoft::Xna::Framework::Design converters), modules/diagnostics/include (CNA/Diagnostics) and modules/inspector/include (CNA/Inspector) exist, but MODULE_SCOPE names none of them. validate_module_scope therefore raises “The runtime C API scope model does not describe this tree” before Doxygen is started, and:

• generate_coverage_inventory.py --check exits 2, so CApiCoverageMatrix fails;
• generate_limitations.py --check builds the same inventory and exits 2, so CApiLimitations fails;
• three tests of tools/c-api/test_coverage_scope.py that read the real repository (test_the_repository_classifies_every_publishing_module and two discover_headers checks) error, so CApiCoverageScopeModel fails;
• check_release_gate.py measures both coverage-closed and limitations-matrix as not met; because release_gate.json records the latter as met, --check reports a disagreement, so CApiReleaseGate fails.

The committed COVERAGE.md (556 headers, 9,355 symbols: 8,363 implemented, 15 partial, 468 planned, 509 not applicable), LIMITATIONS.md and RELEASE_GATE.md (“Not ready. 1 criteria are unmet”, limitations “current”) were generated from a tree without these modules and can be neither regenerated nor verified here. All four tests are registered in every native build with CNA_BUILD_TESTS and Python (cmake/Tests/ModuleProbes.cmake), and general-tests-ci.yml runs an unfiltered ctest whose known-failure list does not name them.

Source locations

• tools/c-api/generate_coverage_inventory.py — MODULE_SCOPE, validate_module_scope, discover_headers, main (exit 2 on RuntimeError)
• tools/c-api/generate_limitations.py — read_rows builds the same inventory through build_inventory
• tools/c-api/check_release_gate.py — check_coverage_has_no_planned_rows, check_limitations_current, evaluate
• tools/c-api/release_gate.json — limitations-matrix recorded as met
• tools/c-api/test_coverage_scope.py — tests that call validate_module_scope and discover_headers on the real repository
• cmake/Tests/ModuleProbes.cmake — CApiCoverageMatrix, CApiCoverageScopeModel, CApiLimitations, CApiReleaseGate outside if(CNA_BUILD_C_API)
• .github/workflows/general-tests-ci.yml — unfiltered ctest; KNOWN_FAILURES lists only EasyGL_GraphicsDevice_ReferenceStencil
• .github/workflows/c-api-coverage-gate.yml — runs --check on pushes touching modules/**/include/CNA/**
• modules/design/include/Microsoft/Xna/Framework/Design.hpp — one of the three unclassified public include trees
• modules/diagnostics/include/CNA/Diagnostics/Diagnostics.hpp — unclassified public include tree
• modules/inspector/include/CNA/Inspector/Agent.hpp — unclassified public include tree
• plans/plan_capi_smoke_stability.md — CNA's own record of three failing generator gates in a local ctest run

Evidence

Read at 009d40f5: generate_coverage_inventory.py, generate_limitations.py, check_release_gate.py, ModuleProbes.cmake and the workflows. Executed with Python 3 against the read-only TARGET tree (no bytecode, nothing generated): validate_module_scope raised and named exactly the three modules; generate_coverage_inventory.py --check and generate_limitations.py --check each exited 2 before Doxygen; the two release-gate measurement functions, called in-process, returned “not met”; test_coverage_scope.py ran 29 tests with 3 errors. check_release_gate.py --check as a whole was not run (it also compiles header probes); its failure follows from evaluate(). CNA's own plan_capi_smoke_stability.md records a 2026-09-18 local ctest -R '^CApi' run with three failing generator gates. CI results are not visible; that general-tests-ci and c-api-coverage-gate go red is inferred from their configuration. Which scope each of the three modules should receive is an owner decision this entry does not make.

Focused reproduction

Executed in the TARGET tree (Python 3, needs neither Doxygen nor a build):

python3 tools/c-api/generate_coverage_inventory.py --check; echo "exit=$?"
# coverage inventory error: The runtime C API scope model does not describe this tree:
#   modules/design/include exists but nothing classifies it. ...
# exit=2
python3 tools/c-api/generate_limitations.py --check; echo "exit=$?"
# limitations error: The runtime C API scope model does not describe this tree:
# exit=2

Current tests

test_coverage_scope.py (CApiCoverageScopeModel) already proves, on fixture trees, that an unclassified module stops the gate: the refusal works as designed. Nothing else checks that the real tree is classified; the four gates that would are the ones failing.

Regression test

Classify design, diagnostics and inspector in MODULE_SCOPE (runtime scope, or out of scope with a recorded owner decision), regenerate COVERAGE.md, LIMITATIONS.md and RELEASE_GATE.md; the four CTests passing again is the regression test, and the existing unknown-module refusal stays the guard for the next module.

Blast radius

C API release tooling and every native CNA test build with Python, where four CTests fail; the published C API coverage numbers are unverifiable at this snapshot, and classifying design as runtime scope would add planned rows. No C route, no runtime behaviour and no C++ API is affected.

Workaround

Read the committed coverage figures as a snapshot of the commit that generated them, not as a measurement of this tree.