CNA-VGAP-056: The checked-in shader bytecode headers of VULKAN, SDL_GPU and the Direct3D renderers come from generators that no CTest or workflow runs and that have no --check mode

CNA snapshot 009d40f5  ·  Known Issues › Verification gaps  ·  source links pinned to 009d40f5

spirv_shaders.hpp (VULKAN, SDL_GPU), hlsl_shaders.hpp (DIRECTX11/12) and the d3d9_*shaders.hpp headers come from hand-run generators with no --check mode and no CTest or workflow, so a shader source edit without regeneration would go unnoticed.

Identifier

CNA-VGAP-056

Category

Verification gap

Subsystem

Testing & evidence

Status

Open

Verified against

CNA 009d40f5 (009d40f5dd085c4e674d3479675fac84b12b3e0a)

Evidence basis

Source-verified: read at TARGET, not executed

Tests touching this area

Yes: see Current tests

Affected contract

The checked-in generated shader headers of the classic renderers (spirv_shaders.hpp for VULKAN and SDL_GPU, hlsl_shaders.hpp for DIRECTX11/12, d3d9_*shaders.hpp for DIRECTX9) and the generators that produce them

Expected behaviour

The graphics-ext shader packages set the standard in this tree: generate_shader_package.py --check behind one reproducibility CTest per package, so a checked-in generated artefact stays derivable from its declared sources. The renderer-family headers are generated the same way and would be held to the same guard.

Actual behaviour at TARGET

Vulkan's and SDL_GPU's compile_shaders.py accept only --output. compile_shaders_hlsl.py (D3D11/12) and compile_shaders_sm2.py (D3D9) are documented as run by hand through MinGW and Wine. None is referenced by a CMake target, CTest or workflow. verify_d3d_shaders_reproducible.sh is a manual no-change regeneration check (D3DCommon whole file, six D3D9 custom arrays) that no workflow runs. verify_hlsl_shaders_native_msvc.py, run by the manual-dispatch d3d-windows-ci.yml, recompiles the HLSL but intentionally does not compare bytes with hlsl_shaders.hpp, and SpirvPayloadValidation runs spirv-val, which checks validity, not freshness. SDL_GPU's generator expands the Vulkan renderer's shadow_sampling.glsl, so one edit reaches two headers; the commit that added that coupling recorded a hand check that Vulkan's header still regenerates byte for byte.

Source locations

• modules/renderers/vulkan/src/shaders/compile_shaders.py — main: only --output is parsed; header says to re-run it by hand
• modules/renderers/sdl-gpu/src/shaders/compile_shaders.py — main: only --output; expands shadow_sampling.glsl from the Vulkan directory
• modules/renderers/common/d3d/src/shaders/compile_shaders_hlsl.py — docstring: not part of any CMake target, run by hand
• modules/renderers/directx9/src/shaders/compile_shaders_sm2.py — docstring: not part of any CMake target, run by hand
• scripts/verify_d3d_shaders_reproducible.sh — manual reproducibility check, referenced by no workflow or CTest
• scripts/verify_hlsl_shaders_native_msvc.py — docstring: intentionally does not byte-diff against hlsl_shaders.hpp
• cmake/Tests/ModuleProbes.cmake — SpirvPayloadValidation (validity only) and the graphics-ext package reproducibility tests

Evidence

Read at 009d40f5, nothing executed. Searched every non-documentation file for references to the four generators and the reproducibility script: only scripts and the generators themselves name them. Git history up to the snapshot read for the Vulkan and SDL_GPU shader directories: commit 74f6296bc edits the shared shadow_sampling.glsl, regenerates only SDL_GPU's header and states that Vulkan's was reproduced by hand. No drift was found or claimed; the gap is the missing guard.

Focused reproduction

No focused reproduction is known. Nothing has been invented here; the evidence above is what exists.

Current tests

None for the generated headers' freshness: no CTest or workflow runs any of the four generators or compares their output with the checked-in headers. SpirvPayloadValidation runs spirv-val, which checks validity, not freshness; the graphics-ext shader packages are guarded separately (CNA-VGAP-017).

Regression test

Give each generator a --check mode (regenerate to a temporary path and compare bytes) and register one reproducibility CTest per generated header, as tools/shader_package/generate_shader_package.py --check does for the graphics-ext packages.

Blast radius

VULKAN, SDL_GPU and the Direct3D renderers: an edit to a shader source that is not followed by a regeneration would go unnoticed. SDL_GPU's generator expands the Vulkan renderer's shadow_sampling.glsl, so a single edit reaches two headers.

Workaround

Re-run the generator by hand after a shader source edit and diff the header (for the Direct3D headers, scripts/verify_d3d_shaders_reproducible.sh does this manually).