Evidence tiers of the native modern GPU renderers

CNA snapshot c1c316b9  ·  Deep Dives › Renderers  ·  source links pinned to c1c316b9

Four of CNA's 14 renderer identities drive a modern explicit GPU API: VULKAN, SDL_GPU, WEBGPU and METAL. They share a vocabulary (immutable pipelines, visible resource transitions, shader delivery as a build concern) but not an implementation, and a successful device creation says little about which CNA features a given run actually proves. This page compares what each identity implements at this snapshot with the evidence behind it, explains why a capability bit is not evidence, describes the explicit-refusal pattern, and collects the defect shapes these renderers exposed. It is for anyone choosing among them, porting to one of them, or reading a test result from one of them.

Four identities, four implementations

The table states what the code does at this snapshot, not only what the capability bits say. "Custom shader intake" is what a ShaderEffect accepts; the stock effects are always the renderer's own precompiled or embedded shaders.

Maturity and category are CNA's own code-owned declarations, not measurements. Every one of the four keeps XNA's SurfaceFormat::Color free of an automatic sRGB encode; how Vulkan and WebGPU arrive there is explained in Vulkan: UNORM on purpose and WebGPU surface format. The user-facing summary, including platform gates and dependencies, is in Rendering backends: modern GPU APIs.

A capability bit is a promise, not evidence

Three separate things are easy to conflate:

1. A capability answer. GraphicsDevice::SupportsCapability says what the renderer claims. For most of the original fourteen members the shared base default is true, so a true answer can mean either an authored yes or a default nobody revisited, while a false answer is always authored (Tutorial 101 explains the asymmetry). VULKAN, SDL_GPU and METAL answer from switches without a default: arm, so a member they do not name is false at the renderer level (METAL names the thirteen original members and decides MultiSampleAntiAliasing per device), while WEBGPU overrides the members it probes and lets the rest fall through to the shared default.
2. A device-level answer versus a renderer-level answer. On SDL_GPU the renderer's own switch answers compute, indirect draw and the float render-target members false, while GraphicsDevice derives them from dedicated queries and format probes and answers true; METAL is the same for the float and half-float members. The device-level answer is the public one; a caller holding the renderer interface directly sees different values.
3. Evidence that the feature renders correctly. A test that creates a device and does not throw proves that the device path executes. It does not prove pixels, and neither does a capability bit. The pages for each renderer name which pixel assertions back which feature.

A capability bit therefore never promotes a no-throw smoke test into feature evidence, and native and browser WebGPU are distinct evidence environments even though they share one identity and one implementation.

What each identity's evidence actually covers

No test registered for these four renderers compares a scene with CNA's 39-scene real-XNA reference corpus. The only complete exact comparison registered as a test is DIRECTX9's; EasyGL gates two scenes at tolerance 0, SOFTWARE registers the same two but fails only on a render failure or a missing reference (CNA-BUG-206), and FNA3D gates only that the corpus renders (see Verification: renderers). CNA's multi-renderer corpus script can render the corpus through any renderer of a multi-renderer build, but no such result for these four is part of the evidence described here. The shared cross-renderer parity corpus, 32 fixtures registered for EasyGL, WEBGPU, SDL_GPU and DIRECTX11, compares each renderer against the fixture's own computed expectations, not against XNA. None of these tests was executed for this page; the table records registrations and CNA's own notes.

The METAL row changed the most in this snapshot: CNA’s Mac mini M4 campaign re-enabled each capability that had been refused (readback, MSAA, multiple render targets, occlusion queries, instancing, custom effects, every surface format) together with native tests under Metal’s validation layers, so its pixel evidence now comes from real Apple hardware rather than from CI. What it still lacks is in the last column (Tutorial 109: what remains).

Explicit refusal as an integration pattern

Because the shared capability default is permissive, a renderer can overclaim by doing nothing. METAL shows the safer pattern: it enumerated the unsupported surface, threw at the boundary, and widened it only when a platform-specific observation proved the new path — readback, occlusion queries, MSAA and the rest were each re-enabled on a Mac mini M4 together with native tests. Its remaining refusals still name their reason, for example "Metal SamplerState.MipMapLevelOfDetailBias: this Metal device ignores MTLSamplerDescriptor.lodBias (measured), so a non-zero bias is refused." and "Metal RenderTarget2D array slices are not supported." in MetalRenderer.mm. Callers get an early diagnostic instead of a later blank frame.

The same pattern appears across this group wherever a feature cannot be delivered:

• SDL_GPU throws from CreateOcclusionQuery() because SDL_gpu has no query commands, instead of returning a null query whose Begin and End would silently do nothing.
• VULKAN throws a named exception when a descriptor pool cannot be grown, instead of drawing with a substituted default texture (descriptor pools).
• WEBGPU refuses a draw that samples or blends a 32-bit float target on an adapter without the needed optional feature, instead of quietly overwriting.

The opposite, silent substitution, is the most expensive defect shape in this group, because nothing fails: Vulkan once drew white sprites when its descriptor pool was full, and WebGPU once sampled a 1 × 1 white cube when a RenderTargetCube was bound as an environment map.

Record the identity and the native driver

For portability work, a result needs two coordinates: which CNA identity ran, and which native driver or device path served it. An abstraction that selects Vulkan on one machine and Metal on another does not turn one successful run into proof of both.

• SDL_GPU: GetDriverNameEXT() returns SDL's driver name, and the renderer logs the backend SDL chose at construction. CNA's own tests force a driver with SDL's SDL_GPU_DRIVER environment variable (an SDL setting, not a CNA option). A Vulkan result says nothing about SDL’s Direct3D 12 or Metal drivers, and a Direct3D 12 driver result under Wine is Wine evidence, not native Windows presentation.
• VULKAN: the physical device matters as well as the API. Several behaviours are device-conditional (wireframe, anisotropy, precise occlusion counts, first-instance indirect draws, BC formats), and the applied present mode can differ from the request; GetAppliedPresentModeEXT() and OcclusionQuery::isPixelCountPreciseEXT() report what the device gave.
• WEBGPU: record native or browser, and for native which wgpu-native package; the browser's shader compiler is Tint and the native one is Naga, which can disagree about the same WGSL.
• METAL: record the machine and whether the console was unlocked. CNA’s macOS evidence comes from one physical Mac mini M4 (Apple GPU, validation layers on, console locked so every window was occluded) and from the iOS Simulator. GitHub’s hosted runner exposes an “Apple Paravirtual device” that cannot run Metal shader validation and needed its GPU tests serialised, so a hosted result is evidence for that device only. An Intel or AMD Mac GPU differs again: it has no packed 16-bit formats and needs managed-storage synchronisation, which CNA implements but no Intel Mac has run.

Defect shapes these renderers taught

The defects recorded for these four renderers are mostly fixed at this snapshot. The shapes recur, and recognising them is the useful part:

• State accepted, then discarded. An interface takes a complete state packet and the implementation consumes part of it. Vulkan's blend and stencil state were the canonical cases, and SDL_GPU's depth bias was captured but omitted from its pipeline key. Tests that vary each field independently catch it.
• Silent substitution. A default resource replaces the one asked for (white textures, a white cube) with no error. The fix is a named refusal, and the test must use values that the substitute cannot produce, such as colours that encode an index.
• Replay in implementation order instead of public order. Both SDL_GPU and WebGPU once replayed every 3D family before every sprite, justified as a typical draw order. A typical order is not a contract. The discriminating test issues the same two opaque draws in both orders and expects different winners.
• An oracle derived from the wrong contract. Vulkan's depth-bias test used OpenGL's depth convention and blamed the driver; WebGPU's cull-mode mapping was inverted to satisfy a probe quad whose winding had been mis-derived; an early Vulkan stencil repair swapped front and back faces on a hypothesis about one software driver. Each looked like a measurement. Derive expectations from the reference contract (XNA and FNA) and confirm a driver-specific explanation on a second driver.
• A review that looks instead of measuring. A manual screenshot review missed WebGPU's translucent sprites rendering opaque, and a pixel-value assertion caught it. Conversely, a captured screenshot caught SDL_GPU's upside-down sprites, which a "did it throw" test could not. Neither a no-throw check nor a visual review is a pixel assertion.
• Validation as evidence with a scope. Turning on SDL_gpu's validation made two tolerated contract violations visible, each of which could hang the Vulkan driver, and then forced the design back to the public resource contract. A clean validation run proves the API usage it exercised, not the pixels.
• An arm that volume cannot reach. Descriptor-pool exhaustion never happened on the tested drivers even with thousands of live resources, so Vulkan's regression injects the failure. A test that only generates volume can pass while executing none of the code it claims to cover.