DIRECTX9: stock-effect bytecode, device lifecycle and oracle findings

CNA snapshot 009d40f5  ·  Deep Dives › Renderers  ·  source links pinned to 009d40f5

DIRECTX9 is the renderer whose stated goal is pixel-for-pixel agreement with the real XNA 4.0 runtime rather than feature parity, and it is the only renderer recorded as matching all 39 scenes of CNA's XNA oracle corpus at tolerance 0 (under Wine and DXVK). This page explains the choices behind that: where its stock-effect bytecode comes from and how close it is to Microsoft's, how it enforces GraphicsProfile from a real capability structure, how the lost-device lifecycle and render targets work, and the two sprite-projection details the oracle forced. It also states where the renderer's reach stops. It is for porters who need XNA-exact output and for maintainers who change this family.

Plain Direct3D 9, its own tables, a compiler-free stock pipeline

DirectX9Renderer.cpp creates its device with plain Direct3DCreate9, deliberately not Direct3DCreate9Ex, and asks GetDeviceCaps before CreateDevice whether the adapter has hardware transform and lighting, choosing hardware or software vertex processing from the answer rather than assuming it. CreateDevice also creates the implicit swap chain, so there is no separate device and swap-chain lifetime as on the DXGI renderers. Three structural decisions follow from the fidelity goal:

• No shared Direct3D helpers. D3DFORMAT is a different enum space from DXGI_FORMAT and Direct3D 9 has no state objects, so this family does not use D3DCommon; render state is a sequence of SetRenderState and SetSamplerState calls with nothing to cache.
• The stock pipeline links only d3d9. The stock shaders are precompiled bytecode embedded in d3d9_shaders.hpp. d3dcompiler is linked only into the isolated cna_renderer_d3d9_effect static library (CMakeLists.txt), which holds the runtime ShaderEffect compiler and the constant-table (CTAB) parser it needs.
• Presentation parameters come from the game. The back-buffer format, depth format, full-screen flag and swap interval arrive through GraphicsRendererCreateArgs and later through UpdatePresentationFormatEXT (see Direct3D presentation and state), instead of the fixed formats the DXGI renderers use.

Compiled XNA effects (.fxb Effect Framework binaries) are a separate, opt-in path: CNA_DIRECTX9_COMPILED_EFFECTS=ON builds D3D9CompiledEffect.cpp and makes SupportsCompiledEffects() true; a default configure reports CompiledEffects false. See Tutorial 128.

Microsoft's stock-effect sources, compiled by CNA

The ten files in src/shaders/xna are Microsoft's XNA 4.0 stock-effect HLSL (Ms-PL), copied byte for byte from the FNA reference tree; verify-d3d9-stock-effects-vendored.sh diffs them against that tree and fails on any change. The six compilable .fx files (BasicEffect, AlphaTestEffect, DualTextureEffect, EnvironmentMapEffect, SkinnedEffect, SpriteEffect) declare 66 entry points, and compile_shaders_sm2.py parses them from the files' own compile vs_2_0 … / compile ps_2_0 … statements instead of a hand-typed list, because an earlier hand-typed table had invented wrong names.

Compilation is an offline step run by hand. It cross-builds a small D3DCompile() tool, fxc_tool.cpp, and runs it under Wine in a prefix that holds the genuine Microsoft d3dcompiler_47.dll (Wine's built-in replacement is not a substitute: it stops with E5017 on an ordinary alpha-test ternary, and a vertex-shader probe it did compile came out as a 1.4 MB fully unrolled blob because it has no relative addressing, where the real compiler's 72-bone skinning shader is 2,160 bytes; plan_dx9.md records this as its D9-1 gate), with exactly one flag, D3DCOMPILE_OPTIMIZATION_LEVEL3. It deliberately bypasses the DXVK wrapper, because compiling never opens a device and the wrapper's DXVK gate would reject the run for an unrelated reason.

How close the bytecode is to Microsoft's

compare_against_fxb.py compares CNA's compiled shaders with the bytecode Microsoft shipped inside FNA's copies of the original .fxb files. It first strips D3D9 comment tokens, because the constant table and the compiler's creator string live in comment blocks and legitimately differ between compiler versions and source paths; a raw byte comparison would report zero matches even for identical instruction streams. The run recorded on 2026-07-14 matched 61 of 66 instruction streams exactly. The five that differ are all per-pixel-lighting vertex shaders (VSBasicPixelLighting, VSBasicPixelLightingTx, VSSkinnedPixelLightingOneBone, …TwoBones, …FourBones), and the cause is the compiler version (Microsoft used the XNA-era D3DCompiler_43, CNA uses d3dcompiler_47), not a flag: six flag combinations were tried and none matched.

The project owner's recorded decision is that CNA compiles its own shaders from Microsoft's sources and the .fxb bytes are a verification oracle only, never shipped. That turned the five differing shaders into an obligation to prove equivalence on real XNA output. CNA's plan records four of them proven on 2026-07-16 at 0/65,536 differing pixels (lit_textured_quad_pixellighting and the one-, two- and four-bone skinned_pixellighting scenes), each first shown to be discriminating by a vertex-lit twin that differs from real XNA by 23,409 pixels. The fifth, untextured VSBasicPixelLighting, is recorded as permanently blocked: it takes a 12-byte position-only vertex, a CNA vertex layout that does not exist (see the next section). These are CNA's dated records; nothing was re-run for this page.

Which stock-effect variants a draw can reach

XNA selects a stock-effect variant with an integer ShaderIndex. DIRECTX9 runs Microsoft's real variants, so a variant is drawable only if a CNA vertex layout supplies the input structure that variant's vertex shader declares. D3D9EffectDraw.cpp checks the combination before drawing BasicEffect:

That is 12 of BasicEffect's 32 variants. Every other combination throws a named no matching CNA vertex layout error rather than drawing with the wrong stride: the position-only (12-byte), position + normal (24 bytes, which collides with the colour + texture layout), position + normal + colour (28) and position + normal + texture + colour (36) inputs have no declaration. AlphaTestEffect (all 8 variants), EnvironmentMapEffect (all 16) and SkinnedEffect (all 18) are fully reachable once PreferPerPixelLighting and specularEnabled reach the draw parameters, which they have since 2026-07-16. A skinned vertex that also carries a colour (stride 56) has no XNA equivalent, because Microsoft's SkinnedEffect bytecode has no colour input; it is routed to CNA's own SkinnedVertexColor3D shader. PbrEffect and SkinnedPbrEffect use CNA's own HLSL in src/shaders/cna.

GraphicsProfile from a real capability structure

DIRECTX9 is the one renderer with a native capability structure to consult, so it is the one where GraphicsProfile reaches a hardware query (D3D9ProfileCapabilities.cpp):

• Construction floor. A device created under HiDef throws NoSuitableGraphicsDeviceException naming the reported versions when D3DCAPS9 reports less than vs_3_0/ps_3_0; Reach has no floor worth checking because every Direct3D 9 HAL device exceeds it.
• Adapter query. The descriptor's IsProfileSupported(HiDef) calls MeetsHiDefFloorEXT on the adapter caps, without creating a device: shader model 3, texture size at least 4096, MaxVolumeExtent at least 256, NumSimultaneousRTs at least 4, and no power-of-two texture restriction. Every other renderer answers true.
• Profile ceilings. Independently of the hardware, a Reach game is held to XNA's profile limits: texture edge 2048 (HiDef 4096), cube edge 512 (4096), volume textures none (256), simultaneous render targets 1 (4).

The profile checks exposed a defect shared by every renderer, which is why it is recorded here. Game's GraphicsDevice member was default-constructed with Reach before GraphicsDeviceManager existed, so graphics.GraphicsProfile = HiDef; graphics.ApplyChanges(); had no path to the live device. Current code closes it in GraphicsDeviceManager::applyToExistingRenderer (GraphicsDeviceManager.cpp), which calls SetGraphicsProfileEXT on the existing device before Reset; the ordering is traced on startup internals and the user view is Tutorial 152.

The lost-device lifecycle

Direct3D 9 is the one API where XNA's DeviceLost, DeviceResetting and DeviceReset events map onto a native mechanism, and the renderer implements it literally:

• When Present() returns D3DERR_DEVICELOST, the renderer marks itself lost and raises Lost. While lost, Present() only polls TestCooperativeLevel(), and any clear, draw or readback throws DeviceLostException.
• When the poll reports D3DERR_DEVICENOTRESET, PerformResetRecovery raises Resetting, releases every registered D3DPOOL_DEFAULT resource (dynamic vertex and index buffers, render targets), releases its own cached depth-stencil surface, calls IDirect3DDevice9::Reset, restores the viewport and default surfaces, and raises Reset. Managed-pool resources survive untouched; default-pool ones recreate lazily on next use, as XNA's dynamic buffers expect the game to refill them.
• Releasing the cached GetDepthStencilSurface() pointer matters: an application-held reference to a losable surface makes Reset fail. The app-initiated resize path had the same bug until it adopted the same release loop.
• A failed Reset is not fatal: recovery stays in the lost state and retries at the next Present(), and a failed resize keeps the previous size and retries every frame.

DXVK rarely loses a device, so the lifecycle is exercised through the framework-reserved debug keys every renderer receives: in Game::PollEvents, after input processing, a non-repeated F9 calls DebugSimulateContextLoss() and F10 calls DebugRestoreContext() (debug keys). On DIRECTX9 the restore goes through the real Reset(). DirectX9_Smoke asserts exact event counts, that Clear() throws while lost, and exact pixels after recovery.

Two neighbouring extension points are not implemented and should not be confused with this path: SetStringMarkerEXT() throws, and SetContextRecoveryEnabled() throws as well, but only after GraphicsDevice has already stored the new value, which changes the shared Texture2D CPU-shadow policy. A caller that catches that exception is left with the policy changed.

Checked native operations

Render-state, viewport, scissor, target and depth-surface calls go through narrow checked wrappers (REMED-GFX-092). A failing HRESULT is reported once, with the operation name and a decoded code; device-loss codes enter the lost-device lifecycle instead of becoming errors; and the affected state category is marked unsafe, so the next clear or draw throws until that complete state is applied again. A render-target transition snapshots every colour slot, the depth surface and the viewport first and rolls back on failure, and only when rollback cannot be proven does it block rendering. The wrappers also give the smoke tests per-instance, one-shot fault injection (InjectNextNativeFailureEXT) without mocking IDirect3DDevice9.

Render targets, textures and formats

• Storage and readback. A RenderTarget2D is a D3DUSAGE_RENDERTARGET texture in D3DPOOL_DEFAULT with exactly one level: mipMap is accepted and ignored, and GetData for a level above 0 throws NotSupportedException. Default-pool memory cannot be locked, so readback copies with GetRenderTargetData() into a system-memory surface and locks that; back-buffer readback does the same and handles the two byte orders the back buffer can have.
• MSAA. The sample count is clamped with IDirect3D9::CheckDeviceMultiSampleType(), all or nothing. A multisampled target draws into a separate multisample surface and is resolved into its sampleable texture with StretchRect() when the binding changes; the smoke test clears a 4x target to (200,210,220,255), unbinds it and requires the resolved texture to read exactly that. Cube render targets ignore the requested sample count.
• MRT. SetRenderTargets binds up to D3DCAPS9::NumSimultaneousRTs surfaces and throws, naming the limit, when asked for more instead of silently binding fewer. A cube face inside a multi-target set is refused. The smoke test proves multi-attachment binding and clear semantics; it does not prove a shader with several colour outputs, so it is not a G-buffer draw claim.
• Volume and cube textures. CreateTexture3D returns null (the renderer-wide "unsupported" answer) when MaxVolumeExtent is 0, and CreateTextureCube when the device lacks D3DPTEXTURECAPS_CUBEMAP. The smoke test writes 32 distinct bytes into the off-origin 2×2×2 box at (1,1,1) of a 4×4×4 volume and requires an exact readback of that box: it proves LockBox() row- and slice-pitch copying in both directions, not a rendering path or mip generation.
• Formats. The family has an internal D3DFORMAT mapping but no ClassifySurfaceFormatEXT override, so the public texture constructors apply the framework rule: SurfaceFormat::Color only, anything else throws. The back buffer honours the requested format, with one substitution: Color maps to D3DFMT_A8B8G8R8, which Direct3D 9 does not allow for a swap chain, so the back buffer is created as D3DFMT_A8R8G8B8.
• Sampling a target as a texture. A RenderTarget2D or RenderTargetCube handed to a stock effect as its texture is a different concrete renderer class from an ordinary texture, and GpuDrawParams carries only the common ITextureRenderer pointer. The stock-effect dispatch used to static_cast that pointer to the plain-texture class, which passed a garbage IDirect3DTexture9* to SetTexture and crashed later on DXVK's shader-compiler thread, while SpriteBatch silently drew untextured. D3D9EffectDraw.cpp and D3D9SpriteBatch.cpp now resolve the native texture with a dynamic_cast to either class (ResolveD3D9TextureEXT, plus a cube counterpart). DirectX9_RenderTarget_EffectSource, DirectX9_RenderTarget_ProducerConsumer and DirectX9_RenderTarget_SamplingOrientation are registered for this renderer as cross-renderer controls.

What the capability answers do and do not say

DIRECTX9 is the only renderer without a SupportsCapability() override, so its answers are the permissive interface defaults (capability reporting). Two consequences are easy to miss. MultiSampleAntiAliasing reports true even when CheckDeviceMultiSampleType() rejects every count above one, because the probe lives in the target-creation path, not in the public query. And AnisotropicFiltering is answered without consulting D3DCAPS9::MaxAnisotropy: ApplySamplerState writes the requested MaxAnisotropy into D3DSAMP_MAXANISOTROPY unclamped (only the compiled-effect path clamps to the caps). OcclusionQuery is honest in practice: the query is created only when CreateQuery(D3DQUERYTYPE_OCCLUSION, nullptr) succeeds. Probe the operation, or read the capability profile, before relying on these answers.

SpriteBatch projection: the half-pixel offset and the zFarPlane sign

Real XNA's Direct3D 9 SpriteBatch bakes a half-texel correction into its orthographic projection, because Direct3D 9 puts texel centres at integer coordinates. D3D9SpriteBatchRenderer::BuildMatrixTransformEXT (D3D9SpriteBatch.cpp) reproduces it: it builds CreateOrthographicOffCenter(0, W, H, 0, 0, -1) and then shifts the translation terms by half a pixel, M41 += -0.5 * M11 and M42 += -0.5 * M22. FNA's modern-API SpriteBatch has no such term, which is why the formula was measured against real XNA rather than copied. The oracle recorded one trap while mutation-testing it: a 1×1 texture stretched over a rectangle stays pixel-exact with the offset removed, because a single texel has nothing to shift between, while the four-colour rotated and flipped scenes diverge by 4,800 pixels. Only a multi-texel scene with sharp content boundaries observes the offset.

The last argument, zFarPlane = -1, is the second finding. The original code passed zFarPlane = 1, which makes the projection's Z row M33 = 1/(zNear - zFar) = -1, M43 = 0, so a sprite's depth became Z' = -layerDepth. Direct3D 9's clip-space Z range is [0, 1], not OpenGL's [-1, 1], so every sprite with a layerDepth above 0 fell outside the clip volume and was removed by the fixed-function clip test with no error, whatever the depth-stencil state. Every earlier scene drew at the default layerDepth 0 and could not see it; the sort-mode scenes, which overlap two translucent sprites at depths 0 and 1, rendered only the first one. With zFarPlane = -1 the Z row is the identity (M33 = 1), XNA's documented [0, 1] range (0 front, 1 back) lands inside the clip volume, and the X/Y half-pixel terms are unaffected. Reverting the fix turns the sort-mode test red, which is the mutation check the oracle README records.

The oracle, as far as this renderer is concerned

CTest D3D9_XNA_Diff runs run-oracle-corpus-diff.sh over every scene with cna_oracle_render and diffs each image against the committed real-XNA reference at tolerance 0; it needs only the Direct3D 9 prefix, not the XNA one. A scene is a small declarative text file that both the XNA program and CNA parse; the simplest, colored3d.scene, is ten key/value lines after its four-line header comment:

width=256
height=256
profile=HiDef
clearcolor=100,149,237,255
vertexcolor=true
lighting=false
primitive=TriangleList
vertex=-0.6,-0.6,0,255,0,0,255
vertex=0.0,0.7,0,0,255,0,255
vertex=0.6,-0.6,0,0,0,255,255

With vertex colour on and lighting off, BasicEffect draws one red, green and blue cornered triangle under identity transforms over Color.CornflowerBlue (100,149,237,255). Matching corners establish vertex-colour plumbing; matching interior pixels also exercise Gouraud interpolation. The same format, with a few more keys, drives the fog, alpha-test, skinning and sprite scenes. Method, provenance, denominators and the exact exit conditions are on Using the XNA oracle as evidence.

What the corpus does not cover matters as much. It has cull-mode scenes but no viewport or scissor scene, and the smoke test's resize check proves only a full-size viewport after reset, so CNA's plan calling Direct3D 9 viewport and scissor "oracle-proven" overstates it: the supportable claim is native implementation plus broad draw coverage. Texture3D has no scene. SpriteSortMode.Immediate differs from Deferred only in when work is submitted, which a raster diff cannot observe, and SpriteSortMode.Texture sorts by an implementation-defined identity hash, so no deterministic expected image can exist. Every result also inherits the host: one GPU, DXVK, and synthesized D3DCAPS9.

Read in this order

1. DirectX9Renderer.hpp: the class comment, the profile hooks and the checked-operation list (some method comments predate later work; trust the .cpp).
2. DirectX9Renderer.cpp: device creation, presentation parameters, reset recovery, clears, targets and state.
3. D3D9EffectDraw.cpp: the stock-effect dispatch and its reachability table.
4. the vendored-shader README, then compare_against_fxb.py.
5. directx9_smoke_test.cpp: what each lettered check proves.