Tutorial 150: Build-Time FX Compilation with cna-content

CNA Tutorials  ·  CNA snapshot 009d40f5

XNA samples ship HLSL .fx source. What CNA's Effect(GraphicsDevice&, bytes) constructor and the XNB EffectReader load (Tutorial 128) is the compiled Direct3D 9 Effect Framework container that the XNA content build produced from it. This tutorial closes that gap end to end: write an effect, compile it into an .xnb with cna-content, and draw with it. The whole pipeline is:

Tint.fx  --cna-content-->  CNA.EffectSourceImporter   (resolves #include tree as build dependencies)
                            CNA.EffectSourceProcessor  (profile / defines / debug policy)
                            external fxc /T fx_2_0     (Windows compiler, or under wine)
                            CNA.XnbEffectWriter        (--format xnb only)
         -->  Content/Effects/Tint.xnb  --ContentManager::Load<std::shared_ptr<Effect>>-->  Effect

Prerequisites

• CNA at the next branch (git clone -b next https://github.com/libcna/cna.git, or check out 009d40f5dd085c4e674d3479675fac84b12b3e0a) and sharp-runtime on its next branch. The cna-content executable is the cna_content_tool CMake target and is part of an ordinary CNA build; the main/develop branches are alpha.1 and do not have it.
• A renderer that supports compiled effects (Tutorial 128): FNA3D always, or one of the other 13 identities with its CNA_*_COMPILED_EFFECTS option on. For example -DCNA_GRAPHICS_RENDERER=OPENGL33 -DCNA_EASYGL_COMPILED_EFFECTS=ON.
• An fxc at profile fx_2_0. It must be Microsoft's legacy compiler from the DirectX SDK (June 2010) — the compiler XNA's own Content Pipeline used. A modern standalone d3dcompiler_47 cannot write a legacy effect at all (it fails with E5017: Aborting due to not yet implemented feature: Write pass assignments) because fxc delegates the fx_2_0 path to d3dx9: the SDK's d3dx9_43.dll and D3DCompiler_43.dll (from its redistributable cabs) must sit beside fxc.exe. On Linux or macOS you run it through wine. CNA records the exact identity it used and the extraction commands in modules/renderers/fna3d/effects/README.md; the compiler is neither vendored nor redistributed by CNA.
• Windows-style target only. The compiled container is Direct3D 9 bytecode. The default XNB platform is windows, so you normally pass nothing; the processor refuses the Windows Phone platform (“The Windows Phone platform does not support custom shaders”), and the XNB writer refuses the Xbox 360 platform (big-endian, unverified) unless you pass --xnb-allow-unverified-xbox. There is no Xbox 360, Windows Phone or Metal shader output.

Step 1: write the effect

Create ContentSource/Effects/Tint.fx. It is ordinary XNA-style HLSL Effect Framework source, in the same shape as CNA's own compiler-tested fixture: a matrix, a colour, a texture and sampler, one vertex and one pixel shader at Shader Model 2.0, and one technique.

// ContentSource/Effects/Tint.fx  (Shader Model 2.0, so it also fits the Reach profile)
float4x4 WorldViewProjection;
float4   Tint = float4(1, 1, 1, 1);

texture Texture;
sampler TextureSampler = sampler_state
{
    Texture   = (Texture);
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU  = Clamp;
    AddressV  = Clamp;
};

struct VSInput
{
    float4 Position : POSITION0;
    float2 TexCoord : TEXCOORD0;
};

struct VSOutput
{
    float4 Position : POSITION0;
    float2 TexCoord : TEXCOORD0;
};

VSOutput MainVS(VSInput input)
{
    VSOutput output;
    output.Position = mul(input.Position, WorldViewProjection);
    output.TexCoord = input.TexCoord;
    return output;
}

float4 MainPS(VSOutput input) : COLOR0
{
    return tex2D(TextureSampler, input.TexCoord) * Tint;
}

technique Tinted
{
    pass P0
    {
        VertexShader = compile vs_2_0 MainVS();
        PixelShader  = compile ps_2_0 MainPS();
    }
}

Two rules worth knowing. CNA does not rewrite your compile vs_2_0 … statements, so Reach versus HiDef (Shader Model 2 versus 3) is your decision expressed in the source. And the importer records every #include "Common.fxh" in the include tree (up to 256 files, each up to 8 MiB, resolved relative to the including file and confined to the source root) as a build dependency, so editing a header rebuilds only the effects that include it. Also drop a PNG at ContentSource/Textures/checker.png; the same build converts it to a Texture2D XNB.

Step 2: compile it with cna-content

The command line is cna-content build <source> -o <output> --format xnb plus the compiler. Tell it where fxc lives, and on a non-Windows machine which launcher to run it through:

# Linux / macOS: run the Windows compiler through wine
cna-content build ContentSource -o Content --format xnb \
    --fx-compiler "$HOME/dxsdk/Utilities/bin/x86/fxc.exe" \
    --fx-compiler-launcher wine

# Windows: name the compiler, no launcher
cna-content build ContentSource -o Content --format xnb ^
    --fx-compiler "C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\Utilities\bin\x86\fxc.exe"

A directory build needs an output directory and preserves extensionless relative content names, so the result is:

Content/
├── .cna-content.lock
├── .cna-content-manifest.json
├── Effects/
│   └── Tint.xnb
└── Textures/
    └── checker.xnb

A single asset can be built explicitly, in which case the output path's extension must match --format: cna-content build ContentSource/Effects/Tint.fx -o Content/Effects/Tint.xnb --format xnb --fx-compiler …. Add --explain to see why each asset was rebuilt or skipped; an unchanged rebuild is a cheap no-op.

How the compiler is found

The compiler and the launcher are two independent axes, each resolved in its own order; the first one that is set wins:

The compiler is probed once per invocation (fxc /?), and the version it reports enters the build fingerprint, so switching compilers rebuilds instead of reusing another compiler's output. Wine is the launcher CNA documents and the only one whose host-to-Windows path translation has been measured. If you build content from CMake and CNA is part of your build, the cna_add_content() function wraps the same tool (TARGET, SOURCE_DIR, OUTPUT_DIR, FORMAT xnb, and so on).

Effect processor parameters

CNA.EffectSourceProcessor accepts exactly three parameters, set per asset in an optional .cna-content.json at the source root (or passed with --config). Unknown names are rejected. Each participates in the build fingerprint.

{
  "format": "CNA.ContentPipeline.Config",
  "version": 1,
  "assets": {
    "Effects/Tint.fx": {
      "parameters": {
        "profile": { "type": "string", "value": "hidef" },
        "defines": { "type": "string", "value": "USE_TINT=1" },
        "debug":   { "type": "bool",   "value": false }
      }
    }
  }
}

When the build fails

• No usable compiler — the first .fx fails with one complete explanation (“no usable effect compiler: …”) that lists the four ways to name a compiler, the wine launcher, the d3dx9_43.dll/D3DCompiler_43.dll requirement and the .fxb alternative, rather than failing once per asset.
• The compiler rejects the source — the build reports 'Tint.fx': the effect compiler (fxc …) rejected it followed by the compiler's own diagnostics, re-emitted with your authored file name and line rather than a temporary path. Warnings are logged, not fatal.
• The compiler wrote something that is not an Effect Framework 9.1 container — refused, never embedded. CNA checks the signature of the output before it will call it an XNA Effect.
• --format cnb — refused for effects. The CNB container reserves an Effect identifier but deliberately has no schema for one, because a .cnb holding Direct3D 9 bytecode would be unloadable on every renderer that is not Direct3D 9. Effects are XNB only.
• A missing #include — reported as Tint.fx(12): included file 'Common.fxh' does not exist (or, if it escapes the source root, refused).

Step 3: load and draw it

At run time nothing about the compiler matters any more: Tint.xnb is an ordinary XNB Effect asset, read by the EffectReader registered by the XNB built-ins. The renderer must report CompiledEffects; check it, then load by extensionless name.

#include <cstdio>
#include <memory>
#include "Microsoft/Xna/Framework/Game.hpp"
#include "Microsoft/Xna/Framework/GraphicsDeviceManager.hpp"
#include "Microsoft/Xna/Framework/Graphics/Effect.hpp"
#include "Microsoft/Xna/Framework/Graphics/EffectParameter.hpp"
#include "Microsoft/Xna/Framework/Graphics/VertexBuffer.hpp"
#include "Microsoft/Xna/Framework/Graphics/VertexPositionTexture.hpp"
#include "Microsoft/Xna/Framework/Content/ContentLoadException.hpp"
#include "CNA/GraphicsCapability.hpp"

using namespace Microsoft::Xna::Framework;
using namespace Microsoft::Xna::Framework::Graphics;
using Microsoft::Xna::Framework::Content::ContentLoadException;

class TintGame final : public Game {
public:
    TintGame() : graphics_(this) {}

protected:
    void LoadContent() override {
        auto& device = getGraphicsDeviceProperty();

        if (!device.SupportsCapability(CNA::GraphicsCapability::CompiledEffects)) {
            throw std::runtime_error(
                "This renderer build cannot load compiled XNA effects "
                "(see Tutorial 128 for the per-renderer build option).");
        }

        getContentProperty().setRootDirectoryProperty("Content");
        try {
            tint_ = getContentProperty().Load<std::shared_ptr<Effect>>("Effects/Tint");
        } catch (const ContentLoadException& e) {
            std::fprintf(stderr, "Could not load Effects/Tint: %s\n", e.what());
            throw;
        }
        checker_ = getContentProperty().Load<Texture2D>("Textures/checker");

        // A quad in clip space, so the identity WorldViewProjection is enough.
        const VertexPositionTexture quad[6] = {
            { Vector3(-0.5f,  0.5f, 0.0f), Vector2(0.0f, 0.0f) },
            { Vector3( 0.5f,  0.5f, 0.0f), Vector2(1.0f, 0.0f) },
            { Vector3( 0.5f, -0.5f, 0.0f), Vector2(1.0f, 1.0f) },
            { Vector3(-0.5f,  0.5f, 0.0f), Vector2(0.0f, 0.0f) },
            { Vector3( 0.5f, -0.5f, 0.0f), Vector2(1.0f, 1.0f) },
            { Vector3(-0.5f, -0.5f, 0.0f), Vector2(0.0f, 1.0f) },
        };
        vb_ = std::make_unique<VertexBuffer>(
            device, VertexPositionTexture::getVertexDeclarationStatic(), 6, BufferUsage::WriteOnly);
        vb_->SetData(quad, 6);

        // Parameters come from the compiled effect's reflection; the subscript returns a pointer.
        if (auto* p = tint_->getParametersProperty()["WorldViewProjection"])
            p->SetValue(Matrix::getIdentityProperty());
        if (auto* p = tint_->getParametersProperty()["Tint"])
            p->SetValue(Vector4(1.0f, 0.6f, 0.3f, 1.0f));
        if (auto* p = tint_->getParametersProperty()["Texture"])
            p->SetValue(&checker_);                    // Texture2D* (a pointer)
    }

    void Draw(const GameTime&) override {
        auto& device = getGraphicsDeviceProperty();
        device.Clear(Color::CornflowerBlue);
        device.SetVertexBuffer(vb_.get());
        for (auto& pass : tint_->getCurrentTechniqueProperty()->getPassesProperty()) {
            pass.Apply();
            device.DrawPrimitives(PrimitiveType::TriangleList, 0, 2);
        }
        // No Present(): Game presents in EndDraw().
    }

private:
    GraphicsDeviceManager          graphics_;
    std::shared_ptr<Effect>        tint_;
    Texture2D                      checker_;
    std::unique_ptr<VertexBuffer>  vb_;
};

int main() { TintGame game; game.Run(); }

The vertex declaration must supply the inputs the effect's vertex shader declares: POSITION0 and TEXCOORD0 here, which is exactly VertexPositionTexture (a Vector3 position is widened to the shader's float4 with w = 1). You should see a tinted checkerboard quad on cornflower blue. If Load throws, the ContentLoadException wraps the real cause: an ArgumentException for a malformed container, or a NotSupportedException when the renderer lacks CompiledEffects.

The no-compiler route: .fxb and existing XNB

If you already have compiled bytes you need no compiler, and none of the steps above about fxc apply:

• A .fxb file (the raw Effect Framework binary, optionally behind XNA 4.0's own wrapper token) is imported by CNA.CompiledEffectImporter, which only checks the container signature: cna-content build ContentSource -o Content --format xnb turns it into Content/Effects/Name.xnb with no --fx-compiler.
• An .xnb Effect that XNA's own content build produced can be loaded directly: put it under your content root and Load<std::shared_ptr<Effect>> it. It is still gated by CompiledEffects on the active renderer.
• An XNA .contentproj can be built as a whole (cna-content build MyProject.contentproj -o Content): it carries its own platform, profile, compression and per-asset processors, and --fx-compiler reaches a project build too.

What this does not do

• No runtime compilation. Handing .fx text, DXBC or MonoGame MGFX to Effect(device, bytes) fails (ArgumentException, or NotSupportedException for MGFX).
• No other shader formats. The compiled container is Direct3D 9 bytecode that CNA's MojoShader-based translators (or, on DIRECTX9, the native device) run; there is no Metal, Xbox 360 or Windows Phone effect output.
• Not every renderer. Fourteen of 25 identities run compiled effects; on the others use a stock effect or a source ShaderEffect (Tutorial 151).
• Not proven against genuine fxc by CNA. If your compile fails on a shader that XNA accepts, treat it as a compatibility report to file rather than a mistake in this tutorial.

Next steps

• Tutorial 128: Load Compiled XNA Effects on Supported Renderers — the runtime side, renderer options and failure taxonomy.
• Tutorial 53: Shader Uniforms and EffectParameter — reflecting techniques, passes, arrays and annotations.
• Effects reference: Compiled XNA effects.