Tutorial 153: Image-Based Lighting with PbrEffect

CNA Tutorials  ·  CNA snapshot 009d40f5

Tutorial 114 lit its metals with up to three directional lights and a flat ambient colour, which is why a mirror-smooth sphere in an empty scene looks dark: there is nothing to reflect. Image-based lighting fixes that by letting a whole environment — a sky, a room, a studio — light the material. At this snapshot both PBR effects accept an ImageBasedLightEXT, and the engine layer can generate everything it needs from a cube map. In this tutorial you build a grid of twelve spheres, metal above and plastic below, with roughness increasing left to right, lit by nothing but a procedural sky.

What image-based lighting adds

A lit surface needs two integrals over the environment: the diffuse light arriving from the hemisphere around its normal, and the specular light reflected toward the viewer, which depends on how rough the surface is. Evaluating either per pixel from scratch is far too slow, so PBR renderers use the split-sum approximation and precompute three products. CNA's PbrEffect takes them in one struct:

ImageBasedLightEXT::IsValidEXT() is true only when all three textures are non-null, the mip count is at least 1 and the intensity is not negative. An invalid bundle is treated as “no environment” and the effect falls back to its flat ambient colour. The struct holds raw pointers: the effect copies the struct, not the textures, so keep the unique_ptrs that own them alive for as long as anything draws with the effect.

Which renderers honour it

Not every renderer shades from an environment. A renderer without the shader variant accepts the ImageBasedLightEXT and ignores it — and because an environment replaces the flat ambient term (adding both would count the same light twice), PbrEffect zeroes its ambient colour whenever a valid environment is set. On a renderer that ignores IBL you therefore get a surface with no ambient light at all: the safer of two wrong pictures, but still wrong. So ask first, exactly as this tutorial's program does.

The query is a method on the device, so the branch is one line:

const bool useIbl = device.SupportsCapability(CNA::GraphicsCapability::ThreeD)
                 && device.SupportsImageBasedLightingEXT();

CNA's own IBL test additionally asks for CustomEffects before it runs, because it also exercises shadow interaction; for the receiving side alone the query above is the one that says whether the shader consumes the environment.

Set up the project

Use the project layout from Tutorial 03 — CNA and sharp-runtime cloned as siblings on their next branches (see Tutorial 02). The only differences are the engine-layer option and one source file:

cmake_minimum_required(VERSION 3.20)
project(IblDemo LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 23)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

set(CNA_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../cna")

# Choose a renderer that honours IBL (see the table above) and turn the engine layer on
# BEFORE add_subdirectory().
set(CNA_GRAPHICS_RENDERER "OPENGLES3" CACHE STRING "CNA graphics renderer")
set(CNA_CNAEXT ON CACHE BOOL "Build the CNA::Graphics engine layer")
add_subdirectory(${CNA_DIR} ${CMAKE_BINARY_DIR}/cna)

add_executable(IblDemo main.cpp)
target_link_libraries(IblDemo PRIVATE CNA)

Forgetting CNA_CNAEXT gives the classic symptom: #include "CNA/Graphics/EnvironmentProcessor.hpp" compiles and CNA::Graphics::EnvironmentProcessor is “not a member”, because every engine-layer header is empty without the option.

Step 1: includes

Put everything in main.cpp. The engine-layer include is the only gated one:

#include "CNA/Graphics/EnvironmentProcessor.hpp"
#include "CNA/GraphicsCapability.hpp"
#include "Microsoft/Xna/Framework/Color.hpp"
#include "Microsoft/Xna/Framework/Game.hpp"
#include "Microsoft/Xna/Framework/GameTime.hpp"
#include "Microsoft/Xna/Framework/GraphicsDeviceManager.hpp"
#include "Microsoft/Xna/Framework/Matrix.hpp"
#include "Microsoft/Xna/Framework/Vector3.hpp"
#include "Microsoft/Xna/Framework/Graphics/BlendState.hpp"
#include "Microsoft/Xna/Framework/Graphics/CubeMapFace.hpp"
#include "Microsoft/Xna/Framework/Graphics/DepthStencilState.hpp"
#include "Microsoft/Xna/Framework/Graphics/GraphicsAdapter.hpp"
#include "Microsoft/Xna/Framework/Graphics/GraphicsDevice.hpp"
#include "Microsoft/Xna/Framework/Graphics/GraphicsProfile.hpp"
#include "Microsoft/Xna/Framework/Graphics/ImageBasedLightEXT.hpp"
#include "Microsoft/Xna/Framework/Graphics/PbrEffect.hpp"
#include "Microsoft/Xna/Framework/Graphics/PrimitiveType.hpp"
#include "Microsoft/Xna/Framework/Graphics/RasterizerState.hpp"
#include "Microsoft/Xna/Framework/Graphics/SurfaceFormat.hpp"
#include "Microsoft/Xna/Framework/Graphics/Texture2D.hpp"
#include "Microsoft/Xna/Framework/Graphics/TextureCube.hpp"
#include "Microsoft/Xna/Framework/Graphics/VertexPositionNormalTangentTexture.hpp"

#include <cmath>
#include <memory>
#include <vector>

using namespace Microsoft::Xna::Framework;
using namespace Microsoft::Xna::Framework::Graphics;
using CNA::Graphics::EnvironmentProcessor;

Step 2: an environment

A real project loads a panorama and converts it (shown below). To keep this tutorial self-contained we build a tiny procedural sky instead: a bright face above, a dark face below and a blue horizon. The cube-map face order is XNA's: face 2 is PositiveY (up) and face 3 is NegativeY (down).

    // A tiny procedural sky: a bright sky face above, a dark ground face below, a blue horizon.
    std::unique_ptr<TextureCube> BuildSkyCube(GraphicsDevice& device, int size)
    {
        auto cube = std::make_unique<TextureCube>(device, size, false, SurfaceFormat::Color);
        const std::size_t texels = static_cast<std::size_t>(size) * size;
        const std::vector<Color> horizon(texels, Color(90, 120, 200, 255));
        const std::vector<Color> sky(texels, Color(190, 220, 255, 255));
        const std::vector<Color> ground(texels, Color(40, 40, 50, 255));
        for (int face = 0; face < 6; ++face)
        {
            const std::vector<Color>& data = face == 2 ? sky : (face == 3 ? ground : horizon);
            cube->SetData(static_cast<CubeMapFace>(face), data.data(), static_cast<int>(data.size()));
        }
        return cube;
    }

Every product the processor makes is an 8-bit SurfaceFormat::Color texture, so the environment's dynamic range is quantised before it is convolved — fine for a sky like this, and a real limit if you were hoping for a true HDR probe.

With a real panorama (a 2:1 equirectangular image) the conversion is one call; a face a quarter of the panorama's width roughly preserves its detail:

Texture2D panorama = getContentProperty().Load<Texture2D>("sky/studio");
std::unique_ptr<TextureCube> cube =
    processor.convertEquirectangular(&panorama, panorama.getWidthProperty() / 4);

Step 3: the three products, and the light

In LoadContent(), after the gate, generate the products and assemble the struct. The sizes here are deliberately small — the work is CPU-side and happens once at load, and a coarse irradiance cube is enough because irradiance is very low-frequency:

        if (useIbl_)
        {
            sky_ = BuildSkyCube(device, 32);

            EnvironmentProcessor processor(device);
            irradiance_ = processor.generateIrradiance(sky_.get(), 16, 16);
            specular_   = processor.generatePrefilteredSpecular(sky_.get(), 64, kMips, 32);
            brdfLut_    = processor.generateBrdfLut(64, 64);

            ImageBasedLightEXT ibl;
            ibl.Irradiance          = irradiance_.get();
            ibl.PrefilteredSpecular = specular_.get();
            ibl.BrdfLut             = brdfLut_.get();
            ibl.PrefilteredMipCount = kMips;
            ibl.Intensity           = 1.0f;
            effect_->setImageBasedLightEXT(ibl);
        }

kMips is 5, so the prefiltered cube carries five roughness levels from mip 0 (a mirror) to mip 4 (fully rough). The PrefilteredMipCount you store must equal the mipCount you generated: the shader maps roughness to mip as roughness * (count - 1).

Step 4: spheres for a PbrEffect

PbrEffect draws a 48-byte vertex: position, normal, a tangent whose w is the handedness, and one UV set. A plain struct of that layout, drawn with the VertexPositionNormalTangentTexture declaration, is the shape CNA's own examples use (do not pass an array of the VertexPositionNormalTangentTexture class itself: it has a virtual base, so its memory layout is not the GPU layout). The sphere is a plain triangle list, so no index buffer is needed:

    // The 48-byte record PbrEffect draws: position, normal, tangent (w = handedness), uv.
    struct PbrVertex
    {
        float x, y, z;
        float nx, ny, nz;
        float tx, ty, tz, tw;
        float u, v;
    };
    static_assert(sizeof(PbrVertex) == 48, "the PBR stream is 48 bytes per vertex");

    // A UV sphere as a plain triangle list, so no index buffer is needed.
    std::vector<PbrVertex> BuildSphere(int rings, int segments)
    {
        const float pi = 3.14159265358979f;
        const auto point = [&](int ring, int seg) {
            const float v = static_cast<float>(ring) / rings;
            const float u = static_cast<float>(seg) / segments;
            const float theta = v * pi;
            const float phi = u * 2.0f * pi;
            const float nx = std::sin(theta) * std::cos(phi);
            const float ny = std::cos(theta);
            const float nz = std::sin(theta) * std::sin(phi);
            // The tangent points along increasing longitude.
            return PbrVertex{nx, ny, nz, nx, ny, nz, -std::sin(phi), 0.0f, std::cos(phi), 1.0f, u, v};
        };

        std::vector<PbrVertex> out;
        out.reserve(static_cast<std::size_t>(rings) * segments * 6);
        for (int r = 0; r < rings; ++r)
        {
            for (int s = 0; s < segments; ++s)
            {
                const PbrVertex a = point(r, s), b = point(r + 1, s);
                const PbrVertex c = point(r + 1, s + 1), d = point(r, s + 1);
                out.insert(out.end(), {a, b, c, a, c, d});
            }
        }
        return out;
    }

The material is set up once and its factors changed per sphere. Notice what is not here: no textures and no lights. The effect's three directional lights are disabled and its base colour is a factor.

        auto& device = getGraphicsDeviceProperty();

        // Ask first. Elsewhere an ImageBasedLightEXT is accepted and ignored -- and because it
        // replaces the flat ambient term, ignoring it leaves the sphere without ambient light.
        useIbl_ = device.SupportsCapability(CNA::GraphicsCapability::ThreeD)
               && device.SupportsImageBasedLightingEXT();

        sphere_ = BuildSphere(24, 48);
        effect_ = std::make_unique<PbrEffect>(device);
        effect_->setViewProperty(Matrix::CreateLookAt(Vector3(0.0f, 0.0f, 12.0f), Vector3::Zero,
                                                      Vector3(0.0f, 1.0f, 0.0f)));
        effect_->setProjectionProperty(
            Matrix::CreatePerspectiveFieldOfView(0.7f, 960.0f / 540.0f, 0.1f, 100.0f));
        effect_->DirectionalLight0.setEnabledProperty(false);
        effect_->DirectionalLight1.setEnabledProperty(false);
        effect_->DirectionalLight2.setEnabledProperty(false);
        effect_->setDiffuseColorProperty(Vector3(0.95f, 0.72f, 0.35f));

Step 5: draw the grid

Two rows, six columns. The top row is metal (metallic = 1), the bottom row is a dielectric (metallic = 0); roughness rises from 0.05 to 1.0 across each row. Each sphere gets its own world matrix and roughness, then Apply() and a draw. There is no Present() call: the Game presents in EndDraw.

    void Draw(const GameTime&) override
    {
        auto& device = getGraphicsDeviceProperty();
        device.Clear(Color(24, 26, 32, 255));
        device.setRasterizerStateProperty(RasterizerState::CullNone);
        device.setDepthStencilStateProperty(DepthStencilState::Default);
        device.setBlendStateProperty(BlendState::Opaque);

        constexpr int kColumns = 6;
        for (int row = 0; row < 2; ++row)
        {
            for (int column = 0; column < kColumns; ++column)
            {
                effect_->setMetallicFactorProperty(row == 0 ? 1.0f : 0.0f);
                effect_->setRoughnessFactorProperty(0.05f + 0.19f * column);
                const float x = (column - (kColumns - 1) * 0.5f) * 2.2f;
                const float y = row == 0 ? 1.3f : -1.3f;
                effect_->setWorldProperty(Matrix::CreateTranslation(x, y, 0.0f));

                effect_->Apply();
                device.DrawUserPrimitives(PrimitiveType::TriangleList, sphere_.data(), 0,
                                          static_cast<int>(sphere_.size()) / 3,
                                          VertexPositionNormalTangentTexture::getVertexDeclarationStatic());
            }
        }
    }

The complete program

Everything above, assembled. The constructor requests the HiDef graphics profile when the adapter offers it, as CNA's engine-layer examples do; the default Reach profile refuses several things the engine layer wants, float render targets among them (Tutorial 152 covers Reach and HiDef). Save it as main.cpp:

#include "CNA/Graphics/EnvironmentProcessor.hpp"
#include "CNA/GraphicsCapability.hpp"
#include "Microsoft/Xna/Framework/Color.hpp"
#include "Microsoft/Xna/Framework/Game.hpp"
#include "Microsoft/Xna/Framework/GameTime.hpp"
#include "Microsoft/Xna/Framework/GraphicsDeviceManager.hpp"
#include "Microsoft/Xna/Framework/Matrix.hpp"
#include "Microsoft/Xna/Framework/Vector3.hpp"
#include "Microsoft/Xna/Framework/Graphics/BlendState.hpp"
#include "Microsoft/Xna/Framework/Graphics/CubeMapFace.hpp"
#include "Microsoft/Xna/Framework/Graphics/DepthStencilState.hpp"
#include "Microsoft/Xna/Framework/Graphics/GraphicsAdapter.hpp"
#include "Microsoft/Xna/Framework/Graphics/GraphicsDevice.hpp"
#include "Microsoft/Xna/Framework/Graphics/GraphicsProfile.hpp"
#include "Microsoft/Xna/Framework/Graphics/ImageBasedLightEXT.hpp"
#include "Microsoft/Xna/Framework/Graphics/PbrEffect.hpp"
#include "Microsoft/Xna/Framework/Graphics/PrimitiveType.hpp"
#include "Microsoft/Xna/Framework/Graphics/RasterizerState.hpp"
#include "Microsoft/Xna/Framework/Graphics/SurfaceFormat.hpp"
#include "Microsoft/Xna/Framework/Graphics/Texture2D.hpp"
#include "Microsoft/Xna/Framework/Graphics/TextureCube.hpp"
#include "Microsoft/Xna/Framework/Graphics/VertexPositionNormalTangentTexture.hpp"

#include <cmath>
#include <memory>
#include <vector>

using namespace Microsoft::Xna::Framework;
using namespace Microsoft::Xna::Framework::Graphics;
using CNA::Graphics::EnvironmentProcessor;

namespace
{
    // The 48-byte record PbrEffect draws: position, normal, tangent (w = handedness), uv.
    struct PbrVertex
    {
        float x, y, z;
        float nx, ny, nz;
        float tx, ty, tz, tw;
        float u, v;
    };
    static_assert(sizeof(PbrVertex) == 48, "the PBR stream is 48 bytes per vertex");

    // A UV sphere as a plain triangle list, so no index buffer is needed.
    std::vector<PbrVertex> BuildSphere(int rings, int segments)
    {
        const float pi = 3.14159265358979f;
        const auto point = [&](int ring, int seg) {
            const float v = static_cast<float>(ring) / rings;
            const float u = static_cast<float>(seg) / segments;
            const float theta = v * pi;
            const float phi = u * 2.0f * pi;
            const float nx = std::sin(theta) * std::cos(phi);
            const float ny = std::cos(theta);
            const float nz = std::sin(theta) * std::sin(phi);
            // The tangent points along increasing longitude.
            return PbrVertex{nx, ny, nz, nx, ny, nz, -std::sin(phi), 0.0f, std::cos(phi), 1.0f, u, v};
        };

        std::vector<PbrVertex> out;
        out.reserve(static_cast<std::size_t>(rings) * segments * 6);
        for (int r = 0; r < rings; ++r)
        {
            for (int s = 0; s < segments; ++s)
            {
                const PbrVertex a = point(r, s), b = point(r + 1, s);
                const PbrVertex c = point(r + 1, s + 1), d = point(r, s + 1);
                out.insert(out.end(), {a, b, c, a, c, d});
            }
        }
        return out;
    }

    // A tiny procedural sky: a bright sky face above, a dark ground face below, a blue horizon.
    std::unique_ptr<TextureCube> BuildSkyCube(GraphicsDevice& device, int size)
    {
        auto cube = std::make_unique<TextureCube>(device, size, false, SurfaceFormat::Color);
        const std::size_t texels = static_cast<std::size_t>(size) * size;
        const std::vector<Color> horizon(texels, Color(90, 120, 200, 255));
        const std::vector<Color> sky(texels, Color(190, 220, 255, 255));
        const std::vector<Color> ground(texels, Color(40, 40, 50, 255));
        for (int face = 0; face < 6; ++face)
        {
            const std::vector<Color>& data = face == 2 ? sky : (face == 3 ? ground : horizon);
            cube->SetData(static_cast<CubeMapFace>(face), data.data(), static_cast<int>(data.size()));
        }
        return cube;
    }
}

class IblGame final : public Game
{
public:
    IblGame() : graphics_(this)
    {
        if (GraphicsAdapter::getDefaultAdapterProperty().IsProfileSupported(GraphicsProfile::HiDef))
            graphics_.setGraphicsProfileProperty(GraphicsProfile::HiDef);
        graphics_.setPreferredBackBufferWidthProperty(960);
        graphics_.setPreferredBackBufferHeightProperty(540);
    }

protected:
    void LoadContent() override
    {
        auto& device = getGraphicsDeviceProperty();

        // Ask first. Elsewhere an ImageBasedLightEXT is accepted and ignored -- and because it
        // replaces the flat ambient term, ignoring it leaves the sphere without ambient light.
        useIbl_ = device.SupportsCapability(CNA::GraphicsCapability::ThreeD)
               && device.SupportsImageBasedLightingEXT();

        sphere_ = BuildSphere(24, 48);
        effect_ = std::make_unique<PbrEffect>(device);
        effect_->setViewProperty(Matrix::CreateLookAt(Vector3(0.0f, 0.0f, 12.0f), Vector3::Zero,
                                                      Vector3(0.0f, 1.0f, 0.0f)));
        effect_->setProjectionProperty(
            Matrix::CreatePerspectiveFieldOfView(0.7f, 960.0f / 540.0f, 0.1f, 100.0f));
        effect_->DirectionalLight0.setEnabledProperty(false);
        effect_->DirectionalLight1.setEnabledProperty(false);
        effect_->DirectionalLight2.setEnabledProperty(false);
        effect_->setDiffuseColorProperty(Vector3(0.95f, 0.72f, 0.35f));

        if (useIbl_)
        {
            sky_ = BuildSkyCube(device, 32);

            EnvironmentProcessor processor(device);
            irradiance_ = processor.generateIrradiance(sky_.get(), 16, 16);
            specular_   = processor.generatePrefilteredSpecular(sky_.get(), 64, kMips, 32);
            brdfLut_    = processor.generateBrdfLut(64, 64);

            ImageBasedLightEXT ibl;
            ibl.Irradiance          = irradiance_.get();
            ibl.PrefilteredSpecular = specular_.get();
            ibl.BrdfLut             = brdfLut_.get();
            ibl.PrefilteredMipCount = kMips;
            ibl.Intensity           = 1.0f;
            effect_->setImageBasedLightEXT(ibl);
        }
        else
        {
            effect_->setAmbientLightColorProperty(Vector3(0.25f, 0.28f, 0.35f));
            effect_->EnableDefaultLighting();
        }
    }

    void Draw(const GameTime&) override
    {
        auto& device = getGraphicsDeviceProperty();
        device.Clear(Color(24, 26, 32, 255));
        device.setRasterizerStateProperty(RasterizerState::CullNone);
        device.setDepthStencilStateProperty(DepthStencilState::Default);
        device.setBlendStateProperty(BlendState::Opaque);

        constexpr int kColumns = 6;
        for (int row = 0; row < 2; ++row)
        {
            for (int column = 0; column < kColumns; ++column)
            {
                effect_->setMetallicFactorProperty(row == 0 ? 1.0f : 0.0f);
                effect_->setRoughnessFactorProperty(0.05f + 0.19f * column);
                const float x = (column - (kColumns - 1) * 0.5f) * 2.2f;
                const float y = row == 0 ? 1.3f : -1.3f;
                effect_->setWorldProperty(Matrix::CreateTranslation(x, y, 0.0f));

                effect_->Apply();
                device.DrawUserPrimitives(PrimitiveType::TriangleList, sphere_.data(), 0,
                                          static_cast<int>(sphere_.size()) / 3,
                                          VertexPositionNormalTangentTexture::getVertexDeclarationStatic());
            }
        }
    }

private:
    static constexpr int kMips = 5;

    GraphicsDeviceManager graphics_;
    bool useIbl_ = false;
    std::vector<PbrVertex> sphere_;
    std::unique_ptr<PbrEffect> effect_;

    // ImageBasedLightEXT stores raw pointers: these must outlive every draw that uses them.
    std::unique_ptr<TextureCube> sky_;
    std::unique_ptr<TextureCube> irradiance_;
    std::unique_ptr<TextureCube> specular_;
    std::unique_ptr<Texture2D>   brdfLut_;
};

int main()
{
    IblGame game;
    game.Run();
    return 0;
}

Build and run

cd my-ibl-demo
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
./build/IblDemo

What CNA's own IBL test establishes about this configuration (and therefore what to look for): an environment lights a surface that has no lights and no ambient colour at all; with the environment detached that same surface is black; roughness changes what a metal reflects, because the prefiltered mip ramp is really indexed; and a flat ambient colour and an environment are exclusive, never summed. So expect:

• Top row (metal). On the left, a nearly mirror-like sphere reflecting the environment's faces as broad colour bands — bright sky above, dark ground below, blue between. Moving right, the reflection blurs into a smooth gradient.
• Bottom row (plastic). Matte spheres brighter on the upper half (irradiance from the bright sky face) than the lower, with a faint sheen that broadens with roughness.
• On a renderer that reports false the program falls back to EnableDefaultLighting() plus a flat ambient, so you still get lit spheres, just without reflections.

Experiments

• Intensity. Set ibl.Intensity = 0.5f. CNA's white-furnace check uses half intensity because at 1.0 a white environment saturates the frame and can only reveal energy loss.
• Add a sun. Enable DirectionalLight0 with setEnabledProperty(true), a direction and a colour. Direct lights add to the environment term; only the flat ambient is replaced.
• Skinned meshes. SkinnedPbrEffect takes the same ImageBasedLightEXT through the same setImageBasedLightEXT(); CNA's test asserts that an identity-skinned quad matches the rigid one.
• See the sky. The engine layer's Skybox(device, cube) draws a cube map behind the scene, either through RenderPipeline::setSkybox() or by calling its draw(); the reference lists the sky and post-processing classes that combine with IBL.
• Shadows. A shadow removes the direct light only and leaves the environment term; Tutorial 154 adds cascaded shadows.

Limits

• Only the renderers in the table honour the environment; elsewhere it is silently ignored (with the ambient zeroed), so always branch on SupportsImageBasedLightingEXT().
• The engine layer is experimental, off by default, not built by any CI workflow, and its revision counter (18) is not an ABI promise.
• The products are 8-bit; irradiance and prefiltered specular are computed with a CPU sweep at load time, so choose sizes and sample counts to suit your load-time budget.
• We did not build or run this program. The API calls are those of CNA's own cnaext_ibl_test and cnaext_showcase_test, and the whole file compiles (syntax only) against the snapshot headers.

Where to go next

• Tutorial 154: Cascaded shadow maps — the other half of the receiving interface.
• Tutorial 114: PBR Materials — the material surface, including the seven maps this tutorial left empty.
• Tutorial 115: CNAEXT and the CNAEXT engine-layer reference.
• Tutorial 116 — post-processing, including the RenderPipeline you would put this scene in.