Tutorial 78: Multi-threading Considerations

CNA Tutorials  ·  Architecture

What is thread-safe in CNA?

Almost nothing in CNA is thread-safe by default. The following are the rules:

The fundamental constraint is that OpenGL and Vulkan (via CNA's backends) require all GPU operations to occur on a single thread that owns the context. Never call any CNA rendering API from a worker thread.

Worker threads for logic

Safe candidates for worker thread offloading:

• Physics simulation (position integration, broad-phase collision)
• AI pathfinding (A* graph search, behaviour tree evaluation)
• Procedural mesh generation (marching cubes, noise terrain)
• Asset decompression and decoding (PNG pixel extraction, OGG decode)
• Frustum culling / visibility determination (read-only scene data)
• Animation blending (matrix palette computation)

All of these produce plain C++ data (positions, matrices, vertex arrays) that the main thread can consume safely if you use double-buffering or a mutex-protected queue.

Job queue with std::jthread (C++23)

std::jthread (C++23) is a thread that automatically joins on destruction, eliminating the need to call join() manually and preventing join-on-destroyed-thread undefined behaviour.

// JobQueue.hpp
#pragma once
#include <thread>
#include <queue>
#include <mutex>
#include <condition_variable>
#include <functional>
#include <vector>
#include <atomic>

class JobQueue {
public:
    using Job = std::function<void()>;

    explicit JobQueue(int threadCount = 0) {
        if (threadCount <= 0)
            threadCount = static_cast<int>(
                std::thread::hardware_concurrency()) - 1;
        threadCount = std::max(1, threadCount);

        for (int i = 0; i < threadCount; ++i) {
            workers_.emplace_back([this](std::stop_token st) {
                while (!st.stop_requested()) {
                    Job job;
                    {
                        std::unique_lock<std::mutex> lock(mutex_);
                        cv_.wait(lock, [&] {
                            return !queue_.empty() || st.stop_requested();
                        });
                        if (st.stop_requested() && queue_.empty()) return;
                        job = std::move(queue_.front());
                        queue_.pop();
                    }
                    job();
                    --pending_;
                }
            });
        }
    }

    // JobQueue destructor automatically joins all jthreads
    ~JobQueue() = default;

    // Submit a job. Thread-safe.
    void Submit(Job job) {
        ++pending_;
        std::lock_guard<std::mutex> lock(mutex_);
        queue_.push(std::move(job));
        cv_.notify_one();
    }

    // Wait until all submitted jobs have completed.
    void WaitAll() {
        while (pending_.load() > 0)
            std::this_thread::yield();
    }

    int Pending() const { return pending_.load(); }

private:
    std::vector<std::jthread>   workers_;
    std::queue<Job>              queue_;
    std::mutex                   mutex_;
    std::condition_variable_any  cv_;
    std::atomic<int>             pending_{0};
};

Game logic on workers, draw on main

The standard pattern is to run expensive Update work on the job queue, then collect results on the main thread and draw:

// In Game::Update — dispatch expensive logic to workers
void Update(const GameTime& gt) override {
    float dt = static_cast<float>(gt.ElapsedGameTime.TotalSeconds());

    // Submit independent AI jobs (each entity is independent)
    for (auto& entity : entities_) {
        jobQueue_->Submit([&entity, dt]() {
            entity.UpdateAI(dt);        // read-only world, writes to entity only
            entity.IntegratePhysics(dt);
        });
    }

    // Meanwhile, do main-thread-only work:
    ProcessInput();
    UpdateCamera(dt);

    // Wait for all worker jobs to complete before drawing
    jobQueue_->WaitAll();

    // Now safe to read all entity positions for culling/rendering
    BuildRenderList();
}

void Draw(const GameTime&) override {
    auto& gd = getGraphicsDeviceProperty();
    gd.Clear(Color::CornflowerBlue);

    for (auto& entry : renderList_) {
        DrawObject(gd, entry);
    }
    gd.Present();
}

Lockless ring buffer for frame data

For high-frequency data exchange between a worker thread and the main thread (e.g. streaming audio sample positions, or particle positions), a single-producer single-consumer lock-free ring buffer avoids mutex overhead entirely:

// LocklessRingBuffer.hpp — SPSC, power-of-two capacity
template <typename T, size_t N>
class LocklessRingBuffer {
    static_assert((N & (N - 1)) == 0, "N must be a power of two");
public:
    // Called from producer thread only
    bool TryPush(const T& value) {
        size_t head = head_.load(std::memory_order_relaxed);
        size_t next = (head + 1) & (N - 1);
        if (next == tail_.load(std::memory_order_acquire)) return false; // full
        buffer_[head] = value;
        head_.store(next, std::memory_order_release);
        return true;
    }

    // Called from consumer thread only
    bool TryPop(T& value) {
        size_t tail = tail_.load(std::memory_order_relaxed);
        if (tail == head_.load(std::memory_order_acquire)) return false; // empty
        value = buffer_[tail];
        tail_.store((tail + 1) & (N - 1), std::memory_order_release);
        return true;
    }

private:
    std::array<T, N>        buffer_{};
    std::atomic<size_t>     head_{0};
    std::atomic<size_t>     tail_{0};
};

Audio thread: SDL3_mixer is thread-safe

CNA's audio system wraps SDL3_mixer. Unlike the rendering API, SDL3_mixer operations are internally thread-safe — you can call SoundEffectInstance::Play() or SoundEffect::CreateInstance() from a worker thread without a mutex. SDL3_mixer manages its own internal lock around the audio callback.

One exception: SoundEffect construction (which decodes the audio file and uploads it to SDL3_mixer) is not guaranteed to be thread-safe. Do this on the main thread in LoadContent or via the BackgroundLoader approach in Tutorial 77 (decode on worker, construct SoundEffect on main).

// Safe: triggering a sound from a physics worker thread
class PhysicsSystem {
public:
    void OnCollision(const CollisionEvent& ev) {
        // SoundEffectInstance::Play is thread-safe in SDL3_mixer
        if (ev.impactSpeed > 5.0f)
            collisionSound_->Play();
    }
private:
    SoundEffectInstance* collisionSound_;  // non-owning, loaded on main
};

Avoiding data races: double-buffering game state

If workers write entity positions while the main thread reads them for rendering, you have a data race. The cleanest solution is double-buffering: workers write to a "back" state buffer while the renderer reads the "front" buffer. At the end of each frame, swap the pointers:

struct EntityState { Vector3 position; float rotation; };

// Two copies: front (renderer reads) and back (workers write)
std::vector<EntityState> stateA_, stateB_;
std::vector<EntityState>* frontState_ = &stateA_;
std::vector<EntityState>* backState_  = &stateB_;

void Update(const GameTime& gt) override {
    float dt = static_cast<float>(gt.ElapsedGameTime.TotalSeconds());

    // Workers write to backState_ — frontState_ is untouched
    for (int i = 0; i < static_cast<int>(entities_.size()); ++i) {
        jobQueue_->Submit([this, i, dt]() {
            (*backState_)[i] = SimulateEntity(entities_[i], dt);
        });
    }
    jobQueue_->WaitAll();

    // Swap: back becomes new front
    std::swap(frontState_, backState_);
}

void Draw(const GameTime&) override {
    // Read from frontState_ — safe, no workers are writing to it now
    for (int i = 0; i < static_cast<int>(entities_.size()); ++i) {
        DrawEntityAt(entities_[i], (*frontState_)[i].position);
    }
}