I need to debug shutdown and lifetime behavior

CNA snapshot 009d40f5  ·  Development › Maintainer Handbook  ·  source links pinned to 009d40f5

Use this recipe for a crash, hang, leak or wrong callback that appears when an object dies: at Dispose, at the end of a scope, at process exit, when a device is torn down under live resources, or when a native or foreign callback fires late. The order in which CNA destroys things is written down once, on Ownership and shutdown, and every owner is mapped on the ownership master map; this page is the procedure: classify the failure shape, pick the tool that can actually see it, reproduce it in the right kind of process, and fix it without merely silencing a sanitizer. Everything was read from TARGET 009d40f5; no test, sanitizer or harness was run for it.

Find the owner: which lifetime is it?

Start by naming the lifetime that ended, not the crash site. The same use-after-free can look identical from a renderer, an audio mixer and a C API handle.

The destruction order to hold in your head

Declaration order in Game.hpp is construction order and its reverse is the teardown script: platform_ (destroyed last), platformInstallation_ (undoes the ambient install if construction throws), platformCapabilities_, eventBatch_, Components_, GraphicsDevice_, Content_, Window_, LaunchParameters_, Services_. Five consequences drive most bugs; the full trace is on the shutdown page.

• The platform outlives everything that borrows it. ~Game uninstalls the ambient accessor in its body, before any member dies, so teardown code must use its stored platform, never GetCurrentPlatform(), which would lazily create a fresh default when nothing is installed (code that must not do that asks HasCurrentPlatform()).
• Components_ is declared before GraphicsDevice_, so a component held through the shared_ptr overload of Add is destroyed after the device, content manager, window and services and must not call back through its Game.
• Game::Dispose(true) does not dispose the Game-owned device. It disposes IDisposable components, then Content_, then the registered device service (which raises DeviceDisposing, where UnloadContent runs). The device is disposed by ~GraphicsDevice or by game code; ~Game calls Dispose(false), which only marks the game disposed.
• The registry is drained while the renderer lives. GraphicsDevice::Dispose(bool) sets isDisposed_ first, raises Disposing, moves resources_ into a local and disposes each resource, then runs destroyNativeResources() (renderer, presenter, window) and releases its one video-subsystem reference. The registry tracks and never deletes.
• A weak token protects late resources. Each GraphicsResource holds a std::weak_ptr<void> of the device's resourceDeviceLifetime_ and checks expired() before calling back into the device; ~GraphicsDevice resets it after Dispose(). It extends nothing: a CNA::Graphics object must not outlive its device, and public resources must not outlive theirs (ADR 0001).

Borrowed and owned pointers

Never infer ownership from a getter or a “manager” name. For each pointer you touch, write down the lender, the borrower and what stops the lender dying first.

Callback barriers

A callback bug is usually decided in creation, generation change or teardown, not in the callback body. Name the barrier the code relies on and keep it on the normal and the failure path:

Before changing any callback, write five answers in the patch: who invokes it and on which thread; which locks are held; whether it can re-enter CNA; who owns its userdata and for how long; and what Stop, Close or Dispose does to in-flight calls. A moved-out object destroyed after the lock scope ends (as the C API registry does), and callbacks run with no lock held (as the sensor and dispatcher code does), are the two patterns that keep barriers deadlock-free.

Catalogue of real failure shapes

Each row is a shape that has occurred in CNA and the current code's answer. “Fixed” rows are teaching material and name the test or harness that pins the answer; “open” rows are source-read gaps recorded for maintainers, not verified defects.

Tooling that exists at TARGET

Sanitizer configurations

CNA_SANITIZE takes a comma-separated list (for example address,undefined), adds -fsanitize=… -fno-omit-frame-pointer -g to every CNA-owned target and to the enabled Sharp Runtime closure, and links the same flag; CNA_SANITIZE_OPTIMIZATION accepts DEFAULT, O0…O3. The configure step (BuildPerformance.cmake) refuses Emscripten, non-GNU/Clang compilers, AddressSanitizer with ThreadSanitizer, ThreadSanitizer with MemorySanitizer, CNA_ENABLE_IPO, and non-full CNA_DEBUG_INFO. Vendored SDL and ENet sub-builds stay uninstrumented (build-performance.md).

The shipped presets are devices-asan (address, O0), devices-tsan (thread, O1) and devices-ubsan (undefined, O1) in CMakePresets.json: Debug, OPENGLES3, tests on and CNA_DEVICES=ON. They are named for the Devices module, and each builds the whole CnaTests aggregate; use them for runtime lifetime work only if the host has a display. There is no runtime-specific sanitizer preset, so a lifetime investigation on another renderer or platform is -DCNA_SANITIZE=address in a separate build tree for that configuration. ASan finds the free, TSan finds the race, UBSan does not find lifetime bugs; run the two that match the hypothesis. CNA's own workflows show the option strings in use: input-ci.yml runs address,undefined with detect_leaks=0:halt_on_error=1, and gltf-sanitizers-ci.yml runs the glTF path with detect_leaks=1 and UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=0:exitcode=1 (without exitcode UBSan prints and the process still exits 0).

• Keep the first report: under ASan preserve the first use-after-free, not the destructor cascade.
• LeakSanitizer noise. CNA's integration notes record leak stacks rooted in the Mesa GLX driver with no CNA frames (integration/lanes/opengl4.md), and a WebGPU stress program's 368 bytes from adapter enumeration, told apart from a leak because the total did not scale when the churn loop was made sixteen times longer. LeakSanitizer swallows buffered stdout (CNA reads such programs' PASS lines in a detect_leaks=0 control run) and cannot run under ptrace.
• No Valgrind guidance was found in the TARGET documentation.

Processes whose exit status is the assertion

What happens after main returns cannot be tested inside a long-running CnaTests process. CNA's answer is a small standalone program spawned by a test, declared in Harnesses.cmake:

Diagnostics resource records and events

With -DCNA_DIAGNOSTICS=STATS or FULL (the default is OFF; the level is a compile definition and changes class layout, so reconfigure and rebuild everything), every GraphicsResource registers a resource record, and GetProvider().CaptureSnapshot() returns them with stable, never-reused ids. Use them to correlate explicit create and dispose cycles with what the registry still holds; do not read them as a GPU allocation census, since state objects and effects appear as zero-byte Unknown records and a resource created during a runtime Off window is never registered. The XNA GraphicsDevice::ResourceCreated and ResourceDestroyed events are the level-independent alternative. See Diagnostics internals.

Step-by-step triage

1. Record the configuration: platform, audio and renderer axes, sanitizer, display, build type, and whether the game was disposed explicitly, destroyed by scope exit or left alive at process exit. Those are different paths and a change must be checked against each.
2. Get the first error under ASan (free) or TSan (race), with its three stacks (access, free, allocation). A bad access in a destructor at exit suggests static destruction; in a member destructor, order or a borrowed pointer.
3. Classify with the owner table. Draw the destruction order and every asynchronous user of the object (device callbacks, sensor sessions, deferred renderer records, foreign threads); an unnamed lender is the finding.
4. Reduce: one game with one resource, then a bare device, then no window (HEADLESS). Use the exit-status harness pattern when the failure needs process teardown.
5. Repeat. One pass does not show a race away: use --gtest_repeat and TSan, and read the skip count, since a window-needing test skips where none can be created.
6. Prefer removing the dependency over ordering it. CNA's fixes for teardown-order bugs were an immortal holder (the registry cannot be ordered), a weak token (a late resource asks whether its device lives), moving state out before iterating, setting the disposed flag before raising events, running callbacks with no lock held, and generation counters that turn a stale pointer into “already gone”. A fix that only reorders declarations needs a test that would fail under the old order.
7. Do not quiet the sanitizer. Several barriers exist because one found a real free or race (the pan-state list, the dynamic-stream flag, the platform stack); a suppression or detect_leaks=0 describes the environment, not a fix.
8. Write the regression in the shape of the failure. Same-frame semantics for component removal; a subprocess exit status for static destruction; a counted-acquisition fake on a real decorated platform for reference balance (a fake that answers every call would have passed before the fix). See Add a regression test.
9. Check the blast radius. The C API child-before-parent gates, the bindings' deferred releases, Emscripten Run(), a second platform backend, and both mixer implementations when a shared contract moves (Blast radius and readiness).

Review checklist

• Which lifetime ended, who lent the pointer, and what stops the lender dying first, written in the description.
• Both teardown paths (explicit dispose and destruction) and the process-exit path considered; the ambient platform not assumed to name this game.
• Callback barrier named, kept on the failure path, and no user code run under a non-recursive lock.
• A regression test whose failure mode matches the bug, with the sanitizer configuration and skip counts stated, and anything not run said plainly.
• No new process-lifetime static that a destructor of a shorter-lived object touches, unless it follows the immortal-holder idiom.

Related: Architectural invariants, Known uncertainty and history, Debugging cookbook, Ownership and shutdown and Ownership and lifetime master map.