Content runtime internals

CNA snapshot 009d40f5  ·  Development › Content internals  ·  source links pinned to 009d40f5

The runtime content module is not one generic file decoder. ContentManager::Load<T> walks a deliberate priority ladder (a compiled XNB first, a compiled CNB second, then a per-type loose-file reader whose resolution can prefer a .cnj sidecar), and each tier has its own registry, validation path and ownership rules. This page traces that ladder function by function at the TARGET snapshot for maintainers who change loading, caching or a reader; the user-level behaviour is on the ContentManager guide and XNB loading guide.

Boundary and dependencies

cna_content is a runtime module. modules/content/CMakeLists.txt links it PUBLIC to cna_graphics_core, cna_audio, cna_media, cna_math and cna_core (plus the sharp-runtime Core.Base, IO and Resources components; Security.Cryptography is PRIVATE), because a loaded asset can construct textures, effects, models, sounds, songs and videos. It does not link the build-time cna_content_pipeline, so a game that only loads content never pulls in FreeType or the build-time media decoders; the other direction of that boundary is on Content pipeline internals. The source tree modules/content/src is split into Xnb/ (binary XNB reader, writer, decompressors, built-in readers), Cnb/ (CNB container and codecs), Xna/ (ContentManager.cpp, ContentReader.cpp and the XNA-shaped orchestration), GltfImport/ (the glTF importer core), Pipeline/ (the canonical build-time engine, which lives in this runtime module so tests and format code share one schema definition) and Internal/. The module's public headers sit in two roots: the XNA-shaped ones under include/Microsoft/Xna/Framework/Content/ and the CNA-owned ones under include/CNA/ (CNA/Content/Cnb, CNA/Content/Pipeline, CNA/Internal/Xnb).

ContentManager holds a borrowed IServiceProvider* and an optional borrowed GraphicsDevice*. getGraphicsDeviceInternal() uses the direct pointer when setGraphicsDevice set one, otherwise asks the provider for IGraphicsDeviceService, and throws ContentLoadException (“no GraphicsDevice is available”) when neither supplies a device, which is how a GPU-requiring load refuses. Nothing in the content module owns the game's GraphicsDevice. Game's constructor calls Content_.setGraphicsDevice(GraphicsDevice_) and then RegisterAllBuiltInXnbReaders() in Game.cpp, so a game's own manager always has a direct device pointer and the 61 built-in XNB readers (60 without native int128) before Initialize(); a standalone manager in a tool, test or C API host has neither until its owner provides them.

An unusual physical ownership edge

The same CMake file compiles tools/gltf_to_cnj/gltf_to_cnj.cpp into cna_content with CNA_GLTF_TO_CNJ_NO_MAIN=1, so the runtime and the command-line tools share one glTF-to-CNJ implementation instead of compiling private copies (the comment names both cna-content and cna_tool_gltf_to_cnb as callers). Vendored cgltf and stb headers and tools/gltf_to_cnj are PRIVATE include roots; zstd (CNA_CNB_HAVE_ZSTD) and Draco (cna_draco) are linked only when available. Moving that tool file, or adding it to another target without checking this declaration, can change runtime behaviour or produce duplicate symbols.

One Load<T> call, function by function

ContentManager::Load<T>(assetName)                        ContentManager.hpp (template)
  → disposed? throw std::runtime_error
  → key = NormalizeKey(name)          '\' → '/', per-byte std::tolower
  → loadedAssets_[{typeid(T), key}] hit? → std::any_cast<T> (no log, no filesystem)
  → log "Loading asset"
  → ResolveExistingAssetPath(BuildAssetPath(name) + ".xnb")
      TryReadAssetBytes(...)  → LoadXnbAsset<T>          tier 1: XNB
  → ResolveExistingAssetPath(BuildAssetPath(name) + ".cnb")
      filesystem::exists  → LoadCnbAsset<T>              tier 2: CNB
  → name ends in ".cnb" and exists → LoadCnbAsset<T>       tier 2b: literal CNB
  → typeReaders_[typeid(T)]  (none → ContentLoadException "No reader registered")
      ResolveAssetPath: literal path → name.cnj → reader extensions → bare path
      LooseFileContentTypeReader<T>::Read(path, *this)   tier 3: loose
      (std::exception → ContentLoadException, ContentLoadException passes through)
  → loadedAssets_[key] = result; return result by value

The whole ladder is a header template in ContentManager.hpp, not in the .cpp. The cache key is AssetCacheKey{std::type_index, normalizedName}: the requested C++ type plus the logical name with backslashes turned into slashes and every byte passed through std::tolower (so ASCII letters in the C locale). It does not include the resolved filesystem path or the RootDirectory, so changing the root after a load does not invalidate the cached entry for the same logical name; Unload() does. Keying by type is what lets Load<T2> of a name cached as T1 reach normal validation and a clear ContentLoadException instead of std::bad_any_cast (the CNB-36 comment in the header, pinned by CnjAssetCacheTypeSafetyTests).

XNB wins even over a caller's literal path or a registered loose reader, and needs no per-T reader on the manager: its root type is decided by the file's own type-reader table through the process-wide ContentTypeReaderManager. Only name + ".xnb" is tried; a caller who spells "foo.xnb" does not get a literal XNB tier, whereas "foo.cnb" does get tier 2b. CNB ranks below XNB and above everything CNA can compile from, because a .cnb is CNA's own compiled artifact; it is self-describing through its asset-type identifier and also needs no reader registered on the manager. Only after both compiled tiers miss does typeReaders_ select a loose reader, and ResolveAssetPath then prefers an existing literal path, then name.cnj (a .cnj sidecar has the final say over a native file of the same name), then the reader's GetExtensions() in order, falling back to the bare path. It checks existence rather than has_extension() because a dot inside a localized asset name such as Flag.en-US is not an extension. Changing this order is a compatibility decision that needs resolver-order tests (CnjResolverOrderTests.cpp), not only a loader unit test.

Path and file services

In ContentManager.cpp, BuildAssetPath normalizes separators in both root and name, treats both as UTF-8 and joins them with PathFromUtf8, so a non-ASCII content root survives on every host. ResolveExistingAssetPath resolves an existing path case-insensitively through ResolveExistingNativePath; on Android a relative path is returned unchanged because it belongs to the packaged-asset namespace, and walking the process working directory would enumerate an unrelated, sandboxed /. TryReadAssetBytes reads an ordinary file with open, size (at most INT32_MAX) and read checks, each failure a ContentLoadException, and otherwise delegates to the current platform's TryLoadFileIgnoringCase (on Android directly for relative paths). For XNB, resolving the tier and reading the bytes are one step, because packaged assets have no reliable existence-only query.

The CNB tiers and the loose tier use std::filesystem::exists and file streams instead, so they do not consult the platform filesystem fallback. To debug a missing asset, record the logical name, the tier that was tried, the resolved UTF-8 path and whether the platform fallback was consulted; UnicodeContentRootTests covers the non-ASCII root.

The CNAEXT content manifest is a different thing from this resolution. RefreshContentManifest() scans the content root once and GetContentManifest() / GetXnbReaderUsageSummary() return that snapshot (the user-level description is in the ContentManager core API table). The header states that the manifest is not consulted by ResolveAssetPath() or Load<T>(), which keep asking the live filesystem so that a file written a moment ago is noticed. The reader-name inventory comes from ScanXnbReaderNames, which parses only an uncompressed, well-formed .xnb and swallows any failure for that one entry; a compressed file therefore appears with an empty reader list. The scan groups files by their path minus the last extension and gives special meaning only to .xnb and .cnj; every other extension, a .cnb included, is listed among the entry's native extensions, so the manifest does not model the CNB tier. Wiring the manifest into the resolver is deliberately deferred in the header comment, because a large test surface depends on the manager noticing files immediately.

XNB: header, reader table and shared resources

LoadXnbAsset<T> refuses input above INT32_MAX bytes, parses the header with ParseXnbHeader (XnbHeader.hpp), cross-checks the file's declared totalLength against the bytes actually read (the value comes from the file and must not drive pointer arithmetic unchecked), requires 14 bytes before a compressed payload, and builds a ContentReader over the uncompressed body, an LZX-decompressed payload, or an LZ4-decompressed payload (MonoGame's single raw block, decoded by the bounded in-tree DecompressXnbLz4Payload in XnbDecompression.cpp). An unknown flag combination (both bits set) is refused.

ContentReader::ReadAsset in ContentReader.hpp runs InitializeTypeReaders, reads the root object, then ReadSharedResources. Initialization has two passes, matching FNA: every reader named in the file's table is created through ContentTypeReaderManager::CreateReader and its version checked with SupportsVersion, and only then is Initialize called on each, so a reader may rely on its peers already existing. An unregistered or version-mismatched reader becomes a ContentLoadException naming the reader, not a null dereference. Shared resources are likewise read in full before any fixup runs. Counts and sizes are bounded before allocation by XnbReadLimits (XnbReadLimits.hpp): maxDecompressedSize 256 MiB, maxStringBytes 1 MiB, maxTypeReaderCount 4,096, maxSharedResourceCount 1,000,000, maxCollectionElementCount 10,000,000 and maxObjectNestingDepth 256, each checked by ContentReader, the type-reader table parser or the decompressors before the corresponding allocation or recursion. The struct also carries maxFileSize (64 MiB), and it is worth knowing where that one bites: the decompressors refuse a compressed payload larger than it, and the canonical source read used by the build-time XNB importer (see below) refuses a larger file. The runtime tier's own whole-file test is the INT32_MAX check in TryReadAssetBytes and LoadXnbAsset, so an uncompressed .xnb between 64 MiB and 2 GiB is not refused by the file-size limit at run time.

External references

External XNB references resolve relative to the referencing logical asset, never the process working directory. ResolveRelativeAssetPath in ContentReader.cpp refuses an absolute reference outright, joins the rest onto the referencing asset's directory, collapses ./.. lexically and refuses a result that climbs above the content root, while a legitimate sibling move such as ../textures/foo from effects/myeffect is allowed. Two paths then diverge:

• The typed ReadExternalReference<T>() (instantiated for Texture2D, TextureCube and shared_ptr<Effect>) calls ContentManager::Load<T>, so it walks the full ladder and shares the typed cache.
• The untyped ReadExternalReference() (the object-typed form, used by ExternalReferenceReader for effect-parameter dictionaries and similar) calls LoadUntypedXnbReference, which caches under std::any (keyed by typeid(std::any) and the normalized name) and insists on a compiled .xnb (no CNB or loose fallback). Its decoder, LoadXnbAssetUntyped, handles uncompressed and LZX payloads and refuses LZ4 with a “not yet supported” ContentLoadException; it does not repeat the typed decoder's “truncated before its compressed-payload size field” guard (see the maintainer notes below).

A change to external-reference policy must preserve both sibling lookup and traversal rejection; ContentReaderExternalReferenceTests.cpp is the first evidence. The reader list and container details for users are on XNB container support.

The canonical read path shared with the build tools

Not every .xnb is read into live objects. The build-time XNB importer, described on the pipeline page, transcodes a supported built-in root to a native asset without a GraphicsDevice. It reuses the same header parser, the same LZX and LZ4 decompressors and the same type-table validation as the runtime tier: DecodeXnbCanonicalAsset in XnbCanonicalData.cpp calls ParseXnbHeader and both decompressors, and reaches a ContentReader through the friend helper XnbCanonicalReaderAccess (XnbCanonicalReaderAccess.hpp), which calls ContentReader::InitializeCanonicalTypeReadersEXT (it parses the reader table and shared-resource count but creates no reader objects) and ReadCanonicalTypeReaderReferenceEXT. A change to a texture, model or sound reader can therefore alter what cna-content accepts as an .xnb source, not only what a game loads, and the 64 MiB maxFileSize above applies to that source read.

CNB: validated container and owned registry

LoadCnbAsset<T> parses and fully validates the file with CnbDocument::ParseFile (the whole file is read and every CRC checked), asks CnbLoaderRegistry::ResolveForDocument for the loader and calls a copy of it with the manager and logical name, then std::any_cast<T>s the result; a type mismatch becomes a ContentLoadException saying which asset type the file holds. The registry in CnbLoaderRegistry.cpp is a process-wide table behind a std::shared_mutex:

• Two entry points partition the identifier space: RegisterBuiltIn accepts only CNA's range (below 0x80000000), Register (the game route behind ContentManager::RegisterCnbLoaderEXT<T>) accepts only the custom range minted by CnbAssetTypeIdFromName, and a custom identifier must equal the hash of its canonical name.
• Re-registering the same identifier with the same name and ownership is tolerated (the first loader is kept); anything else is a std::logic_error.
• For a custom identifier ResolveForDocument also requires the file's own canonical type name (its CMET metadata) to equal the registered one, because a 31-bit hash can collide; a numeric match alone never selects a loader.
• Returning a copy of the loader means a concurrent registration or removal cannot invalidate the callable being run.

That lock does not make a ContentManager thread-safe: its asset cache and loose-reader maps are plain unordered_maps with no instance lock. Every manager constructor calls RegisterBuiltinLoaders(), which registers the twelve built-in loose readers (Texture2D, TextureCube, shared_ptr<Texture3D>, SoundEffect, shared_ptr<Effect>, SpriteFont, Model, AnimationClipEXT, Curve, shared_ptr<SkinnedModelEXT>, Song, Video) on the instance and then the idempotent CNB built-ins: CnbLoaderRegistry::RegisterBuiltIns() for Curve and AnimationClip, which need only their codecs, and eight more in ContentManager.cpp itself (Model v1/v2, Texture2D, TextureCube, Texture3D, SpriteFont, SoundEffect, Song, Video), because those construct objects that need the manager or a GraphicsDevice and must box exactly the type Load<T> asks for (a shared_ptr for Texture3D). A new CNB asset type therefore needs a codec and schema, a registry identity and the exact return type, not just a file extension. The format itself is documented on CNB Format: the loading path.

Three registries, three scopes

For .cnj, RegisterCnjLoader<T>(typeName, factory) dispatches on the envelope's own "type" string rather than on a numeric id. It throws std::invalid_argument for an empty name or factory and std::logic_error if a dedicated reader already owns T or the (T, typeName) pair repeats; the first registration for a T installs a GenericCnjTypeReader<T> (extension .cnj) that looks the type string up in cnjNamedLoaders_.

Cache, disposal and special ownership

loadedAssets_ stores std::any values and Load<T> returns T by value, so caching requires a copy-constructible T. For shared-pointer assets and for XNA value types whose GPU state is shared-owned, the cache and the caller share the underlying object; there is no separate weak texture cache. Unload() is exactly loadedAssets_.clear(): it cannot revoke copies or shared references already handed out. Dispose() routes to the protected virtual Dispose(bool), which calls Unload() only when disposing is true, clears the cache either way and marks the manager disposed; the destructor is defaulted. Do not invent a blanket “ContentManager destroys every GPU object” guarantee.

Load<Audio::SoundEffect> is an explicit specialization. SoundEffect is move-only with a per-owner dispose cascade, so sharing one cached instance would let one caller's disposal stop another caller's playing instances; each call instead returns an independently owned value and nothing is cached. The specialization carries its own copy of the XNB tier, gained its CNB tier later (loaded through shared_ptr<SoundEffect> because std::any cannot hold a move-only type; the CNBF-103A comment records that it was once missing), and then the loose tier. A cache-policy change must cover this special case and ContentManagerTextureCacheCycleTests.

Thread and renderer affinity

The ContentReader constructor acquires a renderer-thread context lease (GraphicsDevice::AcquireRendererThreadContextLease) when the manager has a direct device pointer, because deserializing a texture or effect may create renderer resources; a manager that reaches its device only through the service provider does not take the lease. That is a context and lifetime accommodation, not support for unsynchronized parallel Load calls. Async content loading would have to audit four things explicitly: the manager's unlocked cache and reader maps, the unlocked XNB reader registry, graphics thread affinity, and callback order; the current lease solves none of the first three. Phase-1's Tutorial 77 gives the user-level rule (load on one thread).

Maintainer change route and tests

• New loose format: implement a LooseFileContentTypeReader<T> (LooseFileContentTypeReader.hpp), register it in RegisterBuiltinLoaders or via RegisterTypeReader<T>, decide extension and .cnj precedence, and test missing and malformed files and a wrong requested type.
• New XNB reader: add a process-wide canonical factory with AddTypeCreator and a version contract (or declare it with ReflectiveTypeReaderBuilder<T>), wire built-ins through XnbBuiltInReaders.cpp, then add a fixture proving the type table, shared references and decompression.
• New CNB schema: validate the container before constructing objects, keep the reserved/custom identifier rules, register with the exact boxed type, and test golden vectors plus a runtime Load<T> round trip.
• Any path change: test a Unicode root, packaged lookup, external-reference containment and cache-key behaviour. If the loaded object is graphics- or audio-backed, run those subsystems' lifetime tests too.

Relevant tests (all under modules/content/tests): ContentManagerXnbTests, ContentTypeReaderManagerTests, CnbContentManagerTests, CnjResolverOrderTests, CnjAssetCacheTypeSafetyTests, ContentManagerTextureCacheCycleTests, ContentReaderExternalReferenceTests, UnicodeContentRootTests and CnbSpecConformanceTests. They compile into the content test object group; the focused executable is CnaContentTests, built by the unit-content build preset in CMakePresets.json (its unit configure preset uses the STUB renderer without networking, FFmpeg or Draco). Several texture-reader tests are renderer-gated with CNA_SKIP_IF_RENDERER_IS_NONE_OF (defined in the graphics module's test header RendererTestGate.hpp), so under STUB they skip rather than pass. The most recent edits to the content module before TARGET were confined to four test files, all of which add the OpenGL4 renderer to a renderer list: Texture2DContentTypeReaderTests.cpp and Texture3DTextureCubeContentTypeReaderTests.cpp extend their skip gates, GltfUnlitMaterialTests.cpp extends a CNA_RENDERER_IS condition, and GltfRendererPbrFallbackPolicyTests.cpp updates the expected OpenGL4 texture-binder wording. No production content source changed in that stretch. Tests that spawn a tool process are excluded on Windows, Emscripten, Android and iOS by UnitTests.cmake. None of them was executed for this page; list the built binary's tests or use the configured CTest before claiming a fixture ran. The general test layout is on Test architecture.

Inconsistencies a maintainer should know

• LZ4 through an external reference. A directly loaded LZ4 .xnb decodes, but the same file reached through an untyped external reference is refused by LoadXnbAssetUntyped.
• OpenStream is not on the Load<T> path. The header comment on OpenStream says every .xnb load goes through it, but Load<T> reads with TryReadAssetBytes; only the protected ReadAsset<T> honours an override. ResourceContentManager.cpp overrides only OpenStream, so its public Load<T> still searches files (Phase-1's ContentManager guide says so too).
• The untyped decoder has fewer guards than the typed one. LoadXnbAssetUntyped checks totalLength against the file size but, unlike LoadXnbAsset<T>, not the 14-byte minimum for a compressed file before it builds a four-byte MemoryStream at offset 10 to read the decompressed size. By reading, a compressed-flag file of 10 to 13 bytes reaches that read without a guard, whereas the typed decoder throws ContentLoadException first. No test in ContentReaderExternalReferenceTests.cpp (five cases, all typed) or elsewhere in the content tests names the truncation message, and this was not exercised.
• CNB and packaged assets. The CNB and loose tiers do not use the platform-filesystem fallback that the XNB tier uses; whether a .cnb inside an Android package loads was not established.
• Wrong T for an XNB versus a CNB. A .cnb holding a different asset type than T becomes a ContentLoadException naming what the file holds, and the loose tier wraps any std::exception the same way. The XNB tier has no equivalent wrapper: ContentReader::InnerReadObject<T> does std::any_cast<T> on what the root reader produced, and nothing between it and Load<T> catches std::bad_any_cast, so asking for the wrong type of an .xnb asset can surface that exception (the XNB guide's diagnosis section says the same from the user side).
• Two disposal exceptions. Load<T> on a disposed manager throws std::runtime_error, while ReadAsset<T> throws System::ObjectDisposedException. Dispose(bool) does nothing on a second call because the whole body sits behind the disposed flag.
• SoundEffect loose errors. The generic template wraps a loose reader's std::exception in ContentLoadException; the SoundEffect specialization's loose tier does not.

Source reading order

1. ContentManager.hpp: read the Load<T>, LoadXnbAsset<T>, LoadCnbAsset<T> and ResolveAssetPath templates first; tier priority and cache behaviour live here, not in the .cpp.
2. ContentManager.cpp: constructors, getGraphicsDeviceInternal, path and file services, Dispose(bool), the untyped XNB path, RegisterBuiltinLoaders and the SoundEffect specialization (at the end of the file).
3. ContentReader.hpp and ContentReader.cpp: root dispatch, the two-pass reader table, shared resources, external references and the renderer-thread lease.
4. ContentTypeReaderManager.cpp and CnbLoaderRegistry.cpp: compare the XNB canonical-name table with the CNB numeric/name table, including their different locking.
5. content CMakeLists.txt and content tests: confirm the runtime/tool source edge and pick a behaviour-specific fixture.