CNJ documents: envelope, sourceFile and sidecar rules

CNA snapshot 009d40f5  ·  Deep Dives › Content  ·  source links pinned to 009d40f5

CNJ (.cnj) is CNA's editable JSON content format: one document per asset, optionally with binary sidecars, read at run time by the loose-file readers and taken as input by the CNB compilers. This page gives the rules a document must satisfy at this snapshot: which readers accept which documents, how sourceFile and sidecar paths resolve, which parser reads which field, and where CNJ is weaker than the compiled formats. The format is taught on Tutorial 111 and the descriptor shapes are on the ContentManager guide.

What CNJ is, and what it is not

A .cnj file is plain UTF-8 JSON with no magic bytes, no fixed binary schema and no checksum. It can describe a whole asset (a font's glyph table, a curve, an effect's field values) or act as a metadata layer over a native file through sourceFile. Bulk data lives in sidecars: raw vertex, index, skeleton, clip, morph and volume blobs that the reader opens itself. A shipped .cnj therefore pays a JSON parse and a conversion at every load and has weaker size and structure controls than CNB (see What CNJ does not bound, below). CNB does not embed JSON: the compilers parse a .cnj into the same typed description the runtime builds, then write the frozen CNB schema, so that shared description, not any byte similarity, is the compatibility boundary. At this snapshot only one .cnj file is committed in the repository (a CMake-integration fixture); the tests write their documents at run time.

 name.cnj (read whole, no size limit)
    |
    v
 ParseCnjEnvelope: strict ParseJson of the whole document (no depth limit)
    |   cnjVersion (number) . type (string) . sourceFile (string, optional)
    v
 validate: cnjVersion integral in 1..max for T (Model: 2); type present;
           type equals what T expects (a RegisterCnjLoader type selects the factory)
    |                                                      \
    |                                                       +--> ContentLoadException
    +--> delegating readers: Texture2D, TextureCube, SoundEffect
    |      sourceFile required, resolved from THIS .cnj's folder and contained
    |      --> ContentManager::Load<T>(logical name): full ladder, own cache entry
    |
    +--> self-contained readers: Texture3D, SpriteFont, Effect + 5 stock effects,
           AnimationClip, Curve, Model -- sourceFile refused;
           fields + sidecars resolved from the CONTENT ROOT --> runtime object

 offline: .gltf/.glb --shared import core--> cna_tool_gltf_to_cnj --> Model .cnj + sidecars
          .cnj --shared CnjCanonicalRead readers--> cna_tool_cnj_to_cnb, cna-content --> .cnb

The envelope

ParseCnjEnvelope in CnjEnvelope.hpp never throws: it parses the whole document with CNA::Internal::ParseJson (strict number grammar, escape and surrogate-pair decoding), records a parse failure or a non-object root as a detail string, and picks up three members. A cnjVersion that is not a number counts as missing, a type that is not a string counts as missing, and a sourceFile that is not a string is silently treated as absent. Validation is a separate step: the version must be an integer between 1 and the maximum the type understands (a value of 1.5 is refused, not truncated), and only Model raises that maximum, to 2, for its "bones" array and per-mesh "parentBone"; a ceiling raised for one type does not leak to the others (CnjEnvelopeTests.cpp). type is a cross-check against the T named at the Load<T>() call, not the dispatch key, except for game-registered loaders, where it selects the factory. Because the whole document goes through the strict parser first, malformed JSON is refused even by the readers that later read their fields with substring scanners.

Which reader accepts which document

The sourceFile rule is enforced in code for every built-in reader in ContentManager.cpp; CnjCapabilityMatrixTests.cpp pins it.

The stock-effect branch exists because RegisterTypeReader<T> allows one reader per T and shared_ptr<Effect> was already the custom-GLSL reader's type, so the six effect documents are dispatched inside one reader by "type", which is checked before anything else so a wrong type is reported as such. Song and Video have no CNJ reader, yet the manager's .cnj probe still selects a same-named .cnj for them, as explained on ContentManager resolution rules.

sourceFile: a contained, checked nested load

ResolveCnjSourceFileSafely in CnjSourceFile.hpp applies five refusals, each a ContentLoadException naming the document:

1. an empty value;
2. an absolute (rooted) path;
3. a target outside the content root, decided after weakly_canonical on both the root and the target and compared component by component (so content-evil is not inside content, and symlinks are resolved before the comparison);
4. a target that is itself a .cnj ("sourceFile chaining is not allowed");
5. a target whose sibling target.cnj exists, because the normal resolver would pick that sibling; this also closes a document whose sourceFile points back at itself.

The value is resolved relative to the folder of the .cnj that names it, not relative to the content root. For Content/Textures/hero.cnj the correct spelling is therefore "sourceFile": "hero.png"; "Textures/hero.png" would name Content/Textures/Textures/hero.png. The tests all place the document at the root, where the two bases coincide (CnjSourceFileSafetyTests.cpp, six cases). The result is turned back into a root-relative logical name and loaded with ContentManager::Load<T>, so the target walks the whole ladder (a hero.png.xnb would win) and is cached under its own key.

For Texture2D the colorKey is then applied to the returned texture: pixels whose RGB equal the key get alpha 0 and keep their colour, which is CNA's own rule; the build-time colorKey processor parameter follows XNA's instead and clears the colour, as Content pipeline internals explains. Writing the keyed pixels through Texture2D::SetData(const Color*, int) gives the copy a fresh renderer object rather than writing through the one the cached source shares, which is why a colour-keyed sidecar never alters an explicit load of the native file, in either load order (CnjCacheIsolationTests.cpp).

Three different bases for paths inside documents

• sourceFile: the document's own folder, contained in the root.
• Fields of the self-contained .cnj readers (Model vertices, indices, skeleton, clip and morphTargets; the SpriteFont texture; Effect vertex and fragment; Texture3D data; AnimationClip clipFile): the content root, whatever folder the document is in. This is why cna_tool_gltf_to_cnj's bare sidecar names work only when the conversion writes into the content root.
• Fields of the older .skinnedmodel.json manifest: the manifest's own folder, confined to the root (or, for a manifest loaded by an explicit outside-root path, to that folder).

Every sidecar field is contained: an absolute path, a traversal out of the root or an existing symlink that escapes is refused before the file is opened, and the refusal does not leave a cache entry (ContentPathContainmentTests.cpp, for example ModelRootRelativeBinaryFieldsCannotEscapeContentRoot and EffectVertexTraversalIsRejectedBeforeShaderRead).

Strict fields and legacy scanners

Most readers now read their fields through the shared canonical readers of CnjCanonicalRead.hpp over the strict parse tree: the Texture2D colour key, the Texture3D description, the SpriteFont description, Curve, AnimationClip and the five stock effects. The same functions are called by the .cnj-to-CNB compiler (CnjToCnb.cpp) and the pipeline's CNJ route, so a document means the same thing at run time and at build time; before the consolidation, the colour key was read four different ways and could throw std::out_of_range instead of ContentLoadException. Three readers still take individual fields with substring scanners (ExtractJsonStringField, JsonInt and bracket matching): the custom-GLSL Effect (vertex, fragment), the Model document (mesh entries, bones, animations and material fields) and .skinnedmodel.json. Those scanners cannot misread memory, and the envelope parse has already rejected invalid JSON, but they find a key by text rather than by position in the tree, so a key name that also appears inside a nested object or a string can be picked up in the wrong place.

What CNJ does not bound

• Document size. The whole file is read into memory with no size limit (the CNB reader refuses files over 512 MiB before reading; the runtime XNB tier refuses files over INT32_MAX).
• Nesting depth. JsonParser::ParseValue in Json.hpp recurses through ParseObject and ParseArray with no depth counter, so a document of deeply nested brackets exhausts the stack instead of being refused; the XNB side bounds both type-name and typed-object nesting at 256.
• Integrity. No magic, version-independent schema or checksum distinguishes a truncated or edited file from a valid one beyond what the JSON grammar and field checks catch.
• Fuzzing. The repository's fuzz targets cover the XNB container, the LZX decoder, the XNB writer's inputs and the CNB container; there is no CNJ document fuzzer.

For content from untrusted sources, compile to CNB and load that instead; the comparison of all three formats is on Content input boundaries.

Game-defined document types

RegisterCnjLoader<T>(typeName, factory) installs, on first use for a T, a private GenericCnjTypeReader<T> with the single extension .cnj. It reads the file, runs the baseline envelope validation (JSON, version 1, a type present), looks the type string up among the factories registered for T and passes the raw JSON text plus the manager to the match; an unknown type string is a ContentLoadException. Registration is per manager and fails fast: an empty name or factory is std::invalid_argument, and a T that already has a reader or a repeated (T, typeName) pair is std::logic_error (CnjCustomLoaderTests.cpp). A factory that throws something other than ContentLoadException is wrapped by the loose tier like any other reader failure. Tutorial 46 and Tutorial 148 show the surrounding workflow.

Evidence and what is not proven

Read at this snapshot from CnjEnvelope.hpp, CnjSourceFile.hpp, CnjCanonicalRead.cpp, Json.hpp and the loose readers in ContentManager.cpp; the named tests exist and were read, not run. Not established by any test found: sourceFile resolution from a document in a subfolder, stack exhaustion on deeply nested JSON (a reading of the parser, not an executed input) and key mis-reads by the substring scanners.