Songs, the media library and video: the public contract

CNA snapshot 009d40f5  ·  Deep Dives › Input, audio, media & services  ·  source links pinned to 009d40f5

Microsoft::Xna::Framework::Media in CNA is no longer the thin shell older counts described: it holds file-backed songs and libraries, playlists, a queued music player, visualization data, picture libraries and video that decodes wherever the optional FFmpeg backend is built. This page states the public contract a game can rely on, with the cases where CNA follows XNA against FNA, where a file name resolving is not the same as a file playing, and where the platform decides. The scan, queue and decoder internals are traced for maintainers on Media internals and the FFmpeg video boundary; the task guides are Tutorial 15, Tutorial 122 and Video Playback.

Why audio and media form a cycle

cna_audio and cna_media depend on each other on purpose. FrameworkDispatcher::Update lives in the audio module, because the dynamic-stream list it walks is audio state, yet it advances MediaPlayer and raises the deferred media events; in the other direction MediaPlayer and VideoPlayer play through the audio mixer. The CMake files declare this static-archive cycle and let the linker repeat the archives. Removing either edge would mean moving the dispatcher and the playback responsibilities, so a change on one side needs a review of the other.

The Song contract

A Song is created three ways: by ContentManager (a loose music file or an XNB/CNB reference), by a MediaLibrary scan, or directly (Song.cpp).

• Construction. The CNAEXT constructor Song(fileName, name) throws System::IO::FileNotFoundException naming the path when the file does not exist, including a path the platform cannot even represent. Song::FromUri(name, uri) returns a heap Song* the caller owns. It accepts a plain path or a file: URI and refuses every other scheme with InvalidOperationException ("Only local file URIs are supported for now.", FNA's message); it strips the query and fragment before percent-decoding, so an encoded %3F or %23 stays part of the file name, and it turns file://host/path into a UNC path rather than silently dropping the host (only an empty host or localhost means this machine). The System::Uri overload uses the URI's original string, which keeps relative URIs usable.
• Library links. Album, Artist and Genre are non-owning pointers, null for a directly constructed song and filled only by a library scan from the library's own hierarchy.
• Ratings. IsRated and the 0–10 Rating come from an ID3v2 POPM frame or a Vorbis RATING comment. A source rating of zero means "unrated" in both, so it leaves IsRated false rather than reporting a rating of 0; a non-zero POPM byte maps to round(byte × 10 / 255) clamped to 1–10, and a Vorbis value is read on a 0–100 scale (round(value / 10) clamped to 1–10; a non-numeric value is ignored). Vorbis RATING has no standard scale, so a tagger that writes 0–5 or 0–10 values is misread: RATING=5 becomes a Rating of 1. The mapping is CNA's policy, not an XNA rule: XNA defines only the 0–10 property.
• Protection. IsProtected is always false: the library scan indexes only plain, unencrypted containers, so no protected song can exist.
• Mutable counters. Duration and PlayCount have CNAEXT setters and change during playback: playing replaces the duration with the one the decoder reports and increments the play count, and MediaPlayer::Stop() resets play counts across the current queue. Which object receives those writes depends on the entry point, as traced on Media internals: the queue.
• Identity. The CNAEXT getHandle() returns the path the song plays from. Equals compares handles and GetHashCode hashes the handle, so two equal songs always hash alike; that deliberately avoids FNA's inconsistency of an identity hash paired with a value comparison. ToString() returns the song's name. CNA's source comment records this as an inference from Album, Artist, Genre and Playlist, which all return their names; the decompiled XNA 4.0 Song.ToString() in the xna4-decomp reference confirms it (return Name; after a disposed check), so the inference holds.

std::unique_ptr<Song> intro(Song::FromUri("Intro Theme", "Content/Audio/intro.ogg"));
MediaPlayer::Play(intro.get());   // the player queues its own copy; the caller still owns intro

Syntax-checked against the TARGET headers with <memory> and the Media header included (not executed).

FNA is not the complete API oracle

FNA's Song omits Album, Artist, Genre and ToString(), all of which are part of the XNA 4.0 surface. Reviews that compared CNA only with FNA missed the difference; a comparison with the recorded XNA 4.0 API data found it, and CNA follows XNA. API parity with FNA and API parity with XNA are separate claims, and where they differ the XNA surface is the target.

A name that resolves is not a file that plays

Three lists disagree at this snapshot, and the difference matters when a song is "found" but silent:

So a loose .aac, .wma or .opus file resolves to a Song and then fails to play, and the library scan, whose comment says it indexes only what the bundled mixer can play, still lists .opus files although CNA's configure switches SDL3_mixer's Opus decoder off and CNA's own mixer refuses Ogg Opus by name. A MediaPlayer::Play of an existing file that cannot be decoded does not throw: the player cleans up and leaves its state as it was, so check getStateProperty() rather than assuming the request reached the speakers. Treat the extension lists as naming rules, not as decoder support; re-encode AAC, WMA and Opus music to Ogg Vorbis, MP3 or FLAC.

MediaLibrary in one paragraph

A MediaLibrary is a synchronous, construction-time snapshot: its constructor scans the platform's music and pictures folders (SDL's user folders on SDL3, the Windows known folders on Win32, XDG user directories elsewhere), builds the SongCollection, Album, Artist, Genre, Playlist and picture graphs it owns, and never rescans; there is no Refresh(), so files added later need a new library. Tags come from CNA's own parser (Vorbis comments, Opus tags, ID3v2.3/2.4 with Latin-1, UTF-16 and UTF-8 text, native FLAC comments and pictures), untagged files fall back to file name, parent folder and grandparent folder for title, album and artist, .m3u/.m3u8 playlists are read as UTF-8 with missing or escaping entries skipped, and album art prefers a folder image over embedded art. SavePicture() is the only write and creates Saved Pictures on first use. Tests redirect the roots with an internal override (MediaLibraryPaths::SetMusicRootOverride) so they do not depend on the CI account's files. One lesson from this code is recorded on Media internals: the tag parser once read the track number correctly while the library dropped it when building the Song, which only a fixture with several distinct track numbers checked after library construction could expose; a parser-only test could not.

MediaPlayer: queue, modes and events

• Ownership. MediaQueue::Add(Song*) adopts the pointer, while SongCollection and Playlist (read-only, as in XNA) only point into the library. To keep the queue from deleting library-owned songs, MediaPlayer::Play copies every song it is given and queues the copies.
• Navigation. Without repeat, next and previous clamp at the ends; with IsRepeating the end wraps to the start. Shuffle chooses uniformly from the whole queue, so the same song can play twice in a row.
• End of a song. In a build with a mixer, the mixer's stopped callback sets a flag that the next dispatcher update turns into advancing the queue. Without a mixer (SDL2 or NULL) the player instead compares elapsed time with the duration of the queue's active song and ignores a zero duration. Every queued song is a copy made with the two-argument constructor (MediaPlayer::LoadSong), which carries no duration, and the mixer-less path never sets one, so in those builds a song never auto-advances or ends on its own: the state stays Playing until the game calls Stop or a navigation method.
• Volume and mute. Volume is clamped to 0–1 and IsMuted is independent state: setting the volume while muted stores the new value for the next unmute, and only the mixer track receives zero gain while muted.
• Events. MediaStateChanged and ActiveSongChanged are never raised from the audio thread; they are deferred to FrameworkDispatcher::Update() and coalesced to one raise per pump. Game pumps the dispatcher during its normal update, but a caller using the static media API outside Game (a tool, a test) must call FrameworkDispatcher::Update() itself, or songs never advance and no event fires.

Visualization data

MediaPlayer::GetVisualizationData() reads samples that the mixer's post-mix callback writes into a lock-free single-producer, single-consumer ring of std::atomic<float> with relaxed ordering, so the audio-thread writes and game-thread reads are defined C++ behaviour without locks, allocation or exceptions on the audio thread. A reader may see samples spanning two adjacent callback batches; the resulting one-frame imprecision is accepted. A 512-sample Hann-windowed radix-2 FFT produces XNA's 256 frequency bins (VisualizationFFT.cpp). The magnitudes are scaled by 2/N and the window's coherent gain of one half is deliberately left uncompensated, so by that arithmetic a full-scale sine at a bin centre peaks near 0.5 (the source comment describes the target as about 1.0, and the test asserts only that the peak lies in the tone's own bin and exceeds 0.1). XNA documented no normalization, so treat the values as relative. Disabled or not-yet-fed visualization returns zero-filled arrays. The enabled flag is assigned only from what actually happened when the mixer tap was installed or removed, so it never claims a callback that is absent; the media tests run with a working (dummy-driver) mixer, however, so that failed-installation branch is correct by construction rather than by test, as the test's own comment states.

Video and VideoPlayer

Where the FFmpeg backend was built, VideoPlayer decodes with FFmpeg, uploads each frame into a renderer texture and streams the audio through the mixer. The behaviours a caller sees:

• Play() compares the Video's declared width, height and frame rate with the decoded stream and throws InvalidOperationException on a mismatch (with a one-frame-per-second tolerance on the rate).
• VideoSoundtrackType is metadata only, in CNA as in FNA: no code path ducks or mutes a track because of it.
• Play, Stop, Pause, Resume, GetTexture and the two track switches throw ObjectDisposedException("VideoPlayer") after disposal, FNA's message; the property accessors do not check. Dispose() itself is idempotent, unlike FNA's, because the C++ destructor also calls it and a second explicit disposal must not throw from inside a destructor.
• SetAudioTrackEXT and SetVideoTrackEXT select streams independently: a live audio switch reopens only the audio output at the new track's rate, and a genuine live video switch recreates the frame texture at the new track's size on every switch (ReconfigureVideoOutputForCurrentTrack), whether or not the dimensions differ, and does not seek, so decoding continues from the current position in the new stream. Re-selecting the current track or an out-of-range one does nothing. The queued audio is cleared on every close, even when no audio device or audio track is active, so samples cannot accumulate or leak into a later play.

VideoPlayer player;
player.Play(&introVideo);          // throws NotSupportedException in a build without the backend
player.SetAudioTrackEXT(1);        // switches audio without recreating the video output

Syntax-checked against the TARGET headers with introVideo as a Video (not executed).

Platform availability, then and now

The FFmpeg backend is optional (CNA_ENABLE_VIDEO=AUTO|ON|OFF) and is never built for Windows (MinGW and MSVC), Emscripten, Android or iOS. The alpha.1-era design excluded the video translation units on those targets, so using Video there failed at link time, and a native-Windows preprocessor guard in ContentManager.cpp did not match the CMake condition. Both are history at this snapshot: Video and VideoPlayer are always compiled, a small unavailable decoder keeps the API link-complete, ContentManager.cpp includes the video header unconditionally, and a file-backed video throws NotSupportedException at run time instead. The configure-time switch, the fixtures and what they prove are on Video Playback: platform availability and the FFmpeg boundary's validation section.

Evidence

Source and focused tests establish tag parsing, hierarchy construction, queue ownership, event timing, visualization state, video disposal and multi-track reconfiguration; FFmpeg fixture tests exercise decoding on a configured host. None of that establishes video on the targets where the backend is never built, nor does an extension list establish decoder support. Nothing on this page was executed.