Media internals
Evidence basis: source-verified at the pinned commit; tests exist (not executed for this page). Claims on this page were checked by reading the CNA source at commit 009d40f5; unless a sentence says otherwise, nothing here was built or executed. Test names were located by reading the sources and none was executed. Specialised tag formats, every picture decoder and live audio and video hosts remain unverified, and the hazards listed as source-read maintainer notes have no named test.
modules/media/ implements three things that share a namespace but not a lifetime: MediaLibrary, a snapshot graph of the music and pictures a folder scan found; MediaPlayer, one process-wide music player with its own queue; and Video/VideoPlayer, which orchestrate a decoder, a texture and an audio stream for one playback. This page is for maintainers who change the scan, the graph, the player state machine, the audio-thread callbacks or the video hand-off, and for anyone who has to explain why a song does not play or a frame stays black. The user-level view is in Video Playback, Audio and the Media section of XNA compatibility. Do not treat the library as the live music queue, and do not treat the optional FFmpeg archive as the public media module: the FFmpeg boundary only supplies a decoder.
Physical boundary and the deliberate audio/media cycle
modules/media/CMakeLists.txt builds cna_media with cna_add_module from every src/*.cpp. It links cna_audio and cna_graphics_core publicly and the Sharp Runtime components Core.Base, IO and Uri publicly too (Song::FromUri(string, System::Uri) puts System/Uri.hpp in a public header). cna_input is a private link that only states a real compile dependency: MediaPlayer.cpp includes the FrameworkDispatcher declarations, and those reach into TouchPanel. The CMake comments call the resulting audio-to-media edge an accepted pump cycle, and it is a real one. FrameworkDispatcher::Update lives in modules/audio/src/Xna/FrameworkDispatcher.cpp, because its implementation state is the audio stream list, yet it calls MediaPlayer::Update and raises the deferred media events; in the other direction MediaPlayer calls the audio mixer facade. Changing an entry point on one side means reviewing the other.
The decoder is the one part that varies. VideoDecoder and AudioDurationProbe are declared here (VideoDecoder.hpp, AudioDurationProbe.hpp); the public Video and VideoPlayer implementations always live in cna_media. When CNA_FFMPEG_AVAILABLE is set, the module's CMake file filters out AudioDurationProbeUnavailable.cpp and VideoDecoderUnavailable.cpp, modules/CMakeLists.txt adds the video-ffmpeg subdirectory, and cna_media links cna_video_ffmpeg privately, so the FFmpeg archive supplies the same symbols. Otherwise the two fallback translation units keep the API link-complete without any FFmpeg header. cna_media is one of the runtime parts of the CNA umbrella target, so a game that links CNA gets the module, and the decoder question is settled at configure time, not at run time.
Platform filesystem service (SDL_GetUserFolder | Win32 known folder | XDG user-dirs.dirs) -> MediaLibraryPaths (process-wide override strings) -> MediaLibraryIndex / PictureLibraryIndex / PlaylistParser [caller thread, at construction] -> MediaLibrary-owned Song / Album / Artist / Genre / Playlist / Picture graph MediaPlayer static queue [its own Song copies] [caller thread] -> audio MixerTrack + MixerAudio (streaming) [audio thread: stopped + post-mix callbacks] <- FrameworkDispatcher::Update -> MediaPlayer::Update -> event flags [caller thread, normally Game::Update] VideoPlayer -> VideoDecoder [FFmpeg or unavailable] [caller thread, inside Play and GetTexture] -> RGBA Texture2D::SetDataRGBA + float audio -> MixerStream
| Lifecycle | Owner and lifetime | What it owns | Threads that touch it |
|---|---|---|---|
MediaLibrary | One instance per caller; the scan runs in the constructor and the graph never refreshes | Every Song, Album, Artist, Genre, Playlist, Picture, PictureAlbum and every collection over them, as unique_ptr members | The constructing thread; no locking anywhere in the class |
MediaPlayer | Static service: a static MediaQueue, static state, volume, timer and events; released only by ProgramExit | The queue's own Song copies, one mixer track and one mixer audio handle | The caller thread for every public call; the audio thread for two callbacks that only set an atomic flag or write a lock-free ring |
VideoPlayer | One instance per playback session; Dispose or destruction closes it | A unique_ptr<VideoDecoder>, a unique_ptr<Texture2D> and one mixer stream | The caller thread only; there is no worker |
Which builds have a mixer at all
Music and video audio both go through MixerEngine.hpp (MixerEngine.hpp), and that header only has an implementation where a mixer exists. modules/CMakeLists.txt defines SOUND_ENABLED on the shared build-config target only when CNA_AUDIO_PLATFORM is SDL3 (SDL3_mixer) or ALSA (CNA's own mixer); SDL2 and NULL transport audio but compile the facade without one. The user-facing statement of the same split is the audio implementation table. Two consequences are easy to miss when reading this module:
MediaPlayerstill has a full state machine without a mixer.PlaySongskips loading, track creation, the play count and the mixer-reported duration, and only drives the timer and the state; song end is then detected by comparing elapsed time with the song's known duration (DetectSongEndedByElapsedTime), which never fires for a zero duration.VideoPlayerwraps every audio call in a small shim. Without a mixer,OpenAudioStreamreturns null and every other call is a no-op, so a video plays with its audio silent while decoding and presentation are unaffected.
The library is a construction-time, value-owning snapshot
Where the roots come from
MediaLibraryPaths.cpp resolves the Music and Pictures roots through the current platform's filesystem service (GetCurrentPlatform().GetFileSystem()->GetUserFolder), strips trailing separators, and returns an empty string when the service is absent. Two static override strings (SetMusicRootOverride, SetPictureRootOverride) win over the platform when non-empty; they are process-global and unsynchronized, and tests restore them by passing an empty string. What the platform answers depends on the selected backend, read at 009d40f5:
| Platform | Music / Pictures root |
|---|---|
SDL3 | SDL_GetUserFolder(SDL_FOLDER_MUSIC / SDL_FOLDER_PICTURES) in Sdl3SystemServices.cpp; empty when SDL returns null |
WIN32 | The Windows known folders (FOLDERID_Music, FOLDERID_Pictures) in Win32SystemServices.cpp |
SDL2, X11, WAYLAND, HEADLESS, TERMINAL | The shared StandardFileSystem in StandardFileSystem.cpp, which reads XDG_MUSIC_DIR and XDG_PICTURES_DIR from $XDG_CONFIG_HOME/user-dirs.dirs (or ~/.config/user-dirs.dirs) and returns empty when HOME is unset or the entry is missing |
An empty root is not an error: the scan simply produces an empty library. This is why a library that is empty on a CI host, a container or a bare compositor session says nothing about the scanner.
What the scan indexes and what it costs
MediaLibraryIndex.cpp runs once, in the constructor. It recurses through the music root, tracks the weakly_canonical path of every directory it enters so a symlink cycle terminates, opens directories with skip_permission_denied, and sorts each directory's entries by path before processing them, so the result (including which spelling wins a case-insensitive artist or genre) does not depend on filesystem enumeration order. It indexes only the extensions it treats as playable: .ogg, .oga, .mp3, .wav, .flac and .opus. The last is not actually playable at this snapshot: the vendored SDL_mixer is configured with SDLMIXER_OPUS=OFF (cmake/ThirdPartySDL.cmake) and CNA's own ALSA mixer refuses Ogg Opus by name (DecodeMixerAudio), so an indexed .opus entry cannot be played (see Known Issues). .aac and .m4a are left out on purpose, because the mixer ships no AAC decoder and a library entry that MediaPlayer::Play could never play is worse than no entry. The class comment in MediaLibraryIndex.hpp still lists only four extensions; the source is authoritative. The song path becomes a key in generic (forward-slash) UTF-8 form, because the playlist side looks songs up with a path that also comes out in generic form.
Tags come from AudioTagParser.cpp, a from-scratch reader with no tag-library dependency. ReadTags selects a reader by extension, then fills any field that is still empty from the path:
| Extension | Reader | Also read |
|---|---|---|
.ogg, .oga | Vorbis comments (TryReadVorbisComments) | track number, RATING |
.opus | OpusTags header (TryReadOpusTags) | same comment list |
.flac | Native FLAC VORBIS_COMMENT block (TryReadFlacComments) | embedded art from METADATA_BLOCK_PICTURE |
.mp3 | ID3v2.3/2.4 text frames (TryReadId3v2: title, artist, album, genre, track) with Latin-1, UTF-16 and UTF-8 encodings | POPM rating; embedded art from APIC |
.wav | none | filename and folder fallback only |
The fallback (ApplyFilenameFallback) fills an empty title with the file stem, an empty album with the parent directory name and an empty artist with the grandparent directory name; a genre is never invented. Ratings have no standard source scale, so the mapping to XNA's 0 to 10 range is a documented CNA decision: POPM is 0 to 255 with 0 meaning unrated, and a Vorbis RATING is read as 0 to 100. hasRating is true only when a real rating tag exists, which is what Song.IsRated reports, and is not the same as a non-zero rating. Artists and genres are normalised case-insensitively (a trimmed, lower-cased key; first spelling seen wins), albums are keyed by the exact (artist, album) pair.
Two costs follow from the code and matter for large libraries; neither was measured. ReadTags reads each file completely into memory before parsing (ReadFileBytes), and MediaLibrary then calls AudioDurationProbe::ProbeDurationMS once per song. With the FFmpeg backend that is avformat_open_input plus avformat_find_stream_info and a container-duration read, with no decoding; without it the probe is a stub that returns zero. A zero duration is treated as unknown: the Song is built without one, and album and playlist durations become sums of whatever durations exist. Only a build with FFmpeg gives library songs a duration at scan time.
Graph ownership and construction order
MediaLibrary.cpp builds the graph in one function, MediaLibrary::BuildFromRoots, and the order is part of the design:
- Every indexed file becomes a
Songowned byownedSongs_; aSongCollectionof raw pointers exposes them. - Songs are grouped by genre, artist and
(artist, album)in first-seen order (the scan is already deterministic). A song with an empty artist joins no artist and no album; one with an empty genre joins no genre. - Genres, then artists, then albums are created. An album's duration is the sum of its songs' durations, its genre is the first song's genre, and its art path comes from a folder search.
- The genre and artist album collections are patched in once albums exist.
- A final pass writes the
Songback-references (Album,Artist,Genre) and copies the real track number, rating andIsRatedthat the index had parsed. - Playlists, then pictures and picture albums, are built.
Every group object is a unique_ptr in a member vector, and every collection and back-reference is a raw pointer to a heap object those vectors own. Growing a vector of unique_ptr moves the pointers, never the pointees, so the raw pointers stay valid for as long as the library owns the graph. The public getters return borrowed pointers into that graph: do not cache them past the library's destruction. MediaLibrary::Dispose currently only sets a flag; it neither frees the graph nor reloads it. A maintainer who changes construction must keep the delayed back-reference pass and the collection ownership, and must not take a reference into a temporary vector.
The library is a snapshot, not a media database: it never rescans, so files added after construction are invisible to that instance. Two other places take the same snapshot-time decisions. Album art is looked up per album by scanning the first song's directory case-insensitively for cover, folder, front, album or albumart images (in that precedence, .jpg/.jpeg/.png variants), and when there is no folder image the first member song with embedded art is remembered by path only. Album::GetAlbumArt and GetThumbnail then read the image on demand and return a heap-allocated FileStream or MemoryStream that the caller owns; folder art wins over embedded art, and thumbnails of folder art are box-filtered PNGs no larger than 128 pixels on the long edge (ThumbnailGenerator), falling back to the original file when the source cannot be thumbnailed, and embedded art is returned as extracted. Playlists are .m3u and .m3u8 files found directly in the music root (not recursively); PlaylistParser.cpp skips comment lines, resolves each entry with ResolveContainedPath against the playlist's own directory so absolute and escaping entries are dropped, and skips missing files. MediaLibrary then keeps only entries that match an indexed song path.
Pictures and the one write path
PictureLibraryIndex.cpp recurses through the pictures root with the same cycle guard, permission handling and sorting as the music scan. Every directory becomes a picture-album node, including empty ones; files with .png, .jpg, .jpeg or .bmp extensions are loaded through ImageLoader::Load to read their dimensions, and a file that fails to load is skipped without aborting the scan. MediaLibrary::BuildPictureAlbumTree turns the index into owned PictureAlbum and Picture objects.
SavePicture is the only place the library writes to disk, and it does so lazily. Merely constructing or browsing a library never creates Saved Pictures. On the first save, SavedPictureStore.cpp creates Pictures/Saved Pictures under the pictures root, reduces the caller's name to its last path segment (backslashes are normalised first, and ., .. or an empty result become picture), chooses an extension from magic bytes (PNG, JPEG or BMP, defaulting to .png), appends it to the sanitized name even if the name already had one, and writes the bytes with an ofstream. A failed write returns an empty path and MediaLibrary::SavePicture throws IOException. On success the library tries to load the dimensions (a load failure records 0 by 0), creates the Picture, and appends it to the owned list, the all-pictures collection, the saved-pictures collection and its album, creating the Saved Pictures album node (and a root node, when the pictures root did not exist at construction) through EnsureSavedPicturesAlbum if the scan had not found one. This is another filesystem-authority boundary: tests must use isolated roots and the traversal cases must be re-run before sanitization changes. The code creates or overwrites a filename; it is neither a transactional save nor a race-proof sandbox, and two saves of the same name overwrite each other on disk while both stay in the in-memory collections.
Music queue, audio callback and dispatcher
A static service with its own copies
MediaPlayer.cpp is a static service, not an object owned by each Game. Play(Song*) clears the static MediaQueue (MediaQueue.cpp), which owns unique_ptr<Song> copies made by LoadSong from the source song's handle and name. A copy is built with the two-argument Song constructor, so it carries no duration, no track number, no rating and no album, artist or genre links, and that constructor throws FileNotFoundException when the file no longer exists. The queue therefore never aliases the library: the test PlayEnqueuesADuplicateNotTheOriginalInstance pins that the queued song is a different instance with the same name and handle, and that changing the original's play count does not reach the copy. Which song receives the duration and play-count that PlaySong writes depends on the entry point, and is easy to get wrong: Play(Song*) passes the caller's song to PlaySong, so under a mixer build the mixer-reported duration and the incremented play count land on the caller's (for example the library's) song, while Play(const SongCollection&, index), MoveNext, MovePrevious and automatic advance play queue entries, so there they land on the copies. Stop resets play counts only on the queue's copies. The queue's raw Song* getters are borrowed, and Clear destroys what they point at. Play(const SongCollection&, index) copies every song of the collection, sets the active index and plays it; an out-of-range index leaves the queue filled, the active song null and nothing playing.
PlaySong, Stop and the end of a song
PlaySong first tears down any previous music: it stops and destroys the mixer track, then releases the mixer audio, and clears the ended flag. It then loads the file as streaming audio (LoadMixerAudioFile(handle, predecode=false)), creates a track, attaches the audio, applies volume and mute, installs the stopped callback, records the duration reported by the decoded audio on the song it was given, starts the track, increments that song's play count, and only then resets and starts a steady_clock timer and sets MediaState::Playing. If loading, track creation, attachment or start fails, it cleans up the mixer objects and returns before touching the timer or the state, so the caller must look at getStateProperty() rather than assume a request reached the hardware; a failed Play on a stopped player leaves it stopped, and a failed request while playing leaves the state as it was. Stop is a no-op when already stopped; otherwise it destroys the track before the audio, clears the ended flag, resets the timer and every queued song's play count, and marks the player stopped. Pause and Resume pause and resume the track and the timer. ProgramExit releases any remaining track and audio if a song was ever played. No runtime call site invokes it: within the module tree the only non-test caller is the C API route cna_media_player_program_exit_ext, so a C++ game that wants deterministic release must call it itself.
Ending is a hand-off from the audio thread to the caller thread. OnMusicTrackStopped runs on the audio thread and only stores true into an atomic g_songEnded. MediaPlayer::Update, called from the dispatcher, exchanges the flag back to false on the caller thread; when it was set it counts the finished song and either stops (last song, repeat off) or calls MoveNext. NextSong stops first, then picks the next index: repeat at the last index wraps to zero, shuffle picks uniformly among all indices including the current one (ShuffleCanRepeatTheSameSongIndex), and otherwise the index is clamped to the queue, so a manual MoveNext on the last song replays it rather than stopping. Because Stop and PlaySong clear the flag only after destroying the old track and audio, a flag raised by that teardown is discarded and cannot advance the queue. In a build with no mixer the same Update asks DetectSongEndedByElapsedTime instead.
The dispatcher pump and deferred events
FrameworkDispatcher::Update does six things in a fixed order: it updates every registered DynamicSoundEffectInstance from a snapshot taken under StreamsMutex (so an instance may dispose itself from its own BufferNeeded handler), checks microphone buffers, calls MediaPlayer::Update, raises ActiveSongChanged if its flag is set, then MediaStateChanged if its flag is set, and finally updates the touch panel when a touch device exists. The two flags are plain booleans that MediaPlayer sets from setStateProperty and the play paths, so the events are coalesced (one raise per pump, however many transitions happened) and are never raised from the mixer callback. The pump runs once during Game construction and at the end of the base Game::Update(GameTime&) (see the dispatcher note in the game-loop guide): a derived Update that never calls the base class never advances the queue, ends a song or raises these events unless the program pumps the dispatcher itself. MediaPlayer::ActiveSongChanged and MediaStateChanged are static process-wide handlers, so a subscriber that captures locals must unsubscribe before they die; the dispatcher test uses Add and Remove for that reason.
The visualization tap
Enabling visualization installs a post-mix callback on the mixer. That callback runs on the audio thread for every mixed buffer and must not allocate, lock or throw; VisualizationCapture::Push honours this with a 2,048-sample ring of std::atomic<float> written with relaxed stores and two release-published counters, downmixing to mono by averaging. GetVisualizationData zero-fills the 256 sample and 256 frequency values when disabled or when nothing has been captured, otherwise reads the newest 256 samples and runs a Hann-windowed radix-2 FFT over the newest 512 (VisualizationFFT). The setter separates two facts that can disagree: g_visualizationEnabled is the flag callers see, g_visualizationTapInstalled records whether a callback is really installed. The audio facade documents replacement or removal of the callback as a barrier (the old callback has returned when the call succeeds), and the module relies on it: on disable it clears the ring only after a confirmed uninstall, and on enable it clears the ring before installing. If an uninstall fails, the flag reads false but the tap stays marked installed, the ring is not touched (zeroing a buffer another thread is writing was a real race), and a later disable retries. Enabling with no working mixer leaves both false. A change here must preserve the callback's no-allocation, no-lock, no-exception contract, the reset-after-uninstall barrier and the dispatcher's event timing.
Sharp edges recorded from the source
None of the global queue, settings and timer fields carries a synchronization guarantee, so treat game-thread calls as the supported pattern unless a specific guarantee is proven; the two audio-thread callbacks are the only cross-thread traffic and each communicates through an atomic. The following were read from the source at 009d40f5 and are not covered by a named test; they are recorded as maintainer notes, not as verified failures.
Play(Song*)clears the queue, then reads the song it was given. Passing aSong*that the queue itself owns (for examplegetQueueProperty().getActiveSongProperty()) would read a song thatClearhas just destroyed.Play(Song*)compares the queue's previous first song with the caller's pointer to decide whether to flagActiveSongChanged. The queue holds copies, so for any library or standalone song the pointers differ and the flag is raised on every call, even when the same song is replayed.Play(const SongCollection&, index)never sets theActiveSongChangedflag; playing a collection raises only the state-change event, throughPlaySong.Play(Song*),NextSongand the end-of-queue path do set it.- A
Songcopy that fails to construct (the file vanished after the scan) throws out ofPlayafter the queue was already cleared and possibly partly filled. - The mixer-less profile never sets a duration on queue copies, and the elapsed-time fallback ignores zero durations, so playback in that profile never auto-advances.
Video is an orchestration layer over decode, graphics and audio
Two ways to make a Video
Video.cpp has two constructors with different obligations. The raw-file constructor (Video(fileName, device), reached by FromUriEXT and by the loose-file content reader for .mp4, .ogv, .webm, .mkv, .avi and .mov) first requires the file to exist (FileNotFoundException), then calls RequireVideoDecoderAvailable (NotSupportedException without a backend), then probes the file with a throwaway VideoDecoder for width, height, frame rate and duration; if the probe cannot open the file the dimensions simply stay zero, with no exception. The seven-argument constructor takes trusted metadata and never touches the file or the decoder, which is what the XNB and CNB video readers use, so metadata-only Video objects exist in every build. Neither Video nor VideoPlayer has any synchronization, and the decoder header states that a VideoDecoder must be driven from one thread.
Play, OpenDecoder and GetTexture
VideoPlayer.cpp owns a unique_ptr<VideoDecoder>. Play checks disposal, returns for a null video, and calls RequireVideoDecoderAvailable before assigning anything, so a failed Play in a build without a backend leaves the player stopped with no video. VideoPlayer::OpenDecoder then closes any previous session, opens the file (a decoder that cannot open returns quietly, leaving the player stopped and no exception), and compares the Video's declared width, height and frame rate with what the file reports, throwing InvalidOperationException on a mismatch (frame-rate tolerance one frame per second). Stored track preferences are applied before any output exists, so the texture and the audio stream are built for the track that will actually play. Inside one try block it then creates the frame Texture2D (only when the Video has a graphics device), reopens the audio stream at the active track's sample rate and channel count, decodes the first frame, uploads it with Texture2D::SetDataRGBA and bumps the frame generation, and drains the first audio; any exception calls CloseDecoder and rethrows, so a failed Play does not leave a playing half-state.
GetTexture is what advances playback. There is no worker: decoding, audio feeding, looping and end-of-stream detection all happen inside Play and GetTexture on the caller's thread, and a game that stops calling GetTexture stops feeding audio. The play position is a steady_clock offset, not the audio device's clock. GetTexture returns the current texture without decoding while stopped or paused, otherwise it decodes frames until the last decoded frame is no more than half a frame behind the play position, uploading every decoded frame (skipped frames included), and drains decoded audio into the mixer stream after every NextFrame call, including the one that returns no frame, because the decoder can read trailing audio packets while looking for end of file. At end of stream a looped video seeks to the start and resets the clock; a non-looped video keeps returning the last texture while the mixer stream still has queued bytes, and only when the queue is empty sets the state to Stopped and pauses the stream. It does not close the decoder at that point, so the texture and decoder live until Stop, another Play or disposal. GetFrameGenerationEXT counts frames actually decoded into the texture and is monotonic for the player's whole life (never reset by Stop or a new video), which lets the C API distinguish a repeated frame from a new one. A decode error thrown by the decoder (see the failure table) propagates out of GetTexture uncaught; only OpenDecoder cleans up after an exception.
CloseDecoder destroys the audio stream first, clears the audio buffer (so stale samples cannot reach a later playback), resets the video's back-pointer, and then discards the texture and the decoder. Mid-playback SetAudioTrackEXT and SetVideoTrackEXT reconfigure only the affected output and only when the decoder reports a real switch, so re-selecting the current track or an out-of-range one does nothing.
Ownership and lifetime
A VideoPlayer stores a borrowed Video*, and the Video stores a borrowed VideoPlayer* parent so per-video track selection can reach the live player. Neither owns the other and Play extends no lifetime. The header does not state the ordering rule, but the source implies it: CloseDecoder writes through the stored Video*, so the Video must stay alive and unmoved until the player has been stopped, disposed or destroyed. That is a hazard read from source, not a tested guarantee. Content loading returns Video by value, so callers that keep a Video in a container should hold it where it will not move while playing.
When frames go black, check the boundaries in this order: the decoder's RGBA output, the upload into the texture, the renderer's sampling of that texture, then timing. A decoder unit test proves none of the later boundaries, and a video with no graphics device on its Video yields no texture at all. The decode-side detail is on the FFmpeg boundary page.
Human change and validation map
The module's tests are split by owner, and the names below are as registered in source at 009d40f5; none were executed for these pages. By file: MediaLibraryIndexTests, AudioTagParserTests, PlaylistParserTests, PictureLibraryIndexTests and SavedPictureStoreTests establish scanning, parsing and write behaviour; MediaLibraryTests and the album, artist, genre, picture and playlist tests establish graph ownership and relationships; MediaQueueTests and MediaPlayerTests establish queue, state and dispatcher behaviour; VisualizationTests cover capture and the FFT; the video tests distinguish backend availability from real decode and player behaviour.
| Changing | Tests that pin it | What they do not show |
|---|---|---|
| Roots and scan | MediaLibraryPathsTest.* (override redirects, platform folders with trailing separators stripped), MediaLibraryIndexTest.* (every fixture song, case-variant artists merged, symlink cycle, unreadable subdirectory) | A real user profile; large libraries; non-ASCII filesystems beyond the fixtures |
| Tag parsing | AudioTagParserTest.*: Vorbis, ID3v2.3 and 2.4, all four ID3 encodings, FLAC, Opus, untagged WAV fallback, malformed and truncated input, overflow-inducing frame sizes, POPM and Vorbis ratings, embedded art | Specialized tag formats beyond these; real-world tagger quirks |
| Playlists, pictures | PlaylistParserTest.*, PictureLibraryIndexTest.*, SavedPictureStoreTest.* (traversal, absolute, backslash and dot names; empty root), ThumbnailGeneratorTest.* | Every image decoder; concurrent saves |
| Graph ownership | MediaLibraryTestFixture.ObjectGraphIsInternallyConsistent, SongsPointBackAtTheirOwningAlbumArtistAndGenre, the MediaLibrarySavePictureTest cases and MediaLibrarySavePictureNoPreexistingRootTest (a pictures root that did not exist at construction) and album, artist, genre, picture and playlist suites | Use after Dispose; graph lifetime under other threads |
| Queue and player | MediaQueueTest.*, MediaPlayerTest.* (duplicate-not-original, shuffle repeat, state transitions, dispatcher events), MediaPlayerNoSoundFallbackTest.* (four cases of the pure elapsed-time function) | The failure and hazard cases listed above; the mixer-less profile at run time |
| Visualization | VisualizationFFTTest.*, VisualizationCaptureTest.*, four MediaPlayerTest visualization cases | The no-mixer or failed-install branch (see below) |
| Video | VideoBackendAvailabilityTest.*, VideoTest.*, VideoPlayerTest.*, VideoDecoderTest.* | Real texture upload and audible output on every backend |
Configuration decides what is even compiled. cmake/UnitTests.cmake drops VideoDecoderTests.cpp, VideoTests.cpp, VideoPlayerTests.cpp and the content video XNB suite when CNA_FFMPEG_AVAILABLE is off, and keeps VideoBackendAvailabilityTests.cpp, which compiles exactly one of two mutually exclusive tests: EnabledBuildReportsDecoderAvailable when CNA_VIDEO_AVAILABLE is defined, DisabledBuildRetainsMetadataApiAndRejectsDecoding otherwise. When FFmpeg is available the same file defines CNA_TEST_HAS_AUDIO_DURATION_PROBE for the media test objects so the two suites that assert a probed duration sum (AlbumTests.cpp, PlaylistTests.cpp) know which probe they are linked against. Check the configured test list before claiming playback coverage. Other facts that decide whether a run means anything:
- Fixtures are addressed relative to the repository root (
tests/assets/media/...), and the discovered tests run with that working directory.MediaLibraryTestFixtureredirects both roots to the checked-in trees and clears the overrides inTearDown; the save-picture tests use scratch copies so the fixture tree gains noSaved Picturesfolder. When changing library code, check that any override is still restored. - The focused executable is
CnaMediaTests, anEXCLUDE_FROM_ALLdeveloper target; it is not a separate CTest registration, so CTest reaches these cases only throughCnaTests. MediaPlayerTestsetsSDL_AUDIODRIVER=dummy. Its own comments state that the mixer always exists in that run, so the no-mixer or failed-install visualization branch is correct by construction and not by test. The one test that plays MP3 and FLAC songs to their natural end (Mp3AndFlacSongsPlayThroughCnasOwnMixerToTheirEnd) is compiled only underCNA_AUDIO_PLATFORM_ALSA; no test of the natural-end path under SDL3_mixer is named in this module.- The Linux workflows install the FFmpeg development packages, so their default
AUTOconfiguration would select the backend; thedev,unitandrelease-modulespresets setCNA_ENABLE_VIDEO=OFF. A green preset build is therefore not evidence that video decodes; see the FFmpeg validation section.
For a library change, run the index and graph tests against a disposable music and picture root. For a music change, run MediaPlayerTest plus the dispatcher and audio mixer tests, and a real audio host if the bug concerns audible output. For video, test both an FFmpeg-enabled and a disabled configuration, then texture and audio output on a capable host. For all three, check shutdown and repeated construction or Play, which is how stale static state shows up. The shape of a focused run, not executed here:
# from the repository root, so tests/assets/media/... resolves
cmake --build <build-dir> --target CnaMediaTests
<build-dir>/CnaMediaTests --gtest_filter='MediaPlayerTest.*:MediaQueueTest.*:MediaLibrary*'
What remains unverified. The specialised tag formats beyond ID3v2, Vorbis, Opus and FLAC, every picture decoder, and host-specific sound behaviour still need separate audits; live audio and video output hosts were not exercised for these pages, and nothing here claims the module complete. The findings above come from reading the sources and test files at 009d40f5.
Source-reading order
media/CMakeLists.txt, then the decoder block inmodules/CMakeLists.txt: fix the physical boundary and which decoder implementation entered the build.MediaLibraryPaths.cppandMediaLibraryIndex.cpp: roots, the scan, the extension list and normalisation;AudioTagParser.cpponly when a tag format is in question.MediaLibrary.cpp: graph ownership, the back-reference pass and the saved-picture mutation, withSavedPictureStore.cppfor the write itself.MediaQueue.cpp,MediaPlayer.cppandFrameworkDispatcher.cpp: queue ownership, the audio callbacks and the event pump, with the mixer facade contract inMixerEngine.hpp.VideoPlayer.cppandVideo.cpp: the hand-off, thenMediaPlayerTests.cppto compare the player path against what the tests actually assert.
Cross-cutting rules live in the ownership and lifetime master map and the thread and callback map; the module's place among its siblings is in the module index, and the general change procedure starts at the Maintainer Handbook. For the audio side of the mixer see Audio engine internals.
Deep dives on this topic
Long-form pages that explain the exact semantics, invariants and evidence behind this subject.
- CNA and XNA 4.0: what the compatibility promise covers — What CNA translates and cannot load, which reference settles a disputed XNA question, what CNA deliberately is not at snapshot 009d40f5, and its Ms-PL licence and FNA provenance.
- Songs, the media library and video: the public contract — The caller-visible contract of CNA's Song, MediaLibrary, MediaPlayer, visualization data and VideoPlayer, including where file extensions and decoders disagree and how video availability changed.
Known issues in this area
Current defects, gaps and limitations at this snapshot that touch this subject.
- CNA-BUG-052: SongTypeReader and MediaLibraryIndex offer song formats (.opus, .aac, .wma) that no CNA-configured mixer decodes — Load<Song> resolves .opus, .aac and .wma files and the media library indexes .opus, but neither the vendored SDL3_mixer build nor CNA's ALSA mixer decodes them, so such songs load and then silently fail to play.
- CNA-BUG-137: MediaPlayer::Play(Song*) reads and writes a destroyed Song when given a Song that the queue owns — Play(Song*) clears the queue before it copies and plays the given song, so passing the queue's own song, such as getQueueProperty().getActiveSongProperty(), is a heap use-after-free.
- CNA-BUG-138: MediaPlayer flags ActiveSongChanged on every Play(Song*), even for the same song, and never for Play(const SongCollection&[, index]) — Play(Song*) compares the queue's copy with the caller's pointer, so the event fires on every call, while playing a collection never raises it even when the active song changes.
- CNA-BUG-139: MediaPlayer::Play(Song*) records Duration and PlayCount on the caller's Song instead of the queued copy, and Stop never resets that PlayCount — Play(Song*) hands the caller's Song to PlaySong, so the mixer duration and a PlayCount increment land on the library or content Song while the queue's active copy keeps zero, contrary to MediaLibrary.cpp's comment.
- CNA-BUG-140: MediaPlayer::Play reports no error when a song cannot be played and can leave the player Playing with no track — A mixer load, track or start failure makes PlaySong return silently after the previous track was destroyed, so the state may stay Playing indefinitely, and a vanished file or a missing audio device throws non-XNA excepti
- CNA-BUG-141: MediaPlayer::ProgramExit is documented as called at application exit, but nothing in CNA calls it — Unlike FNA, which hooks ProcessExit, CNA never calls MediaPlayer::ProgramExit outside an explicit C API export, so the music track is not released by CNA at exit and the header's statement is false.
- CNA-BUG-142: Without a mixer (SDL2 or NULL audio) songs never end: queued copies carry no Duration, so the elapsed-time fallback never fires — LoadSong copies only a Song's file and name, so every queued Song has a zero Duration and, in builds without SOUND_ENABLED, DetectSongEndedByElapsedTime never reports an end: MediaPlayer stays Playing and the queue never
- CNA-BUG-143: VideoPlayer and Video keep raw pointers to each other, so destroying a Video before its player is stopped writes to freed memory — VideoPlayer stores a borrowed Video* and Video a borrowed VideoPlayer* parent; CloseDecoder, run by Stop, Dispose and the destructor, writes through the stored Video*, and no header states the ordering rule.
- CNA-BUG-144: VideoPlayer::GetTexture lets the decoder's std::runtime_error escape, undocumented, and leaves the player Playing — A decode, packet, resample or I/O error mid-stream is thrown by VideoDecoder::NextFrame as std::runtime_error through GetTexture, whose header documents only ObjectDisposedException, and the player stays Playing so the n
- CNA-VGAP-006: Media paths without tests: the natural song end on SDL3_mixer, the failed visualization install, MediaLibrary accessors after Dispose and SavePicture's IOException — The song-ended callback path on the default SDL3 audio, the visualization branch where no tap can be installed, MediaLibrary disposal semantics and SavePicture's write-failure exception are implemented but untested.
Related pages
The same subsystem is explained at four altitudes. These are the neighbouring pages at each one.
- User guide
- Video Playback: VideoPlayer · Audio: implemented playback tier · XNA compatibility: Media · Game loop: FrameworkDispatcher::Update
- Architecture
- Audio and input architecture · Runtime lifecycle
- Internals
- FFmpeg video boundary · Audio engine internals · Content runtime internals · SDL3 platform internals
- Maintainer workflow
- I need to debug shutdown and lifetime behavior · I need to add a regression test · Ownership and lifetime master map · Thread and callback map
- Tests and validation
- Test architecture and change recipes · What to test after changing X
- Reference
- Module index · Test target index · CMake option index