Tutorial 122: MediaLibrary and Audio Visualization

CNA Tutorials  ·  Media

Opening the library

#include "Microsoft/Xna/Framework/Media/MediaLibrary.hpp"

using namespace Microsoft::Xna::Framework::Media;

MediaLibrary library;   // scans on construction

Construction resolves the user’s Music and Pictures folders through SDL’s own user-folder API and walks them. The scan is one-shot: the library is a snapshot, not a live view, so recreate it if you expect the folders to have changed.

A second constructor takes a MediaSource*. Sources come only from MediaSource::GetAvailableMediaSources() — the constructor is private — and MediaSourceType has two values, LocalDevice and WindowsMediaConnect.

Browsing what it found

Every catalogue is a collection pointer owned by the library:

SongCollection*     songs     = library.getSongsProperty();
AlbumCollection*    albums    = library.getAlbumsProperty();
ArtistCollection*   artists   = library.getArtistsProperty();
GenreCollection*    genres    = library.getGenresProperty();
PlaylistCollection* playlists = library.getPlaylistsProperty();
PictureCollection*  pictures  = library.getPicturesProperty();
PictureAlbum*       root      = library.getRootPictureAlbumProperty();
PictureCollection*  saved     = library.getSavedPicturesProperty();

Songs carry real metadata, not filename guesses — when the file has tags:

const std::string&  title  = song->getNameProperty();
System::TimeSpan    length = song->getDurationProperty();
SharpRuntime::intcs track  = song->getTrackNumberProperty();
SharpRuntime::intcs rating = song->getRatingProperty();   // XNA's 0-10 scale

// Artist, Album and Genre are objects, not strings - each may be null.
Artist* artist = song->getArtistProperty();
Album*  album  = song->getAlbumProperty();
Genre*  genre  = song->getGenreProperty();

const std::string artistName = artist ? artist->getNameProperty() : std::string{};

That last detail catches people migrating from XNA reflexively: Artist, Album and Genre are real objects with their own song lists, not plain strings, and each one may be absent.

What is parsed

CNA implements the tag readers itself rather than pulling in a tagging library. Four formats are handled:

When a file has no usable tags at all, there is a filename heuristic: title from the file name, album from the parent directory, artist from the one above that. It is a fallback, and it is exactly as reliable as the user’s folder naming.

Durations are probed through FFmpeg’s container and stream metadata — no full decode, so the scan stays fast. On a build without FFmpeg (Windows, Android, Emscripten) the probe returns zero rather than failing, so expect songs with a zero duration there.

Album art

if (album->getHasArtProperty())
{
    System::IO::Stream* art = album->GetAlbumArt();   // caller owns the stream
    // ... decode into a Texture2D ...
}

Art is found in two passes. First, a sidecar image beside the songs — cover, folder, front, album or albumart, in .jpg, .jpeg or .png, matched case-insensitively and in that order of preference. Failing that, embedded art from the first member song that has any: ID3v2 APIC frames in MP3, and FLAC picture blocks, preferring the front-cover type.

GetAlbumArt() and GetThumbnail() both throw System::InvalidOperationException when HasArt is false, so check first. You own the returned stream.

Playlists

Playlists are read from .m3u and .m3u8 files in the top level of the music root — that scan is not recursive, unlike the song and picture scans. .pls is not supported.

One deliberate divergence from the M3U convention is worth knowing: entries with absolute paths, and relative entries that escape the playlist’s own directory via .. or a symlink, are skipped. A playlist is user-supplied data, and CNA will not let it point your game at arbitrary files.

Pictures

Pictures form a real tree: getRootPictureAlbumProperty() gives you the root PictureAlbum, whose getAlbumsProperty() and getPicturesProperty() let you walk down. Each Picture has a name, dimensions, a date, GetImage() and GetThumbnail().

Saving works too — useful for an in-game screenshot or photo mode:

Picture* SavePicture(std::string name, const std::vector<uint8_t>& imageBuffer);
Picture* SavePicture(std::string name, System::IO::Stream* source);

Saved pictures come back through getSavedPicturesProperty(), and a picture’s CNAEXT getTokenEXT() gives you a stable handle you can pass to GetPictureFromToken() later.

Playing a user’s music

MediaPlayer is entirely static — one song at a time, globally:

MediaPlayer::Play((*songs)[0]);            // one song (operator[] yields a Song*)
MediaPlayer::Play(*songs);                 // the whole collection as a queue
MediaPlayer::Play(*songs, 4);              // the queue, starting at index 4

MediaPlayer::setIsShuffledProperty(true);
MediaPlayer::setIsRepeatingProperty(true);
MediaPlayer::setVolumeProperty(0.6f);      // clamped to [0, 1]

MediaPlayer::MoveNext();
MediaPlayer::MovePrevious();
MediaPlayer::Pause();
MediaPlayer::Resume();
MediaPlayer::Stop();

The current queue is available as MediaPlayer::getQueueProperty(), which returns a MediaQueue& with getActiveSongProperty(), getActiveSongIndexProperty(), getCountProperty() and operator[]. Two events, ActiveSongChanged and MediaStateChanged, fire as playback moves along.

Which formats actually play

CNA plays audio through a vendored SDL3_mixer, built with a specific set of decoders. This is what that means in practice:

The AAC gap matters for music libraries specifically, because .m4a is what an iTunes-shaped collection is full of. Offer the user a graceful failure rather than assuming everything in their Music folder will play.

Visualization: a real FFT

This is the genuinely fun part. MediaPlayer can tap the live mixed output and hand you both a waveform and a frequency spectrum — not a decorative approximation, but a real radix-2 FFT over audio captured from the post-mix callback.

#include "Microsoft/Xna/Framework/Media/VisualizationData.hpp"

MediaPlayer::setIsVisualizationEnabledProperty(true);

// Verify it actually took effect - see the warning below.
if (!MediaPlayer::getIsVisualizationEnabledProperty())
{
    // No mixer or no audio device: draw something static instead.
}

Then sample it once per frame:

VisualizationData vis;                    // 256 floats each, zero-initialised
MediaPlayer::GetVisualizationData(vis);

const std::array<float, VisualizationData::Size>& spectrum = vis.getFrequenciesProperty();
const std::array<float, VisualizationData::Size>& waveform = vis.getSamplesProperty();

A minimal bar-graph visualiser, drawn with a one-pixel white texture:

void DrawSpectrum(Graphics::SpriteBatch& spriteBatch,
                  const Graphics::Texture2D& whitePixel,
                  int screenWidth, int screenHeight)
{
    VisualizationData vis;
    MediaPlayer::GetVisualizationData(vis);
    const auto& bins = vis.getFrequenciesProperty();

    const int barWidth = std::max(1, screenWidth / VisualizationData::Size);

    spriteBatch.Begin();
    for (int i = 0; i < VisualizationData::Size; ++i)
    {
        const int height = static_cast<int>(bins[i] * screenHeight * 4.0f);
        spriteBatch.Draw(whitePixel,
                         Rectangle(i * barWidth, screenHeight - height, barWidth - 1, height),
                         Color::White);
    }
    spriteBatch.End();
}

Two details make the difference between a plot and something that looks alive:

• Bins are linear in frequency, and music is not. The FFT runs over a 512-sample Hann-windowed window and yields 256 bins, so bin i is centred on i × sampleRate / 512. Nearly all the visible energy lands in the first few dozen bins. Map bins to bars logarithmically, or plot only the lower part of the range.
• Smooth over time. Raw magnitudes flicker. Keep your own array and ease each bar toward the new value — a simple bar = bar * 0.7f + value * 0.3f is usually enough. A slower fall than rise reads especially well.

With visualization disabled, or before any audio has been captured, both arrays come back zeroed rather than throwing — so a visualiser degrades to a flat line rather than a crash.

Pitfalls

The user’s folders may not exist. On a headless machine, a fresh container, or a locked-down account, the folder lookup can come back with nothing. Every collection will simply be empty. Handle it as a normal case.

You are reading someone’s personal files. Scanning a real Music folder is a privacy-relevant act. Ask before you do it, and do not send what you find anywhere.

Scanning costs real time. A large music folder means a lot of file opens and tag parses. Do it off the critical path, not in the middle of a level load.

Streams are yours to free. GetAlbumArt(), GetThumbnail() and GetImage() hand you ownership.

Where to go next

• Tutorial 15: Background Music
• Tutorial 118: Procedural audio
• Tutorial 121: Video playback
• Audio reference