Tutorial 121: Video Playback with VideoPlayer

CNA Tutorials  ·  Media

CNA’s VideoPlayer is a real decoder, not a placeholder. It opens the file through FFmpeg, decodes video frames and audio, paces playback against presentation timestamps rather than a fixed frame counter, plays the file’s own audio track, and lets you switch between tracks in a multi-track file. What it is not is universally available — and that is the first thing to settle, because the decoder is an optional, build-time choice that shows up at run time as an exception.

Start here: is video in your build?

Without the backend, Video and VideoPlayer still exist and still link on every target, so your code compiles and links everywhere. What changes is behaviour: constructing a file-backed Video for a file that exists, and calling VideoPlayer::Play(), throw System::NotSupportedException (“built without the optional FFmpeg video backend”). Metadata-only Video objects and the XNB VideoReader keep working, so a video asset can be loaded anywhere and simply cannot be played without a decoder.

So there is both a build-time and a run-time way to guard. At build time, CNA defines CNA_VIDEO_AVAILABLE (and the older CNA_FFMPEG_AVAILABLE) on the targets that link the CNA framework when the backend is present. At run time, catch the exception and fall back to a still image or a skip:

#ifdef CNA_VIDEO_AVAILABLE
    // this build can decode video
#else
    // this build has no decoder: show a still image instead
#endif

try {
    player_.Play(&*intro_);
} catch (const System::NotSupportedException&) {
    GoToMainMenu();   // no decoder in this build
}

On Debian and Ubuntu the prerequisites for a Linux build with video are:

sudo apt install libavcodec-dev libavformat-dev libavutil-dev libswresample-dev
cmake -S . -B build -DCNA_ENABLE_VIDEO=ON     # ON: a missing package is an error, not a silent fallback

See Tutorial 124: WebAssembly gotchas for the web side of this, alongside the other things that behave differently in the browser.

Loading a video

There are two routes. Through the content manager, if the file lives in your content root (from a Game the manager is getContentProperty(), and Load returns the Video by value):

// Content/videos/intro.mp4
Media::Video intro = getContentProperty().Load<Media::Video>("videos/intro");

The loose-file reader recognises .mp4, .ogv, .webm, .mkv, .avi and .mov, tried in that order when you omit the extension. The reader is registered in every build, so Load works everywhere; in a no-decoder build it throws System::NotSupportedException for an existing file, exactly like the direct constructor below. Video can also be built ahead of time: the cna-content tool turns those media files into a Video asset (metadata plus a reference to the media file, which is deployed beside it), and the XNB VideoReader reads such an asset without needing a decoder.

Or construct one directly from a path, which is often more convenient for a video that is not a shipped asset:

CNAEXT Video(std::string fileName, Graphics::GraphicsDevice* device);

This constructor probes the file immediately. It throws System::IO::FileNotFoundException if there is nothing there and, in a build without the FFmpeg backend, System::NotSupportedException if there is. A second, seven-argument constructor exists for the XNB path, where duration, dimensions, frame rate and soundtrack type come from the asset metadata rather than the file. Note that Video has no default constructor, so a class that holds one as a member needs std::optional<Video> (or a pointer) and creates it when the file is known.

Playing and drawing

VideoPlayer hands you a Texture2D* for the current frame. Call GetTexture() every frame you draw — it is the call that advances decoding, so skipping it stalls playback.

#include "Microsoft/Xna/Framework/Media/Video/Video.hpp"
#include "Microsoft/Xna/Framework/Media/Video/VideoPlayer.hpp"

using namespace Microsoft::Xna::Framework;
using namespace Microsoft::Xna::Framework::Media;

class IntroScreen
{
    std::optional<Video> intro_;        // Video has no default constructor
    VideoPlayer          player_;
    bool                 hasVideo_ = false;

    void Start(Content::ContentManager& content)
    {
        try
        {
            intro_ = content.Load<Video>("videos/intro");
            player_.setIsLoopedProperty(false);
            player_.setVolumeProperty(1.0f);
            player_.Play(&*intro_);       // throws NotSupportedException without a decoder
            hasVideo_ = true;
        }
        catch (const System::NotSupportedException&)
        {
            hasVideo_ = false;            // no decoder in this build: skip the intro
        }
    }

    void Draw(Graphics::GraphicsDevice& device, Graphics::SpriteBatch& spriteBatch)
    {
        if (!hasVideo_)
        {
            return;
        }

        Graphics::Texture2D* frame = player_.GetTexture();
        if (frame == nullptr)
        {
            return;   // nothing decoded yet
        }

        const auto& viewport = device.getViewportProperty();
        const Rectangle destination(0, 0,
                                    viewport.getWidthProperty(),
                                    viewport.getHeightProperty());

        spriteBatch.Begin();
        spriteBatch.Draw(*frame, destination, Color::White);
        spriteBatch.End();
    }
};

Watch for the end of playback in your update, not your draw:

if (player_.getStateProperty() == MediaState::Stopped)
{
    GoToMainMenu();
}

MediaState has three values: Stopped, Playing and Paused.

How pacing works

Playback is driven by a wall clock, not by how often you happen to call GetTexture(). Each call compares the elapsed play time against the presentation timestamp of the last decoded frame, and decodes forward until it has caught up. A frame that arrives late is skipped rather than shown late, so a stutter in your render loop costs you frames rather than desynchronising the audio.

Consequently, calling GetTexture() more often than the video’s frame rate is free — you simply get the same texture back — and calling it less often makes the player skip. Paused and stopped players return the existing texture without decoding anything.

The video’s own audio

This is genuinely implemented, and it is worth stating plainly because it is the part most often stubbed out elsewhere: the audio stream inside your video file is decoded, resampled and played, in sync with the frames.

player_.setVolumeProperty(0.7f);   // documented range [0, 1]
player_.setIsMutedProperty(true);  // mutes without stopping

Pausing the player pauses the audio device stream too, so there is no drift on resume. At the end of a non-looping video the player waits for the queued audio to drain before it reports Stopped, which is why the transition can lag the last visible frame by a moment.

Multi-track selection (CNAEXT)

Files with several audio streams — a localised release with one track per language, or a commentary track — are addressable by index:

CNAEXT void SetAudioTrackEXT(SharpRuntime::intcs track);
CNAEXT void SetVideoTrackEXT(SharpRuntime::intcs track);

player_.SetAudioTrackEXT(1);   // second audio stream
player_.Play(&intro_);

Set the track before Play() and the preference is applied as the decoder opens. Set it during playback and the player switches, reconfiguring only the output that actually changed. Video carries the same two methods and forwards them to its player.

These are extensions — real XNA has no track selection — so they will not compile under -DCNA_STRICT_XNA_API=ON.

Magenta means “pixel format not handled”

RGBA, RGB24, NV12 and the common planar 8-, 10- and 12-bit YUV layouts convert normally. Anything else lands on the fallback. The fix is on the encoding side: re-encode to a mainstream format, most simply H.264 in yuv420p.

ffmpeg -i exotic.mkv -pix_fmt yuv420p -c:v libx264 -c:a aac intro.mp4

The rest of the surface

On Video itself: getWidthProperty(), getHeightProperty(), getFramesPerSecondProperty(), getDurationProperty() and getVideoSoundtrackTypeProperty(), which returns one of Music, Dialog or MusicAndDialog.

Before you ship a video

• Does every path that can start a video handle System::NotSupportedException (or sit behind CNA_VIDEO_AVAILABLE), so Windows, web, mobile and no-FFmpeg desktop builds still start the game?
• Does the video actually play, rather than showing magenta?
• Can the player skip it, and does the skip stop the player rather than just hiding it?
• If you need video on Linux or macOS, do your build machine and your CI have the four FFmpeg dev packages, and do you configure with -DCNA_ENABLE_VIDEO=ON so a missing package fails the build instead of silently dropping video?
• Have you tested what happens on a machine with no audio device?

Where to go next

• Tutorial 122: MediaLibrary and audio visualization
• Tutorial 124: WebAssembly gotchas
• Tutorial 21: SpriteBatch in Depth
• Video Playback reference
• Platform support reference