Tutorial 121: Video Playback with VideoPlayer

CNA Tutorials  ·  Media

CNA’s VideoPlayer is a real decoder, not a placeholder. It opens the file through FFmpeg, decodes video frames and audio, paces playback against presentation timestamps rather than a fixed frame counter, plays the file’s own audio track, and lets you switch between tracks in a multi-track file. What it is not is universally available — and that is the first thing to settle, because it is a build-time question, not a runtime one.

Start here: is video in your build?

The second point is the one that catches people. There is no runtime “is video available” query to branch on, because the symbols simply are not in the binary. Guard at build time with your own macro:

#if !defined(_WIN32) && !defined(__EMSCRIPTEN__) && !defined(__ANDROID__)
    #define GAME_HAS_VIDEO 1
#endif

On Debian and Ubuntu the prerequisites are:

sudo apt install libavcodec-dev libavformat-dev libavutil-dev libswresample-dev

See Tutorial 124: WebAssembly gotchas for the web side of this, alongside the two other things that behave differently in the browser.

Loading a video

There are two routes. Through the content manager, if the file lives in your content root:

// Content/videos/intro.mp4
Media::Video intro = Content.Load<Media::Video>("videos/intro");

The loose-file reader recognises .mp4, .ogv, .webm, .mkv, .avi and .mov, tried in that order when you omit the extension. Note that the reader is registered only on the platforms where video is compiled in.

Or construct one directly from a path, which is often more convenient for a video that is not a shipped asset:

CNAEXT Video(std::string fileName, Graphics::GraphicsDevice* device);

This constructor probes the file immediately and throws System::IO::FileNotFoundException if there is nothing there. A second, seven-argument constructor exists for the XNB path, where duration, dimensions, frame rate and soundtrack type come from the asset metadata rather than the file.

Playing and drawing

VideoPlayer hands you a Texture2D* for the current frame. Call GetTexture() every frame you draw — it is the call that advances decoding, so skipping it stalls playback.

#include "Microsoft/Xna/Framework/Media/Video/Video.hpp"
#include "Microsoft/Xna/Framework/Media/Video/VideoPlayer.hpp"

using namespace Microsoft::Xna::Framework;
using namespace Microsoft::Xna::Framework::Media;

class IntroScreen
{
    Video       intro_;
    VideoPlayer player_;

    void Start()
    {
        player_.setIsLoopedProperty(false);
        player_.setVolumeProperty(1.0f);
        player_.Play(&intro_);
    }

    void Draw(Graphics::GraphicsDevice& device, Graphics::SpriteBatch& spriteBatch)
    {
        Graphics::Texture2D* frame = player_.GetTexture();
        if (frame == nullptr)
        {
            return;   // nothing decoded yet
        }

        const auto& viewport = device.getViewportProperty();
        const Rectangle destination(0, 0,
                                    viewport.getWidthProperty(),
                                    viewport.getHeightProperty());

        spriteBatch.Begin();
        spriteBatch.Draw(*frame, destination, Color::White);
        spriteBatch.End();
    }
};

Watch for the end of playback in your update, not your draw:

if (player_.getStateProperty() == MediaState::Stopped)
{
    GoToMainMenu();
}

MediaState has three values: Stopped, Playing and Paused.

How pacing works

Playback is driven by a wall clock, not by how often you happen to call GetTexture(). Each call compares the elapsed play time against the presentation timestamp of the last decoded frame, and decodes forward until it has caught up. A frame that arrives late is skipped rather than shown late, so a stutter in your render loop costs you frames rather than desynchronising the audio.

Consequently, calling GetTexture() more often than the video’s frame rate is free — you simply get the same texture back — and calling it less often makes the player skip. Paused and stopped players return the existing texture without decoding anything.

The video’s own audio

This is genuinely implemented, and it is worth stating plainly because it is the part most often stubbed out elsewhere: the audio stream inside your video file is decoded, resampled and played, in sync with the frames.

player_.setVolumeProperty(0.7f);   // documented range [0, 1]
player_.setIsMutedProperty(true);  // mutes without stopping

Pausing the player pauses the audio device stream too, so there is no drift on resume. At the end of a non-looping video the player waits for the queued audio to drain before it reports Stopped, which is why the transition can lag the last visible frame by a moment.

Multi-track selection (CNAEXT)

Files with several audio streams — a localised release with one track per language, or a commentary track — are addressable by index:

CNAEXT void SetAudioTrackEXT(SharpRuntime::intcs track);
CNAEXT void SetVideoTrackEXT(SharpRuntime::intcs track);

player_.SetAudioTrackEXT(1);   // second audio stream
player_.Play(&intro_);

Set the track before Play() and the preference is applied as the decoder opens. Set it during playback and the player switches, reconfiguring only the output that actually changed. Video carries the same two methods and forwards them to its player.

These are extensions — real XNA has no track selection — so they will not compile under -DCNA_STRICT_XNA_API=ON.

Magenta means “pixel format not handled”

RGBA, RGB24, NV12 and the common planar 8-, 10- and 12-bit YUV layouts convert normally. Anything else lands on the fallback. The fix is on the encoding side: re-encode to a mainstream format, most simply H.264 in yuv420p.

ffmpeg -i exotic.mkv -pix_fmt yuv420p -c:v libx264 -c:a aac intro.mp4

The rest of the surface

On Video itself: getWidthProperty(), getHeightProperty(), getFramesPerSecondProperty(), getDurationProperty() and getVideoSoundtrackTypeProperty(), which returns one of Music, Dialog or MusicAndDialog.

Before you ship a video

• Is every VideoPlayer reference behind a build-time guard, so Windows and web builds still link?
• Does the video actually play, rather than showing magenta?
• Can the player skip it, and does the skip stop the player rather than just hiding it?
• Does your build machine have the four FFmpeg dev packages, and does your CI?
• Have you tested what happens on a machine with no audio device?

Where to go next

• Tutorial 122: MediaLibrary and audio visualization
• Tutorial 124: WebAssembly gotchas
• Tutorial 21: SpriteBatch in Depth
• Video Playback reference
• Platform support reference