Network sessions and the SystemLink protocol

CNA snapshot 009d40f5  ·  Deep Dives › Input, audio, media & services  ·  source links pinned to 009d40f5

CNA's Microsoft::Xna::Framework::Net combines one real transport with several compatibility lifecycles: only NetworkSessionType::SystemLink puts bytes on a wire, through the vendored ENet library, while the other session types keep XNA's shape and either simulate the lifecycle or refuse. This page gives the exact session semantics a game can rely on, the SystemLink wire and discovery protocols, the trust model, delivery guarantees, host migration and the evidence behind each. It complements Tutorial 97, which teaches the task, and the network session internals, which trace ownership and the pump for maintainers.

What each session type does

An object that only watches session state cannot tell a real session from a simulated one, which is why this distinction comes first. The transport gate is ENetBackend::RealNetworkingEnabled, true only for SystemLink (ENetBackend.cpp); a second gate in NetworkSession.cpp refuses the matchmaking types.

What a synthetic session keeps is the lifecycle: gamers, StartGame/EndGame, state transitions and their events all work with no socket bound, and Find for LocalWithLeaderboards returns an empty collection at once instead of waiting for a discovery window. NetworkSessionTypePolicyTests.cpp pins both halves: for the two creatable synthetic types, no bound port and no data available after SendData plus Update; for the two matchmaking types, the refusals, including that a refused call leaves no pending action behind. JoinInvited is refused unconditionally with GamerServicesNotAvailableException, and EndJoinInvited rejects every result with ArgumentException so it cannot be used to slip past the refusal; an earlier version fabricated a transport-less PlayerMatch session instead. The static InviteAccepted event is declared and never raised.

Creating, owning and pumping a session

Who owns what

Create and Join return a raw NetworkSession* that the caller owns; Find returns an AvailableNetworkSessionCollection by value; JoinInvited never returns. Dispose() tears down the ENet host, disconnects peers, unregisters discovery and frees every gamer object the session and its transport created, but it does not delete the session object. The caller performs both steps; if a caller deletes without disposing, the destructor disposes first, because a dangling process-wide session pointer would otherwise block every later Create. A custom deleter makes the pairing automatic:

struct SessionDeleter {
    void operator()(NetworkSession* s) const { s->Dispose(); delete s; }
};
using SessionPtr = std::unique_ptr<NetworkSession, SessionDeleter>;

SessionPtr session(NetworkSession::Create(NetworkSessionType::SystemLink, 1, 8));

session->GamerJoined += [](System::Object*, const GamerJoinedEventArgs& e) {
    // Runs now for every gamer already in AllGamers, later from Update() for new joins.
    NetworkGamer* gamer = e.getGamerProperty();
};

// Once per game update; nothing else pumps the transport or raises events.
session->Update();

This and the packet example below were syntax-checked with g++ -std=c++23 -fsyntax-only against the snapshot's headers (sibling sharp-runtime headers for System); they were not built or run. Gamers created by the session are owned by it (LocalNetworkGamer objects in the session, remote gamers in the transport's per-session state); the four gamer collections are non-owning views. Subscribing to GamerJoined replays the handler synchronously for every gamer already in AllGamers, which is how the construction-time local gamers are reported without an extra Update; gamers who join later arrive through the event queue that Update drains.

Argument limits and the gamer count

BeginCreate validates maxLocalGamers in 1–4 (the explicit gamer-list overload has none), maxGamers in 2–MaxSupportedGamers (31) and privateGamerSlots in 0–maxGamers, throwing ArgumentOutOfRangeException. The requested maxGamers is forwarded to the session, so a session created with 8 reports 8, and AddRemoteGamer refuses a gamer beyond it with InvalidOperationException (“Session is full!”). Over the wire that refusal is weak: when the host handles a ClientHello it has already sent the joining client its ServerWelcome and registered its wire ids before AddRemoteGamer throws, and the receive path's catch-all drops the exception, so a join beyond the limit is not turned away with NetworkSessionJoinError::SessionFull (that value is never raised at this snapshot: CNA-BUG-267) and no test joins a full SystemLink session (read from the source; not executed). A joining machine's session, by contrast, is constructed with MaxSupportedGamers and four private slots, values the ported FNA code marks as placeholders, so a client's own MaxGamers does not mirror the host's. MaxPreviousGamers is 100; older departed gamers are evicted first.

One operation, one session, per process

Every Create, Find, Join and JoinInvited overload is a Begin*/End* pair wrapped in a polling loop, and all of them share two class statics: the in-flight NetworkSessionAction* and the live session. Any Begin* throws InvalidOperationException while either is set, so only one operation can be outstanding and only one session can exist in the process: a game cannot browse for sessions while it hosts one, and tests that need two peers use two processes. The action is marked complete at construction (its wait handle is already signalled), its callback runs once right after it is installed, and the matching End* deletes it and clears the static even when session construction throws. The real work happens inside End*: EndCreate constructs the session and starts hosting, EndFind performs the discovery search, EndJoin connects. One oddity: the action answers CompletedSynchronously with false although it completed before Begin* returned.

The completed-at-construction rule exists because of a hang. GamerServicesDispatcher::Update() is empty and UpdateAsync() returns true forever once GamerServices is initialised, so the ported loop while (!result->IsCompleted) UpdateAsync(); never ended once a GamerServicesComponent existed. The same defect was reproduced against the FNA reference source, not only CNA's port. It is pinned out of process, as the GamerServices contract describes.

Join is blocking on native hosts

BeginJoin takes the session type, address and port from the AvailableNetworkSession it is given. On native builds EndJoin connects and then pumps Update on the calling thread, sleeping 1 ms per iteration, until the server welcome has replaced the construction-time host placeholder, the session has ended, or five seconds have passed; a timeout disposes and deletes the half-joined session and throws NetworkSessionJoinException with SessionNotFound, and any other failure is wrapped in the same exception type with its cause. XNA documents Join as blocking, and returning earlier let a faithful caller send its first packet to itself. On Emscripten that wait is compiled out: Join returns before the handshake and the game must keep calling Update.

NetworkSessionJoinError declares three values, SessionNotFound, SessionNotJoinable and SessionFull, and only the first is ever produced at this snapshot: it is what EndJoin reports when the handshake does not complete, and the default of NetworkSessionJoinException, which the catch-all path also uses. SessionFull is never raised (CNA-BUG-267) and SessionNotJoinable has no producer at all, so a refused join (a host that is playing with AllowJoinInProgress false disconnects the hello) cannot be told from an absent host by the error value. Read from the sources at 009d40f5; not executed.

Topology and gamer identity

SystemLink is a star. Every client talks only to the host; application data addressed to another client is relayed by the host to the peer that owns the target's wire id, and never echoed back to its origin. Wire ids are single bytes assigned by the host during the handshake, and ids freed by departed peers are reused before the counter advances, so the byte cannot wrap after 256 cumulative joins. Because the host assigns them, NetworkGamer::Id and IsHost agree across machines; the FNA code CNA ported answers a fixed 0 and true for every gamer. A local gamer's id is provisional until the handshake overwrites it, and it comes from a monotonic counter rather than a collection size, so a remove-then-add cannot produce a duplicate. NetworkGamer::RoundtripTime copies ENet's per-peer measurement, which exists only for directly connected peers: the host sees its clients' round-trip times, and every other gamer (a client's view of the host, or of another client) stays at zero.

The session wire protocol

Session and application messages travel over ENet on a connected channel; NetPacketCodec.hpp defines the leading tag byte:

Encoding reuses PacketWriter/PacketReader, the same BinaryWriter/BinaryReader wrappers games use. Every list count on this wire is one byte, and EncodeCount throws rather than wrapping when a list would exceed 255 entries; MaxSupportedGamers keeps rosters far below that. The host is created with 31 peers and two channels, but every send uses channel 0. Session properties are published by polling: on each pump the host compares the properties with the last snapshot it broadcast (the get-only XNA property is a mutable collection, so edits cannot be observed any other way) and broadcasts a full snapshot when they differ.

A send issued before the handshake has given both its sender and its target a wire id cannot be addressed yet. The backend queues such sends per session (at most 64; the 65th evicts the oldest) and flushes them in order once a ClientHello, ServerWelcome or GamerJoinBroadcast resolves both ends. Entries naming a gamer who leaves or disconnects are purged, and host migration or the end of a client's session clears the queue; every entry that can never be delivered increments a process-wide counter, GetDroppedAppDataCount, which tests and diagnostics read but NetworkSession does not expose.

LAN discovery

ENet has no discovery, so ENetDiscoveryService.cpp runs a separate raw-UDP protocol (NetDiscoveryProtocol.hpp) on the well-known port 61190. A searching client sends a Query (tag 0x01: protocol version, session-type filter) by broadcast and, as a same-machine fallback, by unicast to 127.0.0.1, then polls on the calling thread for a 150 ms window and de-duplicates the Announce replies (tag 0x02) by connect port. A hosting session answers during its own Update when the query's filter matches its type, with its connect port, gamer counts, open private and public slots, host gamertag and advertised properties. The game session itself uses an OS-assigned ENet port on native hosts; discovery is what tells a client which one.

Each reply's QualityOfService carries the measured time between sending the query and receiving that reply as both its average and minimum round-trip time; BytesPerSecondUpstream and BytesPerSecondDownstream stay zero, because connectionless discovery has no established peer to measure throughput on, and IsAvailable keeps its default true.

Parsing untrusted datagrams

Discovery datagrams come from any device on the LAN, so the decoder assumes hostile input. The protocol-version byte is compared with kDiscoveryProtocolVersion (1) before anything else is parsed, so a future format is refused cleanly instead of being misread. Advertised properties travel as sparse (index, value) pairs, and each index must lie in [0, kMaxPropertyIndex), which is 256: a negative index would reach the vector with a huge unsigned offset, and an index near the int32 maximum would drive about two billion Add calls from one crafted packet. A malformed datagram is dropped, and a guard resets the search's result pointer even when decoding throws.

The trust model

There is no authentication or encryption on either wire. The protocol's defence is structural, and its stated threat is a modified client speaking the easily inferred wire format, which needs no man-in-the-middle position:

• Host-only messages are accepted only from the host. ServerWelcome, GamerJoinBroadcast, GamerLeaveBroadcast, StateChangeBroadcast and SessionPropertiesBroadcast are honoured only when they arrive from the peer this side itself connected to; a host never connects out, so it rejects all five. A forged one is logged as a protocol event and the sender is disconnected. Without this, any peer could kick gamers, inject fake ones, corrupt wire-id assignment or force a state change. Every machine needs the check, because even a client owns a real accepting ENet host.
• A repeated hello disconnects the peer. A peer that completed its handshake and sends ClientHello again would otherwise create a fresh batch of gamer objects each time.
• Join-in-progress is enforced. A host in Playing with AllowJoinInProgress false disconnects a new client's hello.
• Malformed packets cannot throw out of Update. A payload that fails to decode is dropped inside a guard.

Each rule has a test in ENetBackendTests.cpp. None of this authenticates a peer: a LAN session is exactly as trustworthy as the machines on the LAN.

Delivery guarantees

XNA defines five SendDataOptions; ENet can express three behaviours, and NetPacketCodec::SendDataOptionsToEnetFlags maps them:

ENet's reliable delivery is always ordered, and reproducing “reliable but unordered” would need another protocol layer, so Reliable receives the stronger ordered guarantee. Code written against XNA's weaker promise still works; code that relies on the ordering is relying on CNA, not on XNA. Two more rules shape what arrives. LocalNetworkGamer::SendData never touches the network: it only queues one event per recipient, and the bytes leave during the next Update. The broadcast overloads address every gamer in AllGamers, including the sender, so a receiver must check sender->getIsLocalProperty(). A reused PacketWriter sends only the bytes before its current position and is then rewound, so a short packet written after a long one does not carry stale trailing bytes.

Packet shapes that surprise

• Colours are four bytes both ways. PacketWriter::Write(Color) writes R, G, B and A as bytes, and PacketReader::ReadColor() reads four bytes back. An earlier version read four floats, so the two were not inverses and a packet holding one colour could not be read; builds from before the fix cannot exchange colour packets with this snapshot.
• ReceiveData(PacketReader&, NetworkGamer*&) returns 0. It fills the reader and rewinds it, but the length it returns is a variable the FNA source declares and never updates; the quirk is preserved and pinned by ReceiveDataIntoPacketReaderReturnsZero. Gate reads on getIsDataAvailableProperty(), or use the byte-vector overload, which returns the copied length (at most the buffer size; an offset that does not fit throws ArgumentException after the packet has already been dequeued).
• The mutable properties indexer appends. On a non-const NetworkSessionProperties, operator[] with an index at or past the end appends one empty slot and returns it, at indexing time, even for a pure read, because a C++ proxy cannot tell a read from a write. It appends at the end rather than extending to the index, as FNA's setter does. Use a const NetworkSessionProperties& or getItem for bounds-checked reads (both throw), and setItem for XNA's exact setter semantics.

PacketWriter writer;
writer.Write(tint);                                   // four bytes
me->SendData(writer, SendDataOptions::ReliableInOrder);

PacketReader reader;
NetworkGamer* sender = nullptr;
while (me->getIsDataAvailableProperty()) {            // not the ReceiveData return value
    me->ReceiveData(reader, sender);
    if (sender != nullptr && !sender->getIsLocalProperty()) {
        Color received = reader.ReadColor();          // four bytes
    }
}

const NetworkSessionProperties& props = session->getSessionPropertiesProperty();
std::optional<int> mode = props[0];                   // const: throws if out of range

Host migration

AllowHostMigration defaults to false; losing the host then ends the session, which is the reference behaviour. With it set, every survivor computes the same successor from the roster it already holds, the lowest remaining wire id, so no election message is needed. Both roles begin the same way: every remote gamer is removed (queuing GamerLeft for each), all wire-id state is reset and queued sends are dropped. The successor then marks its local gamers as host, registers its already-bound ENet host for discovery and raises HostChanged. Every other survivor rediscovers the new host through the normal LAN search, matching by the cached gamertag, and performs an ordinary hello/welcome reconnect. The search is tried three times with its 150 ms window, because when several processes share the discovery port the operating system decides which socket receives a datagram. If nobody reachable is found the session ends as if migration were off.

Consequences for a game: migration is a reconnect, not a hand-over. Remote gamers come back as new NetworkGamer objects through GamerJoined, so pointers held as map keys must be re-keyed; sockets are not preserved; the departed objects stay alive in PreviousGamers until the session is torn down, so earlier event arguments do not dangle. Two hosts with the same gamertag on one LAN are ambiguous to the rediscovery step.

Simulated latency and packet loss

SimulatedLatency and SimulatedPacketLoss are stored-but-inert properties in FNA; in CNA they act, as test controls. Both apply only to application data arriving for one of this session's own local gamers: session-management messages and a host's relay hop between two other peers are never delayed or dropped. Loss is a per-packet probability; exactly 0 and exactly 1 never consult the random generator, which tests can seed. Latency parks packets in a per-session queue released when a later Update observes the due time, so delivery granularity is the frame rate. Neither affects the discovery-time QualityOfService measurement.

What SystemLink still does not do

• Voice and invitations. LocalNetworkGamer::EnableSendVoice and SendPartyInvites have empty bodies; HasVoice, IsTalking, IsMutedByLocalUser and IsPrivateSlot keep their false defaults.
• Machine removal. NetworkMachine::RemoveFromSession() throws NotImplementedException, as FNA's stub does.
• Arbitration and TrueSkill. WriteArbitratedLeaderboard, WriteUnarbitratedLeaderboard and WriteTrueSkill are declared events that nothing raises.
• Internet play. Discovery is LAN broadcast; there is no relay, NAT traversal or matchmaking service.

SystemLink on the web

A browser page has no raw UDP and cannot accept connections. Under Emscripten the discovery functions are empty and Find returns nothing, so an empty result on the web means “cannot search”, not “no hosts yet”. Hosting requests the fixed port 61191, because Emscripten's socket layer does not report an OS-assigned port back; that is only meaningful for a Node.js-run relay or server build, since a browser tab cannot listen. A connecting web client rebuilds its transport state as an outbound-only client, and Join returns before the handshake (see above). CNA's own networking demos are excluded from Emscripten builds. None of the Emscripten paths is covered by a browser test at the snapshot; they are source-level statements.

Lifetime hardening, historically

The session lifecycle carries a sequence of fixed defects, each of which left a guard in the current code. They are listed because the guards are easy to undo by accident:

• Dispose() ran twice and walked gamer pointers the first call had freed, a use-after-free that AddressSanitizer observed while ordinary unit runs passed. It is now idempotent (DisposeCalledTwiceDirectlyIsSafeAndIdempotent).
• After disposal, raw pointers stayed visible through the gamer collections and Host; Dispose now clears all four views and nulls the host (DisposeClearsPreviousGamersSoNoDanglingPointerIsObservable).
• Gamer objects the session created were never destroyed; the session and the transport now own them in registries separate from the non-owning views.
• Completion callbacks were stored and never invoked, including when a callback re-entrantly replaced the action; the callback now runs once, after the action is installed, and Begin* returns the action it created rather than a stale null. For a session Begin*/End* pair that pointer is not safe to use in one case: a callback that calls the matching End* from inside itself makes End* delete the action, and with it the std::function that is running the callback, so the pointer Begin* then returns is already freed and the callback must not touch its captures after End* returns; no test checks either point (source reading, not executed; CNA-BUG-268).
• End* cleared the static action without deleting it, leaking one object per operation; tests now count live actions.
• Local gamer ids derived from a collection size collided after a remove and an add; ids now come from a monotonic counter.
• Application data sent before the handshake was silently dropped, and the first queueing fix did not purge entries when peers left; the bounded, purged queue above replaced both.
• A disposed gamer-collection enumerator's MoveNext() dereferenced a null collection although Current guarded the same state; it now throws NullReferenceException instead.

The lesson the sequence teaches is about evidence: several of these survived green unit suites and were found only under sanitizers or by integration, so a lifetime change here needs a create/dispose run under AddressSanitizer, not only the unit tests.

Evidence and its limits

Focused tests exist for the codec and discovery formats (including malformed and truncated input), the type policy, ownership and disposal, callbacks, id allocation, queue eviction and purging, forged-message rejection, simulated loss and latency with an injected clock and seeded generator, and the preserved compatibility quirks; the suites are listed on the internals page. Cross-process transport evidence comes from TwoProcessLoopbackTest.cpp, which uses posix_spawn to start real host and client processes of net_two_process_harness.cpp, hands the host's bound port to the client through a pipe rather than through discovery (two processes sharing the discovery port is fragile), and exchanges application data under a 20-second outer watchdog; a third case runs a host and two survivors to exercise migration across real processes. These tests are present and were not executed for this page.

What that establishes is SystemLink delivery between processes on the kind of host the tests run on. It does not establish discovery between separate machines, behaviour on Windows (the process-spawning tests are compiled out there, and on Emscripten, Android and iOS, so they do not exist rather than being skipped), anything in a browser, Internet play, voice, or compatibility between peers built from different revisions. Treat multi-machine migration as best effort.