Network session internals

CNA snapshot 009d40f5  ·  Development › Module internals  ·  source links pinned to 009d40f5

modules/net/ contains the XNA-shaped session, gamer, packet and discovery APIs plus CNA's ENet implementation of them. NetworkSession is the public state machine; ENet is the transport, and at this snapshot real transport is gated to NetworkSessionType::SystemLink. Every other session type keeps a synthetic behaviour or is refused: there is no hidden online matchmaking service. This page is for maintainers who need to know how a caller-owned session, its gamer objects, the ENet host and peer maps, the discovery advertisement and the queued events stay mutually valid from Create to Dispose, and where the pump, the wire messages and the tests sit. The user view is Tutorial 97 and the Net row of XNA compatibility.

Build and type policy

modules/CMakeLists.txt enters gamer-services and net only under CNA_ENABLE_NET, which defaults to ON in the root file and is switched off by several presets (see the GamerServices page). modules/net/CMakeLists.txt builds a STATIC library CNA_Net (alias CNA::Net, 25 sources and 29 headers at this snapshot) and links CNA_GamerServices and the target enet publicly, plus the Sharp Runtime components Core.Base, IO, Collections.Core, Runtime and Threading. ENet is vendored in the CNA tree (third_party/enet) and added by cmake/ThirdPartyENet.cmake with EXCLUDE_FROM_ALL; the root file notes that sanitiser instrumentation stays outside the vendored SDL and ENet sub-builds. Neither library is in the CNA umbrella; a game links CNA_GamerServices and CNA_Net itself, and every session is created for SignedInGamer objects that come from the GamerServices layer. In a CNA_ENABLE_NET=OFF build cmake/UnitTests.cmake removes the Net and GamerServices test sources from the aggregate CnaTests, so a green NET=OFF run says nothing about the transport.

The policy lives in two places. ENetBackend::RealNetworkingEnabled returns true only for SystemLink. The public overloads add a second gate, ThrowIfSessionTypeNeedsMatchmakingService, which makes Create and Find throw GamerServicesNotAvailableException for PlayerMatch and Ranked; JoinInvited is refused unconditionally. An API name such as JoinInvited is therefore not evidence of an invitation service.

caller owns NetworkSession* returned by Create / Join
  -> session owns the LocalNetworkGamer objects it creates          [ownedGamers_, unique_ptr registry]
  -> ENetBackend::SessionState owns every REMOTE NetworkGamer        [OwnedRemoteGamers; AddRemoteGamer never takes ownership]
  -> the four public GamerCollection views borrow both kinds
  -> ENetBackend Sessions() map: per-session ENetHostHandle, peers, wire-id maps, queues
  -> ENetHostHandle owns the ENetHost; ENetDiscoveryService holds a process-static UDP socket

one live session per process:  static activeAction_ / activeSession_; every Begin* refuses if either is set

NetworkSession::Update   [caller thread; nothing else pumps]
  -> ENetBackend::PumpSession -> service(0): connect / receive / disconnect -> handshake, app-data
       -> direct-peer round-trip times -> release delayed deliveries -> publish property changes
  -> ENetDiscoveryService::Poll (answer queries)
  -> drain the NetworkEvent queue -> PacketSend delivery / public gamer and state events

Two invariants shape everything below. First, one session per process: the action and session pointers are class statics, and every BeginCreate, BeginFind, BeginJoin and BeginJoinInvited throws InvalidOperationException while either is non-null. A game cannot browse for sessions while it is hosting one, and tests that need two peers use two processes. Second, one thread: the header of ENetBackend.hpp states that the backend and its process-wide session map are single-threaded by contract, and the module contains no std::mutex or std::atomic. See the thread and callback map for the process-wide picture.

Construction, handshake and caller ownership

NetworkSession::Create and Join return raw pointers documented as caller-owned. Every overload is a Begin*/End* pair wrapped in a polling loop. BeginCreate validates its arguments (maxLocalGamers 1–4 (the explicit-gamer-list overload has no such argument), maxGamers 2–MaxSupportedGamers, which is 31, and privateGamerSlots 0–maxGamers), refuses matchmaking types, refuses a second live action or session, and allocates a NetworkSessionAction stored in activeAction_. The action is marked complete at construction and its callback is invoked once, right after activeAction_ is installed, so a re-entrant callback sees consistent state and the Begin* returns the pointer it made, although that pointer is already freed if the callback called the matching End* (End* deletes the action, including the std::function that is running the callback, and the re-entrant test ignores the return value). The constructor comment records this as a deliberate deviation from FNA, where the action only completes through GamerServicesDispatcher::Update; that method is empty in both code bases, so once a GamerServicesComponent exists the polling loop would spin forever; the deviation is pinned by the out-of-process hang tests in the test section. EndCreate builds the session inside a try, deletes the action either way (leaving activeAction_ null even when the constructor throws) and returns activeSession_.

The constructor gathers the local gamers. With an explicit std::vector<SignedInGamer*> it uses those. Without one it walks Gamer::getSignedInGamersProperty() up to maxLocalGamers and takes each non-guest gamer, which with the stub dispatcher means the first stub only. If that list is empty (no Initialize and nothing published) host_ = localGamers_[0] throws from inside EndCreate; the cleanup path keeps the process usable (test FailedCreateDoesNotPermanentlyStrandActiveAction). Each gamer becomes a heap LocalNetworkGamer, is stored in ownedGamers_ and added by pointer to the public local and all-gamer collections. It receives a provisional id from a monotonic counter (never derived from a live collection's size, so remove-then-add cannot collide), an IsHost flag from the factory that built the session, and the first local gamer becomes the placeholder Host. A host starts in Lobby with AllowHostMigration and AllowJoinInProgress false. GamerJoined gets a replay hook: whenever a handler subscribes, it is invoked immediately for every gamer already in AllGamers, which is why a late subscriber is caught up (GamerJoinedReplaysImmediatelyOnSubscriptionForConstructionTimeGamers, GamerJoinedReplaysForALateSubscriber). Later joins are different: AddRemoteGamer and AddLocalGamer enqueue a GamerJoin event delivered from Update. Reading the two paths together, a handler subscribed after a remote gamer has been added but before the next Update would receive that gamer once from the replay and once from the queued event; no test covers that window, so treat it as a caution rather than a proven behaviour. No extra Update call is needed to see the construction-time gamers.

For a SystemLink session the constructor ends by calling ENetBackend::StartHosting. That creates an RAII ENetHostHandle (31 peers, 2 channels, an OS-assigned ephemeral port on native hosts, a fixed port 61191 on Emscripten), reads back the bound port and registers the session with discovery before committing the state to the process map. If UDP registration throws, the half-created host unwinds through the handle's destructor and the map never learns of it (StartHostingRollsBackCleanlyOnDiscoveryRegistrationFailure). ENetLibrary::EnsureInitialized guards enet_initialize with a function-local static; no enet_deinitialize call exists. Note that every SystemLink session starts hosting in its constructor, including one created by Join; a joining session therefore owns an accepting ENet host and is registered for discovery as well, which the REMED-NET-001 comment in the backend acknowledges (“even a client role peer owns a real accepting ENetHost here”).

Join

BeginJoin copies the connect address and port out of the AvailableNetworkSession and takes the session type from it (an earlier FIXME hardcoded PlayerMatch, which would have disabled the transport). EndJoin constructs the joining session with isHost = false, MaxSupportedGamers and four private slots, then, for SystemLink with a non-empty address, calls ENetBackend::ConnectToHost (start hosting if needed, then enet_host_connect and remember the peer as HostPeer). On native builds it then pumps Update on the same thread, sleeping one millisecond per loop, until the host pointer stops being the construction-time placeholder (the server welcome arrived), the session state becomes Ended, or five seconds pass. Expiry disposes and deletes the half-joined session and throws NetworkSessionJoinException with SessionNotFound; any other exception during that phase is wrapped with its cause in the same type. On Emscripten that wait loop is compiled out, so Join returns before the handshake and the caller must pump Update itself; that path was not traced against a browser. BeginJoinInvited validates maxLocalGamers and the single-session rule, then throws GamerServicesNotAvailableException; EndJoinInvited always throws ArgumentException, so it cannot be used to slip past the refusal. NetworkSession::InviteAccepted is declared and never raised. A successful Create or Join does not start background work: everything after that depends on the caller invoking NetworkSession::Update.

Handshake and host-only messages

A client sends ClientHello (its local gamertags) when ENet reports the connect to its HostPeer. The host answers with ServerWelcome: the wire ids it assigned to the client's gamers, a snapshot of the current roster (with each gamer's real host flag) and the current NetworkSessionProperties. It then adds the new gamers as remote gamers of its own session, so GamerJoined is raised for them only after the welcome was sent, and announces them to already-connected peers with GamerJoinBroadcast. Wire ids are single bytes assigned by whichever side hosts the connection; ids reclaimed from departed peers are reused before the counter advances, which stops the byte from wrapping after 256 cumulative joins. The client's HandleServerWelcome overwrites the provisional local ids with the assigned ones, creates remote gamers for the roster, and establishes the welcomed host as Host silently (no HostChanged for an initial join). Every gamer the transport creates goes into OwnedRemoteGamers. The receive path applies these rules, each with a test: a host in Playing with join-in-progress disallowed disconnects a new ClientHello peer; a duplicate ClientHello from an already-handshaked peer disconnects it; the five host-authoritative messages (ServerWelcome, GamerJoinBroadcast, GamerLeaveBroadcast, StateChangeBroadcast, SessionPropertiesBroadcast) are accepted only from the peer this side connected out to, and otherwise the sender is logged and disconnected (HostRejectsForgedServerWelcomeFromNonHostPeer and its siblings); and a payload that fails to decode is dropped inside a try block so a single truncated packet cannot throw out of Update (HostSurvivesTruncatedClientHelloAndContinuesFunctioningAfterward). The full field layout of each message and compatibility between builds are not established by this page.

One Update and one application packet

NetworkSession::Update throws ObjectDisposedException on a disposed session. For SystemLink it calls ENetBackend::PumpSession and then ENetDiscoveryService::Poll. PumpSession services ENet with a zero timeout in a loop, dispatching connect, receive and disconnect; a received packet is wrapped in a guard that destroys it after handling. It then copies ENet's round-trip time into NetworkGamer::RoundtripTime for the gamers of directly connected peers (the host's view of its clients only; every other gamer stays at zero), releases any delayed deliveries whose release time has passed, and, on the transport host, compares the session properties with the last published snapshot and broadcasts a change (the get-only XNA property is a mutable collection, so polling is the only way to notice edits). The session then drains its NetworkEvent queue: a PacketSend whose target is a local gamer is enqueued into that gamer's packet queue with the sender remapped; a remote target goes to ENetBackend::SendAppData; gamer joins and leaves, host changes and state changes raise the public events on the thread that called Update. There is no per-session network worker, but neither the queue nor the backend's process map is safe for concurrent use, so treat Create, Update, Dispose and gameplay sends as a single-owner execution model.

A packet leaves in two steps. LocalNetworkGamer::SendData does not touch the transport: every overload only appends a PacketSend event to the session's queue (the broadcast overloads add one event per gamer in AllGamers, which includes the sender and any other local gamers; the recipient overloads add one). The bytes reach ENet during the next Update. In that loop the transport branch is gated on RealNetworkingEnabled, which is why LocalNetworkGamerTest.SendDataThenReceiveDataRoundtrip, despite its name, asserts that no data becomes available on a Local session. ReceiveData dequeues one packet per call, copies at most data.size() bytes and reports the sender by matching the packet's gamer pointer against the current gamers (FNA's own “bad equality check”, kept). The overload that takes a PacketReader returns 0 as the length while still writing the data, a preserved FNA quirk (ReceiveDataIntoPacketReaderReturnsZero).

Before a client handshake has assigned wire ids, or before a target gamer is known, SendAppData cannot address the packet. The backend queues such sends in a bounded per-session vector (64 entries; when full it evicts the oldest) and delivers them, in order for each sender and target pair, when HandleClientHello, HandleServerWelcome or HandleGamerJoinBroadcast makes both ends resolvable. Every entry that turns out never to be deliverable increments a process-wide counter exposed as GetDroppedAppDataCount: overflow eviction, purge when a named gamer leaves or disconnects, and the whole queue being cleared by host migration or by the client's own session ending. Because SendData itself only queues, this path is reached when a send is drained before its ends are wired (for example after ConnectToHost without waiting, on Emscripten where EndJoin does not wait, or for a gamer introduced later), not after a native Join that already waited for the welcome. Preserve this state transition when changing Join or SendData.

Delivery options map onto ENet flags in NetPacketCodec::SendDataOptionsToEnetFlags: None is unsequenced, InOrder is a plain sequenced unreliable send, and Reliable, ReliableInOrder and Chat are all reliable (ENet has no reliable-unordered primitive). The host is created with two channels, but every send in ENetBackend.cpp uses the default control channel 0. The topology is a star: a client sends everything to the host, which relays to the peer that owns the target's wire id and never echoes a packet back to its origin. NetPacketCodec and NetDiscoveryProtocol are two different wire contracts (session and application messages on a connected channel, versus a LAN query and announcement on a raw UDP socket, protocol version 1); a codec change needs tests with malformed and truncated bytes and a real two-process exchange, not only an in-process send.

Host migration and simulated conditions

The source contains both features; this page records only where they sit. AllowHostMigration is off by default. When a client loses its connection to the host and the flag is set, AttemptHostMigration lets every survivor compute the same successor, the lowest remaining wire id, from its cached roster. The successor promotes itself (marks its local gamers as host, re-registers for discovery, establishes itself as host with HostChanged); the others discard their roster, reset the wire-id maps and pending sends, rediscover the new host through ENetDiscoveryService::FindSessions (up to three attempts, matching by gamertag) and reconnect. There is no wire message for it (tag 0x05 is reserved and unused), it is best effort, and it returns false, ending the session, when disabled or when no reachable host is found. SimulatedLatency and SimulatedPacketLoss are applied in HandleAppData to application data addressed to one of this session's own local gamers only: session-management traffic and a host's relay hop for two other peers are untouched; a loss of exactly 0 or 1 never consults the random generator, and delayed packets wait in a per-session queue until PumpSession observes the release time. Tests exist for all of it in ENetBackendTests.cpp (the tie-break cases run in one process; HostMigrationPromotesOneSurvivorAndTheOtherReconnectsAcrossRealProcesses uses three). Real multi-machine migration behaviour, the statistical behaviour of loss, and every Emscripten path remain unverified by this page.

Discovery is a synchronous, host-specific service

ENetDiscoveryService holds file-static state: one UDP socket, a single registeredHost_ pointer with its connect port, and a temporary result vector that exists only while FindSessions runs. On native builds the socket is bound to port 61190 with SO_REUSEADDR, broadcast and non-blocking options, so several processes on one machine can each bind it; which of them receives a given datagram is up to the operating system, and that is why results are de-duplicated by connect port. FindSessions returns an empty list at once for any type but SystemLink; otherwise it sends the query by broadcast and by loopback, then polls on the calling thread for a 150 ms window, so a single NetworkSession::Find blocks for roughly that long (the work happens in EndFind). Each reply becomes an AvailableNetworkSession whose quality of service carries the measured delay between the query and that reply. ReplyToQuery answers only when the querying filter equals the registered host's session type, and announces the host gamertag (taken from the underlying SignedInGamer, because LocalNetworkGamer::getGamertagProperty always reports the FNA default Stub Gamer), the connect port and slot counts; private slots are reported open because nothing tracks occupancy. NetworkSession::Update calls Poll so a hosting session keeps answering.

Registration and unregistration are explicit, and an unregistered host must not stay advertised. Because the state holds one host pointer, a process that hosted more than one session would advertise only the last registered one; the single-session rule keeps that from arising today. Malformed datagrams are dropped, and a guard object resets the result pointer even when a decode throws, so no dangling result sink survives (MalformedAnnounceDuringSearchIsIgnoredAndDoesNotLeaveADanglingResultsPointer). Emscripten has no raw UDP: the same functions are empty stubs and FindSessions returns nothing, so a browser build compiling proves nothing about LAN discovery. Windows behaviour of the shared-port bind is not established by the Linux two-process tests; the source comments say so themselves.

Teardown order and error boundaries

NetworkSession::Dispose is idempotent and runs in a fixed order. It clears each local gamer's receive queue; for SystemLink it calls ENetBackend::TeardownSession, which sends a disconnect to every known peer (and to the host peer), flushes so the packets actually leave, erases the per-session state (destroying the ENetHostHandle and freeing OwnedRemoteGamers) and unregisters discovery; it then destroys ownedGamers_, clears all four public gamer collections, nulls the host and the static active-session pointer and marks the session disposed. The destructor falls back to Dispose when a caller used delete directly; without that, the process-static session pointer would dangle and, because every later Begin* refuses while it is set, one mismanaged delete would block session creation for the rest of the process (DeletingWithoutDisposeStillAllowsCreatingANewSession, DeletingWithoutDisposeTearsDownRealTransport). Reordering gamer destruction before transport teardown would leave the wire-id maps holding freed gamer pointers. The early return and the clearing of the views are both load-bearing: repeating Dispose once exposed a use-after-free, and the views can otherwise keep pointers into just-freed gamers (DisposeCalledTwiceDirectlyIsSafeAndIdempotent, DisposeClearsPreviousGamersSoNoDanglingPointerIsObservable).

Errors surface as typed XNA exceptions at the session layer (argument validation, InvalidOperationException for state, ObjectDisposedException), as NetworkSessionJoinException for a failed join, and as std::runtime_error from ENet initialisation, host creation, unresolvable addresses and UDP socket creation. ENetHostHandle::Send returns void and silently drops a packet whose allocation or enet_peer_send fails, so not every transport failure can be inferred from the event stream. A departed gamer is moved to PreviousGamers (capped at MaxPreviousGamers, 100, oldest evicted) and the departure raises GamerLeft, or a session-wide SessionEnded when the departing gamer is local. A human debugging “packet never arrives” should check, in order: the type is SystemLink; Update runs on both processes; the handshake finished and wire ids exist; the pre-handshake queue and drop counter; codec decode; local-versus-remote target routing; the receive queue. There is no dedicated receiving thread to blame.

Change-driven evidence and source route

CnaNetTests is the focused target when networking is enabled. It is an EXCLUDE_FROM_ALL executable built from the module's 18 test files (316 test-macro definitions by a line-anchored count; not executed for this page) and linked against CNA_Net; the same files feed CnaTests. cmake --preset unit configures with networking off and therefore has neither. The suites:

• NetworkSessionTests (58 macros): construction, state, factories, events, replay hook, dispose and delete-without-dispose, disposal idempotency, no action leak, and callback re-entrancy. NetworkSessionTypePolicyTests (11): the synthetic types never bind a port and the matchmaking types are refused. LocalNetworkGamerTests, NetworkGamerMachineTests, AvailableNetworkSessionTests, NetworkSessionPropertiesTests, PacketReaderWriterTests and the enum, event-argument and exception suites cover the value types.
• ENetBackendTests.cpp (46): handshake, delivery and relay, the pre-handshake queue and every drop case, gamer leave and session end, migration bookkeeping, state and property broadcasts, forged-message rejection, wire-id reuse, prompt disconnect on dispose, ownership of remote gamers, measured round-trip time, and the latency and loss cases. It shares one session per process, so it uses a raw ENet host to stand in for the second machine.
• ENetDiscoveryServiceTests (8), ENetHostHandleTests (8), ENetLibraryTests (2), NetPacketCodecTests (25) and NetDiscoveryProtocolTests (10) cover discovery, the RAII handle, initialisation and both wire formats.
• TwoProcessLoopbackTest.cpp (3): genuine separate OS processes running net_two_process_harness.cpp as host, client, migration host and migration survivor, plus the discovery-failure rollback. GamerServicesDispatcherHangRegressionTest (4) spawns the dispatcher harness described on the GamerServices page.

The process-spawning tests use POSIX posix_spawn, so UnitTests.cmake filters their sources out of CnaTests on Windows, Emscripten, Android and iOS, and Harnesses.cmake does not build the two-process harness on the first three. On those hosts the cases do not exist; they are not skipped at run time and must not be counted as passing.

Recipes. For a lifetime change, repeat Create and Dispose under AddressSanitizer in addition to the transport tests. For a wire change, test peers built from the same revision and, if compatibility is claimed, the previous one, with malformed and truncated bytes. For a discovery change, test real multi-process bind, announce and find, and the firewall and socket behaviour of the native host. For a pre-handshake or migration change, extend the existing drop-counter cases rather than adding timing-dependent ones; the tests inject a clock and a seeded generator through SetClockForTesting and SeedPacketLossRngForTesting.

Source-reading route

1. NetworkSession.cpp: read the constructor, Dispose, Update, Create/EndCreate and Join/EndJoin first, and draw the raw and owned pointers and the event queue.
2. ENetBackend.cpp with ENetHostHandle.cpp and ENetLibrary.cpp: trace the session map, handshake, packet queue, peer destruction and RAII native ownership.
3. ENetDiscoveryService.cpp: map the process-static socket, host and result state and the host-specific branches before editing Find.
4. LocalNetworkGamer.cpp, NetPacketCodec.cpp and NetDiscoveryProtocol.cpp: the send and receive queues and the two wire contracts.
5. ENetBackendTests.cpp and TwoProcessLoopbackTest.cpp: separate in-process state-machine evidence from genuine OS-process transport evidence.

What this page does not establish

This tour reconstructs ownership, the pump, discovery, join and teardown. It does not fully reconstruct host migration under real network conditions, the scheduling behaviour of simulated loss and latency, every field of every packet, compatibility between protocol versions, or the Emscripten relay and client-only host paths; those need separate source and test audits before the module can be declared complete. Nothing here was executed. The module list is in the module index, ownership across the tree in the ownership map, and the Maintainer Handbook has the change routes. Evidence level: everything above was checked by reading the TARGET sources at 009d40f5.