Tutorial 168: Operating the Gamer Services server

CNA Tutorials  ·  CNA snapshot 3b43bd86

ℹ

What you’ll learn: how to run the server with TLS, provision titles, accounts and content, back it up, upgrade and monitor it, what its limits are, and what it does not protect against.

⚠

Read this first. cna-gamer-services-server is CNA’s own service, not Xbox LIVE. No public instance is operated, and public-Internet deployment has not been independently qualified: running it for other people is your decision and your responsibility. Every command below is quoted from or built on the server’s README, HANDOFF.md and source at commit 4ab12fd; none was run for this page. Connecting a game is Tutorial 164.

What you are running

One C++23 process (MIT licence) serves the whole CNA Gamer Services protocol on one TLS port: the control API (POST /cna/v1), downloads of hash-addressed files (GET /cna/v1/files/<sha256>), the account event channel (WebSocket /cna/v1/events) and the relay that carries online NetworkSession traffic (WebSocket /cna/relay/v1). It keeps everything in one SQLite database in WAL mode, with one writer and in-memory hubs for relay and event channels. Its dependencies come from the system: OpenSSL 3 or newer, Boost 1.74 or newer with Beast, SQLite 3.38 or newer and nlohmann/json 3.11 or newer. Linux is the qualified host. On macOS, CNA’s Apple campaign built it with Homebrew’s Boost on one Apple M4 and ran its suite (37 of 46 tests passed, 9 were skipped for prerequisites macOS lacks — Linux network namespaces and extra loopback addresses — and none failed) after small portability fixes that the server has since merged (they are in its v0.1.0 release and on develop); the server’s own README still lists macOS as unvalidated. Native Windows has not been tested (the Windows database-lock code ran under Wine only).

Not provided, by design: Xbox LIVE compatibility of any kind; self-registration or password changes by players (the admin tool creates accounts and revokes credentials, but has no command to change a password); a store, partner tokens or title-update delivery; TrueSkill; time windows for the ...Recent leaderboard keys; direct peer-to-peer connections; any cluster or multi-server operation.

1. Build and run

cmake --preset dev && cmake --build --preset dev     # needs ccache; at most six compile jobs
build-agent/cna-gamer-services-server --database service.sqlite3 --listen 0.0.0.0 --port 47831 \
    --cert certificate.pem --key private-key.pem --diagnostics-port 47832 --log-format json
# Development only, plain HTTP on a numeric loopback address:
build-agent/cna-gamer-services-server --database service.sqlite3 --insecure-loopback

--help lists every option and --version identifies the binary; tools/qualify.sh quick is the shortest supported build-and-test command. The defaults are --listen 127.0.0.1 and --port 47831. A second server process on the same database refuses to start with DATABASE_IN_USE: the first holds an operating-system lock on <database>.lock, released when it ends, crash included. SIGINT and SIGTERM stop it.

The README’s checklist for a public deployment asks for an unprivileged account that owns only the database directory and the TLS key. The repository ships deployment templates in tools/cna-gamer-services-deploy/: a hardened systemd unit, an unprivileged container image with a Compose file (read-only root file system, the certificate and key as secrets, a health check on /readyz) and an nginx stream example that passes TCP through unchanged, so that the server still terminates TLS. An excerpt of the systemd unit:

[Service]
User=cna-gamer-services
Group=cna-gamer-services
StateDirectory=cna-gamer-services
LoadCredential=certificate.pem:/etc/cna-gamer-services/certificate.pem
LoadCredential=private-key.pem:/etc/cna-gamer-services/private-key.pem
ExecStart=/usr/local/bin/cna-gamer-services-server --database /var/lib/cna-gamer-services/service.sqlite3 --listen 0.0.0.0 --port 47831 --cert %d/certificate.pem --key %d/private-key.pem --diagnostics-listen 127.0.0.1 --diagnostics-port 47832 --log-format json
Restart=on-failure
NoNewPrivileges=yes
ProtectSystem=strict

They are templates: review every path, add your certificate automation and qualify your own staging host. None of them has been installed on a remote production host, the container image was not built during the server’s own qualification, and the nginx example has not been exercised on a public network.

2. TLS, keys and secrets

  • Certificate. Clients require TLS 1.2 or later and verify the chain and the host name, against the system trust store or the CA bundle a title ships (caBundle). Use a public certificate for the DNS name your games use, with its intermediates in the chain file. A renewed certificate needs a restart; clients and relays reconnect by themselves, and live sessions resume their relay connections.
  • One port, no TLS-terminating proxy. Control, relay and events share the TLS port, and the server terminates TLS itself. Put an L4 (TCP) filter in front, not a proxy that terminates TLS, which would also make every player appear to come from the proxy’s address and share its per-address limits.
  • Secrets on disk. The database (with its -wal and -shm files) and the TLS key (mode 0600) belong to the service account only. The server stores scrypt password verifiers and only hashes of access tokens; access tokens last an hour and refresh credentials rotate over 30 days, with a replayed refresh credential revoking its whole family.
  • Logs. The admin tool never prints a credential. By default the server prints one human-readable statistics line a minute; --log-format json emits bounded structured events instead. Neither form contains passwords, tokens, relay tickets, request bodies or client addresses.

3. Titles, accounts and content

build-agent/cna-gamer-services-admin <database> <command> works on the same database as a running server. Passwords and JSON documents are read from standard input:

DB=service.sqlite3
build-agent/cna-gamer-services-admin $DB title my.game "My Game"
build-agent/cna-gamer-services-admin $DB title-minimum-version my.game 1.0.0   # older or unversioned games must update

read -rs PASSWORD   # 8 to 256 bytes; keep shell tracing off
printf '%s\n' "$PASSWORD" | build-agent/cna-gamer-services-admin $DB user alice Alice
unset PASSWORD

printf '%s' '{"key":"first-steps","name":"First Steps","description":"Finish the tutorial.","howToEarn":"Complete the first level.","score":10}' \
  | build-agent/cna-gamer-services-admin $DB achievement my.game
printf '%s' '{"key":"BestScoreLifeTime","mode":0,"ascending":false,"aggregation":"best","arbitrated":false,"columns":{}}' \
  | build-agent/cna-gamer-services-admin $DB leaderboard my.game

# avatars: import the catalog CNA ships (v3, the baseline), then give an account one
build-agent/cna-gamer-services-admin $DB avatar-catalog "$CNA/modules/gamer-services/assets/avatars/v3"
build-agent/cna-gamer-services-admin $DB avatar alice random

Here $CNA is a CNA checkout whose avatar submodule is initialised (git submodule update --init: the catalogs are the cna-avatar-assets repository, checked out at modules/gamer-services/assets/avatars). CNA ships only catalog v3; the development catalogs 1 and 2 left CNA on 15 October 2026, and a later catalog version would be imported after v3. Other commands, all in the README: asset <title> image/png <file> imports a PNG (at most 512×512, 512 KiB) and prints its hash — pass that hash to picture <user> <hash> for a gamer picture, or as an achievement’s picture field; privilege <user> <name> <value> sets an XNA privilege (communication, profileViewing or userContent to everyone, friends or blocked; trade, purchase or premium to allowed or blocked) — the operator’s stand-in for parental controls; game-defaults <user> sets an account’s XNA GameDefaults; revoke-user revokes every credential of an account; inspect and inspect-online <title> print counts only; reset-earned and reset-online clear a title’s earned achievements or its online sessions. A title may have up to 128 leaderboards, each with up to 32 typed columns; the leaderboard key and game mode must match what the game passes to LeaderboardIdentity::Create.

For browser administration, tools/cna-gamer-services-admin-web/ runs a separate web UI on loopback (port 47833 by default), signed in with a password file that only its owner can read. Every change is protected by a per-session CSRF token and carried out by the admin tool, --read-only gives an inspection-only session, and binding it to another address requires --allow-remote with a TLS certificate and key; an SSH tunnel to the loopback listener is the recommended remote access.

4. Backups, updates and monitoring

  • Backups. Use tools/cna-gamer-services-deploy/backup.py while the server runs: it uses SQLite’s online backup API, checks integrity and foreign keys, and publishes a mode-0600 file atomically; it deletes nothing unless --retain is given. Restore with restore.py after stopping the server: it validates the backup and refuses to overwrite an existing database. Never copy the live database file. Keep encrypted copies in another failure domain, schedule backups and try a restore once (the repository’s docs/disaster-recovery.md is the procedure).
  • Durability. Everything a player would notice losing — credentials and revocations, awards, scores, friends, messages, profile and avatar edits — commits with synchronous=FULL; leases, presence and request bookkeeping commit with NORMAL, so a power cut may roll back their last moments.
  • Updating. Opening a database migrates it, in a transaction, to the schema of the server binary; migrations only go forward, and a database from a newer server is refused rather than altered. Take a backup before every upgrade.
  • Monitoring. With --diagnostics-port 47832 the server opens a loopback-only HTTP listener (a non-loopback address is refused): /healthz for liveness, /readyz, ready only when SQLite has foreign keys enabled and exactly the schema this binary expects, and /metrics in Prometheus text format — build and schema, database and table counts, connections and admission refusals, operation and outcome counters and a fixed latency histogram, never labelled by player, address, title or session. The per-minute line still counts responses by code, refused connections, open control connections, attached relay machines and open event channels. Alert on readiness below 1, on INTERNAL_ERROR (storage failed: a full disk, a locked database), on refusals rising, on bursts of RATE_LIMITED and on latency moving against your own baseline.

5. Limits and capacity

LimitValue
Control connections (checked before the TLS handshake)256 in all; 32 held and 600 new a minute per source address; a handshake must finish within 10 s
Sign-ins and refreshes10 a minute per address; an account’s oldest sign-in is signed out past 32 live ones
Relays and event channels1,024 relay machines; 8 event channels per account, 4,096 per server
Recorded requests20,000 a day per account and title (kept in memory, reset by a restart)
Social200 messages an hour; 64 pending invitations per recipient, 32 sent an hour; 1,024 live sessions per title

A game that meets a limit sees the service as unavailable for that call, through XNA’s usual exceptions. Players behind one NAT address share its 32 connections and 10 sign-ins a minute. The README’s committed benchmark measured about 1,200 requests a second (steady mix, p99 118 ms) on one loaded laptop over loopback, while an idle player costs about two requests a minute; it is one machine’s figure at one moment, not a capacity promise. The repository’s load lab (at most six workers; ordinary, sign-in, refresh, session-churn, directory, reconnect, host-migration, event, relay and soak profiles) produces loopback evidence for comparing revisions, not a public-Internet capacity figure.

6. Trust boundaries

⚠

Achievements and scores are authenticated, validated client claims. The server checks the account, the title, provisioned IDs, schemas, ranges and ownership, but it cannot see gameplay: a modified or colluding client can submit plausible fake results, and there is no anti-cheat. Ranked rows count when a strict majority of the reporting machines agree, which colluding machines can satisfy. Do not run competitive events with prizes on these numbers.

  • Hostile traffic. The limits above stop a single address. Nothing limits many addresses together — there is no server-wide handshake rate — so a distributed flood can fill the connection pool or burn CPU. A firewall, connection-rate limiting or a DDoS filter in front is required; the server does not claim DDoS protection.
  • Relay and voice. Relay access needs one-use, short-lived tickets bound to the members, title, session and machine, and the server stamps the sender’s identity on every relayed frame. It does not inspect game data or voice: the session host is trusted to relay game data, and privacy rules for voice are applied by the cooperating CNA clients, with blocks and privileges refreshed on the 30-second heartbeat.
  • Accounts. The operator creates every account and sets its privileges; there is no self-service. Treat the database and its backups as containing personal data.

7. Testing your build

tools/qualify.sh quick builds with ccache and runs the 19 C++ unit and component suites; normal runs every configured test, security an AddressSanitizer/UndefinedBehaviorSanitizer build, performance the Release benchmark smoke, and full adds the documentation checks. The configured set includes the TLS end-to-end and WebSocket relay tests (Python websockets, pinned in tests/requirements.txt), a black-box conformance suite, backup and restore, diagnostics, process-failure (chaos), load-lab and admin-web checks. The 17 tests that drive real CNA processes need CNA’s harness programs from a CNA build, named by CNA_SERVICE_*_HARNESS variables as the README lists; without them they skip (exit 77), which is not a pass. The NAT-isolated variants additionally need unprivileged user namespaces and a slirp4netns binary, which is executed, never linked, so they run on Linux only.

Evidence and qualification

  • CNA’s final audit run of this work (2 October 2026, Linux, server commit 9bd5173 with CNA’s client harnesses): the GamerServices suite 649 passed and the Net suite 524 passed, with no failures; the server’s 40 registered tests: 33 passed, 7 that need Linux network namespaces skipped for lack of a test tool (slirp4netns), none failed. This is the latest Linux run in which the server’s tests drove real CNA processes.
  • The server’s handoff qualification (7 October 2026, Linux loopback, commit 4ab12fd, without a CNA build): 46 configured tests, 29 passed, the 17 that drive real CNA processes skipped (a skip, not a pass), none failed; AddressSanitizer/UndefinedBehaviorSanitizer 19/19 and a focused ThreadSanitizer run 3/3; the Release benchmark smoke completed.
  • Independent conformance: tools/cna-gamer-services-conformance/ drives the real TLS and WebSocket listener through public interfaces only — 20 smoke cases in the test suite plus one deliberate wall-clock relay-ticket-expiry case in the default profile — and checks that all 59 public operations are exercised. It is loopback evidence, not a LAN, NAT or Internet result. The process-failure suite (tools/cna-gamer-services-chaos/, nine cases on scratch data) covers SIGTERM and SIGKILL recovery, ticket refusal, a locked and a full database and hostile connections; it does not prove behaviour after an operating-system crash or a power loss.
  • macOS, recorded by CNA’s early-October Apple campaign on one Apple M4 (before this snapshot; not repeated), on the server’s Apple branch whose fixes are now in its v0.1.0 release: with CNA’s harnesses built against Homebrew curl 8.22, 37 of 46 tests passed, 9 were skipped (seven need Linux network namespaces, two need extra 127.x loopback addresses that macOS does not configure) and none failed. Against Apple’s system libcurl 8.7.1, which has no WebSocket support, the CNA side refuses the relay and the ten tests that drive CNA processes fail. These are recorded runs; this site did not re-run them.
  • Earlier NAT-isolation runs used separate network namespaces on one Linux host: evidence about isolation, not about public-Internet latency, loss or failover. CNA’s manual qualification plan for real networks and devices has not been executed.
  • Not qualified: Windows server hosts and clients (the database lock and the Windows credential store were exercised under Wine only), a real wide-area network with NAT, voice on physical audio devices, and avatar pixels on native Windows drivers and on SDL_GPU’s Metal driver; the deployment templates have not been run on a production host.
  • Browser builds have no service, relay or voice, so browser players cannot use a server at all.

Next

This is the last tutorial of the Gamer Services series. Start again from Tutorial 162, go back to Tutorial 167 for avatars, or read the Gamer Services overview.