Inspector transport internals

CNA snapshot 009d40f5  ·  Development › Module internals  ·  source links pinned to 009d40f5

modules/inspector/ is not part of the Game loop or of the CNA umbrella. It is three things a game opts into explicitly: an application-side agent (CNA::Inspector::Agent) that answers an authenticated, versioned binary protocol on one background thread; a synchronous protocol client; and a separate cna-inspector process that turns that protocol into a loopback-only HTTP API and an embedded browser UI. This page is for maintainers who change the agent, the wire format, the bridge or the preview seam, or who have to explain why the Inspector shows nothing. The user guide is Inspector; the data it transports comes from Diagnostics and profiler internals.

Opt-in composition and host scope

modules/inspector/CMakeLists.txt is always entered by modules/CMakeLists.txt (right after diagnostics), but it defines nothing unless CNA_BUILD_INSPECTOR is ON; the option is declared in the root CMakeLists.txt with default OFF, and no CMake preset or CI workflow at this snapshot turns it on. With the option ON it creates:

The option is a configure-time FATAL_ERROR on Emscripten, Android and iOS; desktop Windows, Linux, macOS and other POSIX desktops are the declared scope. cna_inspector is absent from the list of runtime parts that make up the CNA interface target in modules/CMakeLists.txt, so a game gets Inspector code only by naming CNA::Inspector on its own link line, and the agent exists only after that game calls Agent::Start. There is no static initializer, environment hook or Game integration. The session identity it reports uses the project-wide CNA_PLATFORM_<NAME> definition from PlatformSelection.cmake and NDEBUG (Release/Debug) unless the configuration overrides them.

game process (explicitly links CNA::Inspector)
  Diagnostics process provider (v1)      <- pulled only on request
  Agent::Start(configuration, error)     -> one worker thread, one TCP listener
        ^  binary protocol v1.0, 24-byte header, token in ClientHello
        |  plaintext TCP, loopback unless allowRemote=true
cna-inspector process
  Client (one connection, serialized by a mutex)
  WebBridge: HTTP/1.1 on 127.0.0.1 only, one detached thread per connection (max 32)
        ^  x-cna-inspector-ui-token header on /api/*, exact Host header
browser: embedded HTML/CSS/JS; never receives the agent token

This boundary is the first thing to check when “the Inspector does not appear”: CNA_DIAGNOSTICS=FULL alone starts no agent, CNA_BUILD_INSPECTOR=ON alone links nothing into a game, and a game that links the module but never calls Agent::Start has no thread and no listener. Architecture: build-time pieces outside the runtime closure places the module among the optional development pieces.

Agent ownership, negotiation and shutdown

Start validates everything before a socket exists

Agent.hpp offers two factories. Agent::Start(configuration, error) forwards to the second with Diagnostics::GetProvider(), a function-static provider over diagnostics state that is deliberately allocated once and never freed, so the borrowed reference cannot dangle. Agent::Start(provider, configuration, error) takes any version-1 IDiagnosticsProvider&, which must outlive the agent; the tests use it with a fake. Agent::Impl::Start in Agent.cpp then runs, in order:

1. rejects an empty bindAddress, and a non-loopback one unless allowRemote is true. “Loopback” is a literal string test in InternalSocket.cpp (IsLoopbackAddress): exactly 127.0.0.1, ::1 or localhost; any other spelling, 127.0.0.2 included, counts as remote;
2. rejects more than 64 metadata entries and a token longer than 256 bytes, then generates a missing token: 32 bytes from BCryptGenRandom or /dev/urandom, hex-encoded to 64 characters;
3. clamps maximumRequestsPerSecond to 1–1024, maximumPendingPreviews to 1–16 and previewCooldownMilliseconds to 50–60,000;
4. encodes a throw-away HelloRequest and HelloResponse, so an application name over 4,096 bytes, a metadata key over 128 or value over 1,024 bytes, or invalid UTF-8 fails here as “invalid Inspector identity configuration” rather than later on the wire;
5. creates the listener (SO_REUSEADDR, listen backlog 4, ephemeral port when port is 0) and only then starts the worker std::thread. A thread-creation failure closes the listener again.

Every failure returns nullptr with error set; nothing is left running. The returned std::unique_ptr<Agent> owns the implementation, the listener socket and the worker. It also co-owns the optional previewProvider through its std::shared_ptr, but not anything that provider refers to.

Stop, destruction and the order relative to the Game

Stop is idempotent and noexcept: it clears running_, sets stopping_, shuts down the active client socket under socketMutex_ so that a blocked receive returns, joins the worker, and closes the listener. The destructor calls it. Three consequences matter to a maintainer:

• The idle worker waits in AcceptTcp with a 200 ms timeout and re-checks stopping_, so stopping an idle agent takes at most about that long. A provider call already in progress is not interrupted: join waits for it to return.
• There is a narrow window in which the worker has accepted a connection but not yet published it as activeClient_; a Stop landing there shuts down nothing, and the join then waits for that connection's handshake reads to finish or time out (3 s each for header and payload).
• In the demo the agent is created in main before the Game and destroyed after it, so it keeps serving requests while the game tears down. That is harmless for the process provider, whose state is never freed, but a future preview provider that touches a GraphicsDevice needs the opposite order or its own teardown guard, because the worker may call PollPreview until Stop returns.

The ownership and lifetime master map records the agent as an application-owned object outside the Game's service graph.

Handshake: authenticate first, then look at the provider

The worker serves one connection at a time. The first packet is read with a 3 s timeout, and the checks run in this order: a header that fails ProtocolCodec::DecodeHeader ends the connection silently (no Error packet); an envelope whose major version is not 1 or whose minor version is above 0 gets UnsupportedVersion; anything other than a decodable ClientHello gets InvalidRequest; a hello whose [minimumMajorVersion, maximumMajorVersion] range excludes 1 gets UnsupportedVersion; a token mismatch gets Unauthorized. The token comparison (TokensEqual) walks the longer of the two strings and ORs byte differences, so it does not stop at the first mismatch. Each rejection closes the connection.

Only after authentication does the agent call provider_.CaptureSnapshot(), once, to derive capabilities. Malformed, wrong-version and unauthenticated clients therefore cause no diagnostics collection at all, which two tests pin down (RejectsBadAuthenticationWithoutProviderAccess, InvalidIdentityStringsFailBeforeProviderAccess).

The result is ANDed with the client's requestedCapabilities (all bits by default) and enforced per request afterwards. Note that the derivation reads buildMode, the compile-time CNA_DIAGNOSTICS level, not runtimeMode: a FULL build whose runtime mode was lowered still advertises Events, and the runtime mode is visible only inside each snapshot. The ServerHello also carries the three wire limits, localOnly, the application name and metadata, and the identity strings described under the failure model.

Threads, locks and what the Game thread can feel

No Game, render or instrumentation thread ever calls into the agent, and the agent never touches a GPU object. The coupling is lock contention inside the diagnostics module: the process provider's CaptureSnapshot takes the metric, frame, resource, thread and history mutexes (in FULL it first drains every thread's producer ring into the history under threadMutex and historyMutex), and the frame-completion path on the Game thread takes the frame and metric mutexes. A large snapshot copied on the agent thread can therefore delay a frame end by the length of the copy, but a slow browser or a stalled socket cannot, because network backpressure stops at the agent thread. The diagnostics side of this is described in Diagnostics internals: ownership and concurrency of records; the process-wide map is the thread and callback map.

Demand-driven data path and bounded work

After the ServerHello, the worker loops on ReceivePacket with a 30 s socket timeout, so an authenticated client that goes silent for 30 s is dropped. Each packet's envelope version is re-checked (a change ends the session), then a fixed one-second window counts requests: once maximumRequestsPerSecond is exceeded (default 64) the request is answered with Busy and skipped, but the connection stays open. The window is reset, not sliding, and the Busy reply itself is still sent, so the limit bounds provider work rather than network traffic. RequestRateBackpressureIsBounded checks exactly this with a limit of 2.

Selecting parts therefore reduces bytes on the wire, not the in-process copy: even a metrics-only request copies the whole resource registry inside the provider, as the user guide's performance notes say. Nothing is pushed: an agent with a connected but idle client, or with no client, makes no provider calls. EnabledWithoutClientDoesNotPollProvider observes 75 ms after start; the guarantee itself is structural, because Run calls the provider only inside HandleClient. A capture is a point-in-time copy, not a recording, and a client must advance its event cursor and read the loss fields.

Where the bounds live and what exceeding them does

All limits are constants in Protocol.hpp and the anonymous namespace of Protocol.cpp: payload 8 MiB; 4,096 events per response; 4 MiB of encoded preview data; strings 4,096 bytes unless a field says otherwise (client name 128, token 256, metadata key 128 and value 1,024, error and preview messages 1,024, MIME type 64); 512 metrics; frames up to Diagnostics::FrameHistoryCapacity (240) with at most 64 metrics each; 65,536 resource records; 64 metadata entries; preview dimensions 1–4,096. The encoder enforces the same bounds by throwing std::length_error (or std::invalid_argument for invalid UTF-8), and HandleRequest maps a length_error to LimitExceeded and anything else to InternalError, keeping the connection open.

Resource previews: an extension seam, not GPU access

IResourcePreviewProvider has two methods, RequestPreview and PollPreview, whose documented contract is to start or poll asynchronous readback and return promptly, never waiting for a device or queue. The agent cannot enforce that; it only calls them on its worker thread. No renderer or other module implements the interface at this snapshot (only the benchmark does), so real sessions never negotiate ResourcePreviews. HandlePreviewRequest checks, in this order: capability and provider present (else Unavailable); request decodes, with resource id non-zero, dimensions 1–4,096 and byte limit 1–4 MiB (else InvalidRequest); fewer than maximumPendingPreviews tickets outstanding (else Busy); cooldown since the previous accepted request elapsed (else Busy); the id is present in a fresh snapshot and is a Texture2D, TextureCube, RenderTarget2D or RenderTargetCube (else Unavailable). Only then is the provider called and its answer validated: a pending result needs a non-zero ticket, a non-ready result may carry no image fields, and a ready result needs non-zero dimensions within the request, 1 byte to the request's byte limit, and a PNG, JPEG or WebP MIME type. A ticket the provider reuses is an InternalError.

Pending tickets are kept in pendingPreviews_ together with the original request, and every poll is validated against those original bounds rather than the defaults (PreviewPollHonorsTheOriginalRequestBounds). A ticket that ends (ready, failed, unavailable or invalid) is erased. When a client disconnects, the whole map is cleared without telling the provider; the interface has no cancel call, so a provider must bound its own outstanding readbacks. On the client side, Client::RequestPreview turns an Unavailable error into a normal response with PreviewStatus::Unavailable, which is why OptionalPreviewUnavailableIsANormalResponse expects success. Writing a real provider is a renderer and threading project: resource lifetime, render-thread or queue affinity and device teardown all have to hold while the network worker calls in. A synchronous readback in RequestPreview would put GPU latency on the agent thread and, through the diagnostics locks, near the frame loop.

Wire envelope and compatibility rules

Every packet is a 24-byte little-endian header — magic 0x49414E43 (“CNAI”), major, minor, message type, flags, payload length, request id — followed by the payload. DecodeHeader rejects a wrong magic, an unknown message type, non-zero flags and a payload length above 8 MiB; the receive path allocates the payload buffer from that length before reading, so even an unauthenticated peer can make the agent allocate at most 8 MiB. Payload decoders reject truncation, trailing bytes, out-of-range enumerations, Booleans other than 0 or 1, over-long strings and collections, and invalid UTF-8 (overlong forms, surrogates and code points above U+10FFFF).

The version rule is effectively “exactly 1.0 at both ends”. The agent refuses any envelope with a minor version above 0; Client.cpp refuses any response envelope with a minor above 0 or a mismatched request id, and refuses a session whose providerInterfaceVersion differs from IDiagnosticsProvider::InterfaceVersion. A new message type is not a compatible addition either: an older peer's DecodeHeader treats the unknown type as a malformed header and drops the connection. Any change to a message layout, a bound or a type therefore has to move the agent, the client and the bridge together, and a new JSON field in the bridge does not by itself change the binary contract.

Separate browser bridge and its trust boundary

InspectorMain.cpp parses --agent-host (default 127.0.0.1), --agent-port (required, non-zero), --http-port (0 means ephemeral), --token, --token-file and --allow-remote-agent. The token precedence is: a token file (at most 256 bytes, trailing CR/LF stripped, empty refused) overrides --token, and CNA_INSPECTOR_TOKEN is read only when neither gave one. A non-loopback agent host, by the same literal test as the agent, needs --allow-remote-agent. The URL line is flushed explicitly because Run never returns.

WebBridge.cpp binds its HTTP listener to 127.0.0.1 unconditionally; there is no option to change that, whatever the agent does. Beyond the size and time limits the user guide lists, the parser accepts only an HTTP/1.1 request line with a target of at most 2,048 bytes, refuses any request with a non-zero Content-Length (preview ids travel in the query string), and compares the Host header with 127.0.0.1:<bound port> byte for byte, so a page opened as localhost:<port> gets 400 “invalid Inspector Host header”. That exact-Host rule is the DNS-rebinding defence. Every response is sent with Connection: close, so each request is its own TCP connection.

An /api/ request without the exact x-cna-inspector-ui-token value is answered 404 “not found”, not 401: a browser tab kept from an earlier bridge run (with a fixed --http-port) has a stale token and sees only 404s until it is reloaded. API handlers take clientMutex_, so however many tabs or connections exist, the agent sees one request at a time; static assets do not take the lock. EnsureConnected reconnects lazily on the next API call after the client dropped its connection; a client failure becomes HTTP 503 with the message, which the UI's 500 ms poll loop shows as “reconnecting” (it then refetches /api/session).

Connections are counted under connectionMutex_; the 33rd concurrent one receives a 503 and is closed rather than queued, and each accepted one is served on a detached thread that decrements the count when done. WebBridge::Impl::~Impl waits on a condition variable until that count reaches zero before closing the listener, because the detached threads reference the object. In the shipped cna-inspector, however, Run is an endless accept loop and InspectorMain.cpp installs no signal handler, so the process ends by termination (Ctrl+C) and that destructor path is not reached; it matters only to code that embeds WebBridge differently.

The trust boundary is therefore “processes on this machine that can open the loopback port”: the UI token is in the page served at /, which needs no token. The agent token never leaves the bridge process. See the Inspector security model for the user-facing summary; enabling a remote agent sends that token over plaintext TCP, and remote deployment behaviour is not established by anything on this page.

Failure model and human debugging route

Agent::Start reports configuration, token-generation, identity-encoding, bind/listen and thread-creation failures as nullptr plus an error string. After that, nothing in the module mutates the Game: protocol problems become one of the seven ErrorCode values (InvalidRequest, UnsupportedVersion, Unauthorized, Unavailable, LimitExceeded, Busy, InternalError), every handler is wrapped so a provider exception becomes InternalError, and transport errors end one connection and return the worker to accept. A blank Inspector view has several possible layers, and they are worth testing in this order before touching rendering code:

1. Build data. Was the game compiled with CNA_DIAGNOSTICS=STATS or FULL (Diagnostics modes)? With OFF the session negotiates but snapshots are nearly empty; events and CPU zones need FULL.
2. Agent. Is CNA::Inspector linked, did Agent::Start return non-null, and is the returned pointer still alive? A temporary that is destroyed immediately stops the agent.
3. Handshake. Same port, same token (regenerated on every start unless configured), protocol 1.0 on both sides. Only one client is served at a time: a second bridge connects at TCP level but, with the client's default 3 s timeout, its Connect fails with a receive timeout while the first session lasts.
4. Capability. Does /api/session show the capability the view needs? Resources need STATS, events and zones FULL, previews an installed provider.
5. Provider or wire limits. A persistent LimitExceeded or InternalError on snapshots points at the provider or at an oversized name or label (see the bounds note).
6. Bridge. 400 on every request means the Host header (use 127.0.0.1, not localhost); 404 on /api/ means a missing or stale UI token; 503 carries the agent-side message.
7. Actual loss. Only after all of the above, read producerEventsDropped, eventsDroppedBeforeStart and the history-overwrite counters.

The general tools for the process side (debuggers, sanitizers, logging) are in the debugging cookbook.

What the tests cover, and what they do not

InspectorTests.cpp holds 27 TEST definitions, all against a FakeProvider and an optional FakePreviewProvider over real loopback sockets. UnitTests.cmake drops the file from the test sources unless CNA_BUILD_INSPECTOR is ON; when it is, the tests are compiled into CnaTests, whose cases CTest discovers, and into the focused CnaInspectorTests executable, which is an iteration target and not a separate CTest registration. No CI workflow enables the option. The list below was checked by reading the source at 009d40f5; none of it was executed for this page.

Gaps a maintainer should know before relying on the suite: no test drives WebBridge at all, so the Host check, UI-token enforcement, the 32-connection cap, request parsing and response headers have no automated coverage; no test calls Agent::Stop directly, stops an agent with a client attached, or restarts one (each test's agent is simply destroyed); nothing exercises a real preview provider, a live browser, macOS, or a remote bind. CNA's own docs/inspector.md records a Linux end-to-end run with Chrome and a native MSVC run on Windows and lists macOS runtime validation as pending; those are CNA's records and were not reproduced here.

Where these tests fit in the wider suite is covered by test architecture and change recipes and the test target index; the general procedure for adding one is I need to add a regression test.

Curated implementation route

1. modules/inspector/CMakeLists.txt and inspector_demo.cpp: the opt-in targets, the refused hosts, and how a game links, starts and orders the agent relative to its Game.
2. Agent.hpp then Agent.cpp: provider borrowing, Start validation order, the worker and Stop, authentication before provider access, capability derivation, the rate window and preview validation.
3. Protocol.hpp and Protocol.cpp: message types, capability and error enumerations, every bound, strict UTF-8, and the decoder/encoder symmetry the mutation tests rely on.
4. InternalSocket.cpp: loopback test, secure random, listener and connect, poll()-based waits that survive descriptors past FD_SETSIZE, and exact send/receive with deadlines.
5. Client.hpp and Client.cpp: request ids, envelope checks, which failures disconnect, and the Unavailable-to-status mapping for previews.
6. InspectorMain.cpp and WebBridge.cpp: option and token precedence, the loopback HTTP listener, Host and UI-token checks, the client mutex, detached connections and JSON translation; then WebAssets.cpp for the UI's polling and event-cursor logic.
7. Diagnostics.hpp (IDiagnosticsProvider) and the ProcessProvider in Diagnostics.cpp: what one provider call copies and which locks it takes.
8. InspectorTests.cpp: which of the claims above are pinned by deterministic tests and which still need a live application, browser or renderer.

For the module's place among its siblings see the module index; for choosing a first subsystem to own and the general maintainer workflow, start from the Maintainer Handbook.