CNA-BUG-035: devices-tests.yml's exact-name gtest filters skip seven Microsoft::Devices suites and never match the FileDialog and MessageBox suites

CNA snapshot 009d40f5  ·  Known Issues › Current bugs  ·  source links pinned to 009d40f5

DEVICES_GTEST_FILTER misses 29 of 474 TEST definitions under modules/devices/tests, and CNA_DEVICES_GTEST_FILTER names FileDialogTests.* and MessageBoxTests.*, which match no suite, so the 12 FileDialog and MessageBox cases run in no CI job.

Identifier

CNA-BUG-035

Category

Bug

Subsystem

Build & CI

Status

Open

Verified against

CNA 009d40f5 (009d40f5dd085c4e674d3479675fac84b12b3e0a)

Severity

Low (a triage suggestion, not a project priority)

Evidence basis

Source-verified: read at TARGET, not executed

Tests touching this area

Yes: see Current tests

Affected contract

.github/workflows/devices-tests.yml: DEVICES_GTEST_FILTER (documented as every TEST(...) under modules/devices/tests/Microsoft/Devices/, no more and no fewer) and CNA_DEVICES_GTEST_FILTER (the modules/devices-ext/tests/CNA/Devices/ scope, REMED-BUILD-011)

Expected behaviour

The workflow is the only CI job that builds CnaTests with CNA_DEVICES=ON and a sanitizer (the devices-ubsan preset). Its first filter is documented in the workflow as selecting "every TEST(...) under modules/devices/tests/Microsoft/Devices/, no more and no fewer", kept in step with docs/devices-build.md. Its second filter was added (REMED-BUILD-011) precisely so the CNA::Devices extension suites - FileDialog, MessageBox, Clipboard, SystemTray and the rest - run in CI at all, because the job "silently never compiled that surface" before.

Actual behaviour at TARGET

Matching both filter strings against the TEST/TEST_F definitions at TARGET with gtest's rules (a pattern Suite.* matches that exact suite name only):

• DEVICES_GTEST_FILTER names 21 suites and selects 445 of the 474 definitions under modules/devices/tests/Microsoft/Devices. Seven suites are missing: DevicesShutdownCoordinatorTest (4), DevicesShutdownOrderingTest (3), EnvironmentTests (3), IndependentReferenceCrossCheckTests (2), NativeDiagnosticSinkTest (9), PlatformSensorMigrationTests (4) and PlatformVibrateMigrationTests (4). They are compiled into CnaTests everywhere and run unsanitized in general-tests-ci.yml, but never under this job's UBSan build.
• CNA_DEVICES_GTEST_FILTER lists FileDialogTests.* and MessageBoxTests.*. Since the PLAT-90a refactor (commit 234b6d3e3) the files define the fixtures FileDialogTest and MessageBoxTest plus FileDialogWithoutAServiceTest and MessageBoxWithoutAServiceTest, so neither pattern matches anything and gtest runs the other eight suites without complaint. The dialog tests are wrapped in #ifdef CNA_DEVICES, CNA_DEVICES defaults to OFF, and no other workflow turns it on, so these 12 cases run in no CI job.

The workflow header also still says every CNA::Devices test drives an injected backend through SetBackendForTesting(), the seam PLAT-90a removed in favour of a canned dialog platform.

Source locations

• .github/workflows/devices-tests.yml — DEVICES_GTEST_FILTER and CNA_DEVICES_GTEST_FILTER, and the two CnaTests steps that use them
• modules/devices-ext/tests/CNA/Devices/FileDialogTests.cpp — fixtures FileDialogTest and FileDialogWithoutAServiceTest, inside #ifdef CNA_DEVICES
• modules/devices-ext/tests/CNA/Devices/MessageBoxTests.cpp — fixtures MessageBoxTest and MessageBoxWithoutAServiceTest
• modules/devices/tests/Microsoft/Devices/Detail/DevicesShutdownCoordinatorTests.cpp — DevicesShutdownCoordinatorTest, one of the seven unfiltered suites
• docs/devices-build.md — the same hand-written 21-suite filter, repeated for the ctest and sanitizer recipes
• CMakePresets.json — devices-ubsan preset (CNA_DEVICES=ON, CNA_SANITIZE=undefined)
• docs/devices-thread-safety.md — names devices-tsan as the evidence for the concurrency claims
• modules/devices/tests/Microsoft/Devices/Sensors/Detail/NativeDiagnosticTests.cpp — one of the seven unfiltered suites

Evidence

Checked by reading at 009d40f5; not executed, and no workflow run was inspected. The comparison was mechanical: every TEST/TEST_F/TEST_P definition in the two test directories against the two filter strings. The suite rename is visible in the history of FileDialogTests.cpp: TEST(FileDialogTests, ...) until 2026-08-12, TEST_F(FileDialogTest, ...) since. The option default is in the root CMakeLists.txt (option(CNA_DEVICES ... OFF)); the unfiltered general-tests-ci.yml header records CNA_DEVICES=OFF. An earlier review of this workflow counted four missing suites; three more have been added since without the filter following. The devices-module internals page already lists the seven.

Independently observed as a separate finding (merged): The hand-written filter selects 445 of the 474 TEST definitions under modules/devices/tests, and the only devices CI job uses the devices-ubsan preset, so the devices-tsan evidence the thread-safety document relies on is never produced by CI.

Focused reproduction

Illustrative; not run for this entry. In a devices-ubsan tree:

./cmake-build-devices-ubsan/CnaTests --gtest_list_tests \
    --gtest_filter='FileDialogTests.*:MessageBoxTests.*'      # lists nothing
./cmake-build-devices-ubsan/CnaTests --gtest_list_tests \
    --gtest_filter='FileDialogTest*.*:MessageBoxTest*.*'      # lists the 12 dialog cases

Current tests

The suites themselves are present and complete: FileDialogTests.cpp and MessageBoxTests.cpp (parameter forwarding, callback delivery, the no-service fallback) and the seven Microsoft::Devices suites above. Nothing checks that a workflow filter covers its directory. The input subsystem solved the same drift with one canonical CNA_INPUT_TEST_FILTER in cmake/UnitTests.cmake and ctest -L input; the devices workflow and docs/devices-build.md still copy hand-written strings.

Regression test

Register the devices suites once in CMake (a CnaDevicesTests-style ctest entry with a label, built from the directory rather than typed by hand) and have the workflow run ctest -L devices; or add a check that lists CnaTests --gtest_list_tests per filter and fails when a TEST under either directory is unmatched or when a filter pattern matches nothing.

Blast radius

CI evidence only; no shipped behaviour changes. Under UBSan the shutdown-ordering, environment, diagnostic-sink and platform-migration suites never run; the CNA::Devices::FileDialog and MessageBox contract tests run in no CI job at all. The other eight CNA::Devices suites and the 21 named Microsoft::Devices suites are unaffected.

Workaround

Run the dialog suites locally in a devices-* preset tree, for example ./cmake-build-devices-ubsan/CnaTests --gtest_filter='FileDialog*:MessageBox*:DevicesShutdown*'.