63c)¶

WorkWingman's accessibility testing operates on a four-layer stack: automated static + live-backend axe audits close coverage for machine-checkable WCAG rules, while real NVDA screen-reader flows verify keyboard behaviors and spoken announcements that automation can't assess, and human walkthroughs deliver the qualitative judgment the harness can't automate.

The Four Layers¶

Layer 1: Static Axe Gate (WW-63, Phase 1) — Built App, No Backend¶

What it proves: WCAG 2.2 AA compliance under machine-checkable rules on every route, with the app in its backend-unreachable state (resilience intact, no real data).

Its limits: The backend is offline, so routes that require real data or backend responses render with empty/error states; axe can't verify keyboard focus movement, NVDA announcements, or dialog traps.

How to run:

npm run test:a11y

What happens: - Runs during CI on the frontend job (GitHub Actions). - Sweeps all 22 routes from STATIC_ROUTES (tests/electron-e2e/a11y-shared.ts). - Serves the built app over a local dumb HTTP server (http://127.0.0.1:<random-port>). - Aborts all API traffic to http://127.0.0.1:5211 so the audited state is deterministic (always backend-unreachable). - Runs axe-core with tags ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22a', 'wcag22aa']. - Writes a11y-report.json with violation breakdowns by impact (critical/serious/moderate/minor). - Gate enforcement: Any unauditable route or any critical/serious violation fails the test (ratcheted to baseline 0 on 2026-07-08).

File references: - Test spec: tests/electron-e2e/a11y.spec.ts (lines 15–132) - Shared harness: tests/electron-e2e/a11y-shared.ts (startStaticServer, auditRoute, AXE_TAGS, STATIC_ROUTES) - Report output: tests/electron-e2e/a11y-report.json (gitignored)

Layer 2: Live-Backend Axe Pass (WW-63b) — Real API, Real Data, Keyboard Tests¶

What it proves: WCAG 2.2 AA compliance on every route with the machine owner's real data, param routes with real IDs, and keyboard behaviors (skip-link focus jump, dialog focus trap + Escape, roving tabindex on live-run tabs).

Its limits: NVDA can't verify spoken announcements; the firewall is strictly read-only, so only browsing and opening dialogs (not syncing or mutating) are tested.

How to run:

npm run test:a11y:live

What happens: - Runs locally only—never in CI (the desktop app may be open, and real user data is spoken aloud during NVDA flows). - Requires a running WorkWingman API on http://127.0.0.1:5211. - If the API is already running (dev machine with the Electron app open), it is reused and not killed afterward. - Otherwise, the spec builds the API (fast no-op if already up to date) and spawns src/WorkWingman.Api/bin/Debug/net10.0/WorkWingman.Api.exe directly. - Reads the machine's local API token from WORKWINGMAN_TOKEN_PATH (default: %LOCALAPPDATA%\WorkWingman\api-token) and sends it as the X-WorkWingman-Token header for token-gated routes. - Resolves param-route IDs from real data (first job from GET /api/jobs, first interview retro from GET /api/interview-retro/retros). - Installs a context-scoped read-only firewall: - Aborts every non-GET/HEAD/OPTIONS request to the API origin (both 127.0.0.1 and localhost spellings, port 5211). - Aborts WebSocket connections to the same origin. - Defense in depth: The firewall doesn't trust the app—a component that accidentally POSTed on init or a click handler that leaked a mutation would be caught. - Every test asserts the blocked list is empty; any attempted mutation fails the test outright (not just logged). - Runs axe on the live-data render of all 22 static routes + param variants. - Runs keyboard-behavior tests: - Skip-link test: First Tab lands on the skip link; Enter moves focus to #main. - Sensitive dialog test: Opening the data-backup sync dialog traps focus, Escape closes it, focus restores to the trigger button, and zero POSTs fire (double-checked by the firewall). - Rail tabs roving tabindex: ArrowRight/ArrowLeft move aria-selected, tabindex, and focus between the Steps and Run log tabs. - Writes a11y-live-report.json with axe violations, no-data routes, keyboard results, and param IDs used. - Gate enforcement: Critical/serious violations and unauditable routes fail; blocked non-GET requests fail; all keyboard tests must pass.

File references: - Test spec: tests/electron-e2e/a11y-live.spec.ts (lines 1–578) - Shared harness: tests/electron-e2e/a11y-shared.ts - Read-only firewall: a11y-live.spec.ts lines 219–251 (installReadOnlyFirewall) - Report output: tests/electron-e2e/a11y-live-report.json (gitignored; carries real user data in violation summaries) - Strict mode: WW_A11Y_LIVE_STRICT=1 env var makes skipped keyboard tests FAIL instead of skip (used to prove all skip paths are reachable)

What it proves: NVDA (real screen reader, real eSpeak-ng speech synth) can successfully navigate the app and announces the right text (landmarks, headings, form labels, skip-link, dialog names, no unwanted tagline spam).

Its limits: Desktop focus-stealing is non-negotiable; runs only on Windows; the user's machine must have a real desktop (not headless/RDP-minimized) so Chromium and NVDA both have a window manager to render into. The app's backend can be unreachable or live—both states are auditable.

How to run:

npm run test:a11y:sr

One-time setup per machine:

npx @guidepup/setup

This downloads a portable, Guidepup-patched NVDA build to a temp directory (no admin/elevation needed). Confirms you have the required node modules:

npm i -D @guidepup/guidepup @guidepup/playwright

Firewall rule (fleet bootstrap handles this on all PCs): Loopback TCP port 6837 (Guidepup's local control server) must be reachable. If the bootstrap hasn't run, add the rule manually:

netsh advfirewall firewall add rule name="Guidepup Control Server" dir=in action=allow protocol=tcp localport=6837 remoteip=127.0.0.1

What happens: - Runs locally only—NVDA steals desktop focus (switches foreground app, reads keystrokes, talks), so it is not wired into CI (see testIgnore in playwright.config.ts and playwright.a11y-sr.config.ts). - Uses Guidepup to drive NVDA programmatically (spawn, send keys, read spokenPhraseLog, navigate with SR-specific commands). - Reuses the static server and optionally the API: - If a WorkWingman API is already running on 5211, pages render with real user data (and real names/values are spoken aloud and captured in the report—see PII discipline below). - Otherwise, pages render in the backend-unreachable state (no build/spawn, to stay within the ~3-minute runtime budget NVDA focus-stealing demands). - Runs five serial flows: 1. Queue landmarks & skip-link: Navigate to landmarks on the queue page; first Tab announces the skip-link text; navigate to headings and assert the h1 is announced. 2. Sidebar navigation names: Arrow through 6+ sidebar nav links; every spoken phrase must carry a non-empty accessible name (not bare "link"). 3. Onboarding form labels: Walk back to step 1; Tab through 5+ text fields; each must announce its label (not "edit blank"). 4. Tagline no-spam: Idle for 8s on the queue page; the rotating tagline must announce at most once (zero is also a pass). 5. Sensitive dialog announcement & Escape: (Skipped if no API is running.) Open the data-backup sync dialog; it must announce its name/role; Escape closes it and restores focus to the trigger button; zero POSTs fire. - Asserts on substrings of spoken phrases, not exact strings (NVDA/eSpeak phrasing varies with voice settings and synth changes; exact assertions would flake on cosmetic changes). - Writes a11y-sr-report.json with flow results and sample phrases. - Gate enforcement: Every flow must pass (skip only if precondition is missing and not in strict mode).

File references: - Test spec: tests/electron-e2e/a11y-sr.spec.ts (lines 1–475) - Guidepup setup: npx @guidepup/setup downloads NVDA to the Playwright cache directory - Read-only firewall: a11y-sr.spec.ts lines 133–174 (same pattern as a11y-live, with a special stub for POST /propose-from-linkedin) - Spoken-log polling: a11y-sr.spec.ts lines 188–202 (waitForSpokenMatch polls until a phrase matches, robust against synth speed and OS notifications) - Report output: tests/electron-e2e/a11y-sr-report.json (gitignored; contains real spoken profile data if the API is running) - Strict mode: WW_A11Y_SR_STRICT=1 env var makes the sensitive-dialog flow (and any other flow that would skip on a missing precondition, e.g. no live API on 5211) FAIL instead of skip (mirrors WW_A11Y_LIVE_STRICT in Layer 2, used to prove all skip paths are reachable)

Layer 3b: Audible-Path Check (`audio-loop-mcp`) — Speaker → Loopback → STT¶

What it proves: Layer 3 taps NVDA's spokenPhraseLog at the synthesizer layer — it asserts what NVDA decides to say, not that the audio reaches the speakers. audio-loop-mcp (private C# MCP: WASAPI loopback capture + local Whisper STT + SAPI) closes that gap: it speaks a phrase, records the machine's actual render output, transcribes it locally, and scores heard-vs-spoken. Catches a broken/muted/misrouted audible path a text-layer check can't see.

Its limits: It's a Claude-session MCP tool, not embeddable in the Playwright Node process — so it runs as an operator/agent pre-flight (or a live record_loopback during an NVDA flow → transcribe → assert), not inside a11y-sr.spec.ts. Local-only (no cloud STT).

How it plugs into WW-63: run round_trip_test("Skip to main content. Account needed to continue.") as a pre-flight before an NVDA session — a 100% match confirms the audible path is healthy and Layer 3's synth-layer assertions can be trusted as actually audible. (Verified 2026-07-09: 100% on the app's real skip-link + account-wall phrases.) See the audio-loop-mcp tool repo (andrewjonesdev-tools) for speak / record_loopback / transcribe / listen_and_transcribe / round_trip_test.

Layer 4: Human NVDA/Narrator Walkthrough — Qualitative Judgment¶

The three automated layers close machine-checkable gaps but can't replace a human listening to the app and judging whether the experience is actually usable. Once CI gates are passing:

On Windows: fire up a real NVDA session (or use Narrator) and walk through the app in browse mode + focus mode, navigating with screen-reader commands (heading/landmark hops, roving through form groups, etc.).
Listen for:
Unexpected silences (an element should have announced but didn't).
Overannouncements (redundant labels, status messages announced multiple times).
Misannounced semantics (a button announces as "link", a heading's level is wrong).
Focus jumps that surprise (Escape or form submission doesn't restore focus as expected).

This is the final qualitative gate and the reason all tests remain local—a real user's feedback is irreplaceable.

The Read-Only / PII Discipline¶

Both a11y-live.spec.ts and a11y-sr.spec.ts may render pages against the machine owner's real LocalJsonStore data. The firewall + report gitignore + deletion-on-start regimen protects PII:

Firewall (hard gate): - Every non-GET request to the API is aborted, except the explicitly-stubbed, side-effect-free propose-from-linkedin call in a11y-sr.spec.ts (see the LinkedIn propose stub note below) — that stub is itself recorded and asserted against an exact allowlist, so a NEW unexpected non-GET is still caught and fails the test. - Every WebSocket connect to the API is closed. a11y-sr.spec.ts fails fast at firewall install time if context.routeWebSocket is unavailable on the running Playwright build, rather than degrading to an unblockable observe-only fallback. - Tests assert the blocked list is empty—any mutation attempt fails the test.

Report files (gitignored): - tests/electron-e2e/a11y-live-report.json and a11y-sr-report.json may quote real profile text in axe violation summaries (e.g., "the link 'Andrew Jones' has low contrast"). - Both are in .gitignore so they never commit. - Both are deleted at the start of every fresh run (not retries of the same run). - Each report carries a runId UUID so a later merge-on-write never silently folds a different run's data into this one.

Trace/screenshot/video (off): - For both specs, trace: 'off', screenshot: 'off', video: 'off' (see Playwright config) so failure artifacts never freeze a frame of real profile data.

The one special case — LinkedIn propose stub: - During onboarding, POST /propose-from-linkedin auto-fires if a LinkedIn connection exists. - It's read-only in intent (proposed jobs are never saved locally) but IS a network call to LinkedIn. - The a11y-sr.spec.ts firewall stubs it with an empty proposal set (no external call, no mutation) so the wizard still renders (and NVDA can speak the profile-backed form without a real outbound call).

Mermaid Diagrams¶

Architecture: Four Layers Stacked on the App¶

graph TB
    subgraph "Built Frontend"
        FE["Angular app<br/>(hash routing, built to frontend/dist)"]
    end

    subgraph "Layer 1: Static Axe"
        S["Static HTTP Server<br/>(dumb, no-fallback)"]
        AX1["axe-core<br/>WCAG 2.2 AA rules"]
        AP1["API traffic ABORTED<br/>(deterministic backend-unreachable state)"]
    end

    subgraph "Layer 2: Live Axe + Keyboard"
        API[".NET API on 5211<br/>(real LocalJsonStore)"]
        AX2["axe-core<br/>WCAG 2.2 AA rules"]
        KB["Keyboard Tests<br/>(skip-link, dialog trap, roving tabindex)"]
        FW["Read-Only Firewall<br/>(non-GET aborted, WS closed)"]
    end

    subgraph "Layer 3: NVDA Screen Reader"
        SR["Real NVDA via Guidepup<br/>(eSpeak-ng speech synth)"]
        FLOWS["Five Flows<br/>(landmarks, nav names, labels, tagline, dialog)"]
        SPKEN["Spoken Phrase Log<br/>(asserted on substrings)"]
    end

    subgraph "Layer 4: Human Walkthrough"
        HU["Human with NVDA/Narrator<br/>(browse + focus mode)"]
        JDG["Qualitative Judgment<br/>(usability, surprise silences, overannouncements)"]
    end

    S --> FE
    AX1 --> AP1
    API --> FE
    AX2 --> KB
    FW --> API
    SR --> FE
    SR --> FLOWS
    FLOWS --> SPKEN
    HU --> FE
    HU --> JDG

    style FE fill:#e8f4f8
    style S fill:#fff4e6
    style API fill:#fff4e6
    style SR fill:#e8f5e9
    style HU fill:#f3e5f5

Sequence: NVDA Rig Driving the App¶

sequenceDiagram
    actor H as Human
    participant P as Playwright
    participant CS as Chromium<br/>(built app)
    participant GS as Guidepup<br/>Control Server<br/>(TCP 6837)
    participant NV as NVDA<br/>(real, desktop-stealing)
    participant API as .NET API<br/>(5211)
    participant RWFW as Read-Only<br/>Firewall

    H->>P: npm run test:a11y:sr
    P->>CS: Launch headed Chromium, load built frontend
    P->>GS: Start Guidepup server (TCP 6837)
    GS->>NV: Launch NVDA, attach Guidepup patch
    NV->>P: NVDA steals desktop focus
    P->>CS: navigate to /#/queue
    CS->>API: GET /api/jobs (+ X-WorkWingman-Token header)
    RWFW->>API: (read-only firewall allows GET)
    API->>CS: real jobs JSON
    P->>NV: nvda.keyboard.press('Tab')
    NV->>P: (reads focused element, speaks "skip to main content")
    GS->>P: spokenPhraseLog() returns ["skip to main content", ...]
    P->>P: assert /skip to main content/ in log
    P->>CS: navigate to data-backup (if API running)
    P->>CS: click sync button
    CS->>P: dialog opens (client-side only, no POST)
    P->>NV: nvda.lastSpokenPhrase()
    NV->>P: "dialog, Sync to Drive"
    P->>NV: nvda.press('Escape')
    NV->>CS: Escape key event
    CS->>CS: dialog closes, focus restores to button
    P->>P: assert firewall.blocked === []
    P->>P: writeReport() -> a11y-sr-report.json

Setup Summary¶

Layer	Run	CI?	API	Auth	Runtime	Key Constraint
Static (WW-63)	`npm run test:a11y`	✓ Yes (FE job)	No (aborted)	None	~5 min	Must abort API so state is deterministic
Live (WW-63b)	`npm run test:a11y:live`	✗ No	Yes (reuse or spawn)	`WORKWINGMAN_TOKEN_PATH` or `%LOCALAPPDATA%\WorkWingman\api-token`	~15 min	Real data: gitignore reports, assert firewall.blocked=[]
NVDA (WW-63c)	`npm run test:a11y:sr`	✗ No	Optional (reuse if live)	Same as live	~3 min	Desktop focus stolen; needs real window manager; one-time `npx @guidepup/setup`
Human	—	—	Live, real data	—	∞ (manual)	Qualitative only; no automation

Future: Audio-Loop MCP Integration¶

A planned tool (andrewjonesdev-tools/audio-loop-mcp) will capture the audible output of NVDA runs and cross-verify spoken text against a reference model (ensuring consistency across synth updates and voice configurations). This bridges the gap between "NVDA spoke something" and "NVDA spoke the right thing, deterministically."

References¶

WCAG 2.2: https://www.w3.org/WAI/WCAG22/quickref/
axe-core: https://github.com/dequelabs/axe-core (open-source accessibility testing library)
Guidepup: https://www.guidepup.dev/ (screen-reader automation for Playwright)
NVDA: https://www.nvaccess.org/ (free, open-source screen reader for Windows)
Playwright axe integration: https://github.com/dequelabs/axe-core-npm/tree/develop/packages/playwright

Accessibility Verification Stack (WW-63/63b/63c)¶

The Four Layers¶

Layer 1: Static Axe Gate (WW-63, Phase 1) — Built App, No Backend¶

Layer 2: Live-Backend Axe Pass (WW-63b) — Real API, Real Data, Keyboard Tests¶

Layer 3: NVDA Screen-Reader Automation (WW-63c) — Real NVDA, Spoken Output¶

Layer 3b: Audible-Path Check (audio-loop-mcp) — Speaker → Loopback → STT¶

Layer 4: Human NVDA/Narrator Walkthrough — Qualitative Judgment¶

The Read-Only / PII Discipline¶

Mermaid Diagrams¶

Architecture: Four Layers Stacked on the App¶

Sequence: NVDA Rig Driving the App¶

Setup Summary¶

Future: Audio-Loop MCP Integration¶

References¶

Layer 3b: Audible-Path Check (`audio-loop-mcp`) — Speaker → Loopback → STT¶