Skip to content

WT-a37c · WING-422 · lab test client — drive Live Run and study generation without a real employer

Updated: 2026-08-08T23:10Z Status: BLOCKED — PR #46 held by a FAILED council gate; fixes in flight; WING-461/289 dispatched to a second worktree

1. Goal

Andrew can start the app in a lab mode, pick a synthetic candidate seed, and watch Live Run actually fill an application form — against a fake employer served on his own machine, with no real employer reachable. Same for study generation (flashcards, episodes). Observable definition of done: he sees the automation work, on his screen, end to end, and no traffic leaves the box.

He has never observed either feature run. That is the entire point of this lane — not test coverage, observation.

2. Where the work is

  • Worktree: C:\Users\fives\source\repos\wt-WT-a37c-WING-422
  • Branch: wt/WT-a37c/WING-422 (base: master@cd661d86c, green)
  • PR: #46 — OPEN, not draft (marked ready after master went green), head a1727a518
  • Uncommitted right now: NONE (git status --porcelain → 0 lines)
  • Branch tip a1727a518 == PR head == remote tip. Everything is pushed.
  • This handoff also exists on master at 10fd6c6b5 (E1 metadata carve-out, ruled by WT-0f7d), because Codex finds lanes by listing docs/handoffs/ on master. If you change this file, re-land it to master too or the lane silently reads as uncovered.

3. State — what is TRUE, with receipts

Slice 1 (lab apply-site generator + evidence ledger) — green locally. 

cd tools/pageagent-lab && npm run build   -> BUILD_EXIT=0
node dist/harness/runLabTests.js          -> LAB_EXIT=0, PASS=16 FAIL=0
Includes: ledger first-observation stickiness; page cannot expose a rung recorder; lab rejects POST (HTTP 405) so it can never accept a submission; generator 404s arbitrary job paths; restructured strips stable selectors + ARIA + native/positional label association; shadow reachable only via open shadow root; hostile falls through to manual with the field left untouched.

PR #46 CI: restarted on a1727a518 when the docs commits landed. At last check 25 pending, 0 fail. Every check appears twice — push: {} plus the pull_request event both fire, so each PR burns double runner time on greatroom. Confirmed by WT-0f7d as unintended (see §10).

Master is green at cd661d86c — verified independently, not taken on trust: gh run list → run 31267856350 on wt/regression/master-green at sha cd661d86c = completed success, and master points at that exact sha. (Master's own two runs on the same sha were still in flight; the regression-branch run is the receipt.)

OutboundNavigationPolicy rejects loopback four independent ways — read directly from src/WorkWingman.Infrastructure/Automation/OutboundNavigationPolicy.cs: non-standard port (:41), host without a dot (:44), explicit localhost/.local/.internal (:45-48), IPAddress.IsLoopback (:86). Its only callers are JobPageFetcher and RedirectChainAtsResolver (grep -rn "OutboundNavigationPolicy\."). The apply / Live Run engines never consult it.

Every seed job points at a real employer: grep -l "127.0.0.1\|localhost" src/WorkWingman.Api/DemoSeeds/ → empty.

npm test in tools/pageagent-lab cannot verify anything — filed as WING-455: 

timeout 150 npm test  -> EXIT=124  ELAPSED=150s  LINES=15
grep -c '\[PASS\]\|\[FAIL\]'  -> 0
Root cause, verified in our code (not page-agent): src/model/mockModel.ts:134 hardcodes finish_reason: 'tool_calls' for every response including done (never 'stop'), and :228 clamps cursor = Math.min(cursor + 1, script.length - 1), so once the cursor reaches done (index 12 of 13) it replays done forever. Blocks at run.ts:103 await page.evaluate(); :148 never prints and the finally at :176 never runs, so the un-closed fixture server is a symptom, not a second cause. Third finding: page.on('console') is not wired and the agent runs inside the page, so all step logging goes to the browser console — a looping agent presents as a dead process.

Docs gates verified locally before committing (gate logic read from .github/workflows/ci.yml lines 506-522): two-track pairing GATE1_FAIL=0; reachability satisfied via a docs/README.md row; all internal links in both new docs resolve.

UNVERIFIED: that slice 1 passes CI (25 pending on a1727a518 at last look). Verify with gh pr checks 46.

4. Next actions, in order

  1. Confirm PR #46 CI is green on its own merits. gh pr checks 46 Acceptance: zero fail. A red result now MEANS something — it can no longer be inherited from master. If red, it is ours. Status at last check: 25 pending, 0 fail, on a1727a518 — the run restarted when I pushed the docs commits, so it is a fresh verdict, not a stale pass.
  2. Make the environment immutable — resolve once from argv into an immutable EnvironmentContext; delete the mutable global; never persist it. Fixed order: resolve env → install egress policy → open data dir → open vault → load seeds. Acceptance: a test proves no code path mutates the environment post-startup, and that policy is installed before any possible egress. Do this FIRST — four of the red-team findings collapse once the mutable global is gone.
  3. Browser-boundary loopback allowlist, denying non-approved navigations, subresources, popups, downloads, redirects, WebSockets. Acceptance: one test per traffic class, each asserted separately. Prove it fails by reintroducing the defect.
  4. Seed URL rewrite with preserved provenance — immutable ApplyUrl (loopback in lab) plus immutable SourceApplyUrl (provenance only, never navigable). Title/company/role stay verbatim. Acceptance: passing SourceApplyUrl to a navigation API is a compile or test failure; every lab ApplyUrl resolves in the lab route table.
  5. Gate Live Run by immutable target policy, at every command/worker entry point, before browser launch. Acceptance: real may navigate only the captured real employer target. Demo, sandbox, and persona worlds may navigate only a catalog-derived loopback lab target; the source employer URL remains provenance and is never navigation authority. The gate is enforced at the entry point, not only in UI.
  6. Typed Live Run refusal (LabServerRequired), not false/null/empty. Acceptance: no success path exists on refusal; server-health and URL-rewrite share one gate so there is never a half-lab state.
  7. Lab vault per the threat model (memory-only password), then the capability fixture.

Full spec: docs/technical/lab-environment-threat-model.md (plain sibling in docs/plain/), and the adjudication comment on WING-422.

5. Blockers — and what an agent must REFUSE

  • Andrew must observe the lab himself. The deliverable is him watching Live Run. An agent confirming it works is the failure mode this lane exists to end. Do not mark WING-422 done on an agent's say-so.
  • No agent may click through authenticated prod. Reaching First Flight on app.workwingman.ai needs an invitation code redeemed into a live authenticated surface = credential entry. Being handed a pilot code does not unblock it. app.qa.workwingman.ai cannot substitute (restricted ingress). The loopback route (ops/cloud-run/Run-CloudLocally.ps1) is BLOCKED as of 2026-08-07 — server edition refuses the Firestore emulator, exit 139 at startup.
  • Do not relax OutboundNavigationPolicy to let the lab's loopback URLs through. It is the SSRF control. The lab needs a separate lab-only policy selected structurally.
  • Andrew decision outstanding: he asked the lab ship "with passwords for the keyvaults, and a spot where they are stored". Adjudicated as not possible in that form (§8). He has been told; he has not responded. Do not implement a stored/displayed lab vault password on the assumption he insisted.

6. Traps in this lane

  • npm test in tools/pageagent-lab hangs and reports exit 0 with truncated output. Use node dist/harness/runLabTests.js directly. I reported a green run on it once and had to retract.
  • A slow seat looks exactly like a dead one. Gronk's first review had emitted only its preamble (308 bytes) when I checked; I called it truncated. It was still running and finished as the fullest review (28 KB). Check again before declaring a seat dead.
  • Both seat runs were worth having. The unbounded (repo-reading) run found the OutboundNavigationPolicy loopback conflict — visible only from code. The bounded (design-only) run found the app-HTTP-vs-browser layer error — visible only from the design. Neither alone caught both.
  • agy does not accept --prompt-file (that is grok's). It printed usage and exited 0 — a silent dead seat. Use agy --model <m> --effort <e> -p "$(cat file)". Always check byte count.
  • grok --prompt-file <f> works, but must not also get a positional prompt.
  • Grep counts are not features. callback matched 30 files, nearly all JS callbacks. email matched 2 files, both in WorkWingman.Site — there is genuinely no email seam in the app.
  • Do not rebuild the seeds. src/WorkWingman.Api/DemoSeeds/{firsttimer,switcher,veteran}/ already ships resume.md, story.md (about-me), profile.json, jobs.json, offers.json, benefits.json, retros.json, transcript.json, dar.json, deck-*.json, studycast-source.md, jobs-usajobs-real.json, plus persona-credentials.json (synthetic site logins, not the vault master password — I conflated those two once and had to correct it to Andrew).
  • A new docs/technical/*.md needs a docs/plain/*.md sibling with the SAME filename, AND a link from another doc, or the Docs gate fails on pairing or orphan-reachability. Verify locally against .github/workflows/ci.yml before pushing.
  • "Test Bench mock/live" editions do not exist. docs/technical/editions-matrix.md records that no TestBench/mock/live flags exist and the naming may be conversation-only. Do not invent flags to match the name. Same doc records that cohorts qa, user-testing, internal-testing all fall through RuntimeCapabilities to edition=server-locked, audience=unconfigured.

7. Delegated / dispatched work

All local (this box, greatroom). No fleet-harness send/enqueue jobs were dispatched from this lane. Nothing is in flight. Every output below was READ.

# Kind What I asked Output Read?
1 cavecrew-investigator subagent Locate the pageagent-lab demo hang agent result YES — gave run.ts:103, mockModel.ts:134/:228
2 cavecrew-investigator subagent Map existing demo persona / seed infra agent result YES — the "do not rebuild seeds" finding
3 Jenny seat (agy gemini-3.1-pro --effort high) Design review, attempt 1 scratchpad/jenny-422-design.txt (2.3 KB) YES — FAILED SEAT, usage text, exit 0. Discarded.
4 Jenny seat, attempt 2 (-p) Design review same file, 4.3 KB YES — killed the persona-reuse plan and the shipped vault
5 Gronk seat (grok --prompt-file), unbounded Design red team scratchpad/gronk-422-design.txt (28 KB) YES — found the loopback/SSRF conflict
6 Gronk seat, bounded (no repo) Design red team scratchpad/gronk-422-r2.txt (17.6 KB), 16 findings YES — found the browser-vs-app-layer error
7 Cedric (codex exec -m gpt-5.6-terra) Adjudicate seat disagreements scratchpad/cedric-422.txt (18.9 KB) YES — red team wins all six

Scratchpad dir: C:\Users\fives\AppData\Local\Temp\claude\C--Users-fives-source-repos\d01761ab-89c1-4ddf-a906-4f9c9c4ee1ad\scratchpad\ — session-scoped and will not survive; the durable copies are the WING-422 adjudication comment and docs/technical/lab-environment-threat-model.md.

Background shell jobs (CI watchers) may still be polling; they are read-only gh calls, safe to abandon.

8. Decisions made and why — do not silently reverse these

  1. Egress enforcement goes at the BROWSER boundary, not on the app's HttpClient. Live Run drives a Playwright subprocess that never passes through app HttpClient, so a DelegatingHandler there constrains nothing it was built for while reading as protection. An app-HTTP handler may ship as defense-in-depth only if its name/tests/docs say it does not enforce browser egress.
  2. Persona data does not grant network authority. Persona and sandbox Live Runs are permitted only with a typed, catalog-derived loopback lab target. real retains normal Live Run against the captured employer target. No persona/source URL can be promoted into a navigable target.
  3. No stored and no displayed lab vault password. A committed known password trips the scanner and defeats the vault's promise; a per-run password displayed was ruled worse (screen capture, sharing, recording, clipboard, accessibility trees, crash artifacts) and normalises treating passwords as displayable. Adopted: per-run password in process memory only, never rendered or logged — a crash orphan is inert because nothing ever printed it. What Andrew loses: any viewable/copyable lab vault password. The app opens it, not him.
  4. Keep SourceApplyUrl. Dropping the original employer URL makes "rewrite ran" and "rewrite silently failed" indistinguishable.
  5. Capabilities in lab come from a producer-captured byte-for-byte fixture, not the network (lab denies egress) and not a cached real payload (that imports real entitlements into a synthetic run). Consumer tests consume it unchanged and may never synthesize its shape — that was the WING-305 failure (97/97 green, feature unreachable).
  6. Environment is immutable, argv-resolved, never persisted. A mutable global makes security policy timing-dependent and lets the UI say lab while egress behaves as real.
  7. Coverage is UNCOVERED-by-default, following simple-mode.guard.ts — declaration is opt-out. A touched UNCOVERED surface fails the run; it must not become permanent "known debt".
  8. The lab server must never be evolved to accept submissions. Its 405 is a backstop, not the control. The client invariant (submit located, never clicked) is absolute; lab success means "review step reached", never "HTTP 2xx on apply".
  9. Email is out of scope for this slice — no email seam exists in the app at all. Own slice, own design. Calendar and the interview question bank are real and reusable.

9. Tickets and waiting-on

  • WING-422 — this lane. Adjudicated design is a Jira comment; the durable spec is the threat model doc in-repo.
  • WING-455 (filed by me) — pageagent-lab npm test hangs before any assertion, reported exit 0 twice. The ask that matters is item 3: an assertion-free run must exit non-zero, which fixes the class rather than the one hang.
  • WING-458 (filed by me, outranks this card) — the apply / Live Run navigation has no destination allowlist. OutboundNavigationPolicy's only callers are JobPageFetcher and RedirectChainAtsResolver; IApplyRunDriver.cs:230 hands job.Ats.ApplyUrl straight to page.GotoAsync unchecked, and none of the 14 apply engines reference an allowlist. The apply path does have IsHardBoundaryUrl (:413/:463) and the never-click-submit rule — those are page-identity and action guards, not a destination allowlist. This gap exists in the shipping product with no lab involved; fixing it is not lab work.
  • WING-413 — handed to the UI-coverage lane via WT-0f7d. Passed along with it: it must adopt WING-436's closed outcome vocabulary, or "N still resolving" cannot express its own failure state.

10. Known issues outside this lane (do not re-diagnose)

  • Every CI check runs twice on a PR. push: {} and the pull_request event both fire, so each PR burns double runner time on greatroom (~1 GB free). Confirmed by WT-0f7d as an unintended side effect of the trigger fix that closed the "feature branches run nothing" hole — not a misconfiguration to hunt. Suggested fix: a push: branch filter excluding PR-backed refs. Deliberately deferred: not changing ci.yml again while master is newly green and six PRs are mid-verdict.
  • Seat invocation gotchas (both cost me a round-trip today): agy has no --prompt-file — that is grok's flag; passing it prints usage and exits 0, a silent dead seat. Use agy --model <m> --effort <e> -p "$(cat file)". And a slow seat is indistinguishable from a dead one — check twice and confirm the process exited before declaring a seat dead.
  • Waiting on Andrew: the §5 vault ruling (told, no response), and ultimately his own observation of Live Run, which no agent can substitute for.

11. STOP — read this before touching PR #46 (added 2026-08-08, late)

THE GATE FAILED. #46 IS NOT CLEARED TO LAND OR DEPLOY. If you read only "the council gate ran and seats found things", you will draw the wrong conclusion. Three seats (Gronktayvius/grok, Jenny/gemini-3.1-pro, Cedric/codex) reviewed the diff; two P1s were found by ALL THREE and verified by me in code.

Round 1 findings (fixed in 775644481)

  1. Concurrent duplicate live runs. ApplicationRunService.cs:29-32 returns only the NEWEST non-terminal run; StartRunAsync had no duplicate guard; canStartLiveRun blocked only the SAME job. Start A, start B, and A is startable again — a second browser drives the same application.
  2. The queue bypassed prerequisites Documents enforces. documents.ts:131 gates on referralResolved() && !referralFormOpen() && !referralSaving(); grep -c referral queue.ts was 0. The queue also called approveAndStart on unapproved documents — approving content the user never saw, made worse by WING-333 (documents can be placeholder stubs). approveAndStart is now DELETED, so the capability cannot be called at all.
  3. The launcher did not own its invariants. Two callers, two guard sets, one shared HTTP call. Extracting a shared launcher did NOT extract the invariant — that is exactly how the paths diverged.
  4. My own two docs contradicted each other (found by Cedric alone): the threat model said "only lab may start Live Run; real/sandbox/persona must reject", while the run-target design said real targets the real employer. As written, my invariant forbade the product's core feature. Corrected: real normal; sandbox and every persona REFUSE; lab loopback-only.

Round 2 — P1-A was NOT actually closed

My round-1 fix was frontend check-then-act, which is not a gate. Three bypasses: an in-flight race (two starts before either returns), multi-tab/multi-window (no in-page guard can see another tab), and no server enforcement at all.

Product decision, confirmed by Andrew: exactly ONE active run. Implemented server-side — ApplicationRunService.cs:51 returns ApplicationRunStartResult.Refused(ActiveApplicationRunConflict) and the controller maps it to 409 naming the conflicting job. Verified by me: dotnet build 0 errors, dotnet test 6203 passed / 0 failed. Note IApplicationRunService.cs:18 previously DOCUMENTED the old behaviour — that comment had to change too, or the next reader restores the defect.

Still OPEN from round 2

  • My lab evidence ledger is STILL self-authored (Cedric, P1). runLabTests.ts:36-38 — both fillAndRecordSelector and fillAndRecordInPageAgent execute the same Playwright locator.fill; choosing which helper to call is what stamps the rung. No PageAgent and no application engine ever selects it. This is the third iteration of the same defect by me — page-global, then a parameter, now a function choice. A lab built to catch fabricated evidence is fabricating its own.
  • loadLiveRunReadiness calls getDocuments once per eligible job, and that endpoint returns full resume and cover-letter payloads merely to read approved. At the ~1,400-row scale this component already handles, that is hundreds to thousands of HTTP and disk reads.
  • The parity guard silently skips multi-line AddSingleton<IApplyAutomationEngine> registrations.
  • Lab frame routes accept non-iCIMS and copy-paste-only ATSes, violating the route contract the test claims to enforce.

Re-gate on the NEW diff before landing. A clean round that reviewed a diff which no longer exists proves nothing.

12. Live Run is REACHABLE but UNUSABLE ON DEMO DATA — do not report this as a broken build

Click-verified in a browser (dev server on 4200, real API on 5211):

  • Andrew's real 68 saved jobs: 19 Live-Run-capable, 0 startable — none have documents generated.
  • Marcus Ellery demo persona: byAts: { Unknown: 12 }, all 12 rendering "manual copy-paste", 0 startable.

The app is CORRECT in both cases — AtsKind.Unknown has no registered engine, so copy-paste assist is the right answer. The SEED DATA cannot reach the feature. Anyone demoing this expecting to watch a form fill will get nothing and will report a broken build. It is not broken.

Requirement my own spec was missing: rewriting seed apply URLs to lab sites is NOT sufficient — seeded jobs also need a WIRED ATS KIND, or they stay unreachable wherever the URL points.

13. My own misses, recorded because nobody else will find them

  • tools/pageagent-lab/src/harness/serve.ts legacy startFixtureServer answered 200 to ANY method including POST, so "the lab never accepts a submission" was FALSE on the path run.ts actually uses. Fixed (405 + Allow: GET, HEAD) — but it was my invariant and I missed it.
  • atsCoverageParity.ts derived coverage from engine class names, so a GreenhouseApplyEngine whose Ats => AtsKind.Workable would report the wrong coverage and still pass. It now parses the Ats property and requires the DI type set and the Ats set to agree.
  • I called P1-A closed after verifying a frontend-only guard. Enforcement must live where it is authoritative, not where it is convenient.
  • I wrote a test fixture (getDocuments -> of(null)) that did not match the producer (DocumentsController.cs:15 returns NotFound()), and nearly "fixed" correct code because of it.

14. WING-458 ruling — do not re-litigate

WT-0f7d ruled it does NOT block the cloud push: the gap is pre-existing and ships today, so holding delays unrelated fixes without removing exposure; #46 does not widen it (the lab work drives LAB sites); IsHardBoundaryUrl page-identity guard and never-click-submit remain in force. Escalated to Andrew separately, because "ships today" describes exposure, not consent.

15. The architecture that was never built (WING-461 / WING-289)

electron/main.js:3 claims the app "hosts the automation browser as a WebContentsView the user can watch and take over." Two independent seats confirmed zero executable occurrences of WebContentsView, BrowserView, <webview>, setBrowserView, or contentView.addChildView anywhere under electron/. Reality: BrowserSession selects Visible for apply runs and IApplyRunDriver.cs:227 calls BringToFrontAsync() — a separate top-level Chromium window that steals focus. The Live Run page polls a PNG every 1.5s into an <img>. ClaudePageAdvisor.cs:6-19 makes a similar false claim about an Electron IPC path; production uses LlmAdvisorTransport.

WING-289 (cloud stream, COUNCIL-APPROVED 2026-07-30) sat in To Do for 9 days. Cloud is not inert — it is reachable and broken: apply automation registered unconditionally (Program.cs:623-672), /api/runs* exposed via UnlockOrdinaryDataRoutes=true (ops/cloud-run/env-ladder.json:44,104), Chromium in the image but no Xvfb, the driver still requesting headed, and the UI telling cloud users "Opening the automation browser in its own window…" (live-run.html:153-167) with no edition-conditional rendering anywhere in frontend/src/app/features/live-run/.

In flight: sol (gpt-5.6-sol, xhigh) on WING-461 + WING-289 in an ISOLATED worktree C:\Users\fives\source\repos\_wt\WT-a37c-461 (detached at 775644481) so it cannot collide with the main lane. WING-461 BLOCKS WING-422 on the board. WING-462 is the CI dependency gate, so a code claim and its Jira ticket can never disagree silently again.