Skip to content

FLT-178 — headless console enforcement: findings

Investigation and rollout, 2026-08-05/06. Written up because the diagnosis overturned two prior theories and the evidence should not live only in a transcript.

Root cause (measured, not inferred)

A scheduled task with LogonType=Interactive runs in the user's session, so a console-subsystem executable draws a real window on the desktop.

Settings.Hidden does not prevent this. Per Microsoft's TaskSettings.Hidden documentation it only hides the task's row in the Task Scheduler UI. Every offending fleet task already had Hidden=$true and showed a console anyway.

Control group on greatroom, same machine, same settings, opposite outcome:

Task LogonType Visible console?
UsageGovernorRaftAuthority S4U no
UsageGovernorAdmitAuthority Interactive yes
FleetDispatchServe Interactive yes

Theories that were wrong

Both were mine, both were killed by measurement rather than argument.

  1. "The .NET toolchain grandchildren are flashing." A 120 s trace covering a --no-incremental build plus a 582-test run caught zero visible windows from dotnet, MSBuild worker nodes, vstest, or git. The Set-HeadlessConsoles.ps1 env vars were already doing their job. A PATH shim would have fixed nothing.
  2. "The Claude harness spawns unhidden children." app.asar already contains windowsHide, conhost --headless and bundles node-pty; no harness-spawned window appeared in the trace. The only powershell.exe with a visible window was parented by Andrew's own Windows Terminal.

The instrument was wrong first

v1 of the detector counted conhost.exe launches lacking --headless and reported 74 "flashes". That heuristic is invalid: a process spawned with CREATE_NO_WINDOW still gets a conhost.exe, it simply has no visible window. v1 was counting ordinary hidden background jobs.

v2 polls every 15 ms for top-level ConsoleWindowClass windows where IsWindowVisible() is true — what a human actually sees. It found 7, of which 2 were fleet services and the rest were Andrew's own terminal.

Lesson: a detector that has never been shown a known-bad case is not evidence. See the negative control below.

What S4U costs, and why it was acceptable here

S4U (Kerberos S4U2Self) mints an identity token with no credential material. The token carries the right SID so local ACL checks pass, but the session has no key for:

  • network identity (no TGT / NTLM secret)
  • the user's DPAPI master key, therefore no Windows Credential Manager

Probed on greatroom before converting anything:

  • GH_TOKEN is a plain User environment variable, readable under S4U
  • git ls-remote returned a live SHA with exit=0 from an S4U logon
  • gh auth status reported the Credential Manager account's token was already invalid — GH_TOKEN had been carrying all git auth for some time
  • ssh reached all three peers (exit=0); keys unencrypted on disk, ssh-agent stopped
  • Docker responded fully (server section + 8 containers) from session 0

DPAPI also succeeded, but that result is not trustworthy: Andrew was logged in, so LSASS almost certainly had the master key cached. Do not build on DPAPI under S4U.

Negative control

A checker that returns green on a healthy machine proves nothing — the previous checker did that too, which is why it survived. So the rewritten Check-HeadlessConsoles.ps1 was tested against a deliberately broken machine:

inject:  UsageGovernorScan -> Interactive
checker: NOT COMPLIANT - 1 violation(s)
         UsageGovernorScan is LogonType=Interactive
CHECKER EXIT ON BROKEN STATE: 1
restore: UsageGovernorScan -> S4U

The old checker returned NONE for that exact state.

Incidents caused during rollout

Recorded because they were self-inflicted and the lessons generalise.

  1. FMC API taken down on streaming. A stale instance held fmc-api.log open, blocking restart. Killing it revealed the binary could no longer launch at all: Device Guard/WDAC blocks the self-contained FleetMissionControl.Api.exe. It had survived since 2026-07-19 only because it was loaded before the policy took effect. Repaired by repointing the launcher at the dotnet.exe + managed-DLL form that bedroom already uses (original saved .bak-flt178). Lesson: before killing a long-lived process, confirm its binary still launches.
  2. A stray unelevated FleetDispatch.exe (Explorer-parented, no --host, running since 2026-08-04) had been serving in place of the properly registered elevated task, which had never run. That is why the first elevation attempt failed with Access is denied.

Open fleet bugs found on the way

  • The governor override never reaches fleet-dispatch. GovernorGate.ReadTier reads only state.json's tier field; UsageGovernor.exe override writes a separate override.json. So a logged override — the documented way to unblock work — leaves every non-critical dispatched job deferred with deferred (governor RED). Only --critical bypasses it.
  • FMC launchers are inconsistent across machines and one style is WDAC-blocked; they also append to a fixed log path with no lock handling, so a stale instance silently blocks restarts and reports a misleading "file is being used by another process".
  • Dispatch can wedge while healthy: the listener binds and status works, but HTTP send stops being answered. Bound is not the same as serving.
  • Rejected jobs record no reason — the archive/rejected payloads carry no reason field, so the why has to be inferred from jobClass.

Consequence of this change: a recovery deadlock (IMPORTANT)

Converting a service to S4U + RunLevel Highest puts it in session 0 under an elevated token. An unelevated interactive session then cannot stop or restart it:

schtasks /End /TN FleetDispatchServe   ->  "SUCCESS: ... terminated successfully"
Get-Process FleetDispatch              ->  still alive (pid unchanged)
Stop-Process -Id <pid> -Force          ->  Access is denied

schtasks /End and Stop-ScheduledTask report success and do nothing. This matters because the fleet's only no-UAC elevation path is the dispatch listener itself — so when that service wedges, there is no way to recover it without an elevated shell, which is exactly what the fleet is set up to avoid needing.

Mitigation to add (needs one elevated action to install): register a small FleetRestartDispatch task at RunLevel Highest whose action stops and restarts FleetDispatchServe. Any unelevated session can Start-ScheduledTask a pre-registered Highest task, so that restores a no-UAC recovery path. Until that exists, a wedged elevated fleet service requires Andrew to elevate manually.

This is a real cost of the conversion and was not anticipated in the design. It does not outweigh the fix — the consoles are gone fleet-wide — but it needs the mitigation before the pattern is applied to any further services.