Skip to content

Security — llm-council

Threat model in one paragraph

llm-council spawns locally-installed AI CLIs (claude, codex, agy, ollama) as child processes and relays a user-supplied question to them. It holds no credentials of its own — each CLI authenticates through its vendor's normal login on this machine. The attack surface is therefore: the local config file (controls what gets executed), the prompt path (what the children receive), and the CI pipeline.

Secrets handling

  • There are no secrets in this repository. council.config.json contains model names, CLI commands, and argv templates — configuration, not credentials. No API keys, tokens, or endpoints requiring auth.
  • Seat CLIs use their own vendor auth stores (Claude Max, ChatGPT/Codex, Google AI Pro, local Ollama). This is the house BYO-credentials pattern: the tool never creates, stores, or transports keys.
  • reports/ (council transcripts) is gitignored: transcripts quote seat output verbatim and could echo sensitive context from a question. They stay local.
  • Test fixtures use obviously-fake values (Bogus-generated names, cmd.exe stubs) that cannot match real secret shapes.

Command execution safety

  • Commands and argv come only from the local council.config.json (explicit --config, LLM_COUNCIL_CONFIG, or a file found walking up from the exe/cwd). Nothing executable is derived from network input.
  • Children are started with UseShellExecute = false and ArgumentList (no shell, no string-concatenated command lines, no injection via the question text — the prompt travels through stdin or a single argv entry).
  • Environment-variable expansion applies to the seat command path only (e.g. %LOCALAPPDATA%\agy\bin\agy.exe).
  • Timeouts kill the whole child process tree; a wedged seat cannot wedge the tool.
  • Temp-file answers ({tmpfile}) use fresh GUID-named files in %TEMP%, deleted after reading.

Scan results summary (2026-07-08, pre-commit)

Gate Result
House secret-pattern grep (council_*, Bearer, AKIA*, sk-*, gh?_*, xox*, private keys) clean — zero matches
Codex security review (codex exec, repo root) No secrets, no injection, no unsafe exec. Two supply-chain findings, both fixed same day: semgrep Docker image was tag-mutable → now digest-pinned in ci.yml; NuGet restore could drift → packages.lock.json committed for both projects + dotnet restore --locked-mode in every CI job.
Semgrep (semgrep scan --config auto --error, Docker, digest-pinned — FLT-92) 0 findings locally (117 rules, 24 files). One unsafe-path-combine taint finding was reviewed and suppressed at the sink with an inline justification: the report filename slug is stripped to [a-z0-9-] (covered by tests), so path traversal is impossible. Runs in the CI security job on every push. FLT-92: the runner service account (NETWORK SERVICE) is in docker-users on every home-ci host (council-unanimous; reversible via net localgroup ... /delete); a WSL-native semgrep was rejected — WSL distros are per-Windows-user, and re-pointing the runner identity scored worse (credential exposure / unsupported service-WSL). The CI step preflights docker info and null-guards so an unreachable engine fails closed and honest. Long-term: container CI moves to a Linux runner (FLT-99).

CI supply chain

  • actions/checkout pinned to a full commit SHA (mutable tags blocked by policy and by semgrep's rule).
  • Jobs run on a self-hosted, non-ephemeral Windows runner → the repo stays private and untrusted PR code must not run here.
  • Stryker.NET is a repo-local dotnet tool with a pinned version in .config/dotnet-tools.json.
  • Residual risk (FLT-92): NT AUTHORITY\NETWORK SERVICE is a member of the local docker-users group on every home-ci runner host so the CI security job can reach the Docker engine. Anything else running as that account gains the same Docker API access — on Docker Desktop that is an effective host-escape path, regardless of which account holds it. Revert with net localgroup docker-users "NT AUTHORITY\NETWORK SERVICE" /delete. The grant is retired when container CI moves to a Linux runner (FLT-99).
  • The test suite is hermetic (fake runner + cmd.exe stubs): CI never invokes real AI CLIs. Network use beyond package restore is limited to the security job, which pulls the digest-pinned semgrep image and fetches rules at runtime via --config auto (the digest pins the scanner, not the rules — vendoring a policy pack is folded into FLT-99).