Architecture — llm-council

A single-binary C# CLI (net10.0) that convenes a "council" of AI models by shelling out to their locally-installed, already-authenticated CLIs. No API keys, no network code of its own — every seat is a child process.

Components

graph TD
    subgraph "llm-council.exe"
        P[Program.cs<br/>composition root] --> CL[CommandLine<br/>arg parsing]
        P --> CFG[CouncilConfig<br/>load + walk-up discovery]
        P --> E[CouncilEngine<br/>3-stage orchestration]
        E --> PR[Prompts<br/>stage 1/2/3 templates]
        E --> ICR{{ICliRunner<br/>seam}}
        ICR --> PCR[ProcessCliRunner<br/>child process + timeout + temp-file answers]
        P --> R[Report<br/>markdown + JSON]
    end

    CFGFILE[(council.config.json<br/>seats, tiers, chairman)] --> CFG
    PCR --> CLAUDE[claude -p]
    PCR --> CODEX[codex exec]
    PCR --> AGY[agy -p]
    PCR --> OLLAMA[ollama run]
    R --> REPORTS[(reports/*.md<br/>gitignored)]

    subgraph "Tests (never touch real CLIs)"
        FAKE[FakeCliRunner] -.implements.-> ICR
        STUB[cmd.exe stubs] -.driven by.-> PCR
    end

    subgraph "Front-ends"
        SK1[/skills/llm-council/] --> P
        SK2[/skills/council-code-review/] --> P
    end

Key seam: ICliRunner. CouncilEngine is pure orchestration logic over that interface; ProcessCliRunner is the only place a process is spawned. Unit tests inject FakeCliRunner; the hermetic E2E layer drives the real ProcessCliRunner at cmd.exe builtins.

Seats are entirely config-driven (council.config.json): command, argv template ({model}, {prompt}, {tmpfile} placeholders), prompt transport (stdin vs argv), per-tier model ids and extra args. Adding a seat or surviving a vendor model rename is a JSON edit, no rebuild.

Council run sequence

sequenceDiagram
    participant U as User / skill
    participant P as Program
    participant E as CouncilEngine
    participant S as Seats (claude, codex, agy, ollama)
    participant C as Chairman seat
    participant R as Report

    U->>P: llm-council "question" --tier fable
    P->>E: RunAsync(question)
    par Stage 1 — independent answers
        E->>S: Stage1 prompt (per seat, parallel)
        S-->>E: answers (failures dropped, labels A/B/C…)
    end
    Note over E: answers anonymized;<br/>all-seats-failed → fatal
    par Stage 2 — anonymous peer review
        E->>S: Stage2 prompt (answers w/o author names)
        S-->>E: RANKING + critiques (failures recorded)
    end
    E->>C: Stage3 prompt (deanonymized answers + reviews)
    alt chairman ok
        C-->>E: synthesized verdict
    else chairman missing/crashed
        E-->>E: degrade: keep answers+reviews,<br/>verdict = warning (exit 3 later)
    end
    E-->>P: CouncilRun
    P->>R: WriteMarkdown → reports/, ToJson (--json)
    P-->>U: verdict on stdout, exit 0 / 3

Failure model

Failure	Behavior	Exit
One seat fails / times out / empty output	Excluded; council continues, report notes it	0
Every seat fails	Fatal, no report	1
Chairman missing from config or crashes	Degraded success — all answers + reviews preserved in report/JSON, caller chairs manually	3
Bad usage	Help text	2

Timeouts kill the entire child process tree; a child that never drains stdin (or exits before reading it) is a seat failure, never a hang or a crash of the council itself.