How We Got Here — the council journey and verdict¶
This document is the "how we got there" record: the question, the council of three models convened to answer it, their reasoning, and the unanimous verdict that shaped this blueprint. It exists so that six months from now nobody re-litigates a decision that was already argued out.
The question¶
"I have 4 home Windows 11 PCs running Docker Desktop. I want disaster recovery and orchestration for claude-mem (stateful Postgres + Valkey + worker), my GitHub-Actions self-hosted runner pool, and future stateless daemons. Should I stand up Kubernetes across these machines? If not k8s, what?"
There was a genuine pull toward Kubernetes — partly for the capability, partly because k8s is a marketable job skill and running a real cluster at home is good practice. So the council was asked to separate "good for the job/learning" from "good for this production homelab".
The council¶
A three-seat council reviewed the environment independently, then converged:
- Codex (chair for synthesis)
- gpt-oss:120b (heavyweight local reasoning seat)
- qwen3-coder:30b (code/infra-focused local seat)
The unanimous verdict¶
All three seats agreed: full Kubernetes is overkill and fragile for this environment. The concrete reasoning, deduped across seats:
1. The control plane fights the platform¶
These are Windows machines. A Kubernetes/k3s control plane is Linux, so it runs inside WSL2 or lightweight VMs. That adds a layer that can (and on laptops, does) get suspended, lose its clock, or lose its network on sleep — directly under the component that must stay quorate.
2. Wi-Fi + churning DHCP breaks quorum¶
Kubernetes' control plane is backed by etcd, which needs a stable quorum of members reachable by stable addresses. Home Wi-Fi with churning DHCP leases (and one isolated subnet island) means node addresses change and members intermittently can't see each other. That is close to a worst-case network for a raft/etcd quorum. Swarm's raft has the same sensitivity.
3. Kubernetes does not solve the actual hard problem¶
The genuinely hard, DR-critical thing here is stateful Postgres data. Kubernetes schedules pods; it does not replicate database data. With local disks only (no SAN/NAS/replicated PV), a rescheduled Postgres pod cannot bring its data along. So the marquee k8s benefit — reschedule a failed workload onto another node — doesn't even apply to the one workload that matters. Postgres DR still has to be solved deliberately at the database layer, k8s or not.
4. Heavy ops burden for a 4-node home setup¶
Certificates, etcd backups, upgrades, CNI, ingress, storage classes, RBAC — the day-2 operational surface of real k8s is large. For four home PCs it is a maintenance sink with little payoff.
5. Keep the GitHub-Actions runner OUT of any cluster¶
The self-hosted runner pool is already a set of --ephemeral runners, and
GitHub itself is the HA layer and scheduler — it queues jobs and hands them
to available runners. Wrapping that in k8s would be re-implementing an
orchestrator on top of one that already works. Also, one job needs a Windows
host for DPAPI, which can't run in a Linux pod at all. So the runner stays as
a plain warm pool, outside every cluster considered here.
What the council recommended instead¶
Converged, pragmatic recommendation:
- Fix addressing first. Static DHCP reservations + stable node names (hosts aliases and/or Tailscale MagicDNS). Nothing is stable until this is.
- Unify on Docker Compose in Git. Declarative, diffable, restartable. Health checks + restart policies give single-node self-healing without a control plane to keep quorate.
- Harden the bespoke Postgres DR. Keep the hourly
pg_dumpwarm-standby model, but add a tested promote/failover script, an automated restore-verification step, and an explicit primary-marker so two nodes never both believe they are primary. - Docker Swarm is the only "cluster" worth a later look — and only for stateless services, and only after the network is genuinely stable. Never put Postgres in Swarm.
- k3s only once there are always-on Linux nodes with static IPs. Until then, run k3s purely as a learning lab, not as the ops substrate.
How the verdict maps to this repo¶
| Verdict point | Where it lives |
|---|---|
| Fix addressing first | foundation/ |
| Compose-in-Git baseline | compose/stacks/, compose/scripts/Deploy-ComposeStack.ps1 |
| Hardened Postgres DR (tested promote + restore-verify + primary-marker) | compose/scripts/Backup-PostgresWarmStandby.ps1, Promote-PostgresStandby.ps1, Test-PostgresRestore.ps1, Set-PrimaryMarker.ps1 |
| Swarm for stateless, later | compose/swarm/ |
| Runner stays out of clusters | documented; no runner manifests anywhere |
| k3s as a learning lab, flagged not-prod-here | kubernetes/ (banners throughout) |
| Graduation triggers | docs/when-to-graduate.md |
The full pros/cons per substrate and the RPO/RTO discussion are in
pros-cons.md.