The Token Metrics, In Plain English

(Plain-language twin of METRICS-STANDARD.md. Same rules, no jargon.)

What this is

This system measures one thing: did the AI money we spent turn into work that lasted?

It deliberately does NOT measure how many tokens got used. Not because tokens don't cost money — they do — but because counting tokens tells you nothing by itself, in either direction.

The story behind it

A famous 1975 management paper ("On the Folly of Rewarding A, While Hoping for B") describes a trap: organizations reward what's easy to count, then act surprised when people deliver the countable thing instead of the thing that was actually wanted.

Right now companies are falling into this exact trap with AI:

• Some run usage leaderboards — whoever burns the most AI tokens looks the most "AI-forward." One Disney employee racked up 460,000 AI interactions in nine days to climb a leaderboard. Salesforce engineers asked the AI to summarize documents they'd already read, just to hit usage targets. That's not productivity. That's burning fuel to make the gauge move.
• Others do the opposite — reward spending as little as possible. That fails too: one measured study found the cheapest AI model produced work so weak that, after all the review rounds and fixes, each finished feature cost $17 — while a stronger, "more expensive" model got the same feature done for $7.

Judging by token count is like judging a writer by how many pencils they went through. Lots of pencils doesn't mean a good book. Neither does hoarding them.

The one number that matters

Cost per durable ticket — in plain words:

Take every dollar of AI cost that went into a piece of work — including every do-over, every fix, every review round — and divide by the pieces of work that got finished and stayed finished for two weeks.

The "including every do-over" part is the trick. If you cheap out and the work comes back broken, the cost of fixing it lands right back on the same tally. Cutting corners punishes itself. So does burning tokens on work that never ships — spend with no finished work attached just makes the number worse.

The line you see after every prompt

After each response, one line reports what that exchange actually cost:

⛽ $0.19 ET 12.4k (in 8.1k cw 3.2k cr 1.9M out 1.1k) | 5h 34% YEL wk 61% | cache 78% | saved $0.04 meas + ~$0.11 est | WW-42

Reading left to right:

Piece	Plain meaning
⛽ $0.19	What this exchange cost, in dollars.
ET 12.4k	The workload size in "effective tokens" — output text counts 4× because producing words costs far more than reading them.
in / cw / cr / out	The ingredients: fresh input read, notes written to cache, notes re-read from cache (nearly free), and text produced.
5h 34% YEL	How much of the 5-hour usage window is used, and the traffic-light level (green / yellow / red / blackout).
wk 61%	Same idea for the weekly limit. (wk — means we can't measure it yet — we show a dash instead of guessing.)
cache 78%	How much of the reading came from the cheap cache instead of full price. Higher = cheaper.
saved $0.04 meas + ~$0.11 est	Money saved — see the next section for why there are two numbers.
WW-42	The ticket this spend belongs to, when known.

One deliberate absence: the line never gives a prompt a "value score." A single exchange hasn't produced anything finished yet, and scoring it would just invite gaming. Prompts get a price tag, never a grade.

Two kinds of "savings" — receipts vs. estimates

When tools claim they "saved you money," ask: saved compared to what?

• Measured savings (meas) — real receipts. Example: re-reading cached notes costs a tenth of reading fresh; the difference is arithmetic on actual prices, no guessing. Same for using a cheaper model when the work passed all checks on the first try. (If the cheap model failed and a stronger one had to redo it, that counts as negative savings. Fair is fair.)
• Estimated savings (~est) — educated guesses. Example: compressed phrasing probably saved some output, but nobody priced the uncompressed version, so it's a model, not a receipt.

The two numbers are never added together. The moment estimates get mixed into receipts, the "total savings" figure becomes marketing fiction. And some things get no savings credit at all: when the governor blocks work at high usage, we can't know what the blocked work would have cost, so we log that it happened and claim zero dollars.

Things this system refuses to do

1. No rewards for using more AI. Usage volume is a gas gauge, not a scoreboard.
2. No rewards for using less AI. Frugality theater creates expensive rework.
3. No self-grading. No metric where the AI scores the quality of its own output — that's the 1975 trap with extra steps.
4. No blocking work for costing "too much." The quality gate fails a project only when the measuring is broken (telemetry missing, spend that can't be attributed) — never because a number was big. Punishing big numbers just teaches people to hide them.
5. No invented numbers. Every figure is either a receipt, or clearly labeled an estimate, or a dash.

How each number could be cheated — and what stops it

Every metric here ships with its own counter-measure, because any number that matters will eventually get gamed:

Number	The cheat	The counter
Cost per durable ticket	Slice work into confetti tickets so each looks cheap	Median ticket size is watched; reopened work counts against you
Cost per durable ticket	Starve quality to look thrifty	Do-over costs roll back into the same tally automatically
Commit survival	Never clean up old code (protects "surviving lines")	Cleanup commits are tagged and exempt
Cache savings	Stuff the cache to inflate the hit rate	Cost-per-exchange is watched alongside it
Savings totals	Inflate the "compared to what"	Receipts and estimates permanently separated
Low usage %	Avoid AI even when it would help	Finished-work throughput is watched alongside it

If a new metric ever gets added anywhere, it must bring its own row for this table first. No exceptions — an ungameable-looking metric is just one nobody has attacked yet.

The board's phase breakdown (v1.1)

Tickets on the delivery board show where their AI money went across four life stages: making (research and design before work started), working (building it), reviewing (the review rounds), and coming back (anything spent after it was declared done — shown forever, because problems that return are part of the true cost).

The honesty rules carry over:

• The total is frozen the day the ticket is done. If it comes back, the comeback cost sits next to the total — the original number never gets quietly rewritten.
• If the timeline data isn't good enough to split honestly, the board says [unsplit] and shows only the total. A guessed split is worse than no split.
• Review costs show as "at least $X" — some reviewers (other AI models) aren't metered yet, and we won't pretend a floor is a total.
• Research time is never flagged as waste. Thinking hard before building is what prevents expensive comebacks, so the making column can't trigger a warning in either direction.
• Reopening a ticket honestly looks neutral on the board. Quietly fixing something without reopening it looks worse — the system watches for spend that touches closed work with no reopen on record.