Skip to content

Value Metrics

The Value Metrics section tracks what WorkWingman actually did during the job hunt — and what it honestly saved — so the owner's own hunt becomes the product's marketing proof. The design came out of an LLM-council deliberation (2026-07-10, five frontier seats; consensus synthesis) and its honesty rules are load-bearing: every number must survive a skeptical buyer's audit.

Architecture

flowchart LR
    subgraph Emitters [server-side emitters]
        A[ApplicationRunService\napply.session.started] --> L
        B[PlaywrightApplyRunDriver\nparked / resumed / readyForReview / failed] --> L
        C[JobQueueService\nscrape.session.completed, fit.analyzed] --> L
        D[ApplicationTrackerService\napply.submitted, application.status.changed] --> L
    end
    E[Angular Study page\nstudy.linkOpened via POST /api/metrics/events] --> L
    L[(MetricsEventLog\nmetrics-events-YYYY-MM.json\nappend-only facts)] --> R[ValueMetricsService\nrollup on demand]
    T[(metrics-baselines.json\neditable per-ATS ranges)] --> R
    R --> API[GET /api/metrics/summary] --> UI[Value metrics dashboard]
  • IMetricsEventLog / MetricsEventLog — append-only event log in monthly LocalJsonStore shards (metrics-events-2026-07). Two contracts: append never throws (a metrics failure must never break the feature that emitted it), and events are immutable facts — durations, outcomes, counts. Never derived values.
  • IValueMetricsService / ValueMetricsService — recomputes the whole summary from raw events on every call, against the baseline table. Revise a baseline → every historical "saved" number updates with it. Nothing derived is ever persisted.
  • MetricsController — GET api/metrics/summary, GET/POST api/metrics/baselines[/manual], and POST api/metrics/events which only accepts the client-writable whitelist (study.*, baseline.manualTimed) — a client can't fabricate an apply session.

Event taxonomy (prefix.object.verb)

Type Emitted by Key props
scrape.session.completed JobQueueService resync scrapedCount, addedCount
fit.analyzed JobQueueService enrichment ats, fitLevel, matchedCount, gapCount, adjacentCount
apply.session.started ApplicationRunService ats, company, title, mode
apply.session.parked / resumed Driver, at the resume gate —
apply.session.readyForReview Driver, after the drive rawMs, outcome, fieldsFilled, resumeAttached
apply.session.failed Driver rawMs, outcome=failed
apply.submitted ApplicationTrackerService ats, company, title
application.status.changed ApplicationTrackerService status
study.linkOpened Study page (client) provider, url
study.session.started / stopped client (opt-in timer) —
study.selfReport client minutes (flagged, own series)
baseline.manualTimed ValueMetricsService ats, minutes
document.generated / document.adopted reserved — drafter services (server-emitted; TROI outcome linkage) —

Perverse-incentive audit (Kerr, "On the Folly of Rewarding A, While Hoping for B")

A second council sitting (2026-07-10, five seats) audited the metric set against Steven Kerr's classic: reward systems fixate on objective, visible, easily-measured behaviors (A) while the actual hoped-for outcome (B — a good job landed via quality applications) goes unrewarded. The audit found the original headline layer rewarded throughput: apps-per-hour paid the user to skip the app's own fit analysis, time-saved banked "savings" for applications that should never be sent, and click counts rewarded clicking. The adopted guardrails:

  1. Qualified gating. A qualified application = user-confirmed submission whose fit analysis rated the job Good/TopApplicant (from existing fit.analyzed events — zero new telemetry). The hero count and ALL headline savings are qualified-only.
  2. Savings follow quality. Unqualified or unsubmitted drives earn zero savings and keep their cost; gross (ungated) savings appear only inside the methodology popover — the gap between gross and qualified is itself a spray warning.
  3. Pairing rule. Throughput never renders alone: the hero row always carries response rate, interviews, and offers; spam-punishing denominators (active minutes per qualified submission, responses per 10 qualified) ride along.
  4. Selectivity is success. Fit-analyzed jobs deliberately not pursued render as healthy selectivity, not funnel drop-off — deciding not to apply is the highest-value invisible behavior. Offers are a first-class funnel stage.
  5. Receipts show fit. Every receipt row carries its fit band, so the user's own spray is visible in their own receipts — self-correcting, no surveillance.
  6. Study: timed leads, clicks captioned. "Opened, not studied"; follow-through (sessions per opened resource) shown; raw clicks never headline.
  7. Dashboard copy names the trap: "Faster apply is not better apply."

Kerr root cause worth remembering (named by one seat, adopted): these metrics are dual-use — owner dashboard AND marketing proof — which biases selection toward flattering, defensible volume aggregates. Screen every future metric against that bias.

Honesty rules (why the numbers are defensible)

  1. Active time ≠ wall-clock. WorkWingman's drive parks at mandatory user pauses (account walls, judgement calls) and can sit parked overnight. parked/resumed events bracket every park at the driver's resume gate; the rollup subtracts parked intervals (an unmatched park is clamped at the terminal event). An overnight pause can never inflate "time saved".
  2. Savings are computed, never stored. saved = max(0, baseline(ats) − activeMinutes) per application, shown as a low–high range. The low bound is the headline. Failed runs save nothing and still show their cost.
  3. Baselines calibrate from the owner's own hunt. Shipped ranges (Greenhouse 8–15, Workday 20–30, iCIMS 12–22 min) are labeled uncalibrated estimate until the owner self-times 3 real manual applies per ATS (recorded as baseline.manualTimed events). "Measured against my own N manual applications" is the marketing line worth quoting.
  4. Submission is the user's act. The automation only ever reaches Review & Submit (it never clicks submit); apply.submitted is the user's confirmation via the tracker.
  5. Study clicks are intent, never hours. Deep-link clicks, opt-in timer minutes, and self-reported minutes are three separate series that are never merged; self-report is excluded from marketing-facing numbers. No browser history, no window tracking, no idle detection — only clicks made inside the app are counted.
  6. Every headline traces to source events. The dashboard's receipts table is the audit trail; nothing on the page exists that can't be recomputed from the local log.

Dashboard (/metrics)

Headline tiles (applications submitted · automation active time · est. time saved range with a methodology popover · response rate) → funnel → per-application receipts table (screenshot-ready) → per-ATS median-vs-baseline breakdown + study panel → baseline calibration form. Everything computes locally; nothing leaves the machine.

Future work

  • study.session.* opt-in timer UI (events + rollup already handle it).
  • Marketing export pack: PII-scrubbed aggregates + methodology.md + baseline-table hash in one ZIP (council-recommended; the summary endpoint already exposes the aggregates).
  • Fit-gap closure metric (gap identified → study action → later matched) once enough fit.analyzed + study history accumulates.