Skip to content

WorkWingman — Testing (Plain Language)

How we know WorkWingman works — and how we make sure it keeps working every time we change something.

The smoke-alarm idea

Think of our tests as smoke alarms wired through the whole app. Before we make any change, we press the "test" button on all of them at once. If every alarm stays quiet, the change is safe. If even one starts beeping, we stop and fix it before anything ships.

The point of a smoke alarm isn't to be pretty — it's to go off at exactly the right moment. So we hold our tests to two simple standards:

  • They have to be honest. A test either passes every single time, or something is genuinely wrong. We never let a test "pass if you run it twice." A flaky alarm is a broken alarm.
  • They test what the app does, not how it's wired inside. We check the actual behavior a person would care about — the right thing shows up on screen, the right answer comes back, a locked vault stays locked. That way we can tidy up the plumbing without setting off false alarms.

What's tested today, in plain terms

Right now there are over 140 automatic checks — 91 on the behind-the-scenes engine (the "backend"), 51 on the screens you actually see (the "frontend"), plus a set that click through the real app in a browser and one that launches the actual desktop app. Here's what they watch over:

  • "Try again when the internet hiccups — but don't when it's pointless." When a website is briefly overloaded or the connection drops, the app quietly retries. When the real problem is "you're not logged in" or "that page doesn't exist," it doesn't waste time retrying. Tests confirm it tells those situations apart correctly.

  • "Recognize which job site we're dealing with." Big employers use application systems like Workday, Greenhouse, Lever, and iCIMS. Tests confirm the app can spot each one from a job posting — and pull out exactly which company's Workday it is.

  • "Keep the password vault shut until it's unlocked." Tests confirm that a locked vault flatly refuses to hand out anything, that unlocking and adding a login works and files it correctly, and that asking for a login that doesn't exist gives a clean "not found" instead of misbehaving.

  • "Answer the app's own internal questions — without ever leaking secrets." Tests start the app up for real (in memory, so it's fast) and confirm the normal screens load fine, the vault never spills passwords over the internal connection, and the "install a tool" feature refuses a hostile request but works from the real app.

  • "Check that the tools it needs are present and healthy." On startup the app makes sure the handful of programs it relies on are installed and new enough. Tests confirm it reads version numbers correctly, doesn't crash on a tool it doesn't recognize, lets you launch when only optional extras are missing, and stops you when something required is missing.

  • "Every screen loads and behaves." There's now a test for each of the 10 screens and the core services — the job queue, the "we're not sure, please choose" pop-up, the light/dark theme switch, and more.

  • "Click through the real app like a person." A set of tests opens the actual app in a browser and walks through it — moving between screens, flipping the theme and checking it sticks, and going through the pause-and-ask moment where the app stops to get your answer.

  • "Launch the real desktop app." A test starts the packaged desktop app for real, confirms the window opens and the screens load, and that it can talk to its own engine.

  • "Read job-page clues correctly, and know where layoff facts really came from." Tests feed in sample LinkedIn job-page text and confirm the repost/original-date reader picks up exactly what's there — and stays quiet instead of guessing when it's not. Other tests stand in for the government layoff-notice feed, the SEC filing search, and the tech-layoff tracker, and confirm WorkWingman merges what they say, keeps the most trustworthy answer when two sources disagree, and never trips up just because one source is offline. See job-signals.md for what these two clues actually are.

  • "Deliberately break things to prove the alarms ring." This is the clever part. We don't just have alarms — we test them. A tool quietly introduces small bugs into the engine on purpose, then checks a test notices and beeps. A bug that slips through unnoticed means an alarm is missing. It's the difference between owning a smoke detector and holding a match under it to be sure. (This runs once a week — it's thorough but slow.)

  • "Run everything five times to catch the flaky ones." Every change re-runs the suites several times over; any test that can't make up its mind is treated as a bug to fix, never shrugged off.

Every one of these runs automatically whenever we make a change (the weekly "break things on purpose" check aside), so a mistake gets caught in minutes.

What deeper testing is still planned

Not done yet — the plan, not today's reality:

  • Cover every last corner, including the parts that install other software and drive a real browser — those need a different kind of test (a real sandbox), which is still to come.
  • The break-things-on-purpose check for the screens. It works on the engine today; doing it for the frontend screens needs an extra piece of setup that Angular doesn't provide yet, so it's deliberately held back rather than shipped half-working.
  • Even wider walk-throughs covering the full start-to-finish journey across all screens.

Testing several versions without several times the work

Plain-English companion to the same idea in repo-topology-multi-target.

This describes how it will work, not how it works today. The single build-and-share process is planned, not built. Right now each version is still built on its own, so each one still has to be tested on its own.

WorkWingman ships in several shapes — cloud, Windows, and soon Mac, Linux, and phone. If each one were built separately, each could go wrong in its own way, and every one of those would have to be found separately by a person holding that particular machine. That cost grows every time a version is added.

So everything is built from one snapshot of the code, and every version is assembled from those same finished pieces. The point is not that this prevents bugs — it doesn't. It's that every version gets the same bugs. A bug that behaves identically everywhere gets found once and fixed once, for everyone.

That is what makes a test result travel: "it passed on the test instance" tells you something real about the shared behavior of the Mac app, because it is provably made from the same pieces. It says nothing about whether that app installs cleanly or whether macOS trusts it — which is why the list below still needs a person and a real machine.

Right now, every version needs its own full check. The shared build does not exist yet, so nothing tested on one version can be assumed about another. The list below is what stays per-device after the shared build ships — not what is enough today.

What still needs a real device and a real person:

  • installing and updating on each desktop operating system
  • whether the computer trusts the app on a machine that has never seen it before
  • the parts unique to that shell — how the phone finds the cloud, the share sheet, screen streaming
  • how it fits into the system: where files go, permission prompts, links that open the app
  • a quick run through the main features on that platform (for the desktop versions and the cloud, which each carry their own copy of the engine; the phone app doesn't, so it gets its own shorter check of the phone-specific parts) — some parts of the app genuinely behave differently per operating system, so "it worked on the test instance" is not enough on its own

That list is deliberately short, and it's meant to stay that way. If it starts growing, it means something shared has leaked into one version — the exact problem this setup exists to prevent, showing up as extra testing work instead of as a bug report.