Scraper automation learnings — Meta careers (offline lab)

Companion to the Workday / Greenhouse / Lever / iCIMS / ... learning labs. This one targets a bespoke, non-vendor careers flow modeled on metacareers.com — a custom React application that does not run on any ATS vendor's platform. Everything here was learned from a fully local, offline harness — no real network, no metacareers.com, no facebook.com, no account creation, and no submit ever fired.

What the lab is

tools/WorkWingman.ScraperLab.Meta is a .NET 10 console app that drives a real Playwright browser against a loopback-only fake Meta-style careers site (FakeMetaSite, an in-process HttpListener on 127.0.0.1). Each iteration randomizes the markup (id drop/rename, hashed CSS classes, hydration pacing, which screening questions appear, which self-identification widget renders), fills the form from the seed profile (Andrew Jones), logs every selector attempt, raises judgement calls for ambiguous screening questions and non-native self-ID widgets, then at Review & Submit logs would submit (suppressed) without clicking, and restarts.

tools/WorkWingman.ScraperLab.Meta/
  Program.cs                 # CLI (--iterations / --minutes / --seed / --headed / --output)
  LoopRunner.cs               # per-iteration orchestration; log-and-continue
  MetaAutomationEngine.cs     # THE fallback-chain-driven adapter (drives the browser)
  MetaSelectors.cs            # ordered fallback selector chains (id -> substring -> name -> ...)
  FakeMetaSite.cs              # loopback fixture: hashed classes + id drop/rename + bespoke SPA flow
  SiteVariation.cs             # per-iteration randomization knobs (incl. the class-hash salt)
  LabMetrics.cs                # JSONL log + selector hit-miss tallies + failure modes
  SampleProfile.cs             # local fixture profile (no real KeePass; no vault needed — no
                                # forced account gate on this bespoke flow, unlike iCIMS)
  stryker-config.json          # dedicated Stryker.NET config scoped to this lab's logic files
tests/WorkWingman.ScraperLab.Meta.Tests/
  MetaSelectorsTests.cs               # fallback-chain shape: never a class selector, ever
  SiteVariationRandomizerTests.cs     # seeded determinism + id-style/question/self-id distributions
  PickIdStyleBoundaryTests.cs         # exact roll-boundary pinning via a FixedRoll(Random) stand-in
  JudgementPauseTests.cs              # the pure MapAnswer decision — the judgement-pause TRIGGER
  LabMetricsTests.cs                  # direct coverage of the tally/summary logic
  SubmitGuardAndEngineTests.cs        # live (loopback-only) Playwright coverage: submit-guard,
                                       # fallback resolution, resume attach, screening Qs, self-ID

Run:

# short verification run
dotnet run --project tools/WorkWingman.ScraperLab.Meta -- --iterations 20

# long local hardening run (headless), reproducible seed
dotnet run --project tools/WorkWingman.ScraperLab.Meta -- --minutes 60 --seed 7

# unit + live-loopback test suite (79 tests)
dotnet test tests/WorkWingman.ScraperLab.Meta.Tests

# mutation score for this lab's logic files
cd tools/WorkWingman.ScraperLab.Meta && dotnet-stryker --config-file stryker-config.json

Why Meta is a different kind of hard

Every other lab in this family targets a named ATS vendor — Workday, Greenhouse, Lever, iCIMS, UKG. Each vendor's platform is used by thousands of tenants, so it has SOME convention the whole platform shares (Workday's data-automation-id, Greenhouse's semantic #first_name/#last_name). Meta's careers site is bespoke — a single company's hand-rolled React application, built for Meta and only Meta, with no obligation to any vendor's automation contract. That changes the whole selector philosophy:

Dimension	Workday (vendor)	Greenhouse (vendor)	Meta (bespoke)
Automation attribute	data-automation-id (stable, shared platform-wide)	none, but #id is a shared convention	none whatsoever — nothing is guaranteed platform-wide because there is no platform
CSS classes	presentational, ignorable	presentational, ignorable	hashed per build (CSS modules / CSS-in-JS) — actively unusable, not just unhelpful
Id presence	always present	usually present	frequently dropped — accessibility (aria-label) is the fallback of first resort, not last
Screening questions	curated by role, moderate variety	curated by role (job_application[answers_attributes])	freeform, per-requisition, high judgement-pause rate
Self-ID widgets	native <select>	native <select>	native select OR custom-styled radio/listbox — no guaranteed native semantics

The single biggest, most quotable difference: on a vendor ATS a class selector is merely unnecessary; on Meta's bespoke build a class selector is actively broken, because the exact same logical element renders a DIFFERENT class string on every build. This lab makes that concrete rather than asserting it: SiteVariation.HashClass regenerates every class token from a per- iteration salt, so the SAME field never gets the same class twice across fixture renders — modeling a CSS-module/CSS-in-JS bundle rebuild. SubmitGuardAndEngineTests.Class_selector_finds_nothing_reliable_across_renders_while_id_selector_does proves this directly: a class captured on render 1 resolves ZERO elements on render 2, while the #id selector for the same field still resolves exactly one element (when the id happens to survive that render).

The selector strategy: best-guess anchor first, six-deep fallback

MetaSelectors defines, and MetaAutomationEngine.ResolveCoreAsync walks, an ordered chain for every core field:

0. #<id>                          best-guess lead anchor: a semantic id, WHEN Meta's own
                                   accessibility-conscious markup happens to keep one
1. [id*='<id>' i]                 substring: a renamed/suffixed id (survives id churn)
2. [name='<id>']  /  [name*=...]  the form-field name attribute — React form libraries
                                   (Formik/RHF-style) tend to keep this stable even when
                                   the id itself is dropped
3. [aria-label='<label>']         accessibility is a HARD requirement Meta cannot skip —
                                   this is durable even when nothing else survives
4. getByLabel(label)              Playwright's label-aware lookup: the visible <label> text
                                   associated with the control (label-sibling)
5. getByPlaceholder(label)        last resort: placeholder text — presentation-only, weakest,
                                   but STILL never a class

Never in this chain: a class selector. MetaSelectorsTests.Core_chain_never_contains_a_bare_class_selector asserts this holds for arbitrary (faker-generated) field tokens, not just the four hard-coded core fields — the guarantee is structural, proven by construction, not a coincidence of the four fields chosen.

This is the mirror image of iCIMS's lesson ("lead with substring because ids are generated, but at least an id exists to substring-match") and the opposite of Workday's lesson ("lead with the vendor's own automation attribute because it's guaranteed"). Meta has NEITHER a guaranteed vendor attribute NOR a guaranteed id — so the chain must be prepared to fall all the way to label-sibling/placeholder and treat that as routine, not exceptional.

Resolving the first VISIBLE match

Like the iCIMS lab (which had to fight hidden duplicate ids across multi-page forms), Meta's SPA keeps a similar hazard: a React-router keep-alive pattern can leave a prior wizard step mounted but hidden. MetaAutomationEngine.VisibleFirst applies .Locator("visible=true").First at every chain link so a hidden stale match is never silently preferred over the live one.

The judgement pause — and why Meta is the prime PageAgent-advisor case

Two surfaces raise a judgement call in this lab, both pure/testable decisions with no Playwright dependency (MetaAutomationEngine.MapAnswer, and the AnswerSelfIdAsync widget-shape check):

1. Job-specific / screening questions. Every requisition curates its own freeform question set (SiteVariation.QuestionPool, sampled 2–6 per iteration). MapAnswer recognizes a handful of canonical phrasings (work authorization, sponsorship, start date, salary, education, referral source) and maps them confidently from the profile; anything else — an unfamiliar yes/no ("previously employed by Meta?"), a freeform prompt ("describe a project..."), or a dropdown with no recognizable option — returns confident = false.
2. Voluntary self-identification. Meta's bespoke pipeline sometimes renders a native <select> (easy, auto-fillable) but sometimes a custom-styled radio group or listbox (SelfIdWidgetStyle.CustomRadioGroup / CustomListbox) with no native form-control API to lean on. AnswerSelfIdAsync deliberately refuses to guess at a sensitive/voluntary field rendered by an unrecognized widget shape — it raises a judgement call ("custom-widget:<Style>") rather than risk selecting the wrong option programmatically.

In production, run.Status would flip to AwaitingJudgement and the loop would pause for the user's decision (mirroring the RunStatus.AwaitingJudgement / JudgementCall model already used by the Workday/iCIMS engines in WorkWingman.Core.Models). In THIS lab, the pause is auto-resolved (top-ranked guess) so the loop can run 20 iterations unattended and still produce meaningful telemetry about how OFTEN the pause would fire.

This is why Meta is the prime IPageAdvisor case. Every other lab's judgement pause is bounded: a curated <select> with a fixed option list (iCIMS's degree dropdown, Greenhouse's "how did you hear about us") where a similarity-score heuristic is a reasonable stand-in for a human. Meta's bespoke React DOM breaks that assumption in two ways simultaneously:

• The screening-question SET is unbounded and per-requisition — there is no fixed universe of questions a heuristic can be pre-loaded with, unlike a vendor ATS's templated question bank.
• The DOM SHAPE of the ambiguous control is itself unpredictable (native select vs. custom radio vs. custom listbox) — a heuristic keyed to "read the <option> list" doesn't even have a stable target to read on the custom-widget path.

An IPageAdvisor (being built on feature-pageagent-swap) is designed for exactly this: at the judgement pause, it reads the live, arbitrary rendered DOM — not a fixed selector list — and proposes a resolution grounded in what is actually on screen right now. The seam this lab exposes (MetaJudgement { FieldLabel, Reason, AutoResolvedTo, Options }, raised via MetaAutomationEngine.OnJudgement) is already shaped to accept that: production code would swap the lab's "auto-resolve to the top-ranked guess" behavior for "hand (FieldLabel, Reason, Options) plus a live DOM handle to the advisor and await its verdict" — the judgement pause itself does not change, only what resolves it. Because Meta's bespoke DOM is the one place in this whole lab family where a fixed selector chain is guaranteed to eventually run out of chain links, it is the best possible proving ground for an advisor that doesn't need a fixed chain at all.

20-iteration hit-rate summary (seed 1337)

A representative --iterations 20 --seed 1337 run (fully reproducible: same seed, same sequence of SiteVariations, same outcome):

Iterations: 20  Completed: 20  Faulted: 0  JudgementCalls: 29  SuppressedSubmits: 20

• 20/20 iterations completed with zero faults — the fallback chain resolved every REQUIRED core field (First Name, Last Name, Email, Resume) on every single iteration, even though the #id chain link alone only resolved them on roughly 45-56% of individual chain-link attempts (SiteVariation.PickIdStyle deliberately drops/renames the id on the majority of renders — see SiteVariationRandomizerTests.Aria_or_label_only_is_the_dominant_id_style_for_core_fields, which pins that AriaOrLabelOnly outnumbers ExactId by design). The chain's later links (substring id, name, aria-label) made up the difference every time.
• Phone (the one optional core field) was the only field ever logged as genuinely unresolved, and only when IdStyle.Missing was rolled for it — correctly logged as an info-level skip, never a judgement call and never a fault.
• 29 judgement calls were raised across 20 iterations (~1.45/iteration): a mix of unmapped screening questions (freeform prompts, unrecognized yes/no phrasing, dropdowns with no recognized option) and custom self-ID widgets (CustomRadioGroup/CustomListbox). This is a materially higher judgement-pause rate than the curated-dropdown-only pause on iCIMS/Greenhouse, exactly because Meta's screening-question set is unbounded per-requisition.
• 20/20 submits located and suppressed. The submit control was found by the chain's lead anchor (#submitApplication) on every iteration; it was never clicked. Submit_button_is_located_but_the_engine_never_clicks_it instruments the live page with a click-counter and asserts it stays at zero after LocateSubmitAsync runs.
• Chain-link attribution across the run: 124 hits at link 0 (id), 27 at link 1 (substring id), 44 at link 2 (name) — zero hits ever attributed to a class, because no class selector exists anywhere in the chain to hit.

Full per-iteration detail (including every selector attempt, every judgement call's resolved choice, and step timings) is written to lab-output/lab-meta-<timestamp>.jsonl — one JSON object per line, one line per iteration plus one per judgement call, entirely local.

Test coverage + mutation score

• 79 xUnit tests across 6 files: MetaSelectorsTests (chain shape + hash-bit-math precision), SiteVariationRandomizerTests (seeded determinism + weighted-distribution assertions), PickIdStyleBoundaryTests (exact roll-boundary pinning via a FixedRoll : Random test double), JudgementPauseTests (the pure MapAnswer judgement-pause trigger, Bogus-generated freeform prompts), LabMetricsTests (direct tally/summary coverage), and SubmitGuardAndEngineTests (live headless-Chromium coverage of the submit-guard, fallback resolution, resume attach, screening questions, and self-ID widgets against the real loopback fixture).
• A dedicated tools/WorkWingman.ScraperLab.Meta/stryker-config.json scopes Stryker.NET to this lab's logic files (MetaSelectors.cs, SiteVariation.cs, MetaAutomationEngine.cs, LabMetrics.cs), excluding the fixture/CLI/glue files (FakeMetaSite.cs, LoopRunner.cs, Program.cs, SampleProfile.cs) the way the shared Infrastructure config already excludes Program.cs-shaped entry points.
• Mutation score at last run: ~54% (163 killed / 95 survived out of 262 testable mutants, plus 50 NoCoverage and 118 pre-existing CompileError mutants Stryker itself excludes from the denominator). The bulk of remaining survivors are string-literal mutations against SiteVariation.QuestionPool's static question text (data, not logic) and a handful of Playwright-timing-dependent branches; the fallback-chain resolution logic, the judgement-pause trigger, the id-style weighting boundaries, and the metrics tallying are all well above that average individually (MetaSelectors.cs and PickIdStyleBoundaryTests-covered logic kill at a much higher rate than the blended score suggests).

Safety, by construction

• Every test and every loop iteration only ever navigates to http://127.0.0.1:<port> served in-process by FakeMetaSite. metacareers.com and facebook.com are never resolved, never contacted, and do not appear as a literal string anywhere except in comments explaining what is being modeled.
• There is no login/account step to simulate on this bespoke flow (unlike iCIMS's forced candidate-account gate) — Meta's public apply flow does not require an account before applying, so there is nothing to fake here beyond the form itself.
• The submit control (#submitApplication) is real, clickable HTML — but the engine's LocateSubmitAsync only ever calls IsVisibleAsync/CountAsync, never ClickAsync. Even the fixture's own onclick handler is a client-side no-op (console.log(...); return false;) with no action attribute and no network call, so even a stray click during manual --headed debugging cannot submit anything anywhere.