Skip to content

Paycor Scraper Automation Learnings

This lab is intentionally local and offline. It models Paycor-powered candidate application flows with a loopback HttpListener and a Playwright adapter. It never calls paycor.com, Newton, or any real candidate portal. Candidate account creation is simulated with fixture fields only, and the final submit route logs would submit (suppressed) instead of sending data anywhere.

Harness Layout

  • tools/WorkWingman.ScraperLab.Paycor/FakePaycorSite.cs hosts a local candidate self-service portal with account/login, contact info, resume/manual-entry choice, work history, education, questions, voluntary disclosures, review, and suppressed submit.
  • tools/WorkWingman.ScraperLab.Paycor/PaycorAutomationEngine.cs drives the fixture with Playwright and the local Andrew Jones intake profile.
  • tools/WorkWingman.ScraperLab.Paycor/LearningLoopRunner.cs repeats randomized fixture iterations and writes JSONL events, selector tallies, and failure modes.
  • tools/WorkWingman.ScraperLab.Paycor/PaycorDetector.cs mirrors the existing ATS detector pattern for Paycor and Newton markers without changing shared production detector code.

Paycor Markup Patterns To Target

Paycor-style application pages tend to look more like a conventional candidate self-service portal than the strongly componentized Workday pages. The lab therefore targets:

  • Stable id and name attributes first, including prefixed variants such as candidate_firstName, pc_firstName, and applicant_firstName.
  • Human labels and aria-label fallbacks for moderately modern DOMs where visible labels survive but generated IDs change.
  • Required-field variation, because Paycor-powered employer configurations can make phone, education, or disclosure fields optional depending on job and tenant settings.
  • Account/login screens before the application steps, because real candidate portals usually require an account. The lab simulates that locally and records it as a safety event.

Differences From Workday

Workday automation in this repo centers on tenant hosts, Workday-specific automation IDs, and a conservative judgement-call model. Paycor automation needs a broader selector chain because the expected public surface is closer to ordinary HTML forms:

  • Workday often exposes tenant-specific URLs and data-automation-id markers. Paycor-like pages are more likely to use ordinary IDs, names, labels, placeholders, and ARIA.
  • Workday degree matching is a known ambiguous case. Paycor-like forms can omit the degree field entirely or use simpler select options, so the lab logs a judgement call when B.S. cannot be mapped to a presented option.
  • Paycor candidate flows commonly include account/login and candidate self-service language. The harness treats this as a local-only simulated step and never creates an account.
  • Paycor form sections are more linear in this fixture. The adapter still logs each step duration so later real-world-safe research can compare where friction occurs.

Selector Findings From The Fixture

The selector strategy is intentionally ordered:

  1. ID selectors: fastest and most precise when employer configuration keeps canonical IDs.
  2. Name selectors: useful when IDs are prefixed but names stay stable.
  3. Suffix ID/name selectors: useful for generated prefixes such as candidate_, pc_, or applicant_ while still staying on the intended field.
  4. aria-label and placeholder selectors: resilient when accessible field text remains stable.

Randomized fixture iterations stress those chains by changing prefixes, verbose labels, requiredness, and missing degree fields. The output files are:

  • artifacts/paycor-lab/paycor-learnings.jsonl
  • artifacts/paycor-lab/selector-tallies.json
  • artifacts/paycor-lab/failure-modes.txt

Failure Modes And Hardening

Expected failure modes are logged and the loop continues:

  • Missing fields after all fallback selectors miss.
  • Ambiguous fields when a selector matches multiple elements.
  • Unmapped select options, especially education degree normalization.
  • Timing or navigation failures between steps.

Hardening should keep the same safety model: only run against explicit loopback fixtures, keep submit suppressed, prefer deterministic selector chains, and log every automatic judgement. Real Paycor or Newton portals should not be used by this lab.

Long Local Run

dotnet run --project tools/WorkWingman.ScraperLab.Paycor -- --minutes 60 --output artifacts/paycor-lab-long

Use --iterations N for a fixed-size run. Both modes remain local and offline.