Usajobs Gronktayvius

I'll follow the existing ScraperLab pattern and produce a USAJOBS-only traversal lab spec—first locating the house labs and any federal/portal handling.Searching WorkWingman for ScraperLab, AtsKind, USAJOBS, and related patterns.Narrowing to main WorkWingman tree and reading iCIMS lab + AtsKind.## ScraperLab inventory (WorkWingman/)

Labs (tools/): | Project | Path | |---|---| | Workday base | tools/WorkWingman.ScraperLab/ | | Adp, Amazon, Dayforce, Google, Greenhouse, Icims, Lever, Meta, Microsoft, Oracle, Paycom, Paycor, SuccessFactors, Ukg | tools/WorkWingman.ScraperLab.<Ats>/ |

Tests: | Location | Projects | |---|---| | tests/ | ScraperLab.Google.Tests, Icims.Tests, Meta.Tests, Microsoft.Tests | | tools/ | Adp.Tests, Amazon.Tests, Dayforce.Tests, Greenhouse.Tests, Lever.Tests, Oracle.Tests, Paycom.Tests, SuccessFactors.Tests | | nested | tools/...Paycor/WorkWingman.ScraperLab.Paycor.Tests/ |

Typical lab shape: Fake*Site, *AutomationEngine/*Selectors, FixtureVariation/SiteVariation, LoopRunner, LabMetrics/LabProfile/LabVault, Program.cs. Spec: docs/ATS-LABS-TODO.md.

iCIMS lab structure:

tools/WorkWingman.ScraperLab.Icims/     — FakeIcimsSite, IcimsAutomationEngine, FixtureVariation, LoopRunner…
tests/WorkWingman.ScraperLab.Icims.Tests/ — IcimsSelectorAndJudgementTests, IframeSwitchLogicTests
src/.../IcimsSelectors.cs + IcimsApplyEngine.cs  — production (philosophy lives here)

2. iCIMS selector philosophy + test patterns¶

Philosophy (production, not a separate class):

``3:31:WorkWingman/src/WorkWingman.Infrastructure/Automation/IcimsSelectors.cs /// Why iCIMS is the hardest base (learnings from the offline ScraperLab loop): /// - LEGACY + IFRAME-HEAVY. ... #icims_content_iframe ... second iframe (iApply/portal) /// - LEGACY ASP.NET-style ids. ... stable token is a SUFFIX ... substring/ends-with /// - TABLE-BASED layout. ... attribute + label matching /// - FORCED candidate-account step. ... lab SIMULATES ... NEVER creates a real account /// Contrast with Workday: ... exact-id-first wins; on iCIMS, lead with substring/ends-with

**Tests encode philosophy:**


Defs:
- tests/.../IcimsSelectorAndJudgementTests.cs:7 — class doc — substring/ends-with philosophy
- tests/.../IcimsSelectorAndJudgementTests.cs:14 —

Field_chains_lead_with_a_suffix…` — id$= FirstName - tests/.../IcimsSelectorAndJudgementTests.cs:26 — Theory Degree_similarity — BS/BA bridge - tests/.../IcimsSelectorAndJudgementTests.cs:50 — Exact_match_auto_fills — no judgement - tests/.../IcimsSelectorAndJudgementTests.cs:65 — Low_confidence_degree_raises — AwaitingJudgement - tests/.../IframeSwitchLogicTests.cs:13-67 — DecideFrameContext — Top/Outer/Inner/NotFound - tests/.../IframeSwitchLogicTests.cs:60 — Content_iframe_chain_leads — #icims_content_iframe

**Key test methods (verbatim patterns):**
- `Field_chains_lead_with_a_suffix_or_substring_match_not_an_exact_id` — `Assert.StartsWith("input[id$='FirstName']", IcimsSelectors.FirstName[0])`; no `[id='` lead
- `[Theory][InlineData(...)] Degree_similarity_scores_reasonable_matches_higher` — bridge score BS≥BA
- `Low_confidence_degree_raises_a_judgement_call_instead_of_guessing` — `RunStatus.AwaitingJudgement`
- `DecideFrameContext` matrix — inline / outer / double-nested / NotFound / outer-wins

---

## 3. `AtsKind` — no host-class enum

```184:184:WorkWingman/src/WorkWingman.Core/Models/JobPosting.cs
public enum AtsKind { Unknown, Workday, Greenhouse, Lever, Icims, Ukg, Adp, Paycom, Oracle, Dayforce, SuccessFactors, Paycor, Microsoft, Avature, Amazon, Google, SmartRecruiters, Ashby, Workable, Jobvite, Eightfold, RippleHire, SalesforceCareers, Phenom, Other }

No ats|employer|aggregator|portal|blocked host taxonomy. Classification = host-marker detect only:

Site	File:line	Note
`AtsDetector`	`src/.../AtsDetector.cs:5`	HTML + URL host markers
`DetectFromUrl`	`AtsDetector.cs:377`	host-only classify
`ApplyUrlMarkers`	`AtsDetector.cs:118`	per-kind hosts
`RedirectChainAtsResolver`	`src/.../RedirectChainAtsResolver.cs:16`	≤5 redirect hops
`IAtsResolver`	`src/.../IAtsResolver.cs:6`	resolve interface

4. USAJOBS / federal / ApplyURI¶

Path:line	Symbol	Note
`src/.../UsaJobsSource.cs:12`	`UsaJobsSource`	Search API BYO-key
`UsaJobsSource.cs:104`	`ApplyUri`/`PositionUri`	apply URL pick
`UsaJobsSource.cs:137`	`AtsKind.Unknown`	federal not ATS-kind
`UsaJobsSource.cs:304`	`ApplyUri` DTO	JSON list
`src/.../UsaJobsHiringPathChipMapper.cs:9`	chips	vet/mspouse/etc
`src/.../UsaJobsAccountConnection.cs:11`	login.gov hand-off	never types secrets
`src/.../UsaJobsSavedJobsSource.cs`	saved jobs	`www.usajobs.gov`
`IApplyRunDriver.cs:413`	`IsHardBoundaryUrl`	refuse login.gov / USAJOBS apply
`extension/docs/usajobs-capture.md`	capture notes	Agency portal ≠ apply
`docs/technical/ww-78-`, `ww-81-`, `ww82-*`	recon docs	ApplyURI, deep-link only
`tests/.../UsaJobsSourceTests.cs:23`	fixture	`ApplyURI` + `PositionURI`

Excerpt map:

// UsaJobsSource.cs:104
var applyUrl = d.ApplyUri?.FirstOrDefault(...) ?? d.PositionUri ?? string.Empty;
// Ats = Kind.Unknown; ApplyUrl = applyUrl; TenantHost = host of applyUrl
// IApplyRunDriver: login.gov + usajobs.gov /application|/apply → HARD BOUNDARY, advisor refused

5. Traversal / hop / resolve / apply hand-off¶

Path:line	Symbol	Note
`RedirectChainAtsResolver.cs:124`	hop loop max 5	headers-only GET
`RedirectChainAtsResolver.cs:128`	`DetectFromUrl` per hop	accept known ATS host
`AtsDetector.cs:212`	`UnwrapNestedAtsUrl`	LinkedIn `externalApply?url=`
`AtsDetector.cs:403`	nested URL re-detect	wrapper → real ATS
`IAtsResolver.cs:6`	`ResolveAsync`	cache + resolve
`Program.cs:536`	DI wire	`RedirectChainAtsResolver`
`tests/AtsResolutionTests.cs`	hop guards	port/creds/TOCTOU
ScraperLab hand-off	lab vault/sign-in	local account sim only

No dedicated resolve-job-link / multi-ATS job-link crawler beyond redirect-chain + nested-URL unwrap.

6. Posting-id extraction patterns¶

ATS	Pattern	Where
Greenhouse	API `id` → `greenhouse:{boardToken}:{id}`; URL `boards.greenhouse.io/{co}/jobs/{n}/apply`	`GreenhouseBoardSource.cs:113,129`; lab `FakeGreenhouseSite.cs:37`
Workday	tenant host only (`*.wdN.myworkdayjobs.com`); no posting-id extractor	`AtsDetector` TenantHost; lab `fakeco.wd1.myworkdayjobs.com`
iCIMS	real `/jobs/{n}/…` (e.g. `19357`); fake `jobs/apply?in_iframe=1&icims`	validation doc; `FakeIcimsSite.cs:37`
Lever lab	`PostingId` GUID8 → `{slug}/{id}/apply`	`LeverVariation.cs:15`, `FakeLeverSite.cs:38`
Amazon lab	`RequisitionId` → `applications/apply/{id}`	`AmazonVariation.cs`

Selector philosophy contrast: - Workday: data-automation-id first (WorkdaySelectors.cs:7-11) - Greenhouse: #id first (GreenhouseSelectors.cs:4-6) - iCIMS: id$= / id*= first + iframe switch (IcimsSelectors.cs:25-28)

Key location table¶

Defs:
- src/WorkWingman.Core/Models/JobPosting.cs:184 — `AtsKind` — full ATS enum
- src/WorkWingman.Infrastructure/Automation/AtsDetector.cs:5 — host/HTML detect
- src/WorkWingman.Infrastructure/Automation/AtsDetector.cs:377 — `DetectFromUrl`
- src/WorkWingman.Infrastructure/Automation/RedirectChainAtsResolver.cs:16 — hop resolve
- src/WorkWingman.Infrastructure/Automation/IcimsSelectors.cs:3 — SELECTOR PHILOSOPHY
- src/WorkWingman.Infrastructure/Automation/WorkdaySelectors.cs:3 — data-automation-id-first
- src/WorkWingman.Infrastructure/Automation/GreenhouseSelectors.cs:3 — #id-first chains
- src/WorkWingman.Infrastructure/Sources/UsaJobsSource.cs:12 — federal Search+ApplyUri
- src/WorkWingman.Infrastructure/Automation/IApplyRunDriver.cs:413 — federal hard boundary
- tools/WorkWingman.ScraperLab.Icims/* — offline iCIMS lab
- tests/WorkWingman.ScraperLab.Icims.Tests/* — philosophy assertions
- docs/ATS-LABS-TODO.md:1 — lab template + 8 pending ATSes
- docs/reference/icims-real-tenant-validation.md:23 — ends-with vindicated live

Totals: 14 labs + Workday base, 12 test projects, 1 AtsKind enum (no host-class enum), USAJOBS = source+deep-link (not ATS engine), hop resolve = RedirectChainAtsResolver only.Next I'll pull the iCIMS lab test style and the existing USAJOBS source/recon docs so the portal lab matches house patterns.Live job pages resolve; extracting public announcement signals next.# TRAVERSAL LAB — USAJOBS (Portal class)

Scope: USAJOBS only.
Host class: portal — terminal by design.
House posture already aligned: UsaJobsSource maps AtsKind.Unknown; deep-link only; IsHardBoundaryUrl refuses login.gov and USAJOBS /application + /apply paths.
Lab goal: prove the traversal stops cheaply on a USAJOBS host, never burns hops chasing USA Staffing / Monster, and never invents a federal AtsKind value without evidence.

Evidence labels used below:

Tag	Meaning
OBSERVED	Fetched / measured in this session or already in-repo fixtures
HOUSE	Existing production / test code
INFERRED	Reasoned from docs; not treated as verified selectors

0. Provenance snapshot (this session)¶

Fact	Tag	Evidence
Search API is the discovery path; HTML search shell is not	HOUSE + OBSERVED	`UsaJobsSource` uses `data.usajobs.gov/api/Search`; public search HTML returned filter chrome with 0 job hrefs (SPA/shell)
Fixture ApplyURI stays on usajobs.gov	HOUSE	`UsaJobsSourceTests`: `ApplyURI = https://www.usajobs.gov/job/812345678?PostingChannelID=RESTAPI`
Live announcement HTML is publicly fetchable	OBSERVED	`GET https://www.usajobs.gov/job/812345600` → 200, ~124 KB
Control number / announcement number public	OBSERVED	`Control number = 812345600`, `Announcement number = ST-12556984-24-RLW`
Public agency-system signal (USA Staffing) without login	OBSERVED	`a[href^="https://apply.usastaffing.gov/ViewQuestionnaire/"]` → `…/ViewQuestionnaire/12556984`
Same signal on second live job	OBSERVED	`…/job` path via GetJob for `794559100` also had `ViewQuestionnaire` → USA Staffing
`apply.usastaffing.gov` root title is Application Manager	OBSERVED	Root HTML `<title>Application Manager</title>` (Blazor WASM shell)
Monster Government Solutions named on sample pages	NOT OBSERVED	0 hits for `monster` / `Application Manager` on the two announcement pages
Public page had no plain `>Apply<` CTA in static HTML	OBSERVED	Sign-in chrome + How to Apply section; apply CTA appears auth/JS-gated
Product never automates federal apply	HOUSE	WW-82, `FederalApplyPrepCard`, hard boundary

1. TRAVERSAL PHILOSOPHY (portal)¶

USAJOBS is not an ATS hop. It is the federal job-announcement portal: the Search API and the public announcement page are systems of discovery and packaging. The real submit step lives in an agency application system (often USA Staffing Application Manager; sometimes other vendors) that only becomes reachable after login.gov + the USAJOBS five-step package.

Structurally that means:

Host wins first. Any *.usajobs.gov (and bare usajobs.gov) URL is portal-terminal for traversal. Do not “resolve through” it looking for Greenhouse/iCIMS-style apply engines.
API URL beats HTML apply CTA. The Search payload’s ApplyURI / PositionURI already name the announcement. Prefer those over scraping Apply buttons.
Downstream agency links on the public page are metadata, not next hops. A public ViewQuestionnaire link can name USA Staffing; it must never become a traversal target for apply automation.
Login is a hard stop, not a challenge to pass. No cookies, no login.gov, no applicant session. If the only remaining path needs auth → terminal Portal (or fail closed if host was never confirmed).

Contrast with iCIMS form lab: there, generated ASP.NET ids force suffix-first selectors. Here, the structural truth is portal termination — the “selector philosophy” is “do not select a next hop at all once the host class is portal.”

2. ORDERED RULE CHAIN (most specific first)¶

This is the executable rule order for a TraversalRules.UsaJobs / PortalTraversal static class. Deterministic C# only.

R0 — Input normalization (no browser)¶

// Prefer producer fields in this order (HOUSE MapToJobPosting already does 1 then 2):
// 1) MatchedObjectDescriptor.ApplyURI[0]
// 2) MatchedObjectDescriptor.PositionURI
// 3) constructed https://www.usajobs.gov/job/{MatchedObjectId}  // only if id is all digits

Why this order: ApplyURI is the channel-tagged apply entry (PostingChannelID=RESTAPI in fixture/docs). PositionURI is the canonical public announcement. Fabricating a path without a known control number is forbidden.

R1 — Host class predicate (terminal if true)¶

// Playwright-compatible URL predicates (no DOM needed)
bool IsUsaJobsPortalHost(Uri u) =>
    u.Scheme is "https" or "http"
    && (u.Host.Equals("usajobs.gov", StringComparison.OrdinalIgnoreCase)
        || u.Host.EndsWith(".usajobs.gov", StringComparison.OrdinalIgnoreCase));

Reason first: host class is cheap, stable, and matches frontend normalizeUsajobsUrl + hard-boundary host checks.

R2 — Hard-boundary paths (still portal, never drive)¶

// HOUSE: IApplyRunDriver.IsHardBoundaryUrl — keep identical semantics
bool IsFederalHardBoundary(Uri u) =>
    // login.gov (any path)
    u.Host.Equals("login.gov", StringComparison.OrdinalIgnoreCase)
    || u.Host.EndsWith(".login.gov", StringComparison.OrdinalIgnoreCase)
    // USAJOBS application wizard
    || (IsUsaJobsPortalHost(u)
        && (u.AbsolutePath.StartsWith("/application", StringComparison.OrdinalIgnoreCase)
            || u.AbsolutePath.StartsWith("/apply", StringComparison.OrdinalIgnoreCase)));

R3 — Posting id extraction (dedupe key) — CONFIDENT¶

// Prefer API MatchedObjectId when present (HOUSE Id = MatchedObjectId ?? PositionId)
// From URL, only the control-number path forms:
//   /job/{digits}
//   /GetJob/ViewDetails/{digits}   (legacy; still serves 200 OBSERVED)
static readonly Regex ControlNumberPath = new(
    @"^/(?:job|GetJob/ViewDetails)/(?<id>\d+)/?$",
    RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);

// Query string is NOT a posting id source (PostingChannelID is a channel tag, not a JOA id).

R4 — Optional public agency-system annotation (not a hop)¶

Only after R1 classified portal. Read-only, public page only:

// ORDERED selectors for OPTIONAL FederalAgencySystemHint — never NextUrl
public static readonly string[] AgencySystemHintSelectors =
[
    // 1) Most specific: public questionnaire preview host (OBSERVED)
    "a[href^='https://apply.usastaffing.gov/ViewQuestionnaire/']",
    "a[href*='://apply.usastaffing.gov/ViewQuestionnaire/']",

    // 2) Broader USA Staffing host on page (OBSERVED root is Application Manager)
    "a[href*='://apply.usastaffing.gov/']",

    // 3) Text markers — LAST, and only as weak hint (INFERRED; not verified for Monster)
    // Do NOT implement Monster text rules until a live announcement is captured.
];

Ordering reason: host-bearing anchors beat body text (same lesson as Spectrum multi-marker trap). First match sets AgencySystemHint = UsaStaffing (or similar) and still leaves traversal terminal.

R5 — What is NOT in the chain¶

No “find Apply and click.” No redirect-chain follow beyond confirming usajobs host. No HTML first-marker ATS scan on the announcement body (it can mention many agencies, OPM, help links).

3. NEGATIVE RULES (must never select)¶

Never	Why
Treat `ApplyURI` as unresolved / keep hopping	ApplyURI is the portal answer (HOUSE fixture; product deep-links it)
Resolve `usajobs.gov` → Lever/iCIMS/Greenhouse from page text	Portal text is full of agency noise; first-marker-wins is the Spectrum trap class
Follow `a[href*='apply.usastaffing.gov']` as next hop	OBSERVED public link is ViewQuestionnaire (assessment preview), not apply hand-off; Application Manager shell is auth-bound
Click Apply / open `/application` / `/apply`	HOUSE hard boundary; login required (WW-82)
Touch `login.gov`	Standing product constraint; hard boundary
Invent `AtsKind.UsaStaffing` / `MonsterGov` as terminal ATS for apply engines	Downstream is post-login; no public submit URL; wrong kind would route into commercial form fillers
Use `PostingChannelID` or announcement number alone as dedupe posting id	Channel id is RESTAPI/PCI; announcement numbers are agency formats; control number / MatchedObjectId is the stable JOA key
Scrape search results HTML for jobs	OBSERVED shell has 0 job hrefs; use Search API
Stealth, UA spoofing, CAPTCHA solve, proxies	Hard product line (global traversal constraints)
Treat missing agency hint as failure	Hint is optional metadata; portal terminal still succeeds

4. TERMINATION PREDICATE¶

IF host is usajobs.gov (R1):
    result.HostClass     = Portal
    result.Terminal      = true
    result.AtsKind       = Unknown          // HOUSE today — no federal enum value
    result.ApplyUrl      = canonical announcement URL (strip tracking only if needed)
    result.PostingId     = MatchedObjectId OR path control number OR null
    result.AgencyHint    = UsaStaffing | Unknown   // optional; never blocks terminal
    result.NextAction    = StopTraversal
    // Product path: FederalApplyPrepCard + deep-link out. No engine drive.

ELSE IF host is login.gov OR path is /application|/apply on usajobs:
    result = same Portal/Unknown terminal + HardBoundary = true
    // Do not continue.

ELSE IF host is apply.usastaffing.gov AND arrived via ViewQuestionnaire only:
    // Should not be reached if R1 stops first. If mis-hopped:
    result.HostClass = PortalAdjacentMetadata
    result.Terminal  = true
    result.FailClosed reason = "Agency questionnaire is not an apply hand-off"
    // Never classify as commercial ATS.

ELSE:
    // Not USAJOBS lab's job — hand to other host-class labs.

Keep hopping? Never, once R1 matches.
Fail closed to Unknown? Only if input was not parseable as USAJOBS and not another known class — USAJOBS lab itself should not invent success.

5. END-TO-END HOP MAP (USAJOBS source path)¶

USAJOBS does not start from Adzuna. It starts from the official Search API object (analog of “Hop 0”).

Hop 0 — Search API item (JSON)¶

Input (HOUSE fixture shape):

{
  "MatchedObjectId": "812345678",
  "MatchedObjectDescriptor": {
    "PositionID": "DE-123",
    "PositionTitle": "IT Specialist",
    "PositionURI": "https://www.usajobs.gov/job/812345678",
    "ApplyURI": ["https://www.usajobs.gov/job/812345678?PostingChannelID=RESTAPI"]
  }
}


Know	Control number id, title, org, ApplyURI host = portal
Need	Nothing else to terminate traversal
Action	Parse ApplyURI[0] → R1 host check → TERMINAL Portal
Fail closed	Missing both ApplyURI and PositionURI → drop item (HOUSE already returns null)

No browser. No Playwright. No HTML.

Hop 0b — Live control number (OBSERVED for lab fixtures)¶

Use as additional Theory data (real public page):

Field	Value
Control number	`812345600`
Announcement number	`ST-12556984-24-RLW`
PositionURI shape	`https://www.usajobs.gov/job/812345600`
Agency hint URL	`https://apply.usastaffing.gov/ViewQuestionnaire/12556984`

Hop 1 — Optional public announcement read (annotation only)¶


Know	Already terminal portal
Need	Optional `AgencySystemHint` for UX copy (“often USA Staffing after login”)
Action	`HttpClient` GET announcement (plain browser UA is fine; no stealth). Run R4 selectors on HTML.
Fail closed	403/5xx/empty → leave `AgencyHint = Unknown`; still Portal terminal
Never	Navigate Apply, login, or open Application Manager

Hop 2 — Does not exist for automation¶


Would be	login.gov → USAJOBS 5 steps → “Continue to Agency Site” → USA Staffing / other
Status	Out of scope by product law (WW-82, hard boundary)
Lab asserts	No code path schedules this hop

6. POSTING ID EXTRACTION¶

Source	Pattern	Confidence
Search `MatchedObjectId`	all digits string	CONFIDENT — HOUSE primary `JobPosting.Id`
URL path `/job/{id}` or `/GetJob/ViewDetails/{id}`	`^\d+$`	CONFIDENT — OBSERVED live pages
`PositionID` / announcement number (e.g. `ST-12556984-24-RLW`)	agency-local format	NOT posting id for dedupe — secondary display only
`PostingChannelID=RESTAPI`	query	NOT an id
Questionnaire path `/ViewQuestionnaire/{n}`	digits	UNKNOWN as JOA id — may equal middle of announcement number (OBSERVED 12556984 in both) but do not use as primary dedupe key without API proof
Downstream Application Manager application id	after login	UNKNOWN — never invent

Wrong regex on agency questionnaire