Employer Vanity Gronktayvius
I'll study the existing ScraperLab pattern first, then write a full-depth TRAVERSAL LAB spec for employer vanity career sites only.Searching WorkWingman ScraperLab tests and traversal/ATS-related code.Found ScraperLab under a worktree. Enumerating all lab projects and hunting Icims/Google/Meta/Microsoft plus TRAVERSAL.Main repo is WorkWingman. Digging into lab tests, philosophy assertions, and traversal/ATS hop code.Reading key selector/philosophy tests and ATS resolution sources.# ScraperLab map — TRAVERSAL lab prep
Root: C:\Users\fives\source\repos\WorkWingman
No ScraperLab.Traversal / hop lab exists yet.
1. Lab projects + test files¶
Named targets (tests under tests/)¶
| Lab | Tool project | Test project | Test files |
|---|---|---|---|
| Icims | tools/WorkWingman.ScraperLab.Icims/ |
tests/WorkWingman.ScraperLab.Icims.Tests/ |
IcimsSelectorAndJudgementTests.cs, IframeSwitchLogicTests.cs, AssemblyInfo.cs |
tools/WorkWingman.ScraperLab.Google/ |
tests/WorkWingman.ScraperLab.Google.Tests/ |
GoogleLabTests.cs, FixtureHydrationTests.cs, AssemblyInfo.cs |
|
| Meta | tools/WorkWingman.ScraperLab.Meta/ |
tests/WorkWingman.ScraperLab.Meta.Tests/ |
MetaSelectorsTests.cs, PickIdStyleBoundaryTests.cs, SiteVariationRandomizerTests.cs, JudgementPauseTests.cs, SubmitGuardAndEngineTests.cs, LabMetricsTests.cs, AssemblyInfo.cs |
| Microsoft | tools/WorkWingman.ScraperLab.Microsoft/ |
tests/WorkWingman.ScraperLab.Microsoft.Tests/ |
MicrosoftSelectorTests.cs, MicrosoftEngineDecisionTests.cs, AssemblyInfo.cs |
Pinned in solution: tests/WorkWingman.Tests/SolutionMembershipTests.cs:71-74.
Other labs (tests under tools/ or nested)¶
| Lab | Path | Tests |
|---|---|---|
| Workday (base) | tools/WorkWingman.ScraperLab/ |
no sibling .Tests |
| Greenhouse | tools/WorkWingman.ScraperLab.Greenhouse/ |
tools/...Greenhouse.Tests/ → SelectorLogicTests.cs, FakeGreenhouseSiteTests.cs |
| Lever | tools/WorkWingman.ScraperLab.Lever/ |
tools/...Lever.Tests/ → LeverLabTests.cs |
| Amazon | tools/WorkWingman.ScraperLab.Amazon/ |
tools/...Amazon.Tests/ → AmazonLabTests.cs |
| Dayforce | tools/WorkWingman.ScraperLab.Dayforce/ |
tools/...Dayforce.Tests/ → DayforceLabTests.cs |
| Adp | tools/WorkWingman.ScraperLab.Adp/ |
tools/...Adp.Tests/ → SelectorLogicTests.cs, FakeAdpSiteTests.cs |
| Paycom | tools/WorkWingman.ScraperLab.Paycom/ |
tools/...Paycom.Tests/ → SelectorLogicTests.cs, FakePaycomSiteTests.cs |
| Paycor | tools/WorkWingman.ScraperLab.Paycor/ |
nested ...Paycor.Tests/ → SelectorAndDetectorTests.cs, FakePaycorSiteTests.cs |
| SuccessFactors | tools/WorkWingman.ScraperLab.SuccessFactors/ |
tools/...SuccessFactors.Tests/ → FixtureAndSelectorTests.cs, FakeSuccessFactorsSiteTests.cs |
| Oracle | tools/WorkWingman.ScraperLab.Oracle/ |
tools/...Oracle.Tests/ → FakeOracleSiteTests.cs |
| Ukg | tools/WorkWingman.ScraperLab.Ukg/ |
no dedicated .Tests |
Shared: tools/WorkWingman.TestSupport/ (LoopbackListener, SharedBrowser).
2. Typical lab structure¶
tools/WorkWingman.ScraperLab.<Ats>/
<Ats>Selectors.cs | SelectorChain.cs | SelectorModel.cs # pure data chains
<Ats>AutomationEngine.cs # fill; submit LOCATE only
Fake<Ats>Site.cs # loopback HttpListener HTML
SiteVariation.cs | FixtureVariation.cs | <Ats>Variation.cs
LabProfile.cs, LabVault.cs, LabMetrics.cs, LoopRunner.cs, Program.cs
tests/… or tools/….Tests/
AssemblyInfo.cs → [assembly: AssemblyTrait("Category", "Lab")]
*Selector*Tests.cs # philosophy + chain shape (no browser)
*Engine*/*Judgement* # pause gates
Fake*SiteTests.cs # fixture invariants (no real hosts)
Template doc: docs/ATS-LABS-TODO.md:7-17 — copy Lever; slnx membership required.
Selector ownership split:
- Production chains: src/WorkWingman.Infrastructure/Automation/{Ats}Selectors.cs (Icims/Microsoft/Google/…)
- Lab-local chains: Meta/Greenhouse/Amazon often define own *Selectors.cs in tools
- Lab tests often import Infrastructure selectors (Icims, Microsoft)
Assertion philosophy (not just string equality):
1. Lead-rung rule — first candidate must match ATS’s durable anchor style
2. Negative rules — ban whole-id, bare class, vendor attrs that don’t exist
3. Degrade order — chain widens, never tightens
4. Judgement gate — low confidence → AwaitingJudgement, never guess
5. Fixture safety — no real vendor hosts; submit is no-op
6. Variation determinism — same seed → same variation; boundary rolls pinned with Theory/InlineData
3. House-style excerpts¶
Icims — “no field chain leads with exact whole-id”¶
```13:24:tests/WorkWingman.ScraperLab.Icims.Tests/IcimsSelectorAndJudgementTests.cs [Fact] public void Field_chains_lead_with_a_suffix_or_substring_match_not_an_exact_id() { Assert.StartsWith("input[id$='FirstName']", IcimsSelectors.FirstName[0]); // No field chain should LEAD with an exact whole-id equality selector (id='...'). foreach (var chain in new[] { IcimsSelectors.FirstName, IcimsSelectors.LastName, IcimsSelectors.School }) Assert.DoesNotContain("[id='", chain[0]); }
### Google — no bare `#id` lead
```258:273:tests/WorkWingman.ScraperLab.Google.Tests/GoogleLabTests.cs
public void GoogleSelectors_NoChain_LeadsWithAnExactIdSelector()
{
// ... lead rung must be custom data-fieldid, never bare #id
foreach (var chain in new[] { GoogleSelectors.Field(...), ... })
{
Assert.DoesNotContain(chain.Candidates[0], "#");
Assert.StartsWith("[data-fieldid=", chain.Candidates[0]);
}
}
Meta — never a class selector¶
```27:38:tests/WorkWingman.ScraperLab.Meta.Tests/MetaSelectorsTests.cs public void Core_chain_never_contains_a_bare_class_selector() { for (var i = 0; i < 20; i++) { var chain = MetaSelectors.CoreCssChain(token, token); foreach (var sel in chain) Assert.DoesNotContain(".", sel); } }
### Meta — degrade order
```42:48:tests/WorkWingman.ScraperLab.Meta.Tests/MetaSelectorsTests.cs
public void Core_chain_degrades_id_to_substring_to_name_in_order()
{
Assert.Equal("#email", chain[0]);
Assert.Contains("[id*='email' i]", chain[1]);
Assert.Equal("[name='email']", chain[2]);
}
Microsoft — semantic id → name → aria/label¶
```8:18:tests/WorkWingman.ScraperLab.Microsoft.Tests/MicrosoftSelectorTests.cs public void Contact_chains_lead_with_semantic_id_then_name_then_aria_or_label() { Assert.Equal("input[id='firstName']", MicrosoftSelectors.FirstName[0]); Assert.Equal("input[name='firstName']", MicrosoftSelectors.FirstName[1]); Assert.Contains(MicrosoftSelectors.FirstName, s => s.Contains("aria-label")); }
### Theory/InlineData — degree similarity
```26:38:tests/WorkWingman.ScraperLab.Icims.Tests/IcimsSelectorAndJudgementTests.cs
[Theory]
[InlineData("Bachelor of Science (BS)", "B.S. Game Development", true)]
[InlineData("Bachelor of Arts (BA)", "B.S. Game Development", false)]
public void Degree_similarity_scores_reasonable_matches_higher(...)
Amazon — ban foreign conventions¶
```58:75:tools/WorkWingman.ScraperLab.Amazon.Tests/AmazonLabTests.cs public void AmazonSelectors_chains_never_use_data_automation_id_or_bracketed_names() { foreach (var sel in chain) { Assert.DoesNotContain("data-automation-id", sel); Assert.DoesNotContain("urls[", sel); } }
### Greenhouse — resume scoped to `input` only
```24:31:tools/WorkWingman.ScraperLab.Greenhouse.Tests/SelectorLogicTests.cs
public void Resume_chain_ends_with_any_file_input_as_last_resort()
{
Assert.Equal("input#resume", chain[0]);
Assert.Equal("input[type='file']", chain[^1]);
Assert.All(chain, link => Assert.StartsWith("input", link));
}
Assembly trait¶
```12:12:tests/WorkWingman.ScraperLab.Google.Tests/AssemblyInfo.cs [assembly: AssemblyTrait("Category", "Lab")]
---
## 4. Employer vanity / Radancy / Spectrum / ATS hop
| Thing | Present? | Where |
|-------|----------|--------|
| **ATS multi-hop resolution** | YES | `src/.../RedirectChainAtsResolver.cs` — max 5 hops, headers-only, stop at known ATS host |
| **Apply-link preference** | YES | `AtsDetector.LooksLikeApplyLink` (`AtsDetector.cs:280-285`), `PrefersTheApplyLink_OverAnAtsHostedAsset` tests |
| **Vanity host gate** | YES | `AtsDetector.ApplyUrlMatchesKind` (`AtsDetector.cs:98-115`); comments about Voyager filling vanity ApplyUrl; `ApplyRunDriverTests` InlineData `careers.cigna.com` + Kind=Greenhouse |
| **Tests for hops** | YES | `tests/WorkWingman.Tests/AtsResolutionTests.cs` → class `RedirectChainAtsResolverTests` — `ResolveAsync_FollowsRelativeMultiHopAndStopsAtAtsHop` |
| **Phenom employer-domain** | YES (detect only) | `AtsDetector` Phenom fingerprint; `ApplyUrlMarkers(Phenom)=[]` — no vendor host extract |
| **Radancy** | **No match** | — |
| **Spectrum** | sample company only | `ApplicationTrackerService` seed + vault tenant `spectrum` — not an ATS hop lab |
| **ScraperLab for traversal** | **None** | No `ScraperLab.Traversal` |
Hop core loop:
```124:129:src/WorkWingman.Infrastructure/Automation/RedirectChainAtsResolver.cs
for (var hop = 0; hop < 5; hop++)
{
if (!visited.Add(current.AbsoluteUri)) return Unknown(...);
var detected = AtsDetector.DetectFromUrl(current.AbsoluteUri);
if (detected.Kind != AtsKind.Unknown) return new AtsResolution { ..., Terminal = true };
Contract: src/WorkWingman.Core/Interfaces/IAtsResolver.cs — AtsResolution { Ats, FinalUrl, ResolvedAt, Terminal }.
5. Posting-id extraction¶
| Location | Role |
|---|---|
tools/...Lever/LeverVariation.cs:14-15,58 |
Fake PostingId shapes loopback URL only |
FakeLeverSite.cs:38 |
ApplyUrl => .../{PostingId}/apply |
LeverLabTests.cs:67 |
hardcodes "abcd1234" |
| No general posting-id extractor in Automation | no ExtractPostingId / requisition parser in engines |
Job IDs elsewhere = app-internal queue ids, not ATS posting scrape.
TRAVERSAL lab implications (locate-only, no design)¶
Closest existing code to model:
Defs:
- src/WorkWingman.Infrastructure/Automation/RedirectChainAtsResolver.cs:16 — RedirectChainAtsResolver — multi-hop headers-only resolve
- src/WorkWingman.Infrastructure/Automation/AtsDetector.cs:107 — ApplyUrlMatchesKind — vanity-host drive gate
- src/WorkWingman.Infrastructure/Automation/AtsDetector.cs:280 — LooksLikeApplyLink — prefer apply path tokens
- src/WorkWingman.Core/Interfaces/IAtsResolver.cs:11 — AtsResolution — result DTO
Tests (not ScraperLab tier):
- tests/WorkWingman.Tests/AtsResolutionTests.cs:46 — multi-hop → Greenhouse stop
- tests/WorkWingman.Tests/AtsDetectorTests.cs — host classify / apply-link prefer
- tests/WorkWingman.Tests/ApplyRunDriverTests.cs:90 — vanity host + Kind mismatch
Philosophy pattern for a new TRAVERSAL lab would mirror form labs: assert hop/stop/reject rules, not just final URL strings (e.g. “never follow 304”, “never accept asset URL”, “stop at first ATS hop”, “reject vanity when Kind requires vendor host”).
Form-lab template still Lever (docs/ATS-LABS-TODO.md); hop logic already production-tested outside ScraperLab.
4 named lab test projs under tests/, 11+ more under tools/, 0 Traversal lab.Reading the iCIMS lab pattern and Phenom/Greenhouse labs next — those map closest to vanity-site hand-off.Reading the existing ATS hop resolver and detector so the vanity-site lab spec aligns with production code.# TRAVERSAL LAB — Employer vanity career sites
Scope: host class employer only (Radancy TalentBrew, Phenom, Greenhouse vanity, general class).
House pattern: mirror IcimsSelectors + IcimsSelectorAndJudgementTests — philosophy first, then ordered chains, then negative rules, then theories that fail if someone reorders the chain.
Stack: C# / xUnit / Microsoft.Playwright. Deterministic parse only. No stealth, no fingerprint spoofing, no CAPTCHA solve. Fail closed → Unknown.
Evidence tags in this doc
| Tag | Meaning |
|---|---|
| OBSERVED | Measured by real fetch/walk (your chain facts + production code that exists today) |
| INFERRED | Structural reasoning; not a live selector claim until a fixture or fetch confirms it |
Why this lab exists (gap vs production today)¶
Production already has:
| Piece | What it does | Why vanity breaks it |
|---|---|---|
RedirectChainAtsResolver |
Headers-only, max 5 hops, stop when AtsDetector.DetectFromUrl hits a known ATS host |
Stops on jobs.spectrum.com as Terminal + Unknown — never reads body, never follows apply anchors |
AtsDetector.DetectFromDescriptionPage |
First-marker-wins on raw HTML strings | OBSERVED trap: Spectrum HTML contains icims + lever + radancy + talentbrew; order can mis-label as Lever |
AtsDetector.DetectFromUrl |
Host-only; skips Phenom (no fixed vendor host) | Vanity host never classifies as Greenhouse/iCIMS/etc. until the href host is known |
LooksLikeApplyLink |
Path contains apply/job/career/… |
OBSERVED trap: /application-process matches apply substring |
This lab is the apply-hand-off twin of the form labs: not “fill First Name,” but “pick the real next hop / terminal ATS URL without believing the page’s marketing soup.”
1. TRAVERSAL PHILOSOPHY¶
Structurally true about employer vanity sites
The hostname is a career front (employer brand domain), not the system of record. The page is often a CMS/SPA shell (Radancy TalentBrew, Phenom, Greenhouse-embedded vanity) that embeds multiple vendor strings in scripts, footers, and partner widgets. The durable signal for “where do I apply?” is almost never “which vendor word appears first in the HTML.” It is the navigable hand-off: an anchor href, form action, or iframe src whose parsed host is a known ATS (or whose query leaks a known vendor posting key on the same vanity host).
Selector strategy (mirror of iCIMS)
| Form lab (iCIMS) | Traversal lab (employer vanity) |
|---|---|
| Generated ASP.NET ids → suffix/substring first | Multi-vendor body noise → anchor/form/iframe URL host first |
| Exact whole-id equality must never lead | Raw page-text first-marker-wins must never lead |
| Exact id is a lucky bonus, not the key | Vendor word in body is a soft corroboration, not the hop |
Subclass notes
| Subclass | Role of the vanity host | Terminal? |
|---|---|---|
Radancy TalentBrew (e.g. jobs.spectrum.com) |
Front only; real apply is off-host ATS | Keep hopping via ATS-host anchor |
Phenom (data-ph-at-*, phApp.ddo, phenompeople.com) |
Often is the ATS on the employer domain | Terminal on same host when apply surface is Phenom-native; hop only if a stronger off-host ATS anchor exists |
Greenhouse vanity (e.g. careers.airbnb.com/...?...gh_jid=) |
Host hides vendor; query or embed leaks Greenhouse | Terminal when GH job id + apply surface are proven; prefer boards.greenhouse.io href if present |
| General employer | Unknown CMS | Same rule chain; fail closed if no ATS-host hand-off and no confident same-host ATS fingerprint |
2. ORDERED SELECTOR / RULE CHAIN¶
Most specific first. Reasons on each rung. Playwright-compatible where DOM is needed; pure URL predicates where not.
// tools/WorkWingman.ScraperLab.Traversal.Employer/EmployerVanityTraversalRules.cs
// Pure data — unit-testable without a browser (same posture as IcimsSelectors).
namespace WorkWingman.ScraperLab.Traversal.Employer;
/// <summary>
/// Employer vanity career sites hide the ATS behind a brand host. Body text is multi-vendor noise.
/// Lead with ANCHOR / FORM / IFRAME URL host classification — never first-marker-wins on page text.
/// Contrast: form labs lead with field anchors; this lab leads with hand-off URL structure.
/// </summary>
public static class EmployerVanityTraversalRules
{
/// <summary>
/// Host class for this hop. Vanity career hosts are NOT terminal ATS hosts.
/// </summary>
public const string HostClass = "employer";
// ---- Rung 0: classify "we are on an employer vanity career page" (soft; never picks ATS alone) ----
// OBSERVED (Spectrum): talentbrew / radancy markers present alongside icims/lever.
// INFERRED (general): careers./jobs. brand host + job detail path.
public static readonly string[] FrontEndMarkers =
[
"talentbrew", // Radancy TalentBrew shell
"radancy", // Radancy parent brand
"data-ph-at-", // Phenom DOM stamp
"phApp.ddo", // Phenom JS global
"phenompeople.com", // Phenom widget host
"gh_jid=", // Greenhouse vanity query leak
"boards.greenhouse.io", // Greenhouse embed (host still employer)
"myworkdayjobs.com", // Workday embed/link (host still employer)
];
// ---- Rung 1 (LEAD): collect CANDIDATE hand-off URLs from navigable attributes only ----
// Order: apply-ish anchors → forms → iframes. Never "first text occurrence of lever.co".
public static readonly string[] HandOffDomSelectors =
[
// 1a — apply-shaped anchors (href is the signal; host of href decides Kind)
"a[href*='icims.com' i]",
"a[href*='greenhouse.io' i]",
"a[href*='lever.co' i]",
"a[href*='myworkdayjobs.com' i]",
"a[href*='smartrecruiters.com' i]",
"a[href*='ashbyhq.com' i]",
"a[href*='workable.com' i]",
"a[href*='jobvite.com' i]",
"a[href*='eightfold.ai' i]",
"a[href*='ultipro.com' i]",
"a[href*='dayforce' i]",
"a[href*='successfactors.com' i]",
"a[href*='taleo.net' i]",
"a[href*='oraclecloud.com' i]",
"a[href*='paycomonline' i]",
"a[href*='workforcenow.adp.com' i]",
"a[href*='myjobs.adp.com' i]",
// 1b — same-host apply control whose href is absolute off-host (generic)
"a[href^='http'][href*='apply' i]",
"a[href^='http'][data-automation*='apply' i]",
// 1c — form POST/GET hand-off
"form[action*='icims.com' i]",
"form[action*='greenhouse.io' i]",
"form[action*='lever.co' i]",
"form[action*='myworkdayjobs.com' i]",
// 1d — iframe embed of the real ATS (iCIMS content frame pattern)
"iframe[src*='icims.com' i]",
"iframe[src*='greenhouse.io' i]",
"iframe[src*='lever.co' i]",
"iframe[src*='myworkdayjobs.com' i]",
];
/// <summary>
/// After collecting href/action/src values: score/rank. First match wins only AFTER host gate.
/// </summary>
public static readonly string[] HandOffUrlPredicatesOrdered =
[
// Prefer paths that look like a posting apply, not marketing.
// OBSERVED good: careers-charter.icims.com/jobs/77665/...
// OBSERVED bad: same-host /application-process
"host_is_known_ats AND path_has_posting_shape", // e.g. /jobs/{id}, /job/, /positions/
"host_is_known_ats AND path_has_apply_token", // /apply as path segment, not "application-*"
"host_is_known_ats AND not_asset", // board landing on ATS host (weaker)
"same_host AND query_leaks_vendor_posting_id", // e.g. ?gh_jid=7995199 OBSERVED Airbnb
"same_host AND phenom_native_apply_surface", // Phenom terminal; see §4
];
// ---- Rung 2: same-host vendor leak (Greenhouse vanity) BEFORE any body marker scan ----
// OBSERVED: careers.airbnb.com/positions/7995199?gh_jid=7995199
public static readonly (string QueryKey, string Vendor)[] SameHostPostingQueryLeaks =
[
("gh_jid", "Greenhouse"),
// UNKNOWN: do not invent lever / workday query keys until measured
];
// ---- Rung 3 (LAST, corroboration only): body markers — NEVER used alone to pick Kind ----
// INFERRED order if ever used as tie-break among *already host-qualified* hrefs.
// Explicitly NOT first-marker-wins over the whole document.
public static readonly string[] BodyMarkerCorroborationOnly =
[
"icims.com",
"greenhouse.io",
"myworkdayjobs.com",
"lever.co",
// front-end vendors are NEVER answers:
// "radancy", "talentbrew" — classify front-end only
];
// ---- Hard negatives (see §3) — encoded as ban list the ranker must apply ----
public static readonly string[] BannedPathSubstrings =
[
"/application-process", // OBSERVED Spectrum apply-SHAPED info page
"/applicationprocess",
"/how-to-apply",
"/about-us",
"/benefits",
"/culture",
"/privacy",
"/cookie",
"/legal",
"/login",
"/signin",
"/sign-in",
];
/// <summary>
/// Path segment "apply" is OK; path *prefix* "application" is not enough.
/// Fixes OBSERVED false positive: "/application-process".Contains("apply") == true.
/// </summary>
public static bool PathLooksLikeApplyNotInfo(Uri uri)
{
var path = uri.AbsolutePath.TrimEnd('/').ToLowerInvariant();
foreach (var ban in BannedPathSubstrings)
if (path.Contains(ban, StringComparison.Ordinal)) return false;
// Segment-aware: /apply, /jobs/123/apply — yes
// /application-process — no (banned above)
var segments = path.Split('/', StringSplitOptions.RemoveEmptyEntries);
if (segments.Any(s => s is "apply" or "application" && s == "apply"))
return true;
if (segments.Contains("apply")) return true;
// Posting shapes without the word apply
if (segments.Contains("jobs") || segments.Contains("job") ||
segments.Contains("positions") || segments.Contains("posting"))
return true;
return false;
}
}
Known-ATS host gate (reuse production table)¶
Reuse AtsDetector’s ApplyUrlMarkers / HostMatchesMarker / DetectFromUrl for host classification of a candidate href. Do not invent a parallel host table.
Phenom exception (production already documents this): ApplyUrlMarkers(Phenom) = []. Same-host Phenom terminal is fingerprint-based, not host-based.
Concrete hop algorithm (implementable)¶
function ResolveEmployerVanity(pageUrl, pageHtmlOrDom):
if DetectFromUrl(pageUrl).Kind != Unknown:
return Terminal(that ATS) // already on vendor host; not vanity
candidates = []
for each el matching HandOffDomSelectors (in order):
url = absolute(href|action|src)
if !IsSafePublicHttp(url): continue
if LooksLikeAsset(url): continue
if Path banned (BannedPathSubstrings): continue
if HostMatches known ATS:
candidates.Add(url, source=dom, score=predicate rank)
else if same host AND gh_jid (or known leak) present:
candidates.Add(url, source=query_leak, vendor=Greenhouse, postingId=gh_jid)
if candidates non-empty:
pick highest score; Kind = DetectFromUrl(picked) or Greenhouse if query leak only
return Hop(picked) // caller navigates; next hop re-runs host class
// Same-host Phenom terminal (no off-host ATS)
if ContainsPhenomMarker(html) AND HasPhenomApplySurface(dom):
return Terminal(Phenom, ApplyUrl=pageUrl, PostingId=UNKNOWN unless measured)
// Soft markers alone: FAIL CLOSED
return Unknown
3. NEGATIVE RULES¶
| # | Never do | Why | Evidence |
|---|---|---|---|
| N1 | Never set AtsKind from first vendor substring in raw HTML |
Spectrum body has icims and lever and radancy | OBSERVED |
| N2 | Never prefer body text over an anchor whose host is a known ATS | True answer is href host (careers-charter.icims.com) |
OBSERVED |
| N3 | Never select /application-process (or similar info paths) as the apply hand-off |
Apply-shaped, same host, not ATS | OBSERVED Spectrum |
| N4 | Never treat radancy / talentbrew as terminal ATS Kind |
Front-end vendors | OBSERVED markers; INFERRED class role |
| N5 | Never use production LooksLikeApplyLink as-is for vanity ranking without segment fix |
"application-process".Contains("apply") is true |
OBSERVED bug shape in current code |
| N6 | Never invent posting ids from path when pattern is unconfirmed | Corrupts dedupe key | Policy |
| N7 | Never stealth/UA/TLS spoof or CAPTCHA solve if host blocks automation | Hard product line; fail closed | Constraint |
| N8 | Never treat headers-only terminal Unknown on a vanity host as “done” for drive | That is exactly today’s resolver gap | OBSERVED in RedirectChainAtsResolver |
| N9 | Never drive automation from a Kind set by body marker while ApplyUrl is still the vanity host |
ApplyUrlMatchesKind already blocks drive for host-marker kinds; keep that |
OBSERVED production gate |
4. TERMINATION PREDICATE¶
On an employer vanity hop:
KEEP HOPPING when:
- Best candidate href/action/src host is a known ATS (DetectFromUrl != Unknown)
- OR same-host query leak proves vendor but apply surface still needs navigation
(INFERRED: some GH vanities apply on boards.greenhouse.io after click)
TERMINAL ANSWER when:
A) DetectFromUrl(handOffUrl).Kind is known ATS
AND ApplyUrlMatchesKind(handOffUrl, Kind)
AND not asset
→ Kind + ApplyUrl = handOffUrl + optional PostingId extract
B) Same-host Greenhouse query leak AND no stronger off-host ATS href
AND page is a job detail (path /positions|/jobs|...)
→ Kind = Greenhouse, ApplyUrl = current (or boards URL if later found),
PostingId = gh_jid OBSERVED
C) Phenom fingerprint + Phenom apply surface on same host
AND no stronger off-host ATS hand-off
→ Kind = Phenom, ApplyUrl = current, PostingId = UNKNOWN (until measured)
FAIL CLOSED → Unknown when:
- No host-qualified hand-off
- Only front-end markers (radancy/talentbrew) and no ATS href
- Only banned info paths
- Multiple conflicting ATS hosts in anchors with equal score and no posting-shape winner
- Automation blocked (non-job HTML, challenge page) — do not invent a path around it
Hop budget: reuse resolver’s spirit (e.g. max 5 total chain hops). Vanity body parse is one hop that may emit one new URL; it does not loop on itself.
5. LAB LAYOUT (house style)¶
tools/WorkWingman.ScraperLab.Traversal.Employer/
EmployerVanityTraversalRules.cs # ordered chains + philosophy comments
EmployerVanityHopResolver.cs # pure ranking over candidate URLs + optional HTML
FakeEmployerVanitySite.cs # loopback fixtures (Spectrum-shaped, GH vanity, Phenom)
EmployerVanityVariation.cs # shuffle marker order, inject decoy lever, etc.
LabMetrics.cs, LoopRunner.cs, Program.cs
tools/WorkWingman.ScraperLab.Traversal.Employer.Tests/ # or tests/...
AssemblyInfo.cs # [assembly: AssemblyTrait("Category", "Lab")]
EmployerVanityPhilosophyTests.cs # lead-rung + negative rules (no browser)
EmployerVanityHopResolverTests.cs # Theory/InlineData on measured URLs
FakeEmployerVanitySiteTests.cs # fixture invariants
Fake site must never hit real jobs.spectrum.com in CI; encode the measured shape offline (same as FakeIcimsSite).
Fixture shapes (encode OBSERVED traps)¶
Fixture S — Spectrum / TalentBrew multi-marker (measured chain terminal before iCIMS)
<!-- FakeEmployerVanitySite: SpectrumShape -->
<html>
<head>
<script src="https://cdn.example/talentbrew.js"></script>
<script>/* partner widgets mention lever.co and icims.com in comments */</script>
</head>
<body data-vendor="radancy">
<!-- body text / scripts contain ALL of: icims, lever, radancy, talentbrew -->
<a href="/application-process">Apply</a> <!-- DECOY: apply-shaped info -->
<a href="https://careers-charter.icims.com/jobs/77665/job-title/job">Apply Now</a>
<script src="https://jobs.lever.co/fake-widget.js"></script>
</body>
</html>
Expected: hop → https://careers-charter.icims.com/jobs/77665/..., Kind=Icims, PostingId=77665.
Must not: Kind=Lever, or hop to /application-process.
Fixture G — Greenhouse vanity query leak
<!-- careers.airbnb.com shape — host is employer, not greenhouse.io -->
<link rel="canonical" href="https://careers.airbnb.com/positions/7995199?gh_jid=7995199" />
<!-- optional embed -->
<iframe src="https://boards.greenhouse.io/embed/job_app?token=7995199"></iframe>
Expected: Kind=Greenhouse, PostingId=7995199. Prefer iframe/boards href if present; else same-host + gh_jid.
Fixture P — Phenom native (terminal on employer host)
<body>
<div data-ph-at-job-id="…">…</div>
<script>phApp.ddo = {};</script>
<!-- no off-host ATS anchors -->
<button data-ph-at-text="Apply">Apply</button>
</body>
Expected: Terminal Phenom on same host; PostingId UNKNOWN unless fixture encodes a measured extractor.
Fixture N — only front-end noise
<script src="https://cdn.radancy.net/x.js"></script>
<span>talentbrew</span>
<!-- no ATS anchors -->
Expected: Unknown (fail closed).
6. TEST ASSERTIONS (philosophy, not just strings)¶
Style clone of IcimsSelectorAndJudgementTests / GoogleLabTests / Greenhouse SelectorLogicTests.
using WorkWingman.Core.Models;
using WorkWingman.Infrastructure.Automation;
using WorkWingman.ScraperLab.Traversal.Employer;
namespace WorkWingman.ScraperLab.Traversal.Employer.Tests;
/// <summary>
/// Covers the employer-vanity TRAVERSAL philosophy (anchor-host first, because body text is
/// multi-vendor noise), not form filling. Mirror: iCIMS leads with suffix id; vanity leads with href host.
/// </summary>
public class EmployerVanityPhilosophyTests
{
[Fact]
public void Hand_off_chain_leads_with_ats_host_anchor_not_body_marker_scan()
{
// FIRST selector must be an anchor (or form/iframe) whose match is a URL attribute —
// never a "contains('lever.co')" body scan.
Assert.StartsWith("a[href*", EmployerVanityTraversalRules.HandOffDomSelectors[0]);
Assert.DoesNotContain(
EmployerVanityTraversalRules.HandOffDomSelectors[0],
"body",
StringComparison.OrdinalIgnoreCase);
}
[Fact]
public void No_hand_off_rule_leads_with_raw_page_text_first_marker_wins()
{
// Philosophy negative: body markers exist only as corroboration, never as lead rung.
foreach (var sel in EmployerVanityTraversalRules.HandOffDomSelectors)
{
Assert.False(sel.Contains("talentbrew", StringComparison.OrdinalIgnoreCase));
Assert.False(sel.Contains("radancy", StringComparison.OrdinalIgnoreCase));
}
}
[Fact]
public void Front_end_markers_are_not_terminal_ats_kinds()
{
// Radancy/TalentBrew classify the front, never AtsKind.
Assert.Contains("talentbrew", EmployerVanityTraversalRules.FrontEndMarkers);
Assert.Contains("radancy", EmployerVanityTraversalRules.FrontEndMarkers);
// Sanity: DetectFromUrl must not invent a Kind for a pure brand host.
Assert.Equal(AtsKind.Unknown, AtsDetector.DetectFromUrl("https://jobs.spectrum.com/job/123").Kind);
}
[Fact]
public void Application_process_path_is_banned_and_fails_apply_shape_check()
{
var decoy = new Uri("https://jobs.spectrum.com/application-process");
Assert.Contains("/application-process", EmployerVanityTraversalRules.BannedPathSubstrings);
Assert.False(EmployerVanityTraversalRules.PathLooksLikeApplyNotInfo(decoy));
}
[Fact]
public void Production_LooksLikeApplyLink_false_positive_is_documented_for_application_process()
{
// Guardrail: if someone reuses LooksLikeApplyLink for vanity ranking without the segment fix,
// this test keeps the bug visible. Private method — assert via PathLooksLikeApplyNotInfo contract.
var infoPage = new Uri("https://jobs.spectrum.com/application-process");
var realApply = new Uri("https://careers-charter.icims.com/jobs/77665/foo/job");
Assert.False(EmployerVanityTraversalRules.PathLooksLikeApplyNotInfo(infoPage));
Assert.True(EmployerVanityTraversalRules.PathLooksLikeApplyNotInfo(realApply));
}
[Theory]
[InlineData(
"https://careers-charter.icims.com/jobs/77665/customer-service-rep/job",
AtsKind.Icims,
"77665")]
[InlineData(
"https://boards.greenhouse.io/airbnb/jobs/7995199",
AtsKind.Greenhouse,
"7995199")]
public void Known_ats_hand_off_urls_classify_and_extract_posting_id(
string handOff, AtsKind kind, string postingId)
{
var info = AtsDetector.DetectFromUrl(handOff);
Assert.Equal(kind, info.Kind);
Assert.Equal(postingId, EmployerVanityPostingId.TryExtract(info.Kind, handOff));
}
[Theory]
[InlineData("https://careers.airbnb.com/positions/7995199?gh_jid=7995199", "7995199")]
public void Greenhouse_vanity_query_leak_extracts_gh_jid(string vanityUrl, string expectedId)
{
Assert.Equal(expectedId, EmployerVanityPostingId.TryExtractGreenhouseQuery(vanityUrl));
// Host alone is not Greenhouse.
Assert.Equal(AtsKind.Unknown, AtsDetector.DetectFromUrl(
"https://careers.airbnb.com/positions/7995199").Kind);
}
[Fact]
public void Spectrum_fixture_resolves_to_icims_not_lever()
{
var html = FakeEmployerVanitySite.Render(EmployerVanityVariation.SpectrumMultiMarker);
var result = EmployerVanityHopResolver.ResolveFromHtml(
pageUrl: "https://jobs.spectrum.com/job/example",
html: html);
Assert.Equal(AtsKind.Icims, result.Ats.Kind);
Assert.Contains("icims.com", result.Ats.ApplyUrl, StringComparison.OrdinalIgnoreCase);
Assert.Contains("/jobs/77665", result.Ats.ApplyUrl);
Assert.Equal("77665", result.PostingId);
// NEGATIVE: wrong answer must be provably absent
Assert.NotEqual(AtsKind.Lever, result.Ats.Kind);
Assert.DoesNotContain("lever.co", result.Ats.ApplyUrl, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("/application-process", result.Ats.ApplyUrl, StringComparison.OrdinalIgnoreCase);
}
[Fact]
public void Spectrum_fixture_ignores_application_process_decoy_even_when_it_appears_first()
{
var html = FakeEmployerVanitySite.Render(
EmployerVanityVariation.SpectrumMultiMarker with { DecoyFirst = true });
var result = EmployerVanityHopResolver.ResolveFromHtml(
"https://jobs.spectrum.com/job/example", html);
Assert.DoesNotContain("application-process", result.Ats.ApplyUrl, StringComparison.OrdinalIgnoreCase);
Assert.Equal(AtsKind.Icims, result.Ats.Kind);
}
[Fact]
public void Marker_order_permutation_does_not_change_kind()
{
// Philosophy: body marker order is noise. Fixture injects lever before icims in text;
// href still wins.
foreach (var seed in Enumerable.Range(0, 20))
{
var html = FakeEmployerVanitySite.Render(
EmployerVanityVariation.SpectrumMultiMarker with { MarkerShuffleSeed = seed });
var result = EmployerVanityHopResolver.ResolveFromHtml(
"https://jobs.spectrum.com/job/example", html);
Assert.Equal(AtsKind.Icims, result.Ats.Kind);
}
}
[Fact]
public void Front_end_only_fixture_fails_closed_to_unknown()
{
var html = FakeEmployerVanitySite.Render(EmployerVanityVariation.RadancyOnlyNoAtsHref);
var result = EmployerVanityHopResolver.ResolveFromHtml(
"https://jobs.example-employer.com/job/1", html);
Assert.Equal(AtsKind.Unknown, result.Ats.Kind);
Assert.True(result.FailedClosed);
}
[Fact]
public void Phenom_same_host_is_terminal_when_no_off_host_ats_anchor()
{
var html = FakeEmployerVanitySite.Render(EmployerVanityVariation.PhenomNativeApply);
var result = EmployerVanityHopResolver.ResolveFromHtml(
"https://careers.geaerospace.com/global/en/job/123", html);
Assert.Equal(AtsKind.Phenom, result.Ats.Kind);
Assert.True(result.Terminal);
Assert.Equal("https://careers.geaerospace.com/global/en/job/123", result.Ats.ApplyUrl);
Assert.Null(result.PostingId); // UNKNOWN — do not invent
}
[Fact]
public void Phenom_fingerprint_loses_to_off_host_icims_anchor()
{
// If both exist, ATS host href wins (hand-off > shell fingerprint).
var html = FakeEmployerVanitySite.Render(EmployerVanityVariation.PhenomShellWithIcimsHref);
var result = EmployerVanityHopResolver.ResolveFromHtml(
"https://careers.example.com/job/1", html);
Assert.Equal(AtsKind.Icims, result.Ats.Kind);
Assert.NotEqual(AtsKind.Phenom, result.Ats.Kind);
}
[Theory]
[InlineData("https://jobs.spectrum.com/job/1", true)] // brand jobs. host, unknown ATS
[InlineData("https://careers.airbnb.com/positions/1", true)]
[InlineData("https://careers-charter.icims.com/jobs/77665/x/job", false)] // already ATS
[InlineData("https://boards.greenhouse.io/x/jobs/1", false)]
public void Host_class_employer_only_when_detect_from_url_is_unknown(
string url, bool isEmployerClass)
{
var knownAts = AtsDetector.DetectFromUrl(url).Kind != AtsKind.Unknown;
Assert.Equal(isEmployerClass, !knownAts && EmployerVanityHopResolver.LooksLikeCareerHost(url));
}
}
Philosophy assertions that fail under wrong reorder¶
| If someone later… | Which test dies |
|---|---|
| Leads with body first-marker-wins | Hand_off_chain_leads_with…, Marker_order_permutation…, Spectrum → Lever |
Drops ban on /application-process |
Application_process_path_is_banned…, decoy-first fixture |
| Treats TalentBrew as AtsKind | Front_end_markers_are_not_terminal… |
Uses raw Contains("apply") |
Production_LooksLikeApplyLink_false_positive… |
| Invents Phenom posting id | Phenom_same_host… Assert.Null(PostingId) |
7. POSTING ID EXTRACTION¶
| Vendor | Source | Pattern | Confidence | Status |
|---|---|---|---|---|
| iCIMS | path | icims.com + /jobs/{digits} → $1 |
High | CONFIDENT (OBSERVED 77665) |
| Greenhouse | query on vanity or boards | gh_jid={digits} or /jobs/{digits} on greenhouse.io |
High | CONFIDENT (OBSERVED Airbnb gh_jid; boards path standard) |
| Lever | path on lever.co / jobs.lever.co |
/[company]/{postingId} |
Medium | INFERRED — not measured on this vanity chain; extract only when host already Lever |
| Workday | path on myworkdayjobs.com |
tenant + job number segment | Medium | INFERRED — do not extract until vanity fixture measured |
| Phenom | DOM data-ph-at-job-id etc. |
UNKNOWN | — | UNKNOWN — fingerprint Kind only; no id regex until live harvest |
| Radancy / TalentBrew | — | none | — | N/A — front end, not SoR |
| Other | — | — | — | UNKNOWN |
public static class EmployerVanityPostingId
{
// Only CONFIDENT extractors. Wrong key > no key.
public static string? TryExtract(AtsKind kind, string url) => kind switch
{
AtsKind.Icims => Match(url, @"icims\.com/.*/jobs/(\d+)", 1),
AtsKind.Greenhouse =>
TryExtractGreenhouseQuery(url)
?? Match(url, @"greenhouse\.io/.*/jobs/(\d+)", 1),
_ => null // UNKNOWN — never guess Lever/Workday/Phenom here without measurement
};
public static string? TryExtractGreenhouseQuery(string url)
{
if (!Uri.TryCreate(url, UriKind.Absolute, out var u)) return null;
var q = System.Web.HttpUtility.ParseQueryString(u.Query);
var id = q["gh_jid"];
return string.IsNullOrEmpty(id) || !id.All(char.IsDigit) ? null : id;
}
}
8. How this hop is reached (context only — other classes not fully labbed)¶
Measured chain for orientation:
Adzuna API Job
redirect_url → https://www.adzuna.com/land/ad/{id} [aggregator / blocked posture]
→ (browser, public, headful as shipped) details + apply
→ ZipRecruiter [aggregator — keep hopping]
→ jobs.spectrum.com [EMPLOYER — THIS LAB]
→ careers-charter.icims.com/jobs/77665/... [ats — terminal]
| Hop | Host class | What we know | What we still need | Programmatic action | Fail closed when |
|---|---|---|---|---|---|
| API | — | redirect_url, title, company, Adzuna id |
Real apply destination | Store redirect_url as chain start; do not treat Adzuna id as ATS posting id |
Missing/invalid redirect_url |
| … | aggregator / blocked | (other labs) | … | Headers-only where possible; body only when host class requires it | 403 / challenge HTML → Unknown (no stealth) |
| Employer vanity | employer | Brand URL + multi-marker HTML | True ATS URL + Kind + posting id | This lab’s resolver on page DOM/HTML | No ATS-host hand-off; only decoys; block page |
| ATS | ats | Vendor host + posting path | Drive with existing apply engine | DetectFromUrl + form lab for that Kind |
Host/Kind mismatch (ApplyUrlMatchesKind) |
Adzuna note (in scope for product, out of depth for this lab): traversal may start from API redirect_url; measured CloudFront/consent behavior means hop uses the existing headful Playwright path as a real browser, public pages only, one listing at a time. Employer vanity hop assumes the page HTML is already available (fixture in lab; real page in product after a successful public navigation).
9. Integration sketch with existing resolver¶
RedirectChainAtsResolver (headers-only) → lands on non-ATS FinalUrl, Terminal=true, Kind=Unknown
│
▼ if HostClass.IsEmployerCareer(finalUrl) // new classifier
EmployerVanityHopResolver (HTML/DOM, public) → Hop(atsUrl) | Terminal(Phenom/GH vanity) | Unknown
│
▼ if Hop
DetectFromUrl / drive gate / form ScraperLab for that Kind
Do not fold multi-vendor body scan into DetectFromDescriptionPage without the anchor-first rule — that would reintroduce the Spectrum→Lever trap in production.
10. OBSERVED vs INFERRED checklist¶
| Claim | Tag |
|---|---|
| Chain Adzuna → ZipRecruiter → jobs.spectrum.com → careers-charter.icims.com/77665 | OBSERVED (your measurement) |
| Spectrum HTML markers: icims, lever, radancy, talentbrew | OBSERVED |
| True Spectrum answer only via anchor href host | OBSERVED |
/application-process decoy on Spectrum |
OBSERVED |
Airbnb gh_jid on careers.airbnb.com |
OBSERVED |
LooksLikeApplyLink matches application-process via substring |
OBSERVED (code read) |
| Headers-only resolver stops Unknown on vanity host | OBSERVED (code read) |
| Phenom fingerprint markers | OBSERVED in production AtsDetector |
| Exact Playwright selector list for every Phenom apply button | INFERRED until FakePhenom/live harvest |
| Lever/Workday posting-id regex on vanity pages | INFERRED / UNKNOWN — extract only after host proves vendor |
| “All TalentBrew sites hand off to iCIMS” | INFERRED false — only Spectrum measured; always use href host |
11. Build order (when implementing)¶
EmployerVanityTraversalRules+ philosophy unit tests (no browser) — same day shape as Icims selector tests.FakeEmployerVanitySiteSpectrum multi-marker + decoy; resolver + negative Lever test.- Greenhouse vanity
gh_jid+ boards iframe fixture. - Phenom terminal + Phenom-vs-off-host-href priority.
- Wire optional second stage after
RedirectChainAtsResolverUnknown terminal on career hosts. - Promote confident posting-id extractors only; leave Phenom id UNKNOWN.
One-line philosophy (pin on the lab class)¶
Employer vanity: the host lies, the body gossipes, the anchor tells the truth — and
/application-processis not an apply URL.
That is the traversal twin of “iCIMS ids are generated trees — never lead with exact whole-id.”
I did not write this into a repo file (no ticket/worktree named). If you want it landed under docs/technical/ or scaffolded as tools/WorkWingman.ScraperLab.Traversal.Employer/, say which branch/ticket and I’ll put it in an isolated worktree.