Adzuna Path Cedric

Create tests/WorkWingman.ScraperLab.Traversal.Tests with one static selector/rule class plus xUnit philosophy tests per host class. The resolver returns an explicit terminal state—never an inferred ATS.

public enum TraversalDisposition { Ats, Portal, Blocked, Unknown }

public sealed record TraversalResult(
    TraversalDisposition Disposition,
    Uri? TerminalUrl,
    string? Vendor,
    string? PostingId,
    IReadOnlyList<Uri> Trail,
    string Reason);

Hop 0: Adzuna API result → browser-driven first hop

Treat these fields as evidence only:

API field	Know	Action
redirect_url	Public Adzuna listing entry URL	Validate HTTPS + www.adzuna.com/land/ad/{id}; navigate it in the existing headful Chromium profile.
id	Adzuna listing identity	Use only to corroborate the /land/ad/ → /details/ redirect, never to construct a replacement URL.
adref	Opaque Adzuna token	Preserve for diagnostics only; never decode, alter, or replay it.
title/company/location	Search metadata	Record as diagnostics; never use to identify an ATS or choose a link.

Required browser state: use LaunchPersistentContextAsync with the ordinary, headful application profile at one-listing-at-a-time pace. The profile may contain only normal public-site state acquired through visible browser interaction (including a consent-banner choice); do not inject cookies, use a logged-in account, spoof identity, or create a fresh clean context for this path.

await using var context = await playwright.Chromium.LaunchPersistentContextAsync(
    profileDirectory,
    new() { Headless = false });

var page = context.Pages.FirstOrDefault() ?? await context.NewPageAsync();
await page.GotoAsync(apiResult.RedirectUrl, new()
{
    WaitUntil = WaitUntilState.DOMContentLoaded
});

Expected path:

Adzuna API result
  → redirect_url: www.adzuna.com/land/ad/{id}
  → browser follows 302: www.adzuna.com/details/{id}
  → click the fresh signed aztt apply anchor
  → ZipRecruiter (aggregator)
  → jobs.spectrum.com (employer / Radancy TalentBrew vanity host)
  → careers-charter.icims.com/jobs/77665/... (ATS, iCIMS, posting 77665)

If the /details/ page does not expose exactly one valid fresh aztt apply anchor after the normal profile/consent state is available, return Unknown. A 200 response is not success by itself.

Static rules

internal static class TraversalLabSelectors
{
    // Adzuna: a signed apply handoff is structural evidence; text is not.
    public static readonly string[] AdzunaSignedApply =
    [
        "a[href*='aztt=']:visible"
    ];

    // Employer pages: inspect outbound anchors, never page text/marker scans.
    public static readonly string[] EmployerKnownAtsAnchor =
    [
        "a[href*='.icims.com/jobs/']:visible",
        "a[href*='://boards.greenhouse.io/']:visible",
        "a[href*='://job-boards.greenhouse.io/']:visible",
        "a[href*='://jobs.lever.co/']:visible",
        "a[href*='.myworkdayjobs.com/']:visible"
    ];

    // Only used after the known-ATS-host chain has failed.
    public static readonly string[] StrictApplyLink =
    [
        "a[aria-label*='apply' i][href]:visible",
        "a:has-text('Apply')[href]:visible",
        "a:has-text('Apply now')[href]:visible",
        "a:has-text('Continue to apply')[href]:visible"
    ];

    public static readonly string[] ChallengeEvidence =
    [
        "iframe[src*='challenge' i]",
        "form[action*='challenge' i]",
        "input[name*='captcha' i]",
        "[id*='captcha' i]"
    ];
}

Selectors produce candidates; URL predicates decide whether a candidate is valid. Resolve every href against page.Url before classification.

static bool IsApplicationInformationPage(Uri uri) =>
    uri.AbsolutePath.Contains("application-process",
        StringComparison.OrdinalIgnoreCase);

static bool IsExternalTo(Uri candidate, Uri current) =>
    !candidate.Host.Equals(current.Host, StringComparison.OrdinalIgnoreCase);

static bool TryGetIcmsId(Uri uri, out string id)
{
    id = "";
    if (!uri.Host.EndsWith(".icims.com", StringComparison.OrdinalIgnoreCase))
        return false;

    var match = Regex.Match(uri.AbsolutePath, @"^/jobs/(?<id>\d+)(?:/|$)");
    if (!match.Success)
        return false;

    id = match.Groups["id"].Value;
    return true;
}

static bool TryGetGreenhouseVanityHint(Uri uri, out string id)
{
    id = "";
    var value = HttpUtility.ParseQueryString(uri.Query)["gh_jid"];
    if (value is null || !Regex.IsMatch(value, @"^\d+$"))
        return false;

    id = value;
    return true;
}

Host-class specifications

ats

Traversal philosophy. An ATS is terminal only when the URL itself proves the vendor and exposes a stable posting identity. Page content is not vendor evidence: employer pages can contain arbitrary ATS names, scripts, pixels, and unrelated links. Classify the destination URL before reading DOM markers or clicking another “apply” control.

Ordered rule chain.

1. TryGetIcmsId(uri, out id) — terminal iCIMS result. The /jobs/{numeric-id} path is direct URL evidence.
2. Other vendor parsers only when a documented host-and-id predicate succeeds.
3. Otherwise Unknown, even if the host name resembles an ATS.

Negative rules.

• Never identify ATS vendor from page.ContentAsync(), script text, meta tags, analytics, or a first-marker-wins scan.
• Never convert a vanity hostname into an imagined vendor URL.
• Never treat jobs.spectrum.com as Lever merely because “lever” appears in HTML.

Termination. Return Ats only for a recognized ATS URL with a supported stable ID. Preserve the canonical public posting URL as TerminalUrl; do not click further into a form.

employer

Traversal philosophy. Employer hosts are carriers, not proof of the underlying ATS. The only reliable hand-off evidence is an actual anchor destination, or a vendor-specific URL parameter that is explicitly known to encode an ID. An employer page may include markers for several vendors, so DOM text is deliberately lower than zero confidence.

Ordered rule chain.

1. EmployerKnownAtsAnchor, in listed order, followed by host/path validation and vendor ID extraction.
2. Scan StrictApplyLink, but retain only an external destination that is not an information page and classifies as ATS, aggregator, portal, or blocked.
3. Parse gh_jid as a Greenhouse vendor/ID hint; continue scanning for a real ATS-host anchor.
4. If competing same-rank candidates lead to different destinations, return Unknown; do not pick whichever appeared first.

The known-ATS anchor chain comes first because its href identifies the receiving system. Strict “apply” text is only a fallback because text can label informational pages.

Negative rules.

• Never select /application-process, including when its text or aria-label says “Apply.”
• Never resolve Spectrum to Lever from HTML markers. jobs.spectrum.com is employer-class until its actual anchor points at careers-charter.icims.com.
• A gh_jid parameter does not authorize fabricating boards.greenhouse.io/{tenant}/jobs/{id}; the tenant is unknown.

Termination. An employer page is never itself the ATS result. Terminate only after an anchor reaches a valid ATS URL; otherwise keep hopping through a valid external classified URL, or return Unknown.

aggregator

Traversal philosophy. Aggregators resell listings and are structurally non-terminal. Their visible Apply control is a hand-off mechanism, not a system-of-record identity, and it may lead through another aggregator. Require a validated next destination and retain the full trail for loop detection.

Ordered rule chain.

1. A visible external anchor whose resolved URL passes a registered ATS host/path predicate.
2. A visible StrictApplyLink candidate whose resolved URL is external, is not an information page, and classifies as employer, aggregator, portal, or ATS.
3. Follow normal browser navigation/click and reclassify the resulting URL.

Negative rules.

• Never return ZipRecruiter or Jobcase as the final ATS.
• Never select generic “Learn more,” “company,” category, salary, tracking, or application-process links.
• Never follow a URL that revisits a host/path already in Trail, or exceed a small fixed budget such as six browser hand-offs.

Termination. Keep hopping only for a validated external destination. Return Unknown on ambiguous candidates, loops, missing Apply control, or hop-budget exhaustion.

portal

Traversal philosophy. A portal is terminal because the public job record intentionally hands application completion to an authenticated workflow. It is not an ATS inference failure, and it should not be scraped through login boundaries. Preserve the public portal URL as the result.

Ordered rule chain.

1. URL predicate for a known portal host and public job-record path, e.g. www.usajobs.gov.
2. Stop before login, account-creation, or authenticated application UI.

Negative rules.

• Never click through a sign-in or create-account flow.
• Never reinterpret a portal’s login redirect as an ATS hand-off.
• Never manufacture an ATS vendor or posting ID from a portal URL.

Termination. Return Portal with the public URL. Vendor and PostingId remain null unless separately documented and verified.

blocked

Traversal philosophy. Access refusal is an outcome, not a challenge to defeat. Status 403/429 is decisive; a CloudFront challenge returning 200 is also blocked when the expected public job structure is absent. The resolver has one ordinary-browser attempt and then stops.

Ordered rule chain.

1. Observe navigation response status and final URL.
2. For Adzuna, require the expected /details/{id} URL plus exactly one visible a[href*='aztt='].
3. If status is refusal, challenge evidence is present, or the required structure is absent, classify Blocked or Unknown—never retry with evasive changes.

Negative rules.

• No stealth plugins, navigator.webdriver patching, UA/TLS spoofing, CAPTCHA solving, proxy rotation, cookie injection, or fingerprint modification.
• No HttpClient fallback for Adzuna after browser refusal.
• Never treat Adzuna CloudFront HTTP 200 challenge HTML as a job page.

Termination. Return Blocked for clear refusal/challenge evidence; return Unknown when the evidence is merely incomplete. Neither state may produce a next hop.

ID extraction policy

Vendor	Rule	Confidence / result
iCIMS	Host ends in .icims.com; path begins /jobs/{digits}	Confident. careers-charter.icims.com/jobs/77665/... → 77665.
Greenhouse	Numeric gh_jid query value	Confident as a vendor/ID hint. careers.airbnb.com/positions/7995199?gh_jid=7995199 → Greenhouse, 7995199; still not an ATS URL.
Lever	UNKNOWN	HTML marker is explicitly unsafe; no measured destination URL rule supplied.
Radancy / TalentBrew	UNKNOWN	Spectrum’s page demonstrates that text markers are not sufficient.
Workday	UNKNOWN	Form-selector philosophy does not prove a traversal URL or ID rule.
Google, Meta, Microsoft	UNKNOWN	No traversal identity rule supplied.

xUnit philosophy tests

public sealed class EmployerTraversalLabTests
{
    [Fact]
    public void Known_ats_href_precedes_generic_apply_text()
    {
        Assert.Equal(
            "a[href*='.icims.com/jobs/']:visible",
            TraversalLabSelectors.EmployerKnownAtsAnchor[0]);

        Assert.DoesNotContain(
            TraversalLabSelectors.EmployerKnownAtsAnchor,
            x => x.Contains("has-text", StringComparison.OrdinalIgnoreCase));
    }

    [Theory]
    [InlineData("https://careers-charter.icims.com/jobs/77665/")]
    public void Icms_url_is_terminal_and_extracts_numeric_posting_id(string raw)
    {
        Assert.True(TryGetIcmsId(new Uri(raw), out var id));
        Assert.Equal("77665", id);
    }

    [Fact]
    public void Spectrum_marker_mix_cannot_resolve_to_lever()
    {
        var source = new Uri("https://jobs.spectrum.com/");
        var actualAnchor = new Uri(
            "https://careers-charter.icims.com/jobs/77665/");

        Assert.True(TryGetIcmsId(actualAnchor, out var id));
        Assert.Equal("77665", id);

        Assert.NotEqual("Lever", ClassifyFromPageMarkers(
            source, "icims lever radancy talentbrew"));
    }

    [Theory]
    [InlineData("https://jobs.spectrum.com/application-process")]
    [InlineData("https://jobs.spectrum.com/careers/application-process")]
    public void Application_process_is_never_an_apply_handoff(string raw) =>
        Assert.True(IsApplicationInformationPage(new Uri(raw)));

    [Theory]
    [InlineData("https://careers.airbnb.com/positions/7995199?gh_jid=7995199")]
    public void Greenhouse_vanity_url_yields_hint_not_fabricated_ats_url(string raw)
    {
        Assert.True(TryGetGreenhouseVanityHint(new Uri(raw), out var id));
        Assert.Equal("7995199", id);
        Assert.False(IsRegisteredAtsHost(new Uri(raw)));
    }
}

public sealed class AdzunaTraversalLabTests
{
    [Fact]
    public void Signed_handoff_is_required_before_any_generic_apply_selector() =>
        Assert.Equal("a[href*='aztt=']:visible",
            TraversalLabSelectors.AdzunaSignedApply[0]);

    [Fact]
    public void Challenge_200_without_details_and_aztt_is_not_success()
    {
        var result = ClassifyAdzunaDocument(
            finalUrl: new Uri("https://www.adzuna.com/details/placeholder"),
            statusCode: 200,
            hasSignedApplyAnchor: false,
            hasChallengeEvidence: true);

        Assert.NotEqual(TraversalDisposition.Ats, result.Disposition);
        Assert.NotEqual(TraversalDisposition.Portal, result.Disposition);
    }
}

public sealed class AggregatorTraversalLabTests
{
    [Theory]
    [InlineData("ziprecruiter.com")]
    [InlineData("jobcase.com")]
    public void Aggregators_are_never_terminal_ats(string host) =>
        Assert.False(IsTerminalAtsHost(host));

    [Fact]
    public void Revisited_hop_fails_closed() =>
        Assert.True(HasTraversalLoop([
            new Uri("https://www.ziprecruiter.com/"),
            new Uri("https://www.ziprecruiter.com/")
        ]));
}

public sealed class PortalTraversalLabTests
{
    [Fact]
    public void Portal_does_not_attempt_login_or_ats_discovery()
    {
        var result = ResolvePortal(new Uri("https://www.usajobs.gov/"));
        Assert.Equal(TraversalDisposition.Portal, result.Disposition);
        Assert.Null(result.Vendor);
    }
}

public sealed class BlockedTraversalLabTests
{
    [Theory]
    [InlineData(403)]
    [InlineData(429)]
    public void Refusal_never_produces_next_hop(int statusCode) =>
        Assert.Null(NextHopAfterRefusal(statusCode));

    [Fact]
    public void Challenge_selectors_are_detection_only_not_bypass_mechanisms() =>
        Assert.All(TraversalLabSelectors.ChallengeEvidence,
            selector => Assert.DoesNotContain("solve", selector,
                StringComparison.OrdinalIgnoreCase));
}

The key invariant is simple: Spectrum resolves to iCIMS only because an actual anchor targets careers-charter.icims.com/jobs/77665/...; neither “Lever,” Radancy, nor TalentBrew text can participate in vendor selection.