Skip to content

Scraper Automation Learnings — Oracle (Taleo + Oracle Recruiting Cloud)

Offline learning harness: tools/WorkWingman.ScraperLab.Oracle. Everything is local. A loopback HttpListener serves fixture HTML on 127.0.0.1; Playwright drives a local Chromium against loopback only. There is no real Taleo/Oracle host, no account creation, and no real submit path. The fixture "Submit" is a client-side no-op that renders a SUBMIT SUPPRESSED marker and forwards nowhere.

Why Oracle is two very different targets

"Oracle-powered careers site" is not one thing. Oracle acquired Taleo in 2012 and later shipped a ground-up replacement (Oracle Recruiting Cloud / "Candidate Experience"). The two share almost nothing at the DOM level, so the adapter must detect and handle both.

Aspect Taleo (legacy, *.taleo.net) ORC (modern, *.oraclecloud.com) Workday (for contrast)
Rendering Server-rendered JSF/ASP-style pages, full page reloads SPA-ish, client-side section reveals Heavy SPA, custom web components
DOM structure Nested iframes, <table> layouts Flat semantic <section>/<div> + labels data-automation-id on custom elements
Element ids Long, generated: requisitionListInterface:ID2065:... Stable/semantic: orc-firstName GUID-ish but data-automation-id is stable
Account creation Forced before applying Usually deferred / social-login Forced account, but no iframe
Best selector anchor name= attribute (ids are unusable) id / autocomplete / aria-label data-automation-id
Async fields Rare (postbacks instead) Common (sections injected after load) Very common
Fragility Highest of the three Moderate Moderate–high

Taleo is among the most brittle bases we automate — comparable to iCIMS but Oracle-flavored. The iframe nesting plus generated ids means a naive "find by id" strategy hits 0% on Taleo.

The dual strategy: legacy-iframe + modern-SPA

The OracleAutomationEngine runs one code path with two behaviors selected by mode:

Taleo path

  1. Dismiss forced account creation. The fixture reproduces the "create an account to apply" gate. The engine clicks the local Continue button. No credentials are sent anywhere — it is a client-side hide. (In reality this is where account creation happens; we simulate it locally only.)
  2. Switch into the iframe. WaitForSelectorAsync("iframe#taleoAppFrame") → ContentFrameAsync(), then wait for DOMContentLoaded on the frame. If the variation nests a second iframe (#taleoInnerFrame), the engine locates it via page.Frames (by name, falling back to URL match) and switches again. Every switch is logged and counted (iframeSwitches).
  3. Fill by name=. Because ids are long and salted, the reliable anchor is input[name='email'] etc. The id/autocomplete strategies are still attempted first and logged as misses — that miss record is the learning signal.

ORC path

  1. No iframe, no account gate. Fill directly in the main frame.
  2. Fill by stable id first (#orc-firstName), then autocomplete (given-name, email, tel, organization), then name.
  3. Wait for async-revealed sections. When the variation sets DelayedEducation, the education inputs are injected ~350 ms after load. For school/degree the engine does a bounded WaitForAsync(Attached, 1500ms) before counting the locator — this is the SPA wait that Taleo never needs.

Shared fallback chain

Per field, strategies are tried in reliability order until one hits: id → autocomplete → name → aria-label. A bare aria-label match that resolves to many inputs is treated as ambiguous and skipped (logged as a judgement call) rather than filling the wrong box.

Judgement calls (log-and-continue, never fail loudly)

  • Missing optional field (e.g. MissingPhone drops the phone input): logged as optional/missing, skipped. Not a failure.
  • Blank voluntary EEO (gender, veteran blank in the profile): the engine still attempts them, finds no unambiguous target, and skips. Voluntary fields are left blank by design.
  • Ambiguous selector (>1 match on a weak strategy): skip that strategy, fall through.
  • Required field genuinely unreachable (firstName/lastName/email): this is recorded as a FailureMode so it surfaces in the rollup.

Selector findings (from a representative 20-iteration run, seed 1234)

[Taleo] runs=10 success=10 selectorHitRate=39.0% (87/223) iframeSwitchRate=80% avgMs=633
    strategy id            hits=0    misses=80     <- salted ASP ids: never usable
    strategy name          hits=77   misses=3      <- the workhorse for Taleo
    strategy autocomplete  hits=0    misses=50     <- Taleo has no autocomplete hints
    strategy click         hits=10   misses=0      <- account-panel dismissal
    strategy aria-label    hits=0    misses=3
[Orc]   runs=10 success=10 selectorHitRate=44.2% (57/129) iframeSwitchRate=0% avgMs=455
    strategy id            hits=57   misses=23     <- stable semantic ids carry ORC
    strategy name          hits=0    misses=23     <- never reached (id wins first)
    strategy aria-label    hits=0    misses=23
    strategy autocomplete  hits=0    misses=3

Read this as: on Taleo the id strategy is dead weight (0/80) and name does the work (77/80); on ORC the id strategy wins outright (57/80) and the later strategies are only tried when a section is still rendering. The "low" overall hit rate is expected and healthy — it counts the cheap early strategy misses that the fallback chain is designed to absorb. avgMs is dominated by ORC's async wait and Taleo's iframe navigation.

Failure modes observed / hardened

  • Submit marker in the wrong frame. First cut had the fixture Submit do window.location.href, which navigates the iframe, so the top-level #submitSuppressed wait timed out on every Taleo iframe run (8/10 Taleo failures). Fixed by having the fixture Submit navigate top.location — which is also how a real Taleo submit navigates the whole page. Lesson: when confirming a post-submit state, know which frame the navigation lands in.
  • TIME_WAIT port collisions across rapid iterations: the fixture no longer picks a port at all — LoopbackListener binds a free loopback port and re-picks if it loses the race, reporting the port it actually bound to (WW-86/WW-87). The old mitigation rotated through basePort + iter % 50, which only made a collision less likely and still handed the harness a port nobody had reserved.
  • Ambiguous aria-label filling the wrong input: mitigated by the >1-match skip rule.
  • Nested iframe not yet attached when we look for it: mitigated by a short poll over page.Frames.

Safety invariants (enforced + tested)

  • FakeOracleSite.BaseUrl always starts http://127.0.0.1: (asserted in tests).
  • Fixture HTML never references taleo.net or oraclecloud.com (asserted).
  • The only "submit" is /apply/submit, a no-op that increments SuppressedSubmitCount and returns a static page (asserted). The engine clicks Submit and asserts the suppressed marker — it never posts to any real endpoint.
  • Account creation is a simulated client-side hide; no credentials leave the process.

Running

# 20 iterations (default)
dotnet run --project tools/WorkWingman.ScraperLab.Oracle -- --iterations 20

# time-boxed long run
dotnet run --project tools/WorkWingman.ScraperLab.Oracle -- --minutes 10

# tests
dotnet test tools/WorkWingman.ScraperLab.Oracle.Tests

Learnings are appended as JSONL to bin/.../learnings/oracle-learnings-<timestamp>.jsonl, one object per iteration (mode, variation, per-field selector attempts, hit/miss counts, iframe switches, timing, failure mode, and the SubmitSuppressed flag).