Skip to content

PageAgent vs. Playwright — Feasibility (Technical)

Status: research spike, no code changed. This document evaluates whether Alibaba PageAgent (npm: page-agent, docs: alibaba.github.io/page-agent) could replace or supplement Microsoft.Playwright in WorkWingman's automation layer (src/WorkWingman.Infrastructure/Automation/*). It answers five questions: Claude support, integration architecture, the "one prompt replaces all selectors" thesis, cost/latency/safety tradeoffs, and a recommendation.

Plain-language version: ../plain/pageagent-feasibility.md

0. What PageAgent actually is

PageAgent (22.6k★, MIT, TypeScript monorepo, latest v1.11.0) is "the GUI Agent living in your webpage" — an in-page JS agent that takes a natural-language instruction (agent.execute('Click the login button')) and drives the DOM itself, via an LLM-in-the-loop agentic step cycle. It is explicitly client-side web enhancement, not server-side browser automation — there is no headless-browser server, no Node/Python driver process. It runs inside a page the user already has open, either via <script>/npm import or, for cross-page/multi-tab tasks, an optional Chrome extension.

Monorepo layout (packages/):

Package Role
@page-agent/core Headless agent loop (PageAgentCore), tools, prompts — no UI
@page-agent/page-controller DOM reading/writing: builds a "FlatDomTree", indexes interactive elements, executes actions
@page-agent/llms LLM client abstraction + the one shipped provider, OpenAIClient
@page-agent/page-agent Public page-agent npm package — adds a UI panel on top of core
@page-agent/ui Panel widget + i18n
@page-agent/extension Chrome extension (WXT + React) for multi-tab/page tasks
@page-agent/mcp MCP server — lets an external AI client (Claude Desktop, Copilot) drive a browser through the PageAgent extension
@page-agent/website Docs/landing site

DOM pipeline: Live DOM → FlatDomTree (page-controller) → simplified text description of interactive elements, each given a stable numeric index → sent to the LLM as part of the prompt → LLM picks an index and an action → page-controller executes it (click/type/etc. by index) → tree rebuilt → loop continues. No screenshots, no vision model, no coordinates — this is the "text-based DOM manipulation" the task description calls out, and it's confirmed at the source level (PageAgentCore.ts, page-controller).

1. Claude / Anthropic support

Short answer: not out of the box, but pluggable — with real friction.

What ships today

@page-agent/llms exports exactly one concrete provider client: OpenAIClient. There is no AnthropicClient, ClaudeClient, Bedrock, or Vertex client anywhere in the monorepo. The developer guide's only documented configuration path is three env vars — LLM_MODEL_NAME / LLM_API_KEY / LLM_BASE_URL — pointed at an OpenAI-compatible /chat/completions endpoint (the shown examples are Alibaba's DashScope/Qwen and a local Ollama). "Bring your own LLM" in this project's docs means "bring your own OpenAI-compatible endpoint," not "bring your own provider SDK."

The interface a provider must satisfy

// packages/llms/src/types.ts (paraphrased)
interface LLMClient {
  invoke(
    messages: Message[],
    tools: Record<string, Tool>,
    abortSignal?: AbortSignal,
    options?: InvokeOptions,
  ): Promise<InvokeResult>
}

interface Message {
  role: 'system' | 'user' | 'assistant' | 'tool'
  content?: string | null
  tool_calls?: { id: string; type: 'function'; function: { name: string; arguments: string } }[]
  tool_call_id?: string
  name?: string
}

interface Tool<TParams = any, TResult = any> {
  description?: string
  inputSchema: z.ZodType<TParams>       // Zod schema, converted to JSON Schema per-provider
  execute: (args: TParams) => Promise<TResult>
}

This is a single-tool-call-per-step, forced tool-use loop: PageAgentCore packs every available action into one big "MacroTool" (evaluation_previous_goal, memory, next_goal, action: { [toolName]: toolInput }) and calls invoke() once per agent step, expecting exactly one tool call back. The OpenAIClient reference implementation sends:

POST {baseURL}/chat/completions
Authorization: Bearer {apiKey}
{
  model, messages, tools,
  parallel_tool_calls: false,
  tool_choice: "required"   // or {type:'function', function:{name}} to force a specific tool
}
and reads choice.message.tool_calls[0].function back out. No streaming is used — it's one blocking call per step.

Can Claude satisfy this contract?

Yes, structurally — Claude's Messages API supports forced tool use (tool_choice: {type: "any"} to force some tool call, or {type: "tool", name: "..."} to force a specific one) and returns tool_use content blocks with a name + JSON input, which is semantically equivalent to OpenAI's tool_calls[].function. Claude does not speak the OpenAI wire format directly, though — different endpoint (/v1/messages vs /v1/chat/completions), different envelope (content: [{type:'tool_use', ...}] vs tool_calls: [{function:{...}}]), different single-system-message convention, and a different error/rate-limit response shape than errors.ts expects (NETWORK_ERROR, RATE_LIMIT, SERVER_ERROR, AUTH_ERROR, CONTEXT_LENGTH, CONTENT_FILTER, plus PageAgent-specific NO_TOOL_CALL / INVALID_TOOL_ARGS / INVALID_RESPONSE / INVALID_SCHEMA).

Two concrete integration paths:

Path A — OpenAI-compatible proxy in front of Claude (fastest, zero PageAgent code changes). Point LLM_BASE_URL at any proxy that translates OpenAI chat-completions calls to Anthropic Messages calls (e.g. a small local Express/Node shim, or a hosted compatibility layer). This is the path the task hint already anticipates ("an OpenAI-compatible proxy") and it needs zero changes inside PageAgent — OpenAIClient just talks to a different baseURL. Effort: build and host a thin translation shim (request/response reshaping + SSE-off since PageAgent doesn't stream); this is genuinely a few hundred lines, but it's new infrastructure WorkWingman would own and maintain, not something PageAgent gives you.

Path B — write a real AnthropicClient : LLMClient (clean, more work). Implement invoke() against https://api.anthropic.com/v1/messages directly: - Map messages[] → Anthropic's system (top-level string, pulled out of the array) + messages[] with user/assistant roles only (Anthropic has no system or tool role in the array — tool results become user messages containing a tool_result content block keyed by tool_use_id). - Convert each Tool.inputSchema (Zod) → Anthropic's tool JSON Schema shape ({name, description, input_schema}}) instead of OpenAI's {type:'function', function:{...}} wrapper — the existing zodToOpenAITool() helper is not reusable as-is; a zodToAnthropicTool() sibling is needed (small, since both are JSON Schema underneath). - Set tool_choice: {type: "any"} (equivalent of tool_choice: "required") or {type: "tool", name} for forced single-tool steps. - Parse response.content for the tool_use block (name, input — already an object, not a JSON string, so the JSON.parse() step OpenAIClient does is skipped) instead of choice.message.tool_calls[0]. - Map Anthropic's stop_reason / HTTP errors onto PageAgent's InvokeErrorTypes (overloaded_error → SERVER_ERROR, invalid_request_error on auth → AUTH_ERROR, max_tokens/context overflow → CONTEXT_LENGTH, etc.). - Map usage.input_tokens/output_tokens/cache fields → PageAgent's usage shape (prompt/completion/cached/reasoning tokens).

This is a genuinely small, well-scoped adapter — maybe 150–250 lines plus tests — because the LLMClient interface is narrow and PageAgent's own OpenAIClient is the template. It's the kind of "one obvious replacement point" pattern this repo already uses for other stubs (UnconfiguredGoogleTokenProvider, TemplateDrafter : IClaudeDrafter). Nothing in PageAgent's core loop, tool set, or prompts is OpenAI-specific — the coupling is isolated to one package (@page-agent/llms).

Verdict: Claude can be wired in, but WorkWingman (or a community contributor) has to write the adapter — PageAgent does not ship one, and no GitHub issue/PR reviewed here indicates one is planned. Budget it as a small, contained TypeScript task, not a blocker, but not "free" either.

2. Integration architecture — where would this actually live?

WorkWingman's automation today is entirely server-side C#:

flowchart LR
    Angular["Angular renderer<br/>(Live Run viewer — placeholder today)"]
    API[".NET API<br/>controllers"]
    Engine["WorkdayAutomationEngine<br/>(C#)"]
    Session["BrowserSession<br/>Microsoft.Playwright<br/>persistent Chromium profile"]
    ATS["Workday tenant<br/>(real browser, visible)"]

    Angular -->|HTTP loopback| API --> Engine --> Session --> ATS
    Angular -.polls RunStatus.-> API

BrowserSession (src/WorkWingman.Infrastructure/Automation/BrowserSession.cs) launches a headed, persistent Chromium context via Microsoft.Playwright — this is deliberate: the "Live Run viewer" is meant to mirror a real, visible browser window so the user can watch and intervene. WorkdayAutomationEngine drives that page with Playwright.FillAsync/ClickAsync using selector strings (data-automation-id*='email'), logs every fill with a confidence score, and raises a JudgementCall that pauses the run when confidence is below threshold.

PageAgent is architecturally the opposite shape: it must run inside a JS execution context that has direct DOM access to the target page — i.e., inside the page's own window, or in a content-script-like context with access to that document. It cannot run inside .NET, and it cannot be imported into WorkdayAutomationEngine.cs — there is no cross-language embedding story (no Node-in-.NET here; ClearScript/Jint could run some JS but not a DOM-manipulating agent with no DOM to manipulate).

Where it would have to live, concretely

Two realistic placements, both requiring new plumbing that doesn't exist today:

Option 1 — content script inside an Electron WebContentsView (matches the existing roadmap stub). architecture.md already documents "Live-run browser — the Angular viewer shows a placeholder; in Electron, attach a WebContentsView and drive it from WorkdayAutomationEngine" as a deliberate stub. If that WebContentsView is adopted instead of (or alongside) Playwright's Chromium: - The Electron main process creates a WebContentsView pointed at the Workday apply URL (Electron owns this window instead of Playwright's persistent-context Chromium — losing Playwright's persistent-profile/anti-automation flags unless re-implemented via Electron session partitions and Session.webRequest). - PageAgent (page-agent npm package) is injected into that view's renderer via webContents.executeJavaScript(...) or a Chrome DevTools Protocol (Target.exposeDevToolsProtocol)-style preload script — effectively the same trick Electron already uses for its own preload.js. - The .NET backend cannot call PageAgent's JS API directly — it has to go through Electron. That means a new IPC leg: .NET API → (new) local control channel → Electron main → WebContentsView.webContents.executeJavaScript('agent.execute("...")') and the reverse for results/events (onBeforeStep/onAfterStep/onAskUser callbacks would need to be forwarded back over that same channel as SSE/WebSocket/polling, since the .NET API and Electron main are separate processes today — the app already bridges Angular↔Electron via electron/preload.js's contextBridge, but there is no existing API↔Electron-main channel; today Electron only spawns the API as a child process and loads the Angular build). - Concretely new pieces needed: (a) a local RPC channel .NET↔Electron (named pipe, loopback WebSocket, or a small HTTP callback server in Electron main), (b) a PageAgent injection + result-relay layer in electron/main.js, (c) a rewrite of WorkdayAutomationEngine's public surface to speak "task strings in, step events out" instead of "selector strings in, IPage out."

Option 2 — PageAgent's own MCP server as the bridge (@page-agent/mcp). @page-agent/mcp already exists to let an external AI client (its docs name Claude Desktop and Copilot) drive a browser through the PageAgent Chrome extension over a local WebSocket hub (default port 38401). In theory the .NET API could itself be an MCP client, calling execute_task on that server instead of building custom IPC. But this pulls in the Chrome extension (a third install/permission surface beyond the app itself) and assumes the user's regular Chrome — not Electron's embedded Chromium — is the automation target, which conflicts with the existing "invisible unless watching the Live Run view" model and the persistent-profile design in BrowserSession. This path trades custom IPC for a heavier dependency (extension + its own WebSocket hub) and is a worse fit for an embedded, single-purpose automation view.

What would still be Playwright (if anything)

Playwright's actual value in BrowserSession isn't "clicking things" — it's: - Persistent authenticated profile management (LaunchPersistentContextAsync against a fixed profile dir so LinkedIn/Workday logins survive across runs); - Anti-detection launch flags (--disable-blink-features=AutomationControlled, real UA, SlowMo); - Cookie extraction for non-browser HTTP calls (GetCookieHeaderAsync, handed to Flurl clients); - The only Playwright.NET E2E harness (tests/WorkWingman.E2E) that drives the built Angular app itself (not the ATS) against a stubbed API via route interception — this is unrelated to ATS automation and has no PageAgent equivalent at all; it would not change regardless of what happens to WorkdayAutomationEngine.

If PageAgent replaced the in-page action-taking layer, Playwright (or Electron's own WebContentsView/session APIs) would likely still own outer harness concerns: launching the browsing surface, session/profile persistence, cookie/header extraction for the Flurl-based scraping paths (LinkedInJobsScraper largely doesn't need an LLM at all — it's bulk link harvesting + fallback-selector text extraction, which is a batch/read-only task PageAgent isn't designed for and an LLM call per field would be wasteful for). tests/electron-e2e (the Playwright-for-Electron smoke test) is also unaffected — it tests that the shell boots and reaches the loopback API, nothing about ATS automation.

3. The big thesis — does natural language replace per-ATS selector engineering?

The premise, calibrated to what's actually in this codebase today: WorkWingman currently hand-codes one fallback-selector case in production automation code — WorkdayAutomationEngine.SignInAsync's input[type='text'][data-automation-id*='email'], input[type='email'] — plus a broader, well-documented fallback-chain pattern in LinkedInJobsScraper.FirstTextAsync (title/ company/location/description each tried against 2–4 selector variants because "LinkedIn markup shifts often"). AtsDetector only classifies Greenhouse/Lever/iCIMS/UKG from URL/HTML markers — it does not yet drive any of them. The generalized claim in the research prompt (ADP uses formcontrolname, iCIMS embeds forms in iframes, etc.) is a plausible industry-general pattern — each ATS vendor does have its own DOM conventions — but it is not yet something this repo's code demonstrates first-hand beyond Workday + LinkedIn. Treat the thesis as "would this generalize to ATSes WorkWingman hasn't automated yet," not "this replaces code that's already been painfully hand-tuned across five ATS vendors here."

Is "one natural-language prompt per field, across ATS vendors" plausible? Partially, with real caveats:

  • Plausible upside: PageAgent's DOM-to-text pipeline (FlatDomTree → indexed interactive elements) is vendor-agnostic by construction — it doesn't know or care whether an input has data-automation-id, formcontrolname, or no attribute at all; it just needs the element to be in the accessibility/DOM tree with enough surrounding text (label, placeholder, nearby heading) for the LLM to identify it as "the Education section." That's exactly the generalization Workday's data-automation-id*='email' and ADP's formcontrolname selectors can't give you — a selector chain must be written and maintained per vendor; a natural-language instruction in principle does not.
  • iframes are the sharpest real risk. iCIMS (and many ATS embeds) put the actual application form inside an <iframe>. Playwright has first-class frame_locator support for this. PageAgent's DOM pipeline, as documented, walks the live DOM tree it has access to — same-origin iframes are generally reachable via contentDocument, but cross-origin iframes are opaque to any in-page script for the same reason they're opaque to any other page-embedded JS (browser same-origin policy) — page-agent's own docs/tools list shows no explicit "switch into iframe N" tool, unlike Playwright's explicit FrameLocator API. This is a structural limitation of "runs as page JS," not a bug to be fixed — an in-page agent is bound by the same cross-origin sandboxing as any other script on that page. Multi-page ATS flows crossing origins (SSO redirects, embedded payment/background-check widgets) hit the same wall.
  • No selector chain still means no selector chain to maintain — but it doesn't mean zero per-vendor work. Even "one prompt" needs page-specific instructions/customSystemPrompt context to work well in practice (PageAgent's config exposes exactly this: getPageInstructions(), customSystemPrompt). In effect, hand-written selector fallback chains would be replaced by hand-written prompt/context tuning per ATS quirk — real engineering effort moves from "guess the CSS" to "guess the wording and the guardrails," which is a different and probably smaller-surface problem, but not a zero-effort one.
  • Determinism goes down exactly where selectors made it predictable. A selector chain either matches or doesn't — it's data, testable offline. A natural-language step asks an LLM to look at a DOM snapshot and choose an index; the same page can, in principle, get a different action on a different run (see §4).
  • Confidence-gated judgement calls become harder to reproduce. WorkdayAutomationEngine.TryFillFieldAsync already has an explicit, deterministic confidence/threshold check before trusting a fill. PageAgent's loop bakes reflection into the model's own evaluation_previous_goal/memory free text — there's no numeric confidence value surfaced by the framework the way WorkWingman's RankedOption/JudgementCall model expects. Bridging "LLM free-text self-assessment" into "numeric confidence vs. threshold, ranked alternatives" is new glue code either way.

Verdict on the thesis: plausible as a direction — the core insight (bypass DOM-shape-specific selectors, key off visible text/labels/role) is sound and is exactly why frameworks like this and browser-use (which PageAgent explicitly derives components from) exist. But "replace ALL per-ATS selector engineering with ONE prompt" overstates it: iframes, cross-origin flows, and the loss of a first-class confidence/threshold model mean the realistic outcome is fewer hand-tuned selectors, not zero hand-tuned anything, and a shift of engineering effort from CSS chains to prompt/guardrail chains.

4. Tradeoffs

Cost — LLM calls per application

PageAgent calls the LLM once per agent step (one invoke() per iteration of the execute() loop; maxSteps defaults to 40). A single Workday application, mapped onto WorkdayAutomationEngine.StepLabels, plausibly decomposes into many agent-steps, not one call per page section — because each field-level action (click a text input, type into it, open a dropdown, select an option) is its own step in PageAgent's model, not something batched into one call per section. Rough estimate for one Workday application:

Section Fields/actions Est. agent steps (≈1 LLM call each)
Sign in email, password, submit 3–4
Contact info name, address, phone, email confirm 6–10
Work history 2–3 jobs × (title, employer, dates, description) 15–30
Education school, degree, field, dates 6–10
Skills & languages multi-select / tag inputs 4–8
Portfolio links GitHub + gitfront URLs 2–4
Demographics (EEO) 4–6 dropdowns 4–8
Review & submit (pause, no auto-submit) scroll + verify, no submit call 2–4
Total ~45–80 LLM calls per application

That is a lot more than Playwright's zero LLM calls for the equivalent flow today (Playwright selectors are free at runtime; the only "cost" is the one-time engineering effort to write them). At typical hosted-model pricing this is real but not huge per application (tens of cents to roughly a dollar depending on model tier and prompt-context size — PageAgent resends a running <agent_history> block each step, so cost grows worse than linearly with step count unless the implementation prunes history). At WorkWingman's actual scale (one user, some number of applications per week) this is affordable; it stops being affordable/attractive the moment "every field is an LLM call" meets a job-search user applying to dozens of jobs a day, or a future multi-tenant hosted version.

Latency

Each step is a full round trip to a hosted LLM (or a local Ollama model, per PageAgent's own example — which trades API latency for GPU-bound local inference latency). 45–80 sequential steps (the loop is inherently sequential — each step's action depends on the DOM state produced by the last) at even 1–3 seconds per call is 1–4 minutes of LLM think-time alone per application, before accounting for page load/render waits already in the loop (wait tool, stepDelay). Playwright's equivalent flow runs in seconds once selectors are correct — the 200-800ms SlowMo/PageDelay pacing WorkWingman uses today is deliberately-added human-like delay, not a floor imposed by the automation method.

Determinism / reproducibility

This is the sharpest tradeoff. WorkWingman's testing philosophy is explicit: "Deterministic, always... Flakiness is treated as a defect in the test, not a nuisance to paper over" (docs/technical/testing.md). Playwright selectors are exactly that: the same DOM produces the same fill, every time, and that's provable/testable without a live network call. An LLM-in-the- loop agent is fundamentally not deterministic in that sense — even at low temperature, model updates, provider-side routing, and subtly different DOM snapshots (ads, A/B tests, loading races) can change which index gets clicked. This directly conflicts with the repo's "determinism/churn" CI job, which re-runs suites ×5 and fails on any flip — an LLM-driven automation path is close to un-unit-testable in that framework without heavy mocking of the LLM client itself (which then tests the harness, not the actual field-selection behavior).

The never-auto-submit safety contract

WorkWingman's hard rule — "the final submit step ALWAYS waits for the user (locked behavior)" (WorkdayAutomationEngine class doc) and "It never hits 'submit' without you" (docs/plain/architecture.md) — is currently enforced in code: there is simply no call path that reaches a submit click without a human-triggered resolve on a paused run. PageAgent has no equivalent framework-level guard. Its tool set (click_element_by_index, input_text, select_dropdown_option, scroll*, execute_javascript, wait, ask_user, done) treats a submit button as just another clickable index — nothing in PageAgentCore's hooks (onBeforeStep/onAfterStep/onBeforeTask/onAfterTask) is action-aware enough to intercept "the LLM is about to click the button labeled Submit" without WorkWingman writing that interception itself (e.g., a customTools override that removes/wraps click_element_by_index to inspect the target element's text/role before allowing a click near known submit-like labels, or restricting the tool it forces on the "review" step to ask_user/done only). This is buildable — PageAgent's onAskUser and customTools extension points are exactly the seam you'd use — but it is not a safety net PageAgent gives you for free; today's guarantee comes from WorkdayAutomationEngine simply never calling a submit selector, and any PageAgent integration has to re-earn that guarantee explicitly, per ATS, forever, since the LLM chooses actions dynamically rather than following a fixed code path.

Offline / local-fixture testability

Playwright's route-interception pattern already used in tests/WorkWingman.E2E (StaticSiteServer + StubbedApi, CORS-hardened, driving the built Angular app against fully local fixtures) has no analogue for PageAgent without also mocking the LLM client. You could feed PageAgent a local static HTML fixture of a Workday page (page-controller doesn't care where the DOM comes from), but every step still calls a real or mocked LLM — testing "does PageAgent fill this fixture correctly" means either (a) burning real LLM calls in CI (slow, costs money, non-deterministic — directly against the churn-test philosophy), or (b) mocking LLMClient. invoke() to return canned tool calls, which tests the harness plumbing but not the thing you actually care about (does the model correctly interpret this DOM). Playwright selector tests have no such dilemma — the assertion is "did this selector find and fill the right element."

Failure modes

Failure mode Playwright (today) PageAgent
Selector doesn't match (markup change) Silent skip / fallback chain exhausted → empty field, logged LLM picks some index — could silently fill the wrong field with a plausible-looking but incorrect value
Ambiguous field N/A — selectors are exact or absent LLM may guess confidently and wrongly (hallucinated confidence) rather than surfacing a JudgementCall
Rate limit / auth / outage on the model provider N/A Entire run stalls or errors (NO_TOOL_CALL, RATE_LIMIT, AUTH_ERROR from errors.ts) — a new external dependency in the critical path of every field, not just document drafting
Cross-origin iframe content Explicit FrameLocator API handles it Opaque/inaccessible to in-page JS (see §3)
Model does something destructive Not possible — code path doesn't include destructive selectors Possible unless explicitly guarded (see above)

5. Recommendation

Not a wholesale replacement. Not "not yet" either — a scoped, opt-in hybrid, and only after the Playwright-based Workday path actually exists end-to-end.

Today WorkdayAutomationEngine's apply-automation is itself only partially wired (per connections-and-sso.md: "Seeded metadata... apply automation not wired"), and the Live Run viewer is a documented placeholder. There is no working Playwright ATS-fill baseline yet to "replace." That changes the framing: this isn't yet a rip-and-replace decision, it's a build decision — and PageAgent is a legitimate one of two build paths, not a strict upgrade over the other.

Where PageAgent's strengths land

  • Novel/unautomated ATS vendors the app hasn't hand-coded yet (Greenhouse, Lever, UKG, future ones AtsDetector already classifies but doesn't drive) — where the alternative is writing a new selector chain from scratch anyway. Here PageAgent's marginal cost (an LLM call per field) competes directly against the marginal cost of engineering time to hand-roll selectors for a vendor used rarely. That's a genuinely favorable trade for a long-tail vendor.
  • A fallback path when a known-good selector chain breaks — Workday changes markup; instead of the run failing outright, drop into a PageAgent-driven attempt for that one field/section, flagged clearly in the run log as "AI-assisted fallback" (lower trust, always routes to a JudgementCall-equivalent confirmation regardless of the model's stated confidence).

Where Playwright should stay in charge

  • Workday (the one ATS with committed selectors already) and any other vendor once its selector chain is written and tests exist — deterministic, fast, free, already fits the CI determinism-churn model.
  • The never-auto-submit guarantee — keep this enforced by the harness, not by trusting an LLM (or a PageAgent guard) to reliably self-censor. Concretely: whatever drives the final review/submit step should never route to a tool capable of a submit-labeled click, regardless of which engine filled the preceding fields.
  • LinkedIn scraping (LinkedInJobsScraper) — bulk, read-only, no judgement calls; an LLM call per link would be pure waste against a pattern that already works.
  • The outer harness — profile persistence, anti-detection flags, cookie handoff to Flurl, and both existing E2E suites (tests/WorkWingman.E2E, tests/electron-e2e) are Playwright- shaped problems unrelated to in-page field-filling and are unaffected either way.

Concrete phased plan

  1. Phase 0 (prerequisite, not PageAgent-related): finish the Playwright Workday path. Wire WorkdayAutomationEngine end-to-end against a real tenant behind a feature flag, with the WebContentsView Live Run viewer from the architecture roadmap. This is necessary regardless of PageAgent's fate — right now there's no working baseline to compare against or fall back to.
  2. Phase 1 — spike the Anthropic adapter in isolation. Write and unit-test an AnthropicClient : LLMClient (or a thin OpenAI-compatible proxy, whichever is faster to ship) against @page-agent/llms's existing test patterns (OpenAIClient.test.ts as the template), with no WorkWingman integration yet. This de-risks "can Claude drive PageAgent at all" cheaply and separately from the harder Electron/IPC question.
  3. Phase 2 — prototype the Electron bridge on one throwaway page. Build the minimal .NET ↔ Electron main ↔ WebContentsView control channel against a single local HTML fixture (not a live ATS), proving execute("fill the email field") round-trips step events back to the .NET side. Measure real steps-per-field and $/application on that fixture before touching a live tenant.
  4. Phase 3 — hybrid fallback, one ATS vendor, feature-flagged and off by default. Wire PageAgent as the fallback path only when a Playwright selector chain is absent or has exhausted its fallbacks for a given field, on a vendor Playwright doesn't yet support (Greenhouse or Lever, since AtsDetector already classifies them). Every PageAgent-filled field routes through the existing JudgementCall pause regardless of the model's self- reported confidence, until real-world accuracy data justifies loosening that.
  5. Phase 4 — decide, with data. After N real applications through the hybrid path, compare fill accuracy, $/application, time/application, and judgement-call rate against the Playwright baseline. Only expand PageAgent's footprint (more vendors, higher auto-trust) if the data clears a bar WorkWingman sets in advance — don't let "it worked in the demo" be the bar.

Bottom line: treat PageAgent as a fallback/long-tail-vendor tool bolted onto a Playwright-first architecture, not a Playwright replacement. The natural-language DOM abstraction is real and valuable specifically where hand-written selectors don't exist yet or keep breaking; it is not a safe or cheap substitute for the deterministic, free, already-tested path WorkWingman has for Workday, and it does not come with the never-auto-submit guarantee for free — that has to be re-built and re-verified per integration.