Skip to content

Harness-agnostic grounded research (Technical)

Status: shipped on feat/research-harness. This document covers the research layer that makes WorkWingman's Study & prep research work with any harness — the existing NotebookLM export/MCP paths, the Claude-Project brief, and a new fully-local RAG harness that grounds answers in your prep sources with no external account at all.

Plain-language version: ../plain/research-harness.md

0. Why

The Study screen already had two research surfaces: a Claude Project (JD/resume/gaps loaded as project knowledge) and an opt-in NotebookLM export/publish. Both answer questions inside an external tool and require the user's own account. This branch adds a common seam so research is harness-agnostic, plus a Local RAG harness that answers in-app, grounded and cited, using only a local Ollama — no NotebookLM, no Claude account, no cloud call.

1. The seam

public interface IResearchHarness
{
    string Id { get; }
    string DisplayName { get; }
    Task IngestAsync(NotebookPrepPack pack, CancellationToken ct = default);
    Task<GroundedAnswer> AskAsync(string jobId, string question, CancellationToken ct = default);
}

GroundedAnswer carries the answer text plus the cited source chunks it drew from (GroundedCitation { Title, Snippet, Score }) and a Grounded flag. The citations are the anti-hallucination contract: an answer never invents facts beyond the retrieved chunks, and a harness with nothing ingested / an unreachable backend returns a non-grounded answer with guidance rather than a fabricated one.

Ingest consumes the harness-agnostic NotebookPrepPack — one titled Markdown doc per learning surface (JD & gaps, study plan, ATS/assessment, behavioral), built by NotebookPrepPackBuilder from the same job + study plan every prep surface uses.

2. Harnesses

Id Class Ingest Ask
local-rag LocalRagHarness Embed chunks locally (Ollama all-minilm:l6-v2), persist vectors to JSON Cosine-retrieve top-k, generate a cited answer via the selected ILlmHarness — in-app
notebooklm NotebookLmResearchHarness(ExportNotebookLmPack) Existing compliant local export Points at the user's NotebookLM notebook (answers there)
notebooklm-mcp NotebookLmResearchHarness(McpNotebookLmPublisher) Existing config-gated MCP publish Points at the created notebook
claude-project ClaudeProjectResearchHarness Existing GenerateClaudeStudyProjectAsync brief Points at the Claude Project

The NotebookLM/Claude adapters are thin wrappers over the existing publishers/service — all their guardrails are preserved verbatim: opt-in, per-publish confirmation (enforced in the API/frontend layer, not bypassed), no account creation, deep-link only. LocalRagHarness is the default when Ollama is reachable (ResearchHarnessSelector), because it needs no external account.

3. LocalRAG design

LocalRagHarness (Infrastructure/Research):

  1. Chunk — each source doc is split into ~900-char chunks with 150-char overlap, breaking on whitespace so no chunk splits mid-word (SplitIntoChunks, unit-tested).
  2. Embed — every chunk is embedded via ITextEmbedder → OllamaTextEmbedder (POST http://localhost:11434/api/embeddings, model all-minilm:l6-v2, 384-dim). No key, no egress.
  3. Persist — vectors are stored to research-rag-index JSON via LocalJsonStore, keyed by job id. Ingest is idempotent per pack (a job's chunks are replaced wholesale, never duplicated).
  4. Retrieve — the question is embedded with the same model; stored chunks are ranked by CosineSimilarity and the top-k (default 4) are kept as citations with their scores.
  5. Ground — the retrieved chunks are handed to the selected ILlmHarness (default OllamaLlmHarness, qwen3:8b) under a strict system prompt: answer only from the numbered sources, cite by number, say "not enough information" otherwise. Never fabricate.

Every failure is soft and honest: no embedder → "Local RAG unavailable"; nothing ingested → "nothing ingested"; no match → "no relevant sources"; LLM unreachable → citations still returned with a clear note so the user can read the sources directly. It never invents an answer.

The ILlmHarness contract (frozen)

LocalRAG grounds through the frozen ILlmHarness / ILlmHarnessSelector seam so any generation backend (local Ollama, the user's Claude CLI, a hosted API) can be swapped in without touching retrieval. This branch registers a local OllamaLlmHarness + first-wins LlmHarnessSelector; the canonical richer selector is deduped at merge.

4. Endpoints (StudyController)

  • GET /api/study/research/harnesses — picker list + availability (LocalRAG probes Ollama).
  • POST /api/study/{jobId}/research/ingest { harnessId } — build the pack + ingest. The notebooklm-mcp harness reaches the network under the user's Google login, so this endpoint keeps the same cross-origin ApiToken trust gate as the direct publish-mcp endpoint (defense in depth on top of the frontend confirm). Every other harness is local / deep-link only.
  • POST /api/study/{jobId}/research/ask { question, harnessId } → GroundedAnswer.

5. Frontend

The real Study module (frontend/src/app/features/study) gains a Grounded research panel: a harness picker (chips with availability + default badge), an Ingest button, and — for LocalRAG — an in-app grounded-Q&A box that renders the answer, a Grounded ✓ badge, and each cited source with its relevance score and snippet. External harnesses show a deep-link pointer instead. Loading, error, empty, and not-available states are all handled. Wired against the real API — verified live.

6. Live RAG smoke (verified)

Against the real local Ollama (all-minilm:l6-v2 + qwen3:8b), end-to-end through the HTTP API:

  • Ingest of job-mastercard's prep pack → 4 sources embedded and persisted.
  • Ask "What ATS or assessment platform does this job use?" → grounded answer: "The job uses the Workday ATS platform. This information is specified in source [1]." — top citation ATS & assessment platform (score 0.53), grounded=true, no fabrication. Verified both via curl to the API and through the wired Angular panel in-browser.

7. Tests

  • LocalRagHarnessTests — retrieval + grounding with a deterministic fake embedder + fake LLM (right chunk retrieved; generator handed only retrieved chunks; before-ingest / LLM-unreachable / embedder-unavailable soft paths; idempotent ingest; cosine + chunking math).
  • ResearchHarnessAdapterTests — the NotebookLM/Claude adapters delegate to the existing publishers/service and never fabricate; selector defaults to LocalRAG.
  • ResearchServiceTests — pack building, harness resolution, no-job soft failure, availability.
  • StudyControllerResearchTests — endpoint wiring + the notebooklm-mcp token gate.
  • NotebookPrepPackBuilderTests — new null-safety regression (persisted jobs with null Description/Fit/Ats/Resources no longer NRE the builder, which the research path first exercised).
  • Frontend study.spec.ts — panel load/default, ingest→ask grounded-cited render, harness-switch reset, external-harness confirmation gate, error state.