docs(19-01): complete classification engine pure functions + orchestrator plan

- 19-01-SUMMARY.md: task commits, decisions, deviations, self-check - deferred-items.md: logs 2 pre-existing unrelated itglue-search.test.ts failures discovered during full `npm test` run (out of scope, not fixed)
2026-07-16 08:22:14 -04:00 · 2026-07-16 08:22:14 -04:00 · 63f173ea44
commit 63f173ea44
parent 38c1ae4daf
2 changed files with 143 additions and 0 deletions
--- a/.planning/phases/19-classification-engine/19-01-SUMMARY.md
+++ b/.planning/phases/19-classification-engine/19-01-SUMMARY.md
@ -0,0 +1,128 @@
+---
+phase: 19-classification-engine
+plan: 01
+subsystem: security
+tags: [phishing-triage, deterministic-classifier, vitest, postgres, mimecast]
+
+# Dependency graph
+requires:
+  - phase: 16-eml-mime-evidence-parser
+    provides: "NormalizedMessage / AuthResults shapes, messages.headers JSONB (eml-parser.ts)"
+  - phase: 17-mimecast-blast-radius-lookup
+    provides: "getBlastRadius() delivered/held/rejected/clicked lookup (mimecast-blast-radius.ts)"
+  - phase: 18 (campaign grouping)
+    provides: "campaigns/reports linkage, ORDER BY created_at ASC 'earliest is canonical' convention"
+provides:
+  - "classifyCampaign(campaignId) — deterministic SPAM/UNWANTED/THREAT verdict engine, no LLM calls"
+  - "KNOWN_SIMULATION_SENDERS allowlist (it-support.care, breachsecurenow.com) + domainMatchesAllowlist/isKnownSimulationSender"
+  - "gatherCampaignEvidence(campaignId) — bounded evidence assembly (reports/messages/indicators + one getBlastRadius call)"
+  - "classifications table INSERT (append-only, D-02) with verdict/confidence/reasons/recommendedActions/requiresApproval"
+affects: ["20 (remediation actions)", "phishing/campaigns/{id}/classify route"]
+
+# Tech tracking
+tech-stack:
+  added: []
+  patterns:
+    - "Pure evidence-in / rule-eval / verdict-out module shape (mirrors robotic-classifier.ts, campaign-grouping-service.ts)"
+    - "vi.mock query-router-by-SQL-substring test pattern (mirrors campaign-grouping-service.test.ts)"
+
+key-files:
+  created:
+    - lib/services/campaign-classifier.ts
+    - lib/services/campaign-classifier.test.ts
+    - lib/services/campaign-classifier.fixtures.ts
+  modified: []
+
+key-decisions:
+  - "D-06 allowlist stored as a TypeScript constant (KNOWN_SIMULATION_SENDERS), not a DB table — logic-drift mitigation (T-19-02) via unit-tested pure code"
+  - "domainMatchesAllowlist uses exact-or-proper-subdomain match only (d===allowed || d.endsWith('.'+allowed)) — never .includes() substring matching (T-19-01)"
+  - "D-03 THREAT gate requires BOTH delivered>0/clicked>0 AND (hard auth fail OR known-bad-indicator match spanning >=2 messages) — either signal alone stays at UNWANTED"
+  - "D-04 UNWANTED fires on ANY suspicious signal (single attachment/url indicator, or delivery contained to the reporter(s) only); SPAM only when neither is present"
+  - "ASSUMPTION FLAG (surfaced per plan instruction, needs user confirmation): THREAT recommendedActions escalate to reset_password/isolate_endpoint/disable_forwarding_rule only when blastRadius.clicked>0 — this is a reasoned research proposal (19-RESEARCH.md Open Question #1), NOT an explicit D-08 decision. Does not contradict any locked decision; materially affects what Phase 20 gates approval on."
+
+requirements-completed: [CLASSIFY-01, CLASSIFY-02, CLASSIFY-03, CLASSIFY-04, CLASSIFY-06]
+
+# Metrics
+duration: 15min
+completed: 2026-07-16
+---
+
+# Phase 19 Plan 01: Classification Engine — Pure Rule Functions + classifyCampaign Orchestrator Summary
+
+**Deterministic SPAM/UNWANTED/THREAT classifier with a KnowBe4/Breach-Secure-Now simulation allowlist, D-03 THREAT gate (delivery + malicious signal), D-04 SPAM/UNWANTED split, D-05 confidence scoring, and an append-only `classifications` INSERT — zero LLM calls.**
+
+## Performance
+
+- **Duration:** ~15 min (first commit 08:14:51 → last commit 08:20:58)
+- **Started:** 2026-07-16T08:14:51-04:00
+- **Completed:** 2026-07-16T08:20:58-04:00
+- **Tasks:** 2 completed
+- **Files modified:** 3 (all new)
+
+## Accomplishments
+
+- Pure rule functions (`domainMatchesAllowlist`, `isKnownSimulationSender`, `effectiveAuthResults`, `hasHardAuthFail`, `computeConfidence`, `mapVerdictToActions`, `computeRequiresApproval`) — each independently unit-tested, no DB/Mimecast dependency
+- `classifyCampaign(campaignId)` orchestrator: gathers campaign evidence, applies D-06 simulation short-circuit → D-03 THREAT tier → D-04 SPAM/UNWANTED split → D-05 confidence → D-08 action mapping, and appends one `classifications` row (no `ON CONFLICT`)
+- Positive-path THREAT proven with a real (non-simulation) signal fixture — closes the "always-SPAM stub" blocker the plan explicitly flagged
+- D-03's known-bad-indicator OR-branch proven independently of the auth-fail path (auth PASS + shared indicator across 2 messages → THREAT)
+- D-04's SPAM/UNWANTED boundary proven with two distinct fixtures (not just a single always-one-verdict stub)
+- Synthetic KnowBe4 (`it-support.care`) and Breach Secure Now (`breachsecurenow.com`) simulation fixtures reproduce the Pitfall-1 forwarding-induced auth-verdict inversion, proving both the D-06 allowlist short-circuit and the `authResultsOriginal` precedence in one test
+
+## Task Commits
+
+Each task followed the TDD RED → GREEN cycle with a `test(...)` commit before its `feat(...)` commit:
+
+1. **Task 1: Failing tests + fixtures + pure rule functions**
+   - `f4e6baf` (test) — failing tests + synthetic fixtures for classifier pure rule functions
+   - `3ea6c95` (feat) — implemented pure rule functions (D-05/D-06/D-08); all 30 tests green
+2. **Task 2: classifyCampaign orchestrator + evidence gathering + append-only INSERT**
+   - `f6c954a` (test) — classifyCampaign orchestrator tests (mocked postgres-client + mimecast-blast-radius)
+   - `38c1ae4` (feat) — implemented `gatherCampaignEvidence` + `classifyCampaign`; all 39 tests green
+
+**Plan metadata:** this SUMMARY.md commit (see below)
+
+## Files Created/Modified
+
+- `lib/services/campaign-classifier.ts` — `classifyCampaign` orchestrator, `gatherCampaignEvidence`, pure rule functions, `KNOWN_SIMULATION_SENDERS` constant, `ClassifyResult`/`CampaignEvidence`/`ParsedMessage` types
+- `lib/services/campaign-classifier.test.ts` — 39 tests across 9 pure-function describe blocks + `classifyCampaign` orchestration describe block (mocked `postgres-client`/`mimecast-blast-radius`)
+- `lib/services/campaign-classifier.fixtures.ts` — synthetic KnowBe4/BSN simulation fixtures + non-simulation threat/clean-spam/suspicious-unwanted fixtures (all invented content, no real customer email)
+
+## Decisions Made
+
+- Kept `isKnownSimulationSender`'s parameter type narrowed to a new `SenderIdentity` interface (`{ from: { domain }, returnPath }`) instead of the full `NormalizedMessage`, so both the Task 1 fixtures (typed as full `NormalizedMessage`) and Task 2's bounded `ParsedMessage` (built from `messages.headers` JSONB) can share the same allowlist-check function without type friction.
+- `evaluateSpamVsUnwanted`'s "suspicious signal" definition (any attachment/url indicator present OR delivery contained to the reporter(s) only) was Claude's Discretion per CONTEXT.md D-04 — implemented as an OR of the two conditions so either alone is enough to clear SPAM into UNWANTED, while requiring the stronger D-03 gate (delivery + malicious signal) to reach THREAT.
+- `gatherCampaignEvidence` caps only the human-readable `reportSample` at 10 (T-19-03/CLASSIFY-06 DoS guard for anything embedded in output); the full `messages`/`indicators` arrays are used internally for correctness-critical verdict computation (D-03's cross-message indicator correlation needs the complete set, not a sample) — the bounding requirement is satisfied by keeping `reasons` short and named (never one line per indicator), not by truncating rule-evaluation inputs.
+
+## Deviations from Plan
+
+None — plan executed exactly as written. The ASSUMPTION FLAG called out in the plan's Task 1 action block (click-driven escalation to `reset_password`/`isolate_endpoint`/`disable_forwarding_rule`) was implemented exactly as the plan's code example specified and is documented above under Key Decisions for user visibility, per the plan's own instruction to surface it — it is not a deviation, since the plan explicitly directed this implementation.
+
+## Issues Encountered
+
+- Worktree branch (`worktree-agent-af5d279be12eefb03`) was created from a commit predating this milestone's Phase 16-18 work (missing `eml-parser.ts`, `mimecast-blast-radius.ts`, `campaign-grouping-service.ts`, migration 097, etc.) — confirmed the worktree had zero commits of its own ahead of `master` (`git log HEAD..master` = 333 commits, `master..HEAD` = 0), so `git reset --hard master` was safe and used to bring the worktree in sync with all prerequisite phases before starting.
+- Full `npm test` surfaced 2 pre-existing, unrelated failures in `lib/services/analyzer/itglue-search.test.ts` (`client.getFlexibleAssetsForOrganization is not a function`) — confirmed via `git log` that this test file predates this phase and was not touched by this plan. Logged to `.planning/phases/19-classification-engine/deferred-items.md` per the executor's scope-boundary rule; not fixed.
+
+## User Setup Required
+
+None - no external service configuration required.
+
+## Next Phase Readiness
+
+- `classifyCampaign(campaignId)` is ready to be wired into `POST /api/phishing/campaigns/{id}/classify` (Plan 19-02 / CLASSIFY-05) — the route only needs to call it and return the `ClassifyResult` shape.
+- Phase 20 (remediation actions) can consume `recommendedActions`/`requiresApproval` directly — the D-08 action vocabulary (`no_action`, `warn_user`, `disable_forwarding_rule`, `block_sender`, `purge_message`, `reset_password`, `isolate_endpoint`) is exactly what's produced.
+- **Needs a quick user confirmation** (flagged in the plan, not a blocker): whether `clicked>0`-driven escalation to `reset_password`/`isolate_endpoint`/`disable_forwarding_rule` is the right trigger for Phase 20's approval gating, since it's a reasoned proposal rather than a locked CONTEXT.md decision.
+
+---
+*Phase: 19-classification-engine*
+*Completed: 2026-07-16*
+
+## Self-Check: PASSED
+
+- FOUND: lib/services/campaign-classifier.ts
+- FOUND: lib/services/campaign-classifier.test.ts
+- FOUND: lib/services/campaign-classifier.fixtures.ts
+- FOUND: .planning/phases/19-classification-engine/deferred-items.md
+- FOUND commit: f4e6baf (test: pure rule function tests + fixtures)
+- FOUND commit: 3ea6c95 (feat: pure rule functions)
+- FOUND commit: f6c954a (test: classifyCampaign orchestrator tests)
+- FOUND commit: 38c1ae4 (feat: classifyCampaign orchestrator + evidence gathering)
--- a/.planning/phases/19-classification-engine/deferred-items.md
+++ b/.planning/phases/19-classification-engine/deferred-items.md
@ -0,0 +1,15 @@
+# Deferred Items — Phase 19
+
+Out-of-scope discoveries logged during execution, not fixed per the executor's scope boundary
+(only auto-fix issues directly caused by the current task's changes).
+
+## Plan 19-01
+
+- **`lib/services/analyzer/itglue-search.test.ts`** — 2 pre-existing failing tests
+  (`returns capped, redacted doc snippets when the org is found`,
+  `tolerates per-call failures (configurations errors, flex still returns)`), unrelated to
+  this plan's changes. Root cause appears to be `client.getFlexibleAssetsForOrganization is
+  not a function` — a mock/client-shape mismatch in the analyzer test suite, not touched by
+  `campaign-classifier.ts`/`.test.ts`/`.fixtures.ts`. Confirmed pre-existing via `git log` on
+  the test file (introduced in the original "AI ticket analyzer (phases 1-6)" commit, long
+  before this phase). Full suite otherwise green (358/360 passing before this discovery).