docs(v3.0): generate milestone summary for onboarding

.planning/reports/MILESTONE_SUMMARY-v3.0.md

@@ -0,0 +1,156 @@
+# Milestone v3.0 — Phishing Triage Automation — Project Summary
+
+**Generated:** 2026-07-18
+**Purpose:** Team onboarding and project review
+
+---
+
+## 1. Project Overview
+
+Pulse is Wulf Consulting's internal PSA management dashboard (Next.js 16 + Postgres, syncing Autotask data). The v3.0 milestone's core value proposition:
+
+> A manager/security operator can see every phishing/spam report ticket automatically triaged, deduplicated into campaigns, and classified — with any destructive remediation gated behind explicit human approval.
+
+**Problem it solves:** Wulf's technicians receive a constant stream of Autotask tickets from KnowBe4 phish-alert-button reports, Microsoft's built-in "Report Message" flow, and manual spam/phishing complaints. Historically each ticket was triaged by hand — no deduplication across employees reporting the same campaign, no consistent severity call, no blast-radius visibility into Mimecast, and remediation notes/actions were ad hoc.
+
+**Who it's for:** Wulf's security operators/technicians (triaging tickets day to day) and admins (configuring per-client automation).
+
+**What was built:** An end-to-end pipeline — detect phishing/spam-report tickets → extract `.eml` evidence → look up Mimecast blast radius → deterministically classify (SPAM/UNWANTED/THREAT/USER_AWARENESS) → group duplicates into campaigns → propose (never auto-execute) remediation behind human approval → post a sanitized Autotask triage note → surface it all on a ticket-ID-addressable review page reachable via a real, production-confirmed Autotask LiveLink button → optionally automate the whole chain per-client via an opt-in gate.
+
+**Status:** Shipped 2026-07-17. 9 phases, 30 plans, 69 tasks. All 38 v1 requirements complete.
+
+**Explicit non-goals for this milestone** (deferred to a future v2):
+- No automatic execution of remediation — every action (block sender, purge mailbox, revoke sessions, etc.) is *proposed* and *simulated* only; nothing destructive runs against a real mail/identity system yet (`REMEDEXEC-01..05`).
+- No URL sandbox/detonation or URL reputation lookup (`ENRICH-01`).
+- No LLM-backed classification layer — this milestone is intentionally deterministic (`ENRICH-02` reserves the interface for later).
+- No fully automated ticket closure, no auto-created parent incidents, no auto-closed duplicates.
+- No backfill of historical phishing tickets (~267 pre-existing tickets) — detection is forward-only from ship date.
+
+---
+
+## 2. Architecture & Technical Decisions
+
+| Decision | Why | Phase |
+|---|---|---|
+| **Zero-LLM deterministic classifier** | Structured, size-bounded evidence in → deterministic verdict out. Avoids a prompt-injection surface from attacker-controlled email content ever reaching an LLM. | 19 |
+| **Content-hash idempotency scoped to title+description only** | SHA-256 over `{title, description}` — excludes status/assignee/last-activity so routine ticket churn never re-triggers detection, while evidence (notes/time-entries/attachments) still refreshes on every rescan regardless of hash match. | 15 |
+| **3-tier `.eml` selection** (`rfc.eml` exact → single non-wrapper `message/rfc822` candidate → `OriginatingEmail.eml` fallback) | Empirically derived from sampling 15 real production phishing tickets — content-type alone never disambiguates, since every sampled attachment shares `message/rfc822`. | 16 |
+| **Hand-rolled SPF/DKIM/DMARC tokenizer**, not a library | The obvious library (`mailauth`) only exposes live-verification functions that perform DNS lookups and BIMI HTTP fetches — a hard violation of "never fetch anything from a message." | 16 |
+| **3-tier campaign grouping key**: Message-ID → attachment-hash/URL-domain + subject + sender + 24h window → sender + normalized-subject + client + 24h window | Message-ID is the strongest, cheapest key; the fallbacks catch mass-blast duplicates lacking a shared Message-ID. Campaigns are never merged after the fact — out of scope by design. | 18 |
+| **Mimecast blast-radius as a normalizing abstraction that never throws** | Unconditional fan-out over delivered/held/threat-event lookups, 5-minute Redis cache, degrades to an explicit `unavailable` state on any failure or missing config — the rest of the pipeline never blocks on Mimecast being down. | 17 |
+| **THREAT requires BOTH delivery AND a malicious signal** | An auth failure alone on a message Mimecast fully held (reached no one) isn't a realized threat. Malicious signal = hard SPF/DKIM/DMARC fail OR the same attachment-hash/URL recurring across ≥2 campaign reports. | 19 |
+| **Simulation-vendor allowlist is a code constant, not a DB table** | Every classification rule, including the allowlist, is unit-tested pure code rather than a runtime-editable table that could silently drift. Matches exact domain or proper subdomain only — never substring — so a lookalike domain can't slip through. | 19 |
+| **Confidence = additive point-deduction from 1.0** | Each deduction (no parsed message, no/failed Mimecast lookup, no attachment/URL indicators) names the specific missing evidence in the response, rather than an opaque score. | 19 |
+| **All remediation actions are simulated status-only transitions this milestone** | No real provider methods exist yet (no block/purge on the Mimecast client, no forwarding-rule/reset on the Graph client) — real execution deliberately deferred to a future v2. | 20 |
+| **`acknowledge_user` is the one non-destructive action carve-out** | It's a customer-visible "thanks for reporting" note, not a security action — the only action type the automation gate is allowed to auto-post without human approval. | 19/23 |
+| **Per-company automation gate is 3 independent opt-in booleans** (parse/classify/report), all off by default | Mirrors the "propose, don't execute" safety posture — a client only gets automation once an admin deliberately opts them in. | 23 |
+| **URLs render as inert copy-only text in the review UI, never a clickable link** | Stricter than the triage-note's sanitize-and-describe approach, because this is an interactive page an operator could actually click into. | 22 |
+| **Client-side permission checks call the exact same permission function the server uses** | No bespoke role checks in the UI — the server remains the sole real enforcement boundary; the client check is UX-only. | 22 |
+
+---
+
+## 3. Phases Delivered
+
+| Phase | Name | Status | One-Liner |
+|-------|------|--------|-----------|
+| 15 | Data Model, Detection & Ticket Evidence | ✅ Complete | Durable 7-table schema + idempotent Autotask ticket scanner and base evidence capture |
+| 16 | EML/MIME Evidence Parser | ✅ Complete | Pure, I/O-free RFC822/MIME parser that never executes or fetches anything |
+| 17 | Mimecast Blast Radius Lookup | ✅ Complete | Normalized delivery-data abstraction that degrades gracefully when Mimecast isn't configured |
+| 18 | Campaign Grouping & Phishing Analysis API | ✅ Complete | Duplicate reports accumulate into campaigns automatically; first `/api/phishing/*` routes |
+| 19 | Classification Engine | ✅ Complete | Deterministic SPAM/UNWANTED/THREAT verdicts from bounded evidence — no LLM anywhere |
+| 20 | Remediation, Approval & Audit Safety | ✅ Complete | Proposed-only actions, permission-gated approve/remediate/mark-false-positive, fully audited |
+| 21 | Autotask Triage Note | ✅ Complete | Sanitized, human-readable internal note posted to every linked ticket |
+| 22 | Approval UI (LiveLink) | ✅ Complete* | Ticket-ID-addressable page, confirmed live as a real Autotask LiveLink target |
+| 23 | Classification Disposition + Per-Client Automation Gate | ✅ Complete | Dedicated non-destructive verdict for simulation-vendor reports + per-company opt-in automation |
+
+*Phase 22 is code-complete and verified retroactively, but 5 manual browser click-through checks were never run — see Section 6.
+
+---
+
+## 4. Requirements Coverage
+
+All 38 v1 requirements are marked **Complete** in `v3.0-REQUIREMENTS.md`.
+
+| ID | Phase | Detail |
+|---|---|---|
+| DETECT-01/02/03 | 15/18 | Pattern match on 8 locked substrings; content-hash idempotency; on-demand single-ticket analyze endpoint |
+| EVID-01..04 | 15/16 | Ticket evidence capture; 3-tier `.eml` selection; full RFC822/MIME + auth-result normalization; sanitized body preview, zero-network invariant test-enforced |
+| CAMP-01..03 | 18 | 3-tier grouping key; report/recipient accumulation; campaign list + detail API |
+| BLAST-01/02 | 17 | Normalized blast-radius fan-out; graceful `unavailable` degradation |
+| CLASSIFY-01..06 | 19 | Verdict + confidence + reasons + actions; destructive-action invariant; evidence-naming confidence deductions; sender allowlist; classify API; bounded structured evidence only |
+| REMED-01..06 | 20 | No auto-execution path exists (grep-confirmed); approve API; remediate API (simulated); idempotent double-call handling; mark-false-positive with 409 guard; single audit writer |
+| NOTE-01 | 21 | Sanitized triage note via existing safe Autotask write path, per-ticket failure isolation |
+| ACCESS-01 | 18 | `requirePermission()` convention applied consistently from Phase 18 onward |
+| REVIEW-01..06 | 22 | Ticket-addressable route; timeline; evidence display; classification display; action area wired to APIs; client/server permission parity — all code-verified, live-browser pass outstanding |
+| CLASSDISP-01..03 | 23 | `USER_AWARENESS` verdict; customer-visible acknowledgment note; distinct UI badge |
+| AUTOGATE-01..03 | 23 | Automation-gate table + admin API; admin toggle UI; gated webhook chain (idempotency bug found and fixed pre-close) |
+
+---
+
+## 5. Key Decisions Log
+
+See Section 2 for the full architecture decision table. Notable individual calls worth calling out for new contributors:
+
+- **D-01 (Phase 19):** Deterministic classifier chosen over an LLM specifically to avoid attacker-controlled email content ever reaching a model prompt.
+- **D-05 (Phase 17):** A single global Mimecast client (not per-tenant) was an accepted v1 limitation, documented in the module itself — later partially addressed by a post-ship quick task adding optional per-tenant client injection, though the default automatic path still uses the global client.
+- **D-07 (Phase 18):** The EML parser is only invoked from the on-demand `/analyze` endpoint, not the automatic webhook/cron path — meaning fully automatic detection can only ever reach Tier 3 (weakest) campaign grouping until an operator has explicitly analyzed at least one report in that campaign.
+- **D-09 (Phase 22):** URLs are rendered as inert, copy-only text in the operator review UI — a deliberately stricter posture than the triage-note's sanitize-and-describe approach, because this page is one an operator might actually click around in.
+
+---
+
+## 6. Known Issues, Gaps, and Tech Debt
+
+**Real defects caught and fixed before/around ship:**
+
+1. **Phase 15 (pre-ship):** The webhook trigger originally branched on an Autotask payload field that real webhooks never populate — detection would have silently never fired against real data. Fixed by reading the ticket back from Postgres instead of trusting the payload shape.
+2. **Phase 18 (found via live production verification, 2026-07-16):** Re-analyzing a single-report campaign created a *second* campaign row with the same grouping key, and the abandoned original campaign's report count was never decremented. Fixed with a signal-diverged guard plus a shared decrement helper; proven via 8 regression tests and a live production re-verification.
+3. **Phase 23 (post-ship gap-closure, before milestone close):** Every additional report accumulating into an already-classified simulation campaign re-triggered a duplicate customer-visible "thanks for reporting" note. Fixed via an idempotent, audit-persisting wrapper — plus two second-order bugs found in review of that very fix (a stale-timestamp display bug, and a gap where the *manual* approval path had no awareness of an auto-posted note). All three fixed in one commit before close.
+
+**Process gap:** Phase 22 shipped all 6 plans and was marked complete without ever running formal verification — discovered only during the milestone-close requirements traceability check. A retroactive pass found the code correct (6/6 requirements), but **5 manual browser click-through checks remain outstanding**: full state-machine walkthrough, approve/remediate/mark-false-positive end-to-end, non-privileged role gating, the Mimecast-unavailable rendering state, and visual confirmation of URL inertness/clipboard-copy. Treat the review UI as code-correct but not yet UI-hardened.
+
+**Accepted, still-open items:**
+- No retry if a webhook-triggered ticket isn't yet in Postgres when detection runs (backstop is a cron sweep that ships **disabled by default**).
+- Webhook doesn't re-trigger detection on ticket update (deliberate, not a bug).
+- No floor on the campaign-grouping report-count decrement; a pre-existing race condition in the "already grouped" check.
+- The THREAT-tier "clicked > 0" escalation trigger was flagged as a reasoned proposal rather than an explicitly locked decision — never formally reconfirmed.
+- Two pre-existing, unrelated failing tests (`itglue-search.test.ts`) persist across the whole milestone — confirmed to predate v3.0, out of scope by convention.
+
+**5 post-ship fixes** (2026-07-16 through 2026-07-18, all real production issues caught after ship):
+1. Mimecast blast-radius errors were silently swallowed into a false "clean" result; date-window and multi-tenant handling fixed.
+2. A confidence-percentage display bug showed maximum confidence (1.0) as "1% confidence" — the opposite of its true meaning.
+3. A real KnowBe4 campaign using rotating lookalike domains was misclassified because only one simulation-vendor domain was allowlisted; also fixed a parse double-invocation race and a too-early auto-parse timing issue.
+4. Added a new "mark as accidental report" resolution path (feature addition, not a bug fix).
+5. Mimecast's held-message lookup had no date bounds, so an entire unrelated hold queue could inflate a campaign's blast-radius numbers — fixed with date filtering and sender-relevance matching (false held-message count went from 15 → 1 → 0 across the two fixes).
+
+---
+
+## 7. Getting Started
+
+**Where to look first:**
+- `lib/services/phishing-detector.ts` — the front door; what triggers everything downstream.
+- `lib/services/campaign-grouping-service.ts` — the trickiest logic in the milestone; read its bug history (Section 6, item 2) before touching it.
+- `lib/services/campaign-classifier.ts` — the deterministic rule engine.
+- `app/phishing/tickets/[ticketId]/page.tsx` plus `components/phishing/*` — the operator-facing review surface.
+- `lib/services/phishing-automation-gate.ts` + `app/admin/phishing-automation/page.tsx` — the per-client automation on/off switch.
+
+**Run a manual sweep/analyze:**
+- Single ticket: `POST /api/phishing/tickets/{ticket_id}/analyze`
+- Full reconciliation sweep: `lib/services/phishing-sweep-service.ts` — wired to a daily cron schedule that **ships disabled by default**; enable via `/admin` sync schedules.
+- Reclassify: `POST /api/phishing/campaigns/{id}/classify`
+
+**Tests:** All coverage lives under `lib/services/**` (matching this repo's stated test-coverage convention — no `app/**`/`components/**` coverage exists anywhere in Pulse). Run `npm test`. Whole-repo status at milestone close: 439/441 passing (2 pre-existing, unrelated failures).
+
+**Docs:** `docs/mimecast-api-guide.md` documents the held-message-queue date-range requirement and the gotcha behind the July 2026 false-positive fix — read before making further Mimecast changes.
+
+---
+
+## Stats
+
+- **Timeline:** 2026-07-15 → 2026-07-17 (3 days), plus 5 post-ship follow-up fixes through 2026-07-18
+- **Phases:** 9 / 9 complete
+- **Requirements:** 38 / 38 complete
+- **Plans / Tasks:** 30 plans, 69 tasks
+- **Commits (repo-wide, since v2.0 close 2026-07-12):** 330
+- **Files changed (phishing-specific source paths):** 53 files, 9,382 insertions
+- **Tests:** 439 / 441 passing repo-wide at close
+- **Contributors:** lorentz