diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md deleted file mode 100644 index 28a240d..0000000 --- a/.planning/REQUIREMENTS.md +++ /dev/null @@ -1,254 +0,0 @@ -# Requirements: Pulse — v3.0 Phishing Triage Automation - -**Defined:** 2026-07-14 -**Core Value:** A manager/security operator can see every phishing/spam report ticket -automatically triaged, deduplicated into campaigns, and classified — with any -destructive remediation gated behind explicit human approval. - -## v1 Requirements - -### Detection - -- [x] **DETECT-01**: System scans recent Autotask/Pulse tickets and flags candidates - matching known phishing/spam-report patterns (title/body: "Phishing Report", - "Spam Alert", "Phishing Alert - Email Security Report", "KnowBe4 Phish Alert - Report", "Source: KnowBe4 Phish Alert Button", "userSubmissionsReportMessage", - "reported message destinations", "Microsoft directly") -- [x] **DETECT-02**: Re-scanning does not reprocess a ticket already ingested unless - its source ticket data has changed since last processed (idempotent) -- [x] **DETECT-03**: An operator can trigger analysis of one specific ticket by ID - on demand (`POST /api/phishing/tickets/{ticket_id}/analyze`) instead of waiting - for the scheduled scan - -### Evidence Extraction - -- [x] **EVID-01**: For each candidate ticket, the system extracts ticket ID/number, - company, requester/reporter, title, description, notes, relevant time entries, - and attachment metadata -- [x] **EVID-02**: When multiple `.eml` attachments exist, the system prefers - `rfc.eml` as the original reported message over `OriginatingEmail.eml` - (wrapper/context), matching case-insensitively and by `message/rfc822` - content-type, not filename alone -- [x] **EVID-03**: The system parses the selected original email's RFC822/MIME - structure into normalized headers (From, display name, sender email/domain, - Reply-To, Return-Path, To, Cc, Subject, Date, Message-ID, Received chain, - SPF/DKIM/DMARC authentication results), extracted URLs, and attachment - metadata (name, content-type, size, hash) -- [x] **EVID-04**: The system stores a sanitized/truncated body preview alongside - raw evidence, and never executes or fetches any URL found in a message - -### Campaign Grouping - -- [x] **CAMP-01**: Reports are grouped into a campaign using original Message-ID - first, then attachment-hash/URL-domain + subject + sender + time-window, then - sender + normalized subject + client + time-window as fallback keys -- [x] **CAMP-02**: A campaign can accumulate many linked ticket reports and - recipients over time as duplicates are detected -- [x] **CAMP-03**: An operator can list campaigns and view a single campaign's - full detail (linked reports, messages, indicators, classification history) - via API (`GET /api/phishing/campaigns`, `GET /api/phishing/campaigns/{id}`) - -### Blast Radius (Mimecast) - -- [x] **BLAST-01**: The system can query a Mimecast blast-radius abstraction for - message delivery data (matched/delivered/held/rejected/clicked counts, - per-recipient status) when Mimecast is configured, keyed on message ID, - sender, recipient/reporter, subject, and date window -- [x] **BLAST-02**: When Mimecast is not configured, the system records - `status: unavailable` for that lookup and classification proceeds using - ticket/email evidence alone — it never blocks on missing Mimecast config - -### Classification - -- [x] **CLASSIFY-01**: The system classifies a campaign as exactly one of - `SPAM` / `UNWANTED` / `THREAT`, with confidence, a short summary, - evidence-backed reasons, recommended actions, and a `requires_approval` flag -- [x] **CLASSIFY-02**: Any classification recommending a destructive action - (purge, block, delete, reset, etc.) always sets `requires_approval: true` -- [x] **CLASSIFY-03**: When evidence is incomplete (no Mimecast data, no `.eml`, - etc.), confidence is lowered and the missing evidence is named in the reasons -- [x] **CLASSIFY-04**: Known/expected KnowBe4 security-awareness simulations are - not classified as `THREAT` absent contrary evidence -- [x] **CLASSIFY-05**: An operator can (re-)trigger classification of a campaign - via API (`POST /api/phishing/campaigns/{id}/classify`) -- [x] **CLASSIFY-06**: The classifier accepts structured, size-bounded evidence - (not raw unbounded email bodies) — long bodies are redacted/truncated before - reaching any AI layer, and IT Glue-sourced evidence goes through the existing - redacted search path if referenced - -### Remediation & Approval Safety - -- [x] **REMED-01**: Recommended remediation actions are recorded as `proposed` - but never executed automatically in this milestone -- [x] **REMED-02**: An authorized operator can approve a campaign's remediation - via API (`POST /api/phishing/campaigns/{id}/approve`), recording approver, - timestamp, and the exact approved action parameters -- [x] **REMED-03**: `POST /api/phishing/campaigns/{id}/remediate` proceeds only - for approved actions against a configured, non-destructive-by-default - provider path; otherwise it returns `not_implemented`/an explicit failure — - it never silently succeeds without taking or logging an action -- [x] **REMED-04**: Re-running remediation against an already-completed action - does not duplicate the destructive effect (idempotent) -- [x] **REMED-05**: An operator can mark a campaign as a false positive via API - (`POST /api/phishing/campaigns/{id}/mark-false-positive`) -- [x] **REMED-06**: Every state-changing action (classify, approve, remediate, - mark-false-positive) is recorded as an audit event with actor, event type, - and payload - -### Autotask Integration - -- [x] **NOTE-01**: If Pulse already has a safe Autotask note-writing method, the - system can post an internal triage note summarizing classification, evidence, - blast radius, and recommended actions (sanitized — no raw secrets/tokens/full - malicious URL query strings); otherwise the note text is returned via API - without writing anything to Autotask - -### Access Control - -- [x] **ACCESS-01**: All `/api/phishing/*` endpoints enforce existing Pulse auth - conventions (`requireAuth`/`requirePermission`), with approve/remediate - requiring elevated permission beyond plain read access - -### Approval UI (LiveLink) - -- [x] **REVIEW-01**: A stable, ticket-ID-addressable Pulse route (e.g. - `/phishing/tickets/{ticketId}`) resolves the ticket to its campaign and - renders that campaign's review page, suitable as an Autotask LiveLink target - (LiveLink supplies the ticket ID as dynamic content, not the internal - campaign UUID), authenticated via the existing Better Auth session only — - no separate token or query-param auth scheme -- [x] **REVIEW-02**: The page displays the campaign's timeline — linked - reports, classification history, and audit events (classify/approve/ - remediate/mark-false-positive) — in chronological order -- [x] **REVIEW-03**: The page displays the gathered evidence — parsed EML - headers/URLs/attachments, sanitized body preview, and Mimecast blast-radius - data (including an explicit `unavailable` state when Mimecast isn't - configured) — never rendering a raw/unsanitized body or unredacted secrets -- [x] **REVIEW-04**: The page displays the current classification (SPAM/ - UNWANTED/THREAT), confidence, reasons, and recommended remediation - action(s) -- [x] **REVIEW-05**: An operator can approve, remediate, or mark a campaign as - a false positive directly from the page, calling the existing - `/api/phishing/campaigns/{id}` approve/remediate/mark-false-positive - endpoints and reflecting the resulting state (e.g. a remediated campaign - shows as remediated, not re-offered for approval) -- [x] **REVIEW-06**: An operator without the elevated permission approve/ - remediate already require sees those actions disabled or hidden rather than - a failed request; the page enforces no separate or relaxed permission model - from the underlying APIs - -### Classification Disposition + Automation Gate - -- [x] **CLASSDISP-01**: The classifier assigns a dedicated `USER_AWARENESS` - verdict to campaigns confirmed as phishing-simulation-vendor (KnowBe4/Breach - Secure Now) reports, replacing the previous forced-`UNWANTED` disposition - for these confirmed-simulation cases -- [x] **CLASSDISP-02**: `USER_AWARENESS` maps to a new non-destructive - `acknowledge_user` action that posts a customer-visible thank-you note - (Autotask `noteType: 18`, "Client Portal Note") to the reporting employee -- [x] **CLASSDISP-03**: The campaign review UI surfaces `USER_AWARENESS` and - `acknowledge_user` distinctly from the existing SPAM/UNWANTED/THREAT - verdicts and their actions -- [x] **AUTOGATE-01**: A per-company `phishing_automation_gate` table and - admin-gated `GET`/`PATCH`/`DELETE` API let an operator read and set three - independent opt-in automation flags (`auto_parse`, `auto_classify`, - `auto_report`) per Autotask company, defaulting to all-OFF when no row - exists -- [x] **AUTOGATE-02**: An `/admin/phishing-automation` page lists companies - with three independent per-company `Switch` toggles (one per automation - stage), backed by the `AUTOGATE-01` API -- [x] **AUTOGATE-03**: When a company's automation gate stages are enabled, - the Autotask webhook automatically runs the gated parse→classify→acknowledge - chain for that company's phishing reports, with the `acknowledge_user` - auto-post carve-out narrowly scoped to `USER_AWARENESS` verdicts only — all - other verdicts/actions still require manual approval regardless of gate - state - -## v2 Requirements - -Deferred to future release. Tracked but not in current roadmap. - -### Remediation Execution - -- **REMEDEXEC-01**: Actually execute Mimecast sender/domain/URL block (behind - approval, once REMED-01..06 land and are trusted) -- **REMEDEXEC-02**: Microsoft Graph mailbox search/delete/move for delivered - copies -- **REMEDEXEC-03**: Microsoft Defender/Exchange purge (may be preferable to - Graph — needs its own evaluation) -- **REMEDEXEC-04**: Auto-create/associate an Autotask parent incident ticket -- **REMEDEXEC-05**: Auto-close duplicate reports once a campaign is resolved - -### Enrichment - -- **ENRICH-01**: URL reputation lookup / safe expansion of shortened links - (without detonation) -- **ENRICH-02**: LLM-backed classification layer wired into the - `CLASSIFY-*` rule engine (rule layer ships first; this plugs in behind the - same interface) - -## Out of Scope - -| Feature | Reason | -|---------|--------| -| Automatic tenant-wide mailbox purge | Destructive; requires proven accuracy and explicit approval infrastructure first (REMED-01..04) | -| Automatic password reset / session revocation | Too high-blast-radius for an MVP triage tool; human-in-the-loop only | -| URL detonation / sandbox execution | Security risk of interacting with malicious infra; reputation-only enrichment deferred to v2 | -| Fully automated ticket closure | Accuracy not yet proven; operator closes tickets manually in this milestone | -| Assuming Microsoft Graph is the eventual purge mechanism | Defender/Exchange purge may be preferable; decision deferred to whichever v2 remediation-execution phase picks it up | -| Real customer `.eml` fixtures in tests | Privacy/security — all test fixtures are synthetic | - -## Traceability - -Populated during roadmap creation. - -| Requirement | Phase | Status | -|-------------|-------|--------| -| DETECT-01 | Phase 15 | Complete | -| DETECT-02 | Phase 15 | Complete | -| DETECT-03 | Phase 18 | Complete | -| EVID-01 | Phase 15 | Complete | -| EVID-02 | Phase 16 | Complete | -| EVID-03 | Phase 16 | Complete | -| EVID-04 | Phase 16 | Complete | -| CAMP-01 | Phase 18 | Complete | -| CAMP-02 | Phase 18 | Complete | -| CAMP-03 | Phase 18 | Complete | -| BLAST-01 | Phase 17 | Complete | -| BLAST-02 | Phase 17 | Complete | -| CLASSIFY-01 | Phase 19 | Complete | -| CLASSIFY-02 | Phase 19 | Complete | -| CLASSIFY-03 | Phase 19 | Complete | -| CLASSIFY-04 | Phase 19 | Complete | -| CLASSIFY-05 | Phase 19 | Complete | -| CLASSIFY-06 | Phase 19 | Complete | -| REMED-01 | Phase 20 | Complete | -| REMED-02 | Phase 20 | Complete | -| REMED-03 | Phase 20 | Complete | -| REMED-04 | Phase 20 | Complete | -| REMED-05 | Phase 20 | Complete | -| REMED-06 | Phase 20 | Complete | -| NOTE-01 | Phase 21 | Complete | -| ACCESS-01 | Phase 18 | Complete | -| REVIEW-01 | Phase 22 | Complete | -| REVIEW-02 | Phase 22 | Complete | -| REVIEW-03 | Phase 22 | Complete | -| REVIEW-04 | Phase 22 | Complete | -| REVIEW-05 | Phase 22 | Complete | -| REVIEW-06 | Phase 22 | Complete | -| CLASSDISP-01 | Phase 23 | Complete | -| CLASSDISP-02 | Phase 23 | Complete | -| CLASSDISP-03 | Phase 23 | Complete | -| AUTOGATE-01 | Phase 23 | Complete | -| AUTOGATE-02 | Phase 23 | Complete | -| AUTOGATE-03 | Phase 23 | Complete | - -**Coverage:** -- v1 requirements: 38 total -- Mapped to phases: 38 (Phases 15-23) -- Unmapped: 0 ✓ - ---- -*Requirements defined: 2026-07-14* -*Traceability populated: 2026-07-14 — ROADMAP.md Phases 15-21* -*Last updated: 2026-07-16 — backfilled Phase 23 CLASSDISP-*/AUTOGATE-* entries (23-03)*