diff --git a/.planning/phases/19-classification-engine/19-01-PLAN.md b/.planning/phases/19-classification-engine/19-01-PLAN.md index c02afc9..ea18ca4 100644 --- a/.planning/phases/19-classification-engine/19-01-PLAN.md +++ b/.planning/phases/19-classification-engine/19-01-PLAN.md @@ -14,10 +14,15 @@ requirements: [CLASSIFY-01, CLASSIFY-02, CLASSIFY-03, CLASSIFY-04, CLASSIFY-06] must_haves: truths: - "classifyCampaign(campaignId) returns exactly one of SPAM/UNWANTED/THREAT with confidence, summary, reasons[], recommendedActions[], requiresApproval" - - "Any recommendedActions entry in the destructive set (block_sender, purge_message, reset_password, isolate_endpoint) forces requiresApproval:true; disable_forwarding_rule alone does NOT" - - "Incomplete evidence lowers confidence below 1.0 and names each specific missing source in reasons" - - "A message whose From/Return-Path domain matches KNOWN_SIMULATION_SENDERS is never classified THREAT even when authResults shows a hard fail" + - "The classifier makes zero LLM/Anthropic/OpenRouter calls — pure deterministic rule evaluation only, same style as robotic-classifier.ts (D-01)" + - "Each classify call appends exactly one new classifications row via INSERT with no ON CONFLICT — history is never overwritten, matching the on-demand-only trigger model (D-02)" + - "Any recommendedActions entry in the destructive set (block_sender, purge_message, reset_password, isolate_endpoint) forces requiresApproval:true; disable_forwarding_rule alone does NOT (D-08)" + - "Incomplete evidence lowers confidence below 1.0 and names each specific missing source in reasons (D-05)" + - "A message whose From/Return-Path domain matches KNOWN_SIMULATION_SENDERS is never classified THREAT even when authResults shows a hard fail (D-06)" - "THREAT-tier auth-fail check reads authResultsOriginal first, falling back to authResults only when null" + - "A non-simulation campaign with blastRadius.status==='ok', delivered>0, and an effective hard auth fail is classified THREAT and recommends destructive actions (positive-path THREAT — D-03 both-conditions-met, proven by a real-signal fixture)" + - "The known-bad-indicator OR-branch of D-03 fires THREAT when the same attachment_hash/url value spans >=2 distinct messages in the campaign (even without a hard auth fail), given delivered>0 and a non-allowlist sender" + - "A clean non-simulation campaign (auth pass, no indicators, delivery contained / no delivery-or-click signal) is classified SPAM; a suspicious-but-contained campaign (one indicator match, or delivery contained to the reporter only) is classified UNWANTED — proving the D-04 SPAM/UNWANTED boundary is actually implemented (an always-SPAM stub must fail this)" - "The classifier reads structured columns/headers only — no raw unbounded email body reaches the verdict logic, reasons, or persisted row" artifacts: - path: "lib/services/campaign-classifier.ts" @@ -25,10 +30,10 @@ must_haves: exports: ["classifyCampaign", "KNOWN_SIMULATION_SENDERS", "ClassifyResult"] min_lines: 200 - path: "lib/services/campaign-classifier.test.ts" - provides: "Unit + mocked-integration coverage for CLASSIFY-01/02/03/04/06" + provides: "Unit + mocked-integration coverage for CLASSIFY-01/02/03/04/06, including positive-path THREAT + SPAM/UNWANTED-split verdict assertions" contains: "requires_approval invariant" - path: "lib/services/campaign-classifier.fixtures.ts" - provides: "Synthetic KnowBe4 (it-support.care) and BSN (breachsecurenow.com) NormalizedMessage fixtures with authResultsOriginal=pass / authResults=fail" + provides: "Synthetic KnowBe4 (it-support.care) + BSN (breachsecurenow.com) simulation fixtures AND non-simulation THREAT / clean-SPAM / suspicious-UNWANTED fixtures" contains: "it-support.care" key_links: - from: "lib/services/campaign-classifier.ts" @@ -150,7 +155,13 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report Write `campaign-classifier.test.ts` FIRST (RED), then `campaign-classifier.fixtures.ts`, then implement the pure functions in `campaign-classifier.ts` until green (GREEN). - Fixtures (`campaign-classifier.fixtures.ts`, synthetic only — no real customer email): export a KnowBe4 fixture message with `from.domain='it-support.care'` and a BSN fixture with `from.domain=null, returnPath='bounces...@em8721.breachsecurenow.com'`. EACH fixture sets `authResults = { spf:'fail', dkim:'fail', dmarc:'fail' }` but `authResultsOriginal = { spf:'pass', dkim:'pass', dmarc:'pass' }` — this reproduces the forwarding-induced auth-verdict inversion (Pitfall 1) so the simulation-not-THREAT test proves both the allowlist short-circuit AND the authResultsOriginal precedence. + Fixtures (`campaign-classifier.fixtures.ts`, synthetic only — no real customer email). Export the following named NormalizedMessage-shaped fixtures (all synthetic; Task 2's classifyCampaign tests re-use them by feeding them through the mocked `messages.headers` JSONB path): + - `knowbe4SimMessage`: `from.domain='it-support.care'`, `authResults = { spf:'fail', dkim:'fail', dmarc:'fail' }`, `authResultsOriginal = { spf:'pass', dkim:'pass', dmarc:'pass' }`. + - `bsnSimMessage`: `from.domain=null`, `returnPath='bounces...@em8721.breachsecurenow.com'`, same authResults-fail / authResultsOriginal-pass inversion. + (Both simulation fixtures reproduce the forwarding-induced auth-verdict inversion (Pitfall 1) so the simulation-not-THREAT test proves BOTH the allowlist short-circuit AND the authResultsOriginal precedence.) + - `threatMessage`: a NON-allowlist attacker-style sender (e.g. `from.domain='mlcrosoft.live'` — a real typosquat flavor per 19-RESEARCH.md D-07 finding #3), `authResults = { spf:'fail', dkim:'fail', dmarc:'fail' }`, `authResultsOriginal = null` (so effectiveAuthResults still yields the hard fail). Used with a blast-radius `{status:'ok', delivered:3, clicked:0}` to drive the positive-path THREAT verdict. + - `cleanSpamMessage`: a NON-allowlist bulk/newsletter sender (e.g. `from.domain='mail.example-newsletter.com'`), `authResults = { spf:'pass', dkim:'pass', dmarc:'pass' }`, `authResultsOriginal = null`, no attachment/url indicators. Used with a blast-radius showing NO delivery/click signal (e.g. `{status:'ok', delivered:0, held:1, clicked:0}`) → expected SPAM. + - `suspiciousUnwantedMessage`: a NON-allowlist sender (e.g. `from.domain='promo.some-vendor.net'`) with `authResults = { spf:'pass', dkim:'pass', dmarc:'pass' }`, `authResultsOriginal = null`, paired in Task 2 with exactly ONE url indicator (a single suspicious signal, NOT shared across ≥2 messages) and blast-radius delivery contained to the reporter only (`{status:'ok', delivered:1, clicked:0}` with a single perRecipient entry) → expected UNWANTED (suspicious-but-contained, below the THREAT bar per D-04). Implement in `campaign-classifier.ts` as bare exported functions (no class, no getInstance — mirror groupReportIntoCampaign's module shape). Use the named import `import { postgresClient } from './postgres-client';` and sibling imports `import { getBlastRadius, type BlastRadiusResult } from './mimecast-blast-radius';` / `import type { NormalizedMessage, AuthResults } from './eml-parser';`. @@ -164,6 +175,8 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report - `const DESTRUCTIVE_ACTIONS = new Set(['block_sender','purge_message','reset_password','isolate_endpoint'])`. - `mapVerdictToActions(verdict, evidence)` per the behavior block. `computeRequiresApproval(actions)` = `actions.some(a => DESTRUCTIVE_ACTIONS.has(a))`. + ASSUMPTION FLAG (WARNING from checker — surface to user at/after execution): the `clicked > 0` trigger that escalates a THREAT verdict's actions to include `reset_password` / `isolate_endpoint` / `disable_forwarding_rule` is a REASONED research proposal (19-RESEARCH.md Open Question #1 + Assumptions A-tier), NOT an explicit CONTEXT.md decision — CONTEXT.md D-08 only locks the vocabulary and the OR'd approval invariant, and delegated finer action-mapping to "Claude's Discretion". This choice materially affects what Phase 20 gates approval on, so the executor should note it in the SUMMARY and it is worth a quick user confirmation. It does NOT contradict any locked decision or the CLASSIFY-02 invariant, so it is not a blocker. + Each pure function gets its own `describe` block with direct input/output assertions (no DB/Mimecast mocks needed for these). Do NOT implement `classifyCampaign`/`gatherCampaignEvidence` yet — Task 2 covers those; leave stubs or omit until then. @@ -179,16 +192,18 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report - Test asserts all-three-missing confidence === 0.10 with 3 named reason strings (CLASSIFY-03) - `grep -n "KNOWN_SIMULATION_SENDERS" lib/services/campaign-classifier.ts` shows the constant with both `it-support.care` and `breachsecurenow.com` - `grep -c "\.includes(" lib/services/campaign-classifier.ts` shows domain matching does NOT use `.includes(` for allowlist comparison (endsWith only) + - Fixtures file exports `threatMessage`, `cleanSpamMessage`, and `suspiciousUnwantedMessage` (non-simulation fixtures) in addition to the two simulation fixtures — verify via `grep -n "threatMessage\|cleanSpamMessage\|suspiciousUnwantedMessage" lib/services/campaign-classifier.fixtures.ts` - `npx tsc --noEmit --pretty` exits 0 - All pure rule functions implemented and unit-tested green; fixtures exist with the authResultsOriginal-pass/authResults-fail inversion; tsc clean. + All pure rule functions implemented and unit-tested green; simulation + non-simulation (threat/clean/suspicious) fixtures exist with the authResultsOriginal precedence coverage; tsc clean. Task 2: classifyCampaign orchestrator + evidence gathering + append-only INSERT (RED→GREEN) lib/services/campaign-classifier.ts, lib/services/campaign-classifier.test.ts - - lib/services/campaign-classifier.ts (current state from Task 1 — pure functions to compose) + - lib/services/campaign-classifier.ts (current state from Task 1 — pure functions + fixtures to compose) + - lib/services/campaign-classifier.fixtures.ts (simulation + threat/clean/suspicious fixtures from Task 1) - lib/services/campaign-grouping-service.ts (evidence gather + ORDER BY r.created_at ASC "earliest report is canonical" convention; catch-log-rethrow orchestrator boundary; [CAMPAIGN-GROUPING] log prefix style) - app/api/phishing/campaigns/[id]/route.ts (bulk-fetch-by-id-array shape: reportIds → messages via report_id=ANY, messageIds → indicators via message_id=ANY; headers->>'subject') - migrations/097_phishing_triage_schema.sql (classifications INSERT columns; messages/indicators/reports/campaigns columns) @@ -199,24 +214,36 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report - classifyCampaign returns an object with verdict ∈ {'SPAM','UNWANTED','THREAT'}, numeric confidence, string summary, string[] reasons, string[] recommendedActions, boolean requiresApproval, string id, string campaignId, string createdAt (CLASSIFY-01) - Simulation fixture (it-support.care OR breachsecurenow.com sender) with authResults fail but authResultsOriginal pass + delivered>0: verdict is SPAM or UNWANTED, NEVER THREAT (CLASSIFY-04 — allowlist short-circuits before THREAT tier) - - THREAT only when blastRadius.status==='ok' AND (delivered>0 OR clicked>0) AND (hasHardAuthFail(effectiveAuthResults) on some message OR a known-bad indicator match) AND sender NOT on allowlist (D-03) + - POSITIVE-PATH THREAT (D-03 both conditions): non-allowlist `threatMessage` sender + blastRadius.status==='ok' with delivered>0 + effective hard auth fail → verdict === 'THREAT' AND recommendedActions includes at least 'block_sender' and 'purge_message' (a broken/always-SPAM stub MUST fail this) + - THREAT known-bad-indicator OR-branch: non-allowlist sender + delivered>0 + NO hard auth fail (auth pass) BUT the same attachment_hash/url value on ≥2 distinct messages → verdict === 'THREAT' (D-03 second signal, research A4) + - SPAM vs UNWANTED split (D-04): `cleanSpamMessage` (auth pass, no indicators, no delivery/click signal) → verdict === 'SPAM'; `suspiciousUnwantedMessage` (auth pass, exactly one url indicator not shared across messages, delivery contained to the reporter only) → verdict === 'UNWANTED'. Two distinct fixtures prove the boundary is implemented, not stubbed. - Known-bad indicator match = same attachment_hash or url value on ≥2 distinct messages in the campaign (cross-report correlation — no external reputation lookup) (D-03 interpretation, research A4) - When blastRadius.status!=='ok', or no message parsed, or no indicators: confidence < 1.0 and the specific source named in reasons (CLASSIFY-03) - Persisted reasons/summary contain no raw email body text; evidence arrays are capped (CLASSIFY-06) - Exactly one new classifications row INSERTed per call (append-only, no ON CONFLICT) — verified via the mocked query call args (D-02) - Extend `campaign-classifier.test.ts` with a `describe('classifyCampaign', ...)` block using `vi.mock('./postgres-client')` (queryMock, matching campaign-grouping-service.test.ts) and `vi.mock('./mimecast-blast-radius', () => ({ getBlastRadius: (...) => getBlastRadiusMock(...) }))`. Route mock query responses by distinguishing SQL substring per call (`FROM campaigns`, `FROM reports`, `FROM messages`, `FROM indicators`, `INSERT INTO classifications`) — NOT by call order. Add the named tests: "returns exactly one verdict", "simulation allowlist" (both it-support.care and breachsecurenow.com fixtures, delivered>0, authResults fail → assert verdict !== 'THREAT'), "evidence bounding" (feed a campaign with many indicators; assert persisted reasons is a short bounded array and no reason string contains a raw body). Write these RED first. + Extend `campaign-classifier.test.ts` with a `describe('classifyCampaign', ...)` block using `vi.mock('./postgres-client')` (queryMock, matching campaign-grouping-service.test.ts) and `vi.mock('./mimecast-blast-radius', () => ({ getBlastRadius: (...) => getBlastRadiusMock(...) }))`. Route mock query responses by distinguishing SQL substring per call (`FROM campaigns`, `FROM reports`, `FROM messages`, `FROM indicators`, `INSERT INTO classifications`) — NOT by call order. Build a small local helper that turns a fixture NormalizedMessage into a `messages` row (`{ id, report_id, headers: }`) so the same fixtures drive the orchestrator. + + Write these named tests RED first, then implement until green: + - `"returns exactly one verdict"` — assert result.verdict is one of the three literals and all payload fields present with correct types (CLASSIFY-01). + - `"simulation allowlist"` — BOTH it-support.care (From match) and breachsecurenow.com (Return-Path match, From.domain null) fixtures, delivered>0, authResults fail → assert `result.verdict !== 'THREAT'` for each. + - `"threat tier"` — feed `threatMessage` with `getBlastRadiusMock` → `{status:'ok', delivered:3, clicked:0}`; assert `result.verdict === 'THREAT'` and `result.recommendedActions` includes 'block_sender' and 'purge_message' and `result.requiresApproval === true` (positive-path THREAT — closes the blocker: proves classifyCampaign CAN return THREAT for a real signal). + - `"threat tier known-bad indicator"` — non-allowlist sender across 2 messages sharing one url indicator value, auth PASS, delivered>0, no hard auth fail → assert `result.verdict === 'THREAT'` (D-03 OR-branch, no auth-fail path). + - `"spam vs unwanted tier"` — TWO sub-cases: `cleanSpamMessage` + no-delivery/click blast radius → `result.verdict === 'SPAM'`; `suspiciousUnwantedMessage` + single url indicator + delivery-contained-to-reporter blast radius → `result.verdict === 'UNWANTED'` (proves D-04 boundary is real, not an always-SPAM stub). + - `"evidence bounding"` — feed a campaign with many indicators; assert persisted reasons is a short bounded array and no reason string contains a raw body (CLASSIFY-06). Then implement in `campaign-classifier.ts`: - `interface ClassifyResult { id: string; campaignId: string; verdict: 'SPAM'|'UNWANTED'|'THREAT'; confidence: number; summary: string; reasons: string[]; recommendedActions: string[]; requiresApproval: boolean; createdAt: string }` (export it). - `gatherCampaignEvidence(campaignId)`: SELECT campaign row; SELECT reports `WHERE campaign_id=$1 ORDER BY created_at ASC` (LEFT JOIN contacts for requester_email); bulk-fetch messages `WHERE report_id = ANY($1::uuid[])` selecting `id, report_id, headers, message_id`; bulk-fetch indicators `WHERE message_id = ANY($1::uuid[])` selecting `id, message_id, indicator_type, value`. Parse `headers` JSONB into the NormalizedMessage-shaped fields needed (from.domain, returnPath, authResults, authResultsOriginal, subject). Use the EARLIEST report as canonical sender/subject/date-window for one `getBlastRadius()` call (±24h window around report.created_at); when no report exists, synthesize `{ status:'unavailable', reason:'not_configured' }` without calling Mimecast (research A6). Return a BOUNDED payload: counts + capped sample arrays (cap indicator/report samples at 10), never raw body text (CLASSIFY-06 / T-19-03 DoS guard). - - `classifyCampaign(campaignId)`: gather evidence → `if isKnownSimulationSender(any message) → skip THREAT, run evaluateSpamVsUnwanted only (D-06 short-circuit)` → else `evaluateThreatTier` (D-03) → else `evaluateSpamVsUnwanted` (D-04, SPAM if no suspicious signal, UNWANTED if a suspicious signal present but below THREAT bar) → `computeConfidence` → `mapVerdictToActions` → `computeRequiresApproval` → build a short `summary` string naming the verdict and top reason → INSERT one classifications row `RETURNING id::text AS id, created_at::text AS created_at` with `reasons`/`recommended_actions` as `JSON.stringify(...)::jsonb` → return ClassifyResult. Wrap the orchestrator body in try/catch that logs `[CAMPAIGN-CLASSIFIER]` + err.message (never full payloads — T-17-style) and rethrows. + - `classifyCampaign(campaignId)`: gather evidence → `if isKnownSimulationSender(any message) → skip THREAT, run evaluateSpamVsUnwanted only (D-06 short-circuit)` → else `evaluateThreatTier` (D-03: blastRadius.status==='ok' AND (delivered>0 OR clicked>0) AND (hasHardAuthFail on effectiveAuthResults of some message OR known-bad-indicator match)) → else `evaluateSpamVsUnwanted` (D-04, SPAM if no suspicious signal, UNWANTED if a suspicious signal present — one indicator match, a mild auth issue, or delivery contained to the reporter — but below the THREAT bar) → `computeConfidence` → `mapVerdictToActions` → `computeRequiresApproval` → build a short `summary` string naming the verdict and top reason → INSERT one classifications row `RETURNING id::text AS id, created_at::text AS created_at` with `reasons`/`recommended_actions` as `JSON.stringify(...)::jsonb` → return ClassifyResult. Wrap the orchestrator body in try/catch that logs `[CAMPAIGN-CLASSIFIER]` + err.message (never full payloads — T-17-style) and rethrows. - Known-bad indicator match computed over the gathered indicators: group by (indicator_type, value) for attachment_hash/url types, flag true if any value spans ≥2 distinct message_id. npx vitest run lib/services/campaign-classifier.test.ts -t "returns exactly one verdict" npx vitest run lib/services/campaign-classifier.test.ts -t "simulation allowlist" + npx vitest run lib/services/campaign-classifier.test.ts -t "threat tier" + npx vitest run lib/services/campaign-classifier.test.ts -t "spam vs unwanted tier" npx vitest run lib/services/campaign-classifier.test.ts -t "evidence bounding" npx vitest run lib/services/campaign-classifier.test.ts npx tsc --noEmit --pretty @@ -224,13 +251,16 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report - `npx vitest run lib/services/campaign-classifier.test.ts` passes all describe blocks including classifyCampaign - "simulation allowlist" test passes for BOTH it-support.care (From match) and breachsecurenow.com (Return-Path match, From.domain null) fixtures, each asserting `result.verdict !== 'THREAT'` despite `authResults` hard-fail and delivered>0 - - "returns exactly one verdict" asserts result.verdict is one of the three literals and all six payload fields are present with correct types (CLASSIFY-01) + - "threat tier" test asserts `result.verdict === 'THREAT'`, `recommendedActions` includes 'block_sender' + 'purge_message', and `requiresApproval === true` for the non-allowlist real-signal fixture (positive-path THREAT — the previously-missing coverage; an always-SPAM stub fails here) + - "threat tier known-bad indicator" asserts THREAT fires on the ≥2-message shared-indicator OR-branch with auth PASS (D-03 second signal) + - "spam vs unwanted tier" asserts `cleanSpamMessage → 'SPAM'` AND `suspiciousUnwantedMessage → 'UNWANTED'` (D-04 boundary proven; an always-SPAM stub fails the UNWANTED case) + - "returns exactly one verdict" asserts result.verdict is one of the three literals and all payload fields are present with correct types (CLASSIFY-01) - Mock asserts exactly one `INSERT INTO classifications` query and that it contains no `ON CONFLICT` (append-only, D-02) - `grep -n "authResultsOriginal" lib/services/campaign-classifier.ts` shows the precedence is applied inside the THREAT-tier path - `grep -n "getBlastRadius" lib/services/campaign-classifier.ts` shows exactly one call site guarded by "earliest report exists" - `npm test` full suite green; `npx tsc --noEmit --pretty` exits 0 - classifyCampaign orchestrates evidence→rules→confidence→actions→append-only INSERT; simulation short-circuit and authResultsOriginal precedence proven by tests; full suite + tsc green. + classifyCampaign orchestrates evidence→rules→confidence→actions→append-only INSERT; simulation short-circuit, positive-path THREAT, D-03 indicator OR-branch, and the D-04 SPAM/UNWANTED split all proven by named tests; full suite + tsc green. @@ -248,7 +278,7 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report | Threat ID | Category | Component | Disposition | Mitigation Plan | |-----------|----------|-----------|-------------|-----------------| | T-19-01 | Spoofing | domainMatchesAllowlist | mitigate | Match exact-domain-or-proper-subdomain only (`d===allowed \|\| d.endsWith('.'+allowed)`); never `.includes()` substring — asserted by a test that `it-support.care.attacker.net` does NOT match | -| T-19-02 | Tampering (logic drift) | D-03/D-04/D-06 rule functions | mitigate | Pure, unit-tested TS functions + TS-constant allowlist (not a live-editable DB table); every rule covered by a dedicated test | +| T-19-02 | Tampering (logic drift) | D-03/D-04/D-06 rule functions | mitigate | Pure, unit-tested TS functions + TS-constant allowlist (not a live-editable DB table); every rule covered by a dedicated test — including positive-path THREAT and the SPAM/UNWANTED split, so a silently-weakened verdict path is caught | | T-19-03 | Denial of Service | gatherCampaignEvidence / persisted reasons | mitigate | Bounded evidence payload — counts + sample arrays capped at 10, never full report/message/indicator sets or raw body text embedded verbatim (doubles as CLASSIFY-06) | | T-19-06 | Spoofing (auth-verdict inversion) | effectiveAuthResults | mitigate | Prefer authResultsOriginal over post-forward authResults so a forwarded genuine simulation isn't pushed to THREAT on an invalidated DKIM signature (Pitfall 1) | | T-19-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed this phase (classifier consumes only shipped project code + Node built-ins); Package Legitimacy Gate N/A per 19-RESEARCH.md | @@ -258,7 +288,7 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report - `npx vitest run lib/services/campaign-classifier.test.ts` — all describe blocks green - `npm test` — full suite green (no regression in campaign-grouping / mimecast / eml-parser) - `npx tsc --noEmit --pretty` — clean -- CLASSIFY-01/02/03/04/06 each proven by a named test per 19-VALIDATION.md Per-Task Verification Map +- CLASSIFY-01/02/03/04/06 each proven by a named test per 19-VALIDATION.md Per-Task Verification Map; verdict logic (D-03 THREAT positive path + D-04 SPAM/UNWANTED split) proven by the "threat tier" / "threat tier known-bad indicator" / "spam vs unwanted tier" named tests @@ -266,6 +296,7 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report - destructive action ⇒ requiresApproval:true; disable_forwarding_rule alone ⇒ false (CLASSIFY-02) - incomplete evidence lowers confidence and names the missing source (CLASSIFY-03) - it-support.care / breachsecurenow.com simulation fixtures never classified THREAT absent contrary evidence (CLASSIFY-04) +- a real-signal non-simulation campaign IS classified THREAT with destructive actions, and the SPAM-vs-UNWANTED boundary is exercised by distinct fixtures (positive verdict-logic coverage — D-03/D-04) - evidence is bounded/structured — no raw unbounded body reaches classifier/reasons (CLASSIFY-06) diff --git a/.planning/phases/19-classification-engine/19-RESEARCH.md b/.planning/phases/19-classification-engine/19-RESEARCH.md index 9a2b953..212a76c 100644 --- a/.planning/phases/19-classification-engine/19-RESEARCH.md +++ b/.planning/phases/19-classification-engine/19-RESEARCH.md @@ -712,7 +712,9 @@ directly above. | A5 | TypeScript-constant allowlist (not DB table) is the right storage choice for D-06 | Standard Stack — Alternatives Considered | If wrong, ops will need a code deploy to add/remove a vendor domain rather than an admin UI edit — a process cost, not a correctness risk. Reversible later without a rework (per D-06's own multi-vendor-extensibility requirement). | | A6 | Primary blast-radius call should use the campaign's EARLIEST report as canonical sender/subject/date-window | Code Examples (`gatherCampaignEvidence`) | If wrong (e.g. a later report has better/more-complete data), blast-radius lookup could target a slightly wrong subject/date-window and return fewer/no matches — degrades to `status:'unavailable'`-like under-evidence, not a false verdict, since D-03's gate still requires a real signal to fire. | -## Open Questions +## Open Questions (ALL RESOLVED) + +> All three open questions below are functionally answered by 19-01-PLAN.md and locked for this phase. Each carries an explicit RESOLVED note. The one item that is a reasoned assumption rather than a locked decision (Q1's `clicked > 0` action-escalation trigger) is flagged in 19-01-PLAN.md Task 2 for a quick user confirmation at/after execution. 1. **Should `disable_forwarding_rule` ever be recommended by this phase's rules, given no evidence source in Phases 15-18 signals "a mailbox forwarding rule was created"?** @@ -725,6 +727,7 @@ directly above. reasoned proxy for "possible account compromise, worth checking for a malicious forwarding rule" — but confirm with the user/planner before locking in, since this is inference, not an explicit CONTEXT.md decision. + - **RESOLVED:** Tied to `clicked > 0` on a THREAT verdict (19-01-PLAN.md Task 1 `mapVerdictToActions`, covered by the "threat tier" test). Flagged in 19-01-PLAN.md Task 2 as a reasoned assumption (not a CONTEXT.md decision) worth a quick user confirmation, since it affects what Phase 20 gates approval on. 2. **Should the KnowBe4/BSN allowlist match check `Return-Path` via a parsed domain, or does `NormalizedMessage` need a new `returnPathDomain` field?** @@ -736,6 +739,7 @@ directly above. - Recommendation: do it locally in `campaign-classifier.ts` (as shown in Code Examples) — Phase 16 is marked "read, does not modify" in this phase's canonical_refs, and the split is a one-line, low-risk operation not worth reopening an already-shipped, tested module for. + - **RESOLVED:** Return-Path domain is parsed locally in `campaign-classifier.ts` (`.split('@')[1]` inside `isKnownSimulationSender`, 19-01-PLAN.md Task 1) — `eml-parser.ts` is NOT modified. 3. **Is a campaign's blast-radius lookup re-run on every `/classify` call, or cached/reused across calls within the same campaign?** @@ -747,6 +751,7 @@ directly above. logic needed in this phase. - Recommendation: no action needed; call `getBlastRadius()` on every `classifyCampaign()` invocation and let its existing cache absorb repeat calls. + - **RESOLVED:** No new caching added this phase; `getBlastRadius()`'s existing 5-minute Redis cache is relied on as-is (19-01-PLAN.md Task 2 calls it once per `classifyCampaign`). ## Environment Availability diff --git a/.planning/phases/19-classification-engine/19-VALIDATION.md b/.planning/phases/19-classification-engine/19-VALIDATION.md index b9ae271..ddb7395 100644 --- a/.planning/phases/19-classification-engine/19-VALIDATION.md +++ b/.planning/phases/19-classification-engine/19-VALIDATION.md @@ -2,7 +2,7 @@ phase: 19 slug: classification-engine status: draft -nyquist_compliant: false +nyquist_compliant: true wave_0_complete: false created: 2026-07-16 --- @@ -73,6 +73,6 @@ created: 2026-07-16 - [ ] Wave 0 covers all MISSING references - [ ] No watch-mode flags - [ ] Feedback latency < 5s (automated) — manual route check is a pre-merge/pre-verify gate, not per-commit -- [ ] `nyquist_compliant: true` set in frontmatter +- [x] `nyquist_compliant: true` set in frontmatter **Approval:** pending