docs(19): revise plans for D-03/D-04 verdict coverage + decision citations
This commit is contained in:
parent
ef55e6dfd3
commit
6981dcc628
3 changed files with 54 additions and 18 deletions
|
|
@ -14,10 +14,15 @@ requirements: [CLASSIFY-01, CLASSIFY-02, CLASSIFY-03, CLASSIFY-04, CLASSIFY-06]
|
|||
must_haves:
|
||||
truths:
|
||||
- "classifyCampaign(campaignId) returns exactly one of SPAM/UNWANTED/THREAT with confidence, summary, reasons[], recommendedActions[], requiresApproval"
|
||||
- "Any recommendedActions entry in the destructive set (block_sender, purge_message, reset_password, isolate_endpoint) forces requiresApproval:true; disable_forwarding_rule alone does NOT"
|
||||
- "Incomplete evidence lowers confidence below 1.0 and names each specific missing source in reasons"
|
||||
- "A message whose From/Return-Path domain matches KNOWN_SIMULATION_SENDERS is never classified THREAT even when authResults shows a hard fail"
|
||||
- "The classifier makes zero LLM/Anthropic/OpenRouter calls — pure deterministic rule evaluation only, same style as robotic-classifier.ts (D-01)"
|
||||
- "Each classify call appends exactly one new classifications row via INSERT with no ON CONFLICT — history is never overwritten, matching the on-demand-only trigger model (D-02)"
|
||||
- "Any recommendedActions entry in the destructive set (block_sender, purge_message, reset_password, isolate_endpoint) forces requiresApproval:true; disable_forwarding_rule alone does NOT (D-08)"
|
||||
- "Incomplete evidence lowers confidence below 1.0 and names each specific missing source in reasons (D-05)"
|
||||
- "A message whose From/Return-Path domain matches KNOWN_SIMULATION_SENDERS is never classified THREAT even when authResults shows a hard fail (D-06)"
|
||||
- "THREAT-tier auth-fail check reads authResultsOriginal first, falling back to authResults only when null"
|
||||
- "A non-simulation campaign with blastRadius.status==='ok', delivered>0, and an effective hard auth fail is classified THREAT and recommends destructive actions (positive-path THREAT — D-03 both-conditions-met, proven by a real-signal fixture)"
|
||||
- "The known-bad-indicator OR-branch of D-03 fires THREAT when the same attachment_hash/url value spans >=2 distinct messages in the campaign (even without a hard auth fail), given delivered>0 and a non-allowlist sender"
|
||||
- "A clean non-simulation campaign (auth pass, no indicators, delivery contained / no delivery-or-click signal) is classified SPAM; a suspicious-but-contained campaign (one indicator match, or delivery contained to the reporter only) is classified UNWANTED — proving the D-04 SPAM/UNWANTED boundary is actually implemented (an always-SPAM stub must fail this)"
|
||||
- "The classifier reads structured columns/headers only — no raw unbounded email body reaches the verdict logic, reasons, or persisted row"
|
||||
artifacts:
|
||||
- path: "lib/services/campaign-classifier.ts"
|
||||
|
|
@ -25,10 +30,10 @@ must_haves:
|
|||
exports: ["classifyCampaign", "KNOWN_SIMULATION_SENDERS", "ClassifyResult"]
|
||||
min_lines: 200
|
||||
- path: "lib/services/campaign-classifier.test.ts"
|
||||
provides: "Unit + mocked-integration coverage for CLASSIFY-01/02/03/04/06"
|
||||
provides: "Unit + mocked-integration coverage for CLASSIFY-01/02/03/04/06, including positive-path THREAT + SPAM/UNWANTED-split verdict assertions"
|
||||
contains: "requires_approval invariant"
|
||||
- path: "lib/services/campaign-classifier.fixtures.ts"
|
||||
provides: "Synthetic KnowBe4 (it-support.care) and BSN (breachsecurenow.com) NormalizedMessage fixtures with authResultsOriginal=pass / authResults=fail"
|
||||
provides: "Synthetic KnowBe4 (it-support.care) + BSN (breachsecurenow.com) simulation fixtures AND non-simulation THREAT / clean-SPAM / suspicious-UNWANTED fixtures"
|
||||
contains: "it-support.care"
|
||||
key_links:
|
||||
- from: "lib/services/campaign-classifier.ts"
|
||||
|
|
@ -150,7 +155,13 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
<action>
|
||||
Write `campaign-classifier.test.ts` FIRST (RED), then `campaign-classifier.fixtures.ts`, then implement the pure functions in `campaign-classifier.ts` until green (GREEN).
|
||||
|
||||
Fixtures (`campaign-classifier.fixtures.ts`, synthetic only — no real customer email): export a KnowBe4 fixture message with `from.domain='it-support.care'` and a BSN fixture with `from.domain=null, returnPath='bounces...@em8721.breachsecurenow.com'`. EACH fixture sets `authResults = { spf:'fail', dkim:'fail', dmarc:'fail' }` but `authResultsOriginal = { spf:'pass', dkim:'pass', dmarc:'pass' }` — this reproduces the forwarding-induced auth-verdict inversion (Pitfall 1) so the simulation-not-THREAT test proves both the allowlist short-circuit AND the authResultsOriginal precedence.
|
||||
Fixtures (`campaign-classifier.fixtures.ts`, synthetic only — no real customer email). Export the following named NormalizedMessage-shaped fixtures (all synthetic; Task 2's classifyCampaign tests re-use them by feeding them through the mocked `messages.headers` JSONB path):
|
||||
- `knowbe4SimMessage`: `from.domain='it-support.care'`, `authResults = { spf:'fail', dkim:'fail', dmarc:'fail' }`, `authResultsOriginal = { spf:'pass', dkim:'pass', dmarc:'pass' }`.
|
||||
- `bsnSimMessage`: `from.domain=null`, `returnPath='bounces...@em8721.breachsecurenow.com'`, same authResults-fail / authResultsOriginal-pass inversion.
|
||||
(Both simulation fixtures reproduce the forwarding-induced auth-verdict inversion (Pitfall 1) so the simulation-not-THREAT test proves BOTH the allowlist short-circuit AND the authResultsOriginal precedence.)
|
||||
- `threatMessage`: a NON-allowlist attacker-style sender (e.g. `from.domain='mlcrosoft.live'` — a real typosquat flavor per 19-RESEARCH.md D-07 finding #3), `authResults = { spf:'fail', dkim:'fail', dmarc:'fail' }`, `authResultsOriginal = null` (so effectiveAuthResults still yields the hard fail). Used with a blast-radius `{status:'ok', delivered:3, clicked:0}` to drive the positive-path THREAT verdict.
|
||||
- `cleanSpamMessage`: a NON-allowlist bulk/newsletter sender (e.g. `from.domain='mail.example-newsletter.com'`), `authResults = { spf:'pass', dkim:'pass', dmarc:'pass' }`, `authResultsOriginal = null`, no attachment/url indicators. Used with a blast-radius showing NO delivery/click signal (e.g. `{status:'ok', delivered:0, held:1, clicked:0}`) → expected SPAM.
|
||||
- `suspiciousUnwantedMessage`: a NON-allowlist sender (e.g. `from.domain='promo.some-vendor.net'`) with `authResults = { spf:'pass', dkim:'pass', dmarc:'pass' }`, `authResultsOriginal = null`, paired in Task 2 with exactly ONE url indicator (a single suspicious signal, NOT shared across ≥2 messages) and blast-radius delivery contained to the reporter only (`{status:'ok', delivered:1, clicked:0}` with a single perRecipient entry) → expected UNWANTED (suspicious-but-contained, below the THREAT bar per D-04).
|
||||
|
||||
Implement in `campaign-classifier.ts` as bare exported functions (no class, no getInstance — mirror groupReportIntoCampaign's module shape). Use the named import `import { postgresClient } from './postgres-client';` and sibling imports `import { getBlastRadius, type BlastRadiusResult } from './mimecast-blast-radius';` / `import type { NormalizedMessage, AuthResults } from './eml-parser';`.
|
||||
|
||||
|
|
@ -164,6 +175,8 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
- `const DESTRUCTIVE_ACTIONS = new Set(['block_sender','purge_message','reset_password','isolate_endpoint'])`.
|
||||
- `mapVerdictToActions(verdict, evidence)` per the behavior block. `computeRequiresApproval(actions)` = `actions.some(a => DESTRUCTIVE_ACTIONS.has(a))`.
|
||||
|
||||
ASSUMPTION FLAG (WARNING from checker — surface to user at/after execution): the `clicked > 0` trigger that escalates a THREAT verdict's actions to include `reset_password` / `isolate_endpoint` / `disable_forwarding_rule` is a REASONED research proposal (19-RESEARCH.md Open Question #1 + Assumptions A-tier), NOT an explicit CONTEXT.md decision — CONTEXT.md D-08 only locks the vocabulary and the OR'd approval invariant, and delegated finer action-mapping to "Claude's Discretion". This choice materially affects what Phase 20 gates approval on, so the executor should note it in the SUMMARY and it is worth a quick user confirmation. It does NOT contradict any locked decision or the CLASSIFY-02 invariant, so it is not a blocker.
|
||||
|
||||
Each pure function gets its own `describe` block with direct input/output assertions (no DB/Mimecast mocks needed for these). Do NOT implement `classifyCampaign`/`gatherCampaignEvidence` yet — Task 2 covers those; leave stubs or omit until then.
|
||||
</action>
|
||||
<verify>
|
||||
|
|
@ -179,16 +192,18 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
- Test asserts all-three-missing confidence === 0.10 with 3 named reason strings (CLASSIFY-03)
|
||||
- `grep -n "KNOWN_SIMULATION_SENDERS" lib/services/campaign-classifier.ts` shows the constant with both `it-support.care` and `breachsecurenow.com`
|
||||
- `grep -c "\.includes(" lib/services/campaign-classifier.ts` shows domain matching does NOT use `.includes(` for allowlist comparison (endsWith only)
|
||||
- Fixtures file exports `threatMessage`, `cleanSpamMessage`, and `suspiciousUnwantedMessage` (non-simulation fixtures) in addition to the two simulation fixtures — verify via `grep -n "threatMessage\|cleanSpamMessage\|suspiciousUnwantedMessage" lib/services/campaign-classifier.fixtures.ts`
|
||||
- `npx tsc --noEmit --pretty` exits 0
|
||||
</acceptance_criteria>
|
||||
<done>All pure rule functions implemented and unit-tested green; fixtures exist with the authResultsOriginal-pass/authResults-fail inversion; tsc clean.</done>
|
||||
<done>All pure rule functions implemented and unit-tested green; simulation + non-simulation (threat/clean/suspicious) fixtures exist with the authResultsOriginal precedence coverage; tsc clean.</done>
|
||||
</task>
|
||||
|
||||
<task type="tdd" tdd="true">
|
||||
<name>Task 2: classifyCampaign orchestrator + evidence gathering + append-only INSERT (RED→GREEN)</name>
|
||||
<files>lib/services/campaign-classifier.ts, lib/services/campaign-classifier.test.ts</files>
|
||||
<read_first>
|
||||
- lib/services/campaign-classifier.ts (current state from Task 1 — pure functions to compose)
|
||||
- lib/services/campaign-classifier.ts (current state from Task 1 — pure functions + fixtures to compose)
|
||||
- lib/services/campaign-classifier.fixtures.ts (simulation + threat/clean/suspicious fixtures from Task 1)
|
||||
- lib/services/campaign-grouping-service.ts (evidence gather + ORDER BY r.created_at ASC "earliest report is canonical" convention; catch-log-rethrow orchestrator boundary; [CAMPAIGN-GROUPING] log prefix style)
|
||||
- app/api/phishing/campaigns/[id]/route.ts (bulk-fetch-by-id-array shape: reportIds → messages via report_id=ANY, messageIds → indicators via message_id=ANY; headers->>'subject')
|
||||
- migrations/097_phishing_triage_schema.sql (classifications INSERT columns; messages/indicators/reports/campaigns columns)
|
||||
|
|
@ -199,24 +214,36 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
<behavior>
|
||||
- classifyCampaign returns an object with verdict ∈ {'SPAM','UNWANTED','THREAT'}, numeric confidence, string summary, string[] reasons, string[] recommendedActions, boolean requiresApproval, string id, string campaignId, string createdAt (CLASSIFY-01)
|
||||
- Simulation fixture (it-support.care OR breachsecurenow.com sender) with authResults fail but authResultsOriginal pass + delivered>0: verdict is SPAM or UNWANTED, NEVER THREAT (CLASSIFY-04 — allowlist short-circuits before THREAT tier)
|
||||
- THREAT only when blastRadius.status==='ok' AND (delivered>0 OR clicked>0) AND (hasHardAuthFail(effectiveAuthResults) on some message OR a known-bad indicator match) AND sender NOT on allowlist (D-03)
|
||||
- POSITIVE-PATH THREAT (D-03 both conditions): non-allowlist `threatMessage` sender + blastRadius.status==='ok' with delivered>0 + effective hard auth fail → verdict === 'THREAT' AND recommendedActions includes at least 'block_sender' and 'purge_message' (a broken/always-SPAM stub MUST fail this)
|
||||
- THREAT known-bad-indicator OR-branch: non-allowlist sender + delivered>0 + NO hard auth fail (auth pass) BUT the same attachment_hash/url value on ≥2 distinct messages → verdict === 'THREAT' (D-03 second signal, research A4)
|
||||
- SPAM vs UNWANTED split (D-04): `cleanSpamMessage` (auth pass, no indicators, no delivery/click signal) → verdict === 'SPAM'; `suspiciousUnwantedMessage` (auth pass, exactly one url indicator not shared across messages, delivery contained to the reporter only) → verdict === 'UNWANTED'. Two distinct fixtures prove the boundary is implemented, not stubbed.
|
||||
- Known-bad indicator match = same attachment_hash or url value on ≥2 distinct messages in the campaign (cross-report correlation — no external reputation lookup) (D-03 interpretation, research A4)
|
||||
- When blastRadius.status!=='ok', or no message parsed, or no indicators: confidence < 1.0 and the specific source named in reasons (CLASSIFY-03)
|
||||
- Persisted reasons/summary contain no raw email body text; evidence arrays are capped (CLASSIFY-06)
|
||||
- Exactly one new classifications row INSERTed per call (append-only, no ON CONFLICT) — verified via the mocked query call args (D-02)
|
||||
</behavior>
|
||||
<action>
|
||||
Extend `campaign-classifier.test.ts` with a `describe('classifyCampaign', ...)` block using `vi.mock('./postgres-client')` (queryMock, matching campaign-grouping-service.test.ts) and `vi.mock('./mimecast-blast-radius', () => ({ getBlastRadius: (...) => getBlastRadiusMock(...) }))`. Route mock query responses by distinguishing SQL substring per call (`FROM campaigns`, `FROM reports`, `FROM messages`, `FROM indicators`, `INSERT INTO classifications`) — NOT by call order. Add the named tests: "returns exactly one verdict", "simulation allowlist" (both it-support.care and breachsecurenow.com fixtures, delivered>0, authResults fail → assert verdict !== 'THREAT'), "evidence bounding" (feed a campaign with many indicators; assert persisted reasons is a short bounded array and no reason string contains a raw body). Write these RED first.
|
||||
Extend `campaign-classifier.test.ts` with a `describe('classifyCampaign', ...)` block using `vi.mock('./postgres-client')` (queryMock, matching campaign-grouping-service.test.ts) and `vi.mock('./mimecast-blast-radius', () => ({ getBlastRadius: (...) => getBlastRadiusMock(...) }))`. Route mock query responses by distinguishing SQL substring per call (`FROM campaigns`, `FROM reports`, `FROM messages`, `FROM indicators`, `INSERT INTO classifications`) — NOT by call order. Build a small local helper that turns a fixture NormalizedMessage into a `messages` row (`{ id, report_id, headers: <fixture> }`) so the same fixtures drive the orchestrator.
|
||||
|
||||
Write these named tests RED first, then implement until green:
|
||||
- `"returns exactly one verdict"` — assert result.verdict is one of the three literals and all payload fields present with correct types (CLASSIFY-01).
|
||||
- `"simulation allowlist"` — BOTH it-support.care (From match) and breachsecurenow.com (Return-Path match, From.domain null) fixtures, delivered>0, authResults fail → assert `result.verdict !== 'THREAT'` for each.
|
||||
- `"threat tier"` — feed `threatMessage` with `getBlastRadiusMock` → `{status:'ok', delivered:3, clicked:0}`; assert `result.verdict === 'THREAT'` and `result.recommendedActions` includes 'block_sender' and 'purge_message' and `result.requiresApproval === true` (positive-path THREAT — closes the blocker: proves classifyCampaign CAN return THREAT for a real signal).
|
||||
- `"threat tier known-bad indicator"` — non-allowlist sender across 2 messages sharing one url indicator value, auth PASS, delivered>0, no hard auth fail → assert `result.verdict === 'THREAT'` (D-03 OR-branch, no auth-fail path).
|
||||
- `"spam vs unwanted tier"` — TWO sub-cases: `cleanSpamMessage` + no-delivery/click blast radius → `result.verdict === 'SPAM'`; `suspiciousUnwantedMessage` + single url indicator + delivery-contained-to-reporter blast radius → `result.verdict === 'UNWANTED'` (proves D-04 boundary is real, not an always-SPAM stub).
|
||||
- `"evidence bounding"` — feed a campaign with many indicators; assert persisted reasons is a short bounded array and no reason string contains a raw body (CLASSIFY-06).
|
||||
|
||||
Then implement in `campaign-classifier.ts`:
|
||||
- `interface ClassifyResult { id: string; campaignId: string; verdict: 'SPAM'|'UNWANTED'|'THREAT'; confidence: number; summary: string; reasons: string[]; recommendedActions: string[]; requiresApproval: boolean; createdAt: string }` (export it).
|
||||
- `gatherCampaignEvidence(campaignId)`: SELECT campaign row; SELECT reports `WHERE campaign_id=$1 ORDER BY created_at ASC` (LEFT JOIN contacts for requester_email); bulk-fetch messages `WHERE report_id = ANY($1::uuid[])` selecting `id, report_id, headers, message_id`; bulk-fetch indicators `WHERE message_id = ANY($1::uuid[])` selecting `id, message_id, indicator_type, value`. Parse `headers` JSONB into the NormalizedMessage-shaped fields needed (from.domain, returnPath, authResults, authResultsOriginal, subject). Use the EARLIEST report as canonical sender/subject/date-window for one `getBlastRadius()` call (±24h window around report.created_at); when no report exists, synthesize `{ status:'unavailable', reason:'not_configured' }` without calling Mimecast (research A6). Return a BOUNDED payload: counts + capped sample arrays (cap indicator/report samples at 10), never raw body text (CLASSIFY-06 / T-19-03 DoS guard).
|
||||
- `classifyCampaign(campaignId)`: gather evidence → `if isKnownSimulationSender(any message) → skip THREAT, run evaluateSpamVsUnwanted only (D-06 short-circuit)` → else `evaluateThreatTier` (D-03) → else `evaluateSpamVsUnwanted` (D-04, SPAM if no suspicious signal, UNWANTED if a suspicious signal present but below THREAT bar) → `computeConfidence` → `mapVerdictToActions` → `computeRequiresApproval` → build a short `summary` string naming the verdict and top reason → INSERT one classifications row `RETURNING id::text AS id, created_at::text AS created_at` with `reasons`/`recommended_actions` as `JSON.stringify(...)::jsonb` → return ClassifyResult. Wrap the orchestrator body in try/catch that logs `[CAMPAIGN-CLASSIFIER]` + err.message (never full payloads — T-17-style) and rethrows.
|
||||
- `classifyCampaign(campaignId)`: gather evidence → `if isKnownSimulationSender(any message) → skip THREAT, run evaluateSpamVsUnwanted only (D-06 short-circuit)` → else `evaluateThreatTier` (D-03: blastRadius.status==='ok' AND (delivered>0 OR clicked>0) AND (hasHardAuthFail on effectiveAuthResults of some message OR known-bad-indicator match)) → else `evaluateSpamVsUnwanted` (D-04, SPAM if no suspicious signal, UNWANTED if a suspicious signal present — one indicator match, a mild auth issue, or delivery contained to the reporter — but below the THREAT bar) → `computeConfidence` → `mapVerdictToActions` → `computeRequiresApproval` → build a short `summary` string naming the verdict and top reason → INSERT one classifications row `RETURNING id::text AS id, created_at::text AS created_at` with `reasons`/`recommended_actions` as `JSON.stringify(...)::jsonb` → return ClassifyResult. Wrap the orchestrator body in try/catch that logs `[CAMPAIGN-CLASSIFIER]` + err.message (never full payloads — T-17-style) and rethrows.
|
||||
- Known-bad indicator match computed over the gathered indicators: group by (indicator_type, value) for attachment_hash/url types, flag true if any value spans ≥2 distinct message_id.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>npx vitest run lib/services/campaign-classifier.test.ts -t "returns exactly one verdict"</automated>
|
||||
<automated>npx vitest run lib/services/campaign-classifier.test.ts -t "simulation allowlist"</automated>
|
||||
<automated>npx vitest run lib/services/campaign-classifier.test.ts -t "threat tier"</automated>
|
||||
<automated>npx vitest run lib/services/campaign-classifier.test.ts -t "spam vs unwanted tier"</automated>
|
||||
<automated>npx vitest run lib/services/campaign-classifier.test.ts -t "evidence bounding"</automated>
|
||||
<automated>npx vitest run lib/services/campaign-classifier.test.ts</automated>
|
||||
<automated>npx tsc --noEmit --pretty</automated>
|
||||
|
|
@ -224,13 +251,16 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
<acceptance_criteria>
|
||||
- `npx vitest run lib/services/campaign-classifier.test.ts` passes all describe blocks including classifyCampaign
|
||||
- "simulation allowlist" test passes for BOTH it-support.care (From match) and breachsecurenow.com (Return-Path match, From.domain null) fixtures, each asserting `result.verdict !== 'THREAT'` despite `authResults` hard-fail and delivered>0
|
||||
- "returns exactly one verdict" asserts result.verdict is one of the three literals and all six payload fields are present with correct types (CLASSIFY-01)
|
||||
- "threat tier" test asserts `result.verdict === 'THREAT'`, `recommendedActions` includes 'block_sender' + 'purge_message', and `requiresApproval === true` for the non-allowlist real-signal fixture (positive-path THREAT — the previously-missing coverage; an always-SPAM stub fails here)
|
||||
- "threat tier known-bad indicator" asserts THREAT fires on the ≥2-message shared-indicator OR-branch with auth PASS (D-03 second signal)
|
||||
- "spam vs unwanted tier" asserts `cleanSpamMessage → 'SPAM'` AND `suspiciousUnwantedMessage → 'UNWANTED'` (D-04 boundary proven; an always-SPAM stub fails the UNWANTED case)
|
||||
- "returns exactly one verdict" asserts result.verdict is one of the three literals and all payload fields are present with correct types (CLASSIFY-01)
|
||||
- Mock asserts exactly one `INSERT INTO classifications` query and that it contains no `ON CONFLICT` (append-only, D-02)
|
||||
- `grep -n "authResultsOriginal" lib/services/campaign-classifier.ts` shows the precedence is applied inside the THREAT-tier path
|
||||
- `grep -n "getBlastRadius" lib/services/campaign-classifier.ts` shows exactly one call site guarded by "earliest report exists"
|
||||
- `npm test` full suite green; `npx tsc --noEmit --pretty` exits 0
|
||||
</acceptance_criteria>
|
||||
<done>classifyCampaign orchestrates evidence→rules→confidence→actions→append-only INSERT; simulation short-circuit and authResultsOriginal precedence proven by tests; full suite + tsc green.</done>
|
||||
<done>classifyCampaign orchestrates evidence→rules→confidence→actions→append-only INSERT; simulation short-circuit, positive-path THREAT, D-03 indicator OR-branch, and the D-04 SPAM/UNWANTED split all proven by named tests; full suite + tsc green.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
|
@ -248,7 +278,7 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-19-01 | Spoofing | domainMatchesAllowlist | mitigate | Match exact-domain-or-proper-subdomain only (`d===allowed \|\| d.endsWith('.'+allowed)`); never `.includes()` substring — asserted by a test that `it-support.care.attacker.net` does NOT match |
|
||||
| T-19-02 | Tampering (logic drift) | D-03/D-04/D-06 rule functions | mitigate | Pure, unit-tested TS functions + TS-constant allowlist (not a live-editable DB table); every rule covered by a dedicated test |
|
||||
| T-19-02 | Tampering (logic drift) | D-03/D-04/D-06 rule functions | mitigate | Pure, unit-tested TS functions + TS-constant allowlist (not a live-editable DB table); every rule covered by a dedicated test — including positive-path THREAT and the SPAM/UNWANTED split, so a silently-weakened verdict path is caught |
|
||||
| T-19-03 | Denial of Service | gatherCampaignEvidence / persisted reasons | mitigate | Bounded evidence payload — counts + sample arrays capped at 10, never full report/message/indicator sets or raw body text embedded verbatim (doubles as CLASSIFY-06) |
|
||||
| T-19-06 | Spoofing (auth-verdict inversion) | effectiveAuthResults | mitigate | Prefer authResultsOriginal over post-forward authResults so a forwarded genuine simulation isn't pushed to THREAT on an invalidated DKIM signature (Pitfall 1) |
|
||||
| T-19-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed this phase (classifier consumes only shipped project code + Node built-ins); Package Legitimacy Gate N/A per 19-RESEARCH.md |
|
||||
|
|
@ -258,7 +288,7 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
- `npx vitest run lib/services/campaign-classifier.test.ts` — all describe blocks green
|
||||
- `npm test` — full suite green (no regression in campaign-grouping / mimecast / eml-parser)
|
||||
- `npx tsc --noEmit --pretty` — clean
|
||||
- CLASSIFY-01/02/03/04/06 each proven by a named test per 19-VALIDATION.md Per-Task Verification Map
|
||||
- CLASSIFY-01/02/03/04/06 each proven by a named test per 19-VALIDATION.md Per-Task Verification Map; verdict logic (D-03 THREAT positive path + D-04 SPAM/UNWANTED split) proven by the "threat tier" / "threat tier known-bad indicator" / "spam vs unwanted tier" named tests
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
|
|
@ -266,6 +296,7 @@ campaigns (id, campaign_key, group_method, first_seen_at, last_seen_at, report
|
|||
- destructive action ⇒ requiresApproval:true; disable_forwarding_rule alone ⇒ false (CLASSIFY-02)
|
||||
- incomplete evidence lowers confidence and names the missing source (CLASSIFY-03)
|
||||
- it-support.care / breachsecurenow.com simulation fixtures never classified THREAT absent contrary evidence (CLASSIFY-04)
|
||||
- a real-signal non-simulation campaign IS classified THREAT with destructive actions, and the SPAM-vs-UNWANTED boundary is exercised by distinct fixtures (positive verdict-logic coverage — D-03/D-04)
|
||||
- evidence is bounded/structured — no raw unbounded body reaches classifier/reasons (CLASSIFY-06)
|
||||
</success_criteria>
|
||||
|
||||
|
|
|
|||
|
|
@ -712,7 +712,9 @@ directly above.
|
|||
| A5 | TypeScript-constant allowlist (not DB table) is the right storage choice for D-06 | Standard Stack — Alternatives Considered | If wrong, ops will need a code deploy to add/remove a vendor domain rather than an admin UI edit — a process cost, not a correctness risk. Reversible later without a rework (per D-06's own multi-vendor-extensibility requirement). |
|
||||
| A6 | Primary blast-radius call should use the campaign's EARLIEST report as canonical sender/subject/date-window | Code Examples (`gatherCampaignEvidence`) | If wrong (e.g. a later report has better/more-complete data), blast-radius lookup could target a slightly wrong subject/date-window and return fewer/no matches — degrades to `status:'unavailable'`-like under-evidence, not a false verdict, since D-03's gate still requires a real signal to fire. |
|
||||
|
||||
## Open Questions
|
||||
## Open Questions (ALL RESOLVED)
|
||||
|
||||
> All three open questions below are functionally answered by 19-01-PLAN.md and locked for this phase. Each carries an explicit RESOLVED note. The one item that is a reasoned assumption rather than a locked decision (Q1's `clicked > 0` action-escalation trigger) is flagged in 19-01-PLAN.md Task 2 for a quick user confirmation at/after execution.
|
||||
|
||||
1. **Should `disable_forwarding_rule` ever be recommended by this phase's rules, given no
|
||||
evidence source in Phases 15-18 signals "a mailbox forwarding rule was created"?**
|
||||
|
|
@ -725,6 +727,7 @@ directly above.
|
|||
reasoned proxy for "possible account compromise, worth checking for a malicious forwarding
|
||||
rule" — but confirm with the user/planner before locking in, since this is inference, not an
|
||||
explicit CONTEXT.md decision.
|
||||
- **RESOLVED:** Tied to `clicked > 0` on a THREAT verdict (19-01-PLAN.md Task 1 `mapVerdictToActions`, covered by the "threat tier" test). Flagged in 19-01-PLAN.md Task 2 as a reasoned assumption (not a CONTEXT.md decision) worth a quick user confirmation, since it affects what Phase 20 gates approval on.
|
||||
|
||||
2. **Should the KnowBe4/BSN allowlist match check `Return-Path` via a parsed domain, or does
|
||||
`NormalizedMessage` need a new `returnPathDomain` field?**
|
||||
|
|
@ -736,6 +739,7 @@ directly above.
|
|||
- Recommendation: do it locally in `campaign-classifier.ts` (as shown in Code Examples) — Phase
|
||||
16 is marked "read, does not modify" in this phase's canonical_refs, and the split is a
|
||||
one-line, low-risk operation not worth reopening an already-shipped, tested module for.
|
||||
- **RESOLVED:** Return-Path domain is parsed locally in `campaign-classifier.ts` (`.split('@')[1]` inside `isKnownSimulationSender`, 19-01-PLAN.md Task 1) — `eml-parser.ts` is NOT modified.
|
||||
|
||||
3. **Is a campaign's blast-radius lookup re-run on every `/classify` call, or cached/reused across
|
||||
calls within the same campaign?**
|
||||
|
|
@ -747,6 +751,7 @@ directly above.
|
|||
logic needed in this phase.
|
||||
- Recommendation: no action needed; call `getBlastRadius()` on every `classifyCampaign()`
|
||||
invocation and let its existing cache absorb repeat calls.
|
||||
- **RESOLVED:** No new caching added this phase; `getBlastRadius()`'s existing 5-minute Redis cache is relied on as-is (19-01-PLAN.md Task 2 calls it once per `classifyCampaign`).
|
||||
|
||||
## Environment Availability
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
phase: 19
|
||||
slug: classification-engine
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
nyquist_compliant: true
|
||||
wave_0_complete: false
|
||||
created: 2026-07-16
|
||||
---
|
||||
|
|
@ -73,6 +73,6 @@ created: 2026-07-16
|
|||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 5s (automated) — manual route check is a pre-merge/pre-verify gate, not per-commit
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
- [x] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue