From f2b3602ca1a856f77289e6786a776b0b87b4d724 Mon Sep 17 00:00:00 2001 From: lorentz Date: Thu, 16 Jul 2026 08:05:15 -0400 Subject: [PATCH] docs(19): add pattern map --- .../19-classification-engine/19-PATTERNS.md | 333 ++++++++++++++++++ 1 file changed, 333 insertions(+) create mode 100644 .planning/phases/19-classification-engine/19-PATTERNS.md diff --git a/.planning/phases/19-classification-engine/19-PATTERNS.md b/.planning/phases/19-classification-engine/19-PATTERNS.md new file mode 100644 index 0000000..25b6c41 --- /dev/null +++ b/.planning/phases/19-classification-engine/19-PATTERNS.md @@ -0,0 +1,333 @@ +# Phase 19: Classification Engine - Pattern Map + +**Mapped:** 2026-07-16 +**Files analyzed:** 3 (2 new source files, 1 new test file — no existing files modified) +**Analogs found:** 3 / 3 + +## File Classification + +| New/Modified File | Role | Data Flow | Closest Analog | Match Quality | +|--------------------|------|-----------|-----------------|----------------| +| `lib/services/campaign-classifier.ts` | service | transform (evidence-in / rule-eval / verdict-out, CRUD-adjacent: reads + one INSERT) | `lib/services/robotic-classifier.ts` (rule-eval style) + `lib/services/mimecast-blast-radius.ts` (pure-function module shape, discriminated-union result) + `lib/services/campaign-grouping-service.ts` (evidence-gather-then-orchestrate shape) | exact (composite of 3 role-adjacent analogs — no single file covers both "deterministic rule evaluation" and "gather campaign evidence") | +| `app/api/phishing/campaigns/[id]/classify/route.ts` | route (controller) | request-response | `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` | exact | +| `lib/services/campaign-classifier.test.ts` | test | transform (unit, mocked I/O) | `lib/services/campaign-grouping-service.test.ts` + `lib/services/mimecast-blast-radius.test.ts` | exact | + +## Pattern Assignments + +### `lib/services/campaign-classifier.ts` (service, transform/rule-eval) + +**Analogs:** `lib/services/robotic-classifier.ts` (D-01's named architectural analog — rule-eval style only, NOT the DB-driven rule-cache wrapper), `lib/services/mimecast-blast-radius.ts` (pure-function module doc-comment style + discriminated-union result), `lib/services/campaign-grouping-service.ts` (evidence-gather-then-pure-logic module shape, same schema tables) + +**File header/doc-comment pattern** (mirrors `mimecast-blast-radius.ts` lines 1-35 and `campaign-grouping-service.ts` lines 1-21) — a substantial doc-comment stating load-bearing invariants up front (ephemerality, known limitations, D-06 rationale) is this codebase's convention for phishing-triage service modules, not just a JSDoc blurb: +```typescript +// Source: lib/services/mimecast-blast-radius.ts lines 1-35 (style to follow) +/** + * Mimecast blast-radius lookup abstraction (Phase 17). + * + * ... + * Three load-bearing facts: + * (a) EPHEMERAL (D-03): ... + * (b) ... IS BEST-EFFORT (D-02): ... + * (c) KNOWN LIMITATION — ... (D-05): ... + */ +``` + +**Imports pattern** (mirrors `campaign-grouping-service.ts` lines 22-25 and `mimecast-blast-radius.ts` lines 37-44 — no relative `../` chains, sibling `lib/services/*` imports directly by filename, no path alias needed inside `lib/services/`): +```typescript +// Source: lib/services/campaign-grouping-service.ts lines 22-25 +import type { PoolClient } from 'pg'; +import { postgresClient } from './postgres-client'; +// (this phase additionally needs, following the same sibling-import convention:) +// import { getBlastRadius, type BlastRadiusResult } from './mimecast-blast-radius'; +// import type { NormalizedMessage, AuthResults } from './eml-parser'; +``` + +**Deterministic rule-evaluation style (D-01)** — `evaluateContains()` precedent: `.toLowerCase()` + string comparison ONLY, no regex/eval, small pure functions each doing exactly one check: +```typescript +// Source: lib/services/robotic-classifier.ts lines 206-219 (evaluateContains) +private evaluateContains( + fieldValue: string | number, + matchValue: any, + caseSensitive: boolean +): boolean { + const text = String(fieldValue); + const searchText = caseSensitive ? text : text.toLowerCase(); + const patterns = Array.isArray(matchValue) ? matchValue : [matchValue]; + + return patterns.some((pattern: string) => { + const searchPattern = caseSensitive ? String(pattern) : String(pattern).toLowerCase(); + return searchText.includes(searchPattern); + }); +} +``` +Adapt to a bare exported function (no class, per D-01/Pattern 2 below) — e.g. `isKnownSimulationSender()`, `hasHardAuthFail()`, `evaluateThreatTier()` each following this one-check-per-function, string-only-comparison shape. Domain-suffix matching (not substring) is required for the sender-domain allowlist — see Security Domain note below; do NOT reuse `evaluateContains`'s bare `.includes()` for domain matching. + +**Single-exported-orchestrator shape (D-01/D-02, Pattern 2)** — no class, no singleton instance, mirrors `groupReportIntoCampaign()` and `detectPhishingTicket()`: +```typescript +// Source: lib/services/campaign-grouping-service.ts lines 160-163 (signature shape to mirror) +export async function groupReportIntoCampaign( + reportId: string, + opts?: { skipIfAlreadyGrouped?: boolean } +): Promise { +``` +This phase's equivalent: `export async function classifyCampaign(campaignId: string): Promise` — single entry point the new route calls directly (no class, no `getInstance()`). + +**Evidence-gathering / bulk-fetch pattern** (bulk-fetch-by-id-array shape, reportIds → messagesRes → messageIds → indicatorsRes) — copy verbatim shape from `app/api/phishing/campaigns/[id]/route.ts` lines 86-116 (already used in RESEARCH.md's Code Examples section) and `campaign-grouping-service.ts`'s `ORDER BY r.created_at ASC` "earliest report is canonical" convention (lines 176-181, 217-218): +```typescript +// Source: app/api/phishing/campaigns/[id]/route.ts lines 96-116 (bulk-fetch-by-id-array shape) +const reportIds = reportsRes.rows.map((r) => r.id); +const messagesRes = reportIds.length + ? await postgresClient.query( + `SELECT id::text, report_id::text, message_id, headers->>'subject' AS subject + FROM messages WHERE report_id = ANY($1::uuid[])`, + [reportIds] + ) + : { rows: [] as MessageRow[] }; +const messageIds = messagesRes.rows.map((m) => m.id); +const indicatorsRes = messageIds.length + ? await postgresClient.query( + `SELECT id::text, message_id::text, indicator_type, value, metadata + FROM indicators WHERE message_id = ANY($1::uuid[])`, + [messageIds] + ) + : { rows: [] as IndicatorRow[] }; +``` + +**Discriminated-union result shape** (mirrors `BlastRadiusResult`, the precedent CONTEXT.md explicitly names for shaping this phase's own evidence-completeness signal): +```typescript +// Source: lib/services/mimecast-blast-radius.ts lines 60-71 +export type BlastRadiusResult = + | { status: 'unavailable'; reason: 'not_configured' | 'lookup_failed'; error?: string } + | { + status: 'ok'; + matched: number; + delivered: number; + held: number; + rejected: number; + clicked: number; + perRecipient: Array<{ recipient: string; status: 'delivered' | 'held' | 'rejected' | 'unknown' }>; + source: 'fan-out'; + }; +``` + +**Error handling pattern** (mirrors `getBlastRadius()`'s try/catch — log message only, never full payloads; and `groupReportIntoCampaign()`'s catch-log-rethrow at the orchestrator boundary): +```typescript +// Source: lib/services/mimecast-blast-radius.ts lines 188-194 +} catch (err) { + // T-17-02: log err.message ONLY, never full Mimecast response bodies + const message = err instanceof Error ? err.message : String(err); + console.error('[MIMECAST-BLAST-RADIUS] lookup failed', message); + return { status: 'unavailable', reason: 'lookup_failed', error: message }; +} +``` +```typescript +// Source: lib/services/campaign-grouping-service.ts lines 475-478 +} catch (error) { + console.error('[CAMPAIGN-GROUPING] Failed to group report into campaign', reportId, error); + throw error; +} +``` +Use a `[CAMPAIGN-CLASSIFIER]` log prefix, matching the `[MIMECAST-BLAST-RADIUS]`/`[CAMPAIGN-GROUPING]`/`[PHISHING-DETECT]` bracketed-module-name convention. + +**Append-only INSERT pattern** (`classifications` row, migration 097 exact columns — confirmed exact column list/types from `migrations/097_phishing_triage_schema.sql` lines 114-124: `verdict TEXT`, `confidence NUMERIC`, `summary TEXT`, `reasons JSONB`, `recommended_actions JSONB`, `requires_approval BOOLEAN NOT NULL DEFAULT false`): +```typescript +// Adapt from lib/services/phishing-detector.ts lines 210-229's INSERT-with-RETURNING shape +await postgresClient.query( + `INSERT INTO classifications ( + campaign_id, verdict, confidence, summary, reasons, recommended_actions, requires_approval + ) VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb, $7) + RETURNING id::text AS id, created_at::text AS created_at`, + [campaignId, verdict, confidence, summary, JSON.stringify(reasons), JSON.stringify(recommendedActions), requiresApproval] +); +``` +Note: no `ON CONFLICT` — this is append-only (D-02), unlike `phishing-detector.ts`'s `ON CONFLICT (ticket_id) DO UPDATE` upsert, which does NOT apply here. + +**Auth-verdict precedence (Pitfall 1, load-bearing)** — reads `NormalizedMessage.authResultsOriginal` (from `eml-parser.ts` lines 104, 269-271) FIRST, falling back to `authResults`: +```typescript +function effectiveAuthResults(message: { headers: { authResults: AuthResults; authResultsOriginal: AuthResults | null } }): AuthResults { + return message.headers.authResultsOriginal ?? message.headers.authResults; +} +function hasHardAuthFail(effective: AuthResults): boolean { + return effective.spf === 'fail' || effective.dkim === 'fail' || effective.dmarc === 'fail'; +} +``` + +**Named-pattern-constant precedent (D-06/D-07 allowlist shape)** — mirrors `KNOWN_PHISHING_PATTERNS` (`phishing-detector.ts` lines 26-35): a locked, exported, commented `readonly` constant, NOT a DB table: +```typescript +// Source: lib/services/phishing-detector.ts lines 26-35 (shape to mirror for KNOWN_SIMULATION_SENDERS) +export const KNOWN_PHISHING_PATTERNS: readonly string[] = [ + 'Phishing Report', + 'Spam Alert', + 'Phishing Alert - Email Security Report', + 'KnowBe4 Phish Alert Report', + 'Source: KnowBe4 Phish Alert Button', + 'userSubmissionsReportMessage', + 'reported message destinations', + 'Microsoft directly', +]; +``` +Do NOT reuse or extend this constant — it is the REPORT-pattern list (proves how a ticket was reported), a different signal from the NEW sender-domain allowlist this phase adds (proves who sent the original message). See RESEARCH.md's exact `KNOWN_SIMULATION_SENDERS` proposal (vendor-keyed array of domain arrays) — same "locked TS constant with a doc-comment citing ticket evidence" shape. + +--- + +### `app/api/phishing/campaigns/[id]/classify/route.ts` (route/controller, request-response) + +**Analog:** `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` (full file, 91 lines — read in full, reproduced below in relevant part) + +**Imports pattern** (lines 11-16 — path-alias `@/lib/...` imports, unlike `lib/services/*` sibling imports): +```typescript +// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 11-16 +import { NextRequest, NextResponse } from 'next/server'; +import { requirePermission } from '@/lib/auth-utils'; +import postgresClient from '@/lib/services/postgres-client'; +import { detectPhishingTicket, type DetectableTicket } from '@/lib/services/phishing-detector'; +import { parseAndStoreMessage } from '@/lib/services/phishing-eml-service'; +import { groupReportIntoCampaign } from '@/lib/services/campaign-grouping-service'; +``` +This phase's route imports `classifyCampaign` from `@/lib/services/campaign-classifier` instead. + +**Auth pattern (exact, reuse verbatim — Phase 18 D-06 convention, same `'analyze'` action, NOT a new permission)** (lines 18-23): +```typescript +// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 18-23 +export async function POST( + request: NextRequest, + { params }: { params: Promise<{ ticket_id: string }> } +) { + const { error } = await requirePermission('phishing', 'analyze'); + if (error) return error; +``` +For `.../campaigns/[id]/classify/route.ts`, `params` is `Promise<{ id: string }>` instead of `{ ticket_id: string }` (matches `campaigns/[id]/route.ts`'s param name). + +**UUID validation guard (V5, copy verbatim from the campaign detail route since it's a campaign id, not a ticket_id)**: +```typescript +// Source: app/api/phishing/campaigns/[id]/route.ts lines 11, 66-70 +const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; +// ... +const { id } = await params; +if (!UUID_RE.test(id)) { + return NextResponse.json({ error: 'Invalid campaign id' }, { status: 400 }); +} +``` +Note: `analyze/route.ts` validates a numeric `ticket_id` (`Number.isFinite`), which does NOT apply here — this route's id is a campaign UUID, so follow `campaigns/[id]/route.ts`'s `UUID_RE` guard instead, not `analyze/route.ts`'s numeric check. + +**Core request-response + 404 pattern** (lines 44-48, adapted — campaign-not-found mirrors ticket-not-found): +```typescript +// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 44-48 (shape) +const r = row.rows[0]; +if (!r) { + return NextResponse.json({ error: 'Ticket not found' }, { status: 404 }); +} +``` +Adapt: `if (!campaign) return NextResponse.json({ error: 'Campaign not found' }, { status: 404 });` — same shape as `campaigns/[id]/route.ts` lines 79-82. + +**Success response shape** (lines 77-82 — flat camelCase JSON, no wrapper envelope): +```typescript +// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 77-82 +return NextResponse.json({ + reportId: detection.reportId, + campaignId: grouped?.campaignId ?? null, + groupMethod: grouped?.groupMethod ?? null, + created: grouped?.created ?? false, +}); +``` +This phase's equivalent: `{ id, campaignId, verdict, confidence, summary, reasons, recommendedActions, requiresApproval }` (per RESEARCH.md's response shape). + +**Error handling pattern** (lines 83-89 — try/catch wraps the whole handler body, 500 with `message`): +```typescript +// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 83-89 +} catch (err) { + console.error('[PHISHING-ANALYZE] Failed to analyze ticket', ticketId, err); + return NextResponse.json( + { error: 'Failed to analyze ticket', message: err instanceof Error ? err.message : 'Unknown error' }, + { status: 500 } + ); +} +``` +Use a `[PHISHING-CLASSIFY]` log prefix for this route, matching `[PHISHING-ANALYZE]`/`[PHISHING-CAMPAIGN-DETAIL]`'s per-route bracketed-name convention. + +--- + +### `lib/services/campaign-classifier.test.ts` (test, transform) + +**Analogs:** `lib/services/campaign-grouping-service.test.ts` (mocking `postgresClient` via `vi.mock` factory, hoisting discipline) + `lib/services/mimecast-blast-radius.test.ts` (mocking a sibling service module entirely) + +**Mock-before-import pattern** (lines 1-14 — `vi.mock` factory MUST be declared before the module-under-test import; `// eslint-disable-next-line import/first` comment documents why): +```typescript +// Source: lib/services/campaign-grouping-service.test.ts lines 1-14 +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +// Mock postgresClient BEFORE importing the module under test. +const queryMock = vi.fn(); +const transactionMock = vi.fn(); +vi.mock('./postgres-client', () => ({ + postgresClient: { + query: (...args: unknown[]) => queryMock(...args), + transaction: (...args: unknown[]) => transactionMock(...args), + }, +})); + +// eslint-disable-next-line import/first -- imported after vi.mock hoisting +import { normalizeSubject, extractUrlDomain, groupReportIntoCampaign } from './campaign-grouping-service'; +``` +This phase's test additionally needs `vi.mock('./mimecast-blast-radius', ...)` (per RESEARCH.md's Wave 0 Gaps section) — follow `mimecast-blast-radius.test.ts`'s own `vi.mock('./mimecast-client', ...)` / `vi.mock('./redis-client', ...)` sibling-module-mocking shape (see lines 25-32 of that file) for how to mock `getBlastRadius` itself as a dependency of `campaign-classifier.ts`. + +**Per-call routed mock rows (matches SQL-substring discipline)** — when a module makes multiple sequential/branching queries (as `campaign-classifier.ts`'s evidence-gathering will), route mock responses by a distinguishing SQL substring per call rather than by call order: +```typescript +// Source: lib/services/campaign-grouping-service.test.ts lines 81-90 (pattern) +function makeClient(rows: MockRows) { + return { + query: vi.fn(async (sql: string, params?: unknown[]) => { + clientCalls.push({ sql, params: params ?? [] }); + if (sql.includes('requester_contact_id, company_id, created_at')) { + return { rows: rows.ownReport ?? [], rowCount: rows.ownReport?.length ?? 0 }; + } + if (sql.includes('FROM messages') && sql.includes('WHERE report_id = $1')) { + return { rows: rows.ownMessage ?? [], rowCount: rows.ownMessage?.length ?? 0 }; + } + // ... one branch per distinct query + }), + }; +} +``` + +**Pure-function unit tests** (mirrors `describe('normalizeSubject', ...)` / `describe('extractUrlDomain', ...)` blocks, lines 16-50) — this phase's pure rule functions (`hasHardAuthFail`, `isKnownSimulationSender`, `computeConfidence`, `mapVerdictToActions`, `computeRequiresApproval`) should each get their own `describe` block with direct input/output assertions, no mocking needed for these (only `classifyCampaign()` itself needs the DB/Mimecast mocks). + +## Shared Patterns + +### Auth/Permission (all `/api/phishing/*` routes) +**Source:** `lib/auth-utils.ts` `requirePermission(resource, action)`, established Phase 18 D-06, reused verbatim from `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` lines 22-23. +**Apply to:** `app/api/phishing/campaigns/[id]/classify/route.ts` — same `'analyze'` action, not a new permission (confirmed in `lib/permissions.ts` line 34: `phishing: ["read", "analyze", "approve", "remediate"]`). +```typescript +const { error } = await requirePermission('phishing', 'analyze'); +if (error) return error; +``` + +### Error Handling (route + service boundary) +**Source:** every `app/api/phishing/**/route.ts` file — try/catch around the full handler body, `NextResponse.json({ error, message }, { status: 500 })` on unexpected failure; every `lib/services/*.ts` orchestrator function — catch-log-with-context-then-rethrow (service layer never swallows into a 200; the route layer converts to the JSON error envelope). +**Apply to:** both new files. + +### Bracketed module-name logging +**Source:** `[MIMECAST-BLAST-RADIUS]`, `[CAMPAIGN-GROUPING]`, `[PHISHING-DETECT]`, `[PHISHING-ANALYZE]`, `[PHISHING-CAMPAIGN-DETAIL]` — every phishing-triage module/route prefixes `console.error`/`console.debug` calls with its own bracketed name. +**Apply to:** use `[CAMPAIGN-CLASSIFIER]` in the service, `[PHISHING-CLASSIFY]` in the route. + +### UUID path-param validation (V5) +**Source:** `app/api/phishing/campaigns/[id]/route.ts` lines 11, 66-70 — `UUID_RE` regex guard before any query, returns 400 on malformed input rather than letting an unhandled Postgres error surface as a 500. +**Apply to:** `app/api/phishing/campaigns/[id]/classify/route.ts` (this route's `id` is a campaign UUID, not a ticket_id — do NOT copy `analyze/route.ts`'s numeric `Number.isFinite(ticketId)` check, which validates a different id type). + +### camelCase API response transform +**Source:** every `app/api/phishing/**/route.ts` GET/POST handler — snake_case DB columns manually mapped to camelCase JSON keys at the response boundary (e.g. `campaigns/[id]/route.ts` lines 126-165: `campaign_key` → `campaignKey`, `first_seen_at` → `firstSeenAt`). +**Apply to:** the classify route's JSON response (`campaign_id` → `campaignId`, `requires_approval` → `requiresApproval`, `recommended_actions` → `recommendedActions`). + +### Discriminated-union "evidence completeness" signal +**Source:** `BlastRadiusResult` (`lib/services/mimecast-blast-radius.ts` lines 60-71) — `status: 'ok' | 'unavailable'` shape, explicitly cited in CONTEXT.md as precedent for D-05's confidence-deduction triggers. +**Apply to:** `campaign-classifier.ts`'s internal evidence-completeness flags (`hasAnyMessage`, `blastRadiusStatus`, `hasAttachmentOrUrlIndicators`). + +## No Analog Found + +None. Every file this phase creates has at least one close, directly-applicable analog already in the codebase (all from Phases 15-18 of the same phishing-triage milestone). No RESEARCH.md-only patterns were needed as a fallback. + +## Metadata + +**Analog search scope:** `lib/services/*.ts` (phishing-triage modules: `robotic-classifier.ts`, `mimecast-blast-radius.ts`, `campaign-grouping-service.ts`, `phishing-detector.ts`, `eml-parser.ts`, `phishing-eml-service.ts`), `app/api/phishing/**/route.ts`, `lib/permissions.ts`, `migrations/097_phishing_triage_schema.sql`, existing `*.test.ts` files under `lib/services/` +**Files scanned:** 11 read in full (or targeted section) + 1 migration + 1 permissions file +**Pattern extraction date:** 2026-07-16