docs(19): add pattern map

2026-07-16 08:05:15 -04:00 · 2026-07-16 08:05:15 -04:00 · f2b3602ca1
commit f2b3602ca1
parent 0be109d4cd
1 changed files with 333 additions and 0 deletions
--- a/.planning/phases/19-classification-engine/19-PATTERNS.md
+++ b/.planning/phases/19-classification-engine/19-PATTERNS.md
@ -0,0 +1,333 @@
+# Phase 19: Classification Engine - Pattern Map
+
+**Mapped:** 2026-07-16
+**Files analyzed:** 3 (2 new source files, 1 new test file — no existing files modified)
+**Analogs found:** 3 / 3
+
+## File Classification
+
+| New/Modified File | Role | Data Flow | Closest Analog | Match Quality |
+|--------------------|------|-----------|-----------------|----------------|
+| `lib/services/campaign-classifier.ts` | service | transform (evidence-in / rule-eval / verdict-out, CRUD-adjacent: reads + one INSERT) | `lib/services/robotic-classifier.ts` (rule-eval style) + `lib/services/mimecast-blast-radius.ts` (pure-function module shape, discriminated-union result) + `lib/services/campaign-grouping-service.ts` (evidence-gather-then-orchestrate shape) | exact (composite of 3 role-adjacent analogs — no single file covers both "deterministic rule evaluation" and "gather campaign evidence") |
+| `app/api/phishing/campaigns/[id]/classify/route.ts` | route (controller) | request-response | `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` | exact |
+| `lib/services/campaign-classifier.test.ts` | test | transform (unit, mocked I/O) | `lib/services/campaign-grouping-service.test.ts` + `lib/services/mimecast-blast-radius.test.ts` | exact |
+
+## Pattern Assignments
+
+### `lib/services/campaign-classifier.ts` (service, transform/rule-eval)
+
+**Analogs:** `lib/services/robotic-classifier.ts` (D-01's named architectural analog — rule-eval style only, NOT the DB-driven rule-cache wrapper), `lib/services/mimecast-blast-radius.ts` (pure-function module doc-comment style + discriminated-union result), `lib/services/campaign-grouping-service.ts` (evidence-gather-then-pure-logic module shape, same schema tables)
+
+**File header/doc-comment pattern** (mirrors `mimecast-blast-radius.ts` lines 1-35 and `campaign-grouping-service.ts` lines 1-21) — a substantial doc-comment stating load-bearing invariants up front (ephemerality, known limitations, D-06 rationale) is this codebase's convention for phishing-triage service modules, not just a JSDoc blurb:
+```typescript
+// Source: lib/services/mimecast-blast-radius.ts lines 1-35 (style to follow)
+/**
+ * Mimecast blast-radius lookup abstraction (Phase 17).
+ *
+ * ...
+ * Three load-bearing facts:
+ * (a) EPHEMERAL (D-03): ...
+ * (b) ... IS BEST-EFFORT (D-02): ...
+ * (c) KNOWN LIMITATION — ... (D-05): ...
+ */
+```
+
+**Imports pattern** (mirrors `campaign-grouping-service.ts` lines 22-25 and `mimecast-blast-radius.ts` lines 37-44 — no relative `../` chains, sibling `lib/services/*` imports directly by filename, no path alias needed inside `lib/services/`):
+```typescript
+// Source: lib/services/campaign-grouping-service.ts lines 22-25
+import type { PoolClient } from 'pg';
+import { postgresClient } from './postgres-client';
+// (this phase additionally needs, following the same sibling-import convention:)
+// import { getBlastRadius, type BlastRadiusResult } from './mimecast-blast-radius';
+// import type { NormalizedMessage, AuthResults } from './eml-parser';
+```
+
+**Deterministic rule-evaluation style (D-01)** — `evaluateContains()` precedent: `.toLowerCase()` + string comparison ONLY, no regex/eval, small pure functions each doing exactly one check:
+```typescript
+// Source: lib/services/robotic-classifier.ts lines 206-219 (evaluateContains)
+private evaluateContains(
+  fieldValue: string | number,
+  matchValue: any,
+  caseSensitive: boolean
+): boolean {
+  const text = String(fieldValue);
+  const searchText = caseSensitive ? text : text.toLowerCase();
+  const patterns = Array.isArray(matchValue) ? matchValue : [matchValue];
+
+  return patterns.some((pattern: string) => {
+    const searchPattern = caseSensitive ? String(pattern) : String(pattern).toLowerCase();
+    return searchText.includes(searchPattern);
+  });
+}
+```
+Adapt to a bare exported function (no class, per D-01/Pattern 2 below) — e.g. `isKnownSimulationSender()`, `hasHardAuthFail()`, `evaluateThreatTier()` each following this one-check-per-function, string-only-comparison shape. Domain-suffix matching (not substring) is required for the sender-domain allowlist — see Security Domain note below; do NOT reuse `evaluateContains`'s bare `.includes()` for domain matching.
+
+**Single-exported-orchestrator shape (D-01/D-02, Pattern 2)** — no class, no singleton instance, mirrors `groupReportIntoCampaign()` and `detectPhishingTicket()`:
+```typescript
+// Source: lib/services/campaign-grouping-service.ts lines 160-163 (signature shape to mirror)
+export async function groupReportIntoCampaign(
+  reportId: string,
+  opts?: { skipIfAlreadyGrouped?: boolean }
+): Promise<GroupReportResult | null> {
+```
+This phase's equivalent: `export async function classifyCampaign(campaignId: string): Promise<ClassifyResult>` — single entry point the new route calls directly (no class, no `getInstance()`).
+
+**Evidence-gathering / bulk-fetch pattern** (bulk-fetch-by-id-array shape, reportIds → messagesRes → messageIds → indicatorsRes) — copy verbatim shape from `app/api/phishing/campaigns/[id]/route.ts` lines 86-116 (already used in RESEARCH.md's Code Examples section) and `campaign-grouping-service.ts`'s `ORDER BY r.created_at ASC` "earliest report is canonical" convention (lines 176-181, 217-218):
+```typescript
+// Source: app/api/phishing/campaigns/[id]/route.ts lines 96-116 (bulk-fetch-by-id-array shape)
+const reportIds = reportsRes.rows.map((r) => r.id);
+const messagesRes = reportIds.length
+  ? await postgresClient.query<MessageRow>(
+      `SELECT id::text, report_id::text, message_id, headers->>'subject' AS subject
+         FROM messages WHERE report_id = ANY($1::uuid[])`,
+      [reportIds]
+    )
+  : { rows: [] as MessageRow[] };
+const messageIds = messagesRes.rows.map((m) => m.id);
+const indicatorsRes = messageIds.length
+  ? await postgresClient.query<IndicatorRow>(
+      `SELECT id::text, message_id::text, indicator_type, value, metadata
+         FROM indicators WHERE message_id = ANY($1::uuid[])`,
+      [messageIds]
+    )
+  : { rows: [] as IndicatorRow[] };
+```
+
+**Discriminated-union result shape** (mirrors `BlastRadiusResult`, the precedent CONTEXT.md explicitly names for shaping this phase's own evidence-completeness signal):
+```typescript
+// Source: lib/services/mimecast-blast-radius.ts lines 60-71
+export type BlastRadiusResult =
+  | { status: 'unavailable'; reason: 'not_configured' | 'lookup_failed'; error?: string }
+  | {
+      status: 'ok';
+      matched: number;
+      delivered: number;
+      held: number;
+      rejected: number;
+      clicked: number;
+      perRecipient: Array<{ recipient: string; status: 'delivered' | 'held' | 'rejected' | 'unknown' }>;
+      source: 'fan-out';
+    };
+```
+
+**Error handling pattern** (mirrors `getBlastRadius()`'s try/catch — log message only, never full payloads; and `groupReportIntoCampaign()`'s catch-log-rethrow at the orchestrator boundary):
+```typescript
+// Source: lib/services/mimecast-blast-radius.ts lines 188-194
+} catch (err) {
+  // T-17-02: log err.message ONLY, never full Mimecast response bodies
+  const message = err instanceof Error ? err.message : String(err);
+  console.error('[MIMECAST-BLAST-RADIUS] lookup failed', message);
+  return { status: 'unavailable', reason: 'lookup_failed', error: message };
+}
+```
+```typescript
+// Source: lib/services/campaign-grouping-service.ts lines 475-478
+} catch (error) {
+  console.error('[CAMPAIGN-GROUPING] Failed to group report into campaign', reportId, error);
+  throw error;
+}
+```
+Use a `[CAMPAIGN-CLASSIFIER]` log prefix, matching the `[MIMECAST-BLAST-RADIUS]`/`[CAMPAIGN-GROUPING]`/`[PHISHING-DETECT]` bracketed-module-name convention.
+
+**Append-only INSERT pattern** (`classifications` row, migration 097 exact columns — confirmed exact column list/types from `migrations/097_phishing_triage_schema.sql` lines 114-124: `verdict TEXT`, `confidence NUMERIC`, `summary TEXT`, `reasons JSONB`, `recommended_actions JSONB`, `requires_approval BOOLEAN NOT NULL DEFAULT false`):
+```typescript
+// Adapt from lib/services/phishing-detector.ts lines 210-229's INSERT-with-RETURNING shape
+await postgresClient.query(
+  `INSERT INTO classifications (
+     campaign_id, verdict, confidence, summary, reasons, recommended_actions, requires_approval
+   ) VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb, $7)
+   RETURNING id::text AS id, created_at::text AS created_at`,
+  [campaignId, verdict, confidence, summary, JSON.stringify(reasons), JSON.stringify(recommendedActions), requiresApproval]
+);
+```
+Note: no `ON CONFLICT` — this is append-only (D-02), unlike `phishing-detector.ts`'s `ON CONFLICT (ticket_id) DO UPDATE` upsert, which does NOT apply here.
+
+**Auth-verdict precedence (Pitfall 1, load-bearing)** — reads `NormalizedMessage.authResultsOriginal` (from `eml-parser.ts` lines 104, 269-271) FIRST, falling back to `authResults`:
+```typescript
+function effectiveAuthResults(message: { headers: { authResults: AuthResults; authResultsOriginal: AuthResults | null } }): AuthResults {
+  return message.headers.authResultsOriginal ?? message.headers.authResults;
+}
+function hasHardAuthFail(effective: AuthResults): boolean {
+  return effective.spf === 'fail' || effective.dkim === 'fail' || effective.dmarc === 'fail';
+}
+```
+
+**Named-pattern-constant precedent (D-06/D-07 allowlist shape)** — mirrors `KNOWN_PHISHING_PATTERNS` (`phishing-detector.ts` lines 26-35): a locked, exported, commented `readonly` constant, NOT a DB table:
+```typescript
+// Source: lib/services/phishing-detector.ts lines 26-35 (shape to mirror for KNOWN_SIMULATION_SENDERS)
+export const KNOWN_PHISHING_PATTERNS: readonly string[] = [
+  'Phishing Report',
+  'Spam Alert',
+  'Phishing Alert - Email Security Report',
+  'KnowBe4 Phish Alert Report',
+  'Source: KnowBe4 Phish Alert Button',
+  'userSubmissionsReportMessage',
+  'reported message destinations',
+  'Microsoft directly',
+];
+```
+Do NOT reuse or extend this constant — it is the REPORT-pattern list (proves how a ticket was reported), a different signal from the NEW sender-domain allowlist this phase adds (proves who sent the original message). See RESEARCH.md's exact `KNOWN_SIMULATION_SENDERS` proposal (vendor-keyed array of domain arrays) — same "locked TS constant with a doc-comment citing ticket evidence" shape.
+
+---
+
+### `app/api/phishing/campaigns/[id]/classify/route.ts` (route/controller, request-response)
+
+**Analog:** `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` (full file, 91 lines — read in full, reproduced below in relevant part)
+
+**Imports pattern** (lines 11-16 — path-alias `@/lib/...` imports, unlike `lib/services/*` sibling imports):
+```typescript
+// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 11-16
+import { NextRequest, NextResponse } from 'next/server';
+import { requirePermission } from '@/lib/auth-utils';
+import postgresClient from '@/lib/services/postgres-client';
+import { detectPhishingTicket, type DetectableTicket } from '@/lib/services/phishing-detector';
+import { parseAndStoreMessage } from '@/lib/services/phishing-eml-service';
+import { groupReportIntoCampaign } from '@/lib/services/campaign-grouping-service';
+```
+This phase's route imports `classifyCampaign` from `@/lib/services/campaign-classifier` instead.
+
+**Auth pattern (exact, reuse verbatim — Phase 18 D-06 convention, same `'analyze'` action, NOT a new permission)** (lines 18-23):
+```typescript
+// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 18-23
+export async function POST(
+  request: NextRequest,
+  { params }: { params: Promise<{ ticket_id: string }> }
+) {
+  const { error } = await requirePermission('phishing', 'analyze');
+  if (error) return error;
+```
+For `.../campaigns/[id]/classify/route.ts`, `params` is `Promise<{ id: string }>` instead of `{ ticket_id: string }` (matches `campaigns/[id]/route.ts`'s param name).
+
+**UUID validation guard (V5, copy verbatim from the campaign detail route since it's a campaign id, not a ticket_id)**:
+```typescript
+// Source: app/api/phishing/campaigns/[id]/route.ts lines 11, 66-70
+const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
+// ...
+const { id } = await params;
+if (!UUID_RE.test(id)) {
+  return NextResponse.json({ error: 'Invalid campaign id' }, { status: 400 });
+}
+```
+Note: `analyze/route.ts` validates a numeric `ticket_id` (`Number.isFinite`), which does NOT apply here — this route's id is a campaign UUID, so follow `campaigns/[id]/route.ts`'s `UUID_RE` guard instead, not `analyze/route.ts`'s numeric check.
+
+**Core request-response + 404 pattern** (lines 44-48, adapted — campaign-not-found mirrors ticket-not-found):
+```typescript
+// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 44-48 (shape)
+const r = row.rows[0];
+if (!r) {
+  return NextResponse.json({ error: 'Ticket not found' }, { status: 404 });
+}
+```
+Adapt: `if (!campaign) return NextResponse.json({ error: 'Campaign not found' }, { status: 404 });` — same shape as `campaigns/[id]/route.ts` lines 79-82.
+
+**Success response shape** (lines 77-82 — flat camelCase JSON, no wrapper envelope):
+```typescript
+// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 77-82
+return NextResponse.json({
+  reportId: detection.reportId,
+  campaignId: grouped?.campaignId ?? null,
+  groupMethod: grouped?.groupMethod ?? null,
+  created: grouped?.created ?? false,
+});
+```
+This phase's equivalent: `{ id, campaignId, verdict, confidence, summary, reasons, recommendedActions, requiresApproval }` (per RESEARCH.md's response shape).
+
+**Error handling pattern** (lines 83-89 — try/catch wraps the whole handler body, 500 with `message`):
+```typescript
+// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts lines 83-89
+} catch (err) {
+  console.error('[PHISHING-ANALYZE] Failed to analyze ticket', ticketId, err);
+  return NextResponse.json(
+    { error: 'Failed to analyze ticket', message: err instanceof Error ? err.message : 'Unknown error' },
+    { status: 500 }
+  );
+}
+```
+Use a `[PHISHING-CLASSIFY]` log prefix for this route, matching `[PHISHING-ANALYZE]`/`[PHISHING-CAMPAIGN-DETAIL]`'s per-route bracketed-name convention.
+
+---
+
+### `lib/services/campaign-classifier.test.ts` (test, transform)
+
+**Analogs:** `lib/services/campaign-grouping-service.test.ts` (mocking `postgresClient` via `vi.mock` factory, hoisting discipline) + `lib/services/mimecast-blast-radius.test.ts` (mocking a sibling service module entirely)
+
+**Mock-before-import pattern** (lines 1-14 — `vi.mock` factory MUST be declared before the module-under-test import; `// eslint-disable-next-line import/first` comment documents why):
+```typescript
+// Source: lib/services/campaign-grouping-service.test.ts lines 1-14
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+
+// Mock postgresClient BEFORE importing the module under test.
+const queryMock = vi.fn();
+const transactionMock = vi.fn();
+vi.mock('./postgres-client', () => ({
+  postgresClient: {
+    query: (...args: unknown[]) => queryMock(...args),
+    transaction: (...args: unknown[]) => transactionMock(...args),
+  },
+}));
+
+// eslint-disable-next-line import/first -- imported after vi.mock hoisting
+import { normalizeSubject, extractUrlDomain, groupReportIntoCampaign } from './campaign-grouping-service';
+```
+This phase's test additionally needs `vi.mock('./mimecast-blast-radius', ...)` (per RESEARCH.md's Wave 0 Gaps section) — follow `mimecast-blast-radius.test.ts`'s own `vi.mock('./mimecast-client', ...)` / `vi.mock('./redis-client', ...)` sibling-module-mocking shape (see lines 25-32 of that file) for how to mock `getBlastRadius` itself as a dependency of `campaign-classifier.ts`.
+
+**Per-call routed mock rows (matches SQL-substring discipline)** — when a module makes multiple sequential/branching queries (as `campaign-classifier.ts`'s evidence-gathering will), route mock responses by a distinguishing SQL substring per call rather than by call order:
+```typescript
+// Source: lib/services/campaign-grouping-service.test.ts lines 81-90 (pattern)
+function makeClient(rows: MockRows) {
+  return {
+    query: vi.fn(async (sql: string, params?: unknown[]) => {
+      clientCalls.push({ sql, params: params ?? [] });
+      if (sql.includes('requester_contact_id, company_id, created_at')) {
+        return { rows: rows.ownReport ?? [], rowCount: rows.ownReport?.length ?? 0 };
+      }
+      if (sql.includes('FROM messages') && sql.includes('WHERE report_id = $1')) {
+        return { rows: rows.ownMessage ?? [], rowCount: rows.ownMessage?.length ?? 0 };
+      }
+      // ... one branch per distinct query
+    }),
+  };
+}
+```
+
+**Pure-function unit tests** (mirrors `describe('normalizeSubject', ...)` / `describe('extractUrlDomain', ...)` blocks, lines 16-50) — this phase's pure rule functions (`hasHardAuthFail`, `isKnownSimulationSender`, `computeConfidence`, `mapVerdictToActions`, `computeRequiresApproval`) should each get their own `describe` block with direct input/output assertions, no mocking needed for these (only `classifyCampaign()` itself needs the DB/Mimecast mocks).
+
+## Shared Patterns
+
+### Auth/Permission (all `/api/phishing/*` routes)
+**Source:** `lib/auth-utils.ts` `requirePermission(resource, action)`, established Phase 18 D-06, reused verbatim from `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` lines 22-23.
+**Apply to:** `app/api/phishing/campaigns/[id]/classify/route.ts` — same `'analyze'` action, not a new permission (confirmed in `lib/permissions.ts` line 34: `phishing: ["read", "analyze", "approve", "remediate"]`).
+```typescript
+const { error } = await requirePermission('phishing', 'analyze');
+if (error) return error;
+```
+
+### Error Handling (route + service boundary)
+**Source:** every `app/api/phishing/**/route.ts` file — try/catch around the full handler body, `NextResponse.json({ error, message }, { status: 500 })` on unexpected failure; every `lib/services/*.ts` orchestrator function — catch-log-with-context-then-rethrow (service layer never swallows into a 200; the route layer converts to the JSON error envelope).
+**Apply to:** both new files.
+
+### Bracketed module-name logging
+**Source:** `[MIMECAST-BLAST-RADIUS]`, `[CAMPAIGN-GROUPING]`, `[PHISHING-DETECT]`, `[PHISHING-ANALYZE]`, `[PHISHING-CAMPAIGN-DETAIL]` — every phishing-triage module/route prefixes `console.error`/`console.debug` calls with its own bracketed name.
+**Apply to:** use `[CAMPAIGN-CLASSIFIER]` in the service, `[PHISHING-CLASSIFY]` in the route.
+
+### UUID path-param validation (V5)
+**Source:** `app/api/phishing/campaigns/[id]/route.ts` lines 11, 66-70 — `UUID_RE` regex guard before any query, returns 400 on malformed input rather than letting an unhandled Postgres error surface as a 500.
+**Apply to:** `app/api/phishing/campaigns/[id]/classify/route.ts` (this route's `id` is a campaign UUID, not a ticket_id — do NOT copy `analyze/route.ts`'s numeric `Number.isFinite(ticketId)` check, which validates a different id type).
+
+### camelCase API response transform
+**Source:** every `app/api/phishing/**/route.ts` GET/POST handler — snake_case DB columns manually mapped to camelCase JSON keys at the response boundary (e.g. `campaigns/[id]/route.ts` lines 126-165: `campaign_key` → `campaignKey`, `first_seen_at` → `firstSeenAt`).
+**Apply to:** the classify route's JSON response (`campaign_id` → `campaignId`, `requires_approval` → `requiresApproval`, `recommended_actions` → `recommendedActions`).
+
+### Discriminated-union "evidence completeness" signal
+**Source:** `BlastRadiusResult` (`lib/services/mimecast-blast-radius.ts` lines 60-71) — `status: 'ok' | 'unavailable'` shape, explicitly cited in CONTEXT.md as precedent for D-05's confidence-deduction triggers.
+**Apply to:** `campaign-classifier.ts`'s internal evidence-completeness flags (`hasAnyMessage`, `blastRadiusStatus`, `hasAttachmentOrUrlIndicators`).
+
+## No Analog Found
+
+None. Every file this phase creates has at least one close, directly-applicable analog already in the codebase (all from Phases 15-18 of the same phishing-triage milestone). No RESEARCH.md-only patterns were needed as a fallback.
+
+## Metadata
+
+**Analog search scope:** `lib/services/*.ts` (phishing-triage modules: `robotic-classifier.ts`, `mimecast-blast-radius.ts`, `campaign-grouping-service.ts`, `phishing-detector.ts`, `eml-parser.ts`, `phishing-eml-service.ts`), `app/api/phishing/**/route.ts`, `lib/permissions.ts`, `migrations/097_phishing_triage_schema.sql`, existing `*.test.ts` files under `lib/services/`
+**Files scanned:** 11 read in full (or targeted section) + 1 migration + 1 permissions file
+**Pattern extraction date:** 2026-07-16