docs(18): create phase plan (3 plans, 2 waves)

This commit is contained in:
lorentz 2026-07-15 16:45:29 -04:00
parent ffdd86a0ed
commit 808604e861
4 changed files with 577 additions and 1 deletions

View file

@ -418,7 +418,10 @@ summarizes classification, blast radius, and recommended/approved remediation st
3. `POST /api/phishing/tickets/{ticket_id}/analyze` runs detection + evidence extraction + campaign grouping for one specific ticket on demand and returns the resulting campaign linkage, instead of waiting for the next scheduled scan
4. `GET /api/phishing/campaigns` lists campaigns and `GET /api/phishing/campaigns/{id}` returns full detail (linked reports, messages, indicators, classification history)
5. Every `/api/phishing/*` route introduced in this phase calls `requireAuth()` (or `requirePermission()`) and rejects an unauthenticated/unauthorized request with 401/403 — establishing the auth convention every later phishing endpoint (Phases 19-21) must also follow
**Plans**: TBD
**Plans**: 3 plans (2 waves)
- [ ] 18-01-PLAN.md — Campaign grouping service (tiered match + transactional find-or-create) + tests + phishing permission resource (CAMP-01, CAMP-02, ACCESS-01)
- [ ] 18-02-PLAN.md — POST /api/phishing/tickets/{id}/analyze + wire groupReportIntoCampaign into webhook + cron sweep automatic paths (DETECT-03, CAMP-01, CAMP-02, ACCESS-01)
- [ ] 18-03-PLAN.md — GET /api/phishing/campaigns list + GET /api/phishing/campaigns/{id} nested detail (CAMP-03, ACCESS-01)
**UI hint**: no
### Phase 19: Classification Engine

View file

@ -0,0 +1,221 @@
---
phase: 18-campaign-grouping-phishing-analysis-api
plan: 01
type: tdd
wave: 1
depends_on: []
files_modified:
- lib/services/campaign-grouping-service.ts
- lib/services/campaign-grouping-service.test.ts
- lib/permissions.ts
autonomous: true
requirements: [CAMP-01, CAMP-02, ACCESS-01]
must_haves:
truths:
- "normalizeSubject strips repeated Re:/Fwd:/Fw: prefixes case-insensitively, lowercases, and trims"
- "extractUrlDomain returns the hostname for a valid URL and null (never throws) for a malformed URL"
- "A second report whose tiered key matches an existing campaign increments that campaign's report_count and updates last_seen_at, links reports.campaign_id, and never creates a second campaign"
- "groupReportIntoCampaign called with skipIfAlreadyGrouped short-circuits (returns null) when reports.campaign_id is already set"
- "The phishing permission resource exists in lib/permissions.ts statement with actions read/analyze/approve/remediate; admin+super-admin roles grant read+analyze, user role grants read only"
artifacts:
- path: "lib/services/campaign-grouping-service.ts"
provides: "groupReportIntoCampaign(reportId, opts), normalizeSubject, extractUrlDomain"
exports: ["groupReportIntoCampaign", "normalizeSubject", "extractUrlDomain", "GroupReportResult"]
- path: "lib/services/campaign-grouping-service.test.ts"
provides: "Unit tests for pure helpers + mocked-DB grouping behavior (CAMP-01, CAMP-02)"
contains: "vi.mock('./postgres-client'"
- path: "lib/permissions.ts"
provides: "phishing resource + role grants (D-05)"
contains: "phishing:"
key_links:
- from: "lib/services/campaign-grouping-service.ts"
to: "postgresClient.transaction"
via: "find-or-create wrapped in a single transaction (Pitfall 2 — no UNIQUE on campaign_key)"
pattern: "postgresClient\\.transaction"
- from: "lib/permissions.ts statement"
to: "phishing resource key"
via: "keyof typeof statement picks up phishing for requirePermission type-checking"
pattern: "phishing:\\s*\\["
---
<objective>
Build the shared campaign-grouping core that every trigger site (webhook, cron sweep, on-demand /analyze) will call, plus the `phishing` permission vocabulary the API routes gate on. This is the foundation plan — no dependents exist yet, and both Wave 2 plans depend on the artifacts here.
Purpose: `groupReportIntoCampaign(reportId)` is the single source of truth for tiered campaign matching (CAMP-01) and campaign accumulation (CAMP-02). The `phishing` resource in `lib/permissions.ts` (D-05) establishes the full action vocabulary once so Phase 20 only adds role grants (ACCESS-01 foundation).
Output: `lib/services/campaign-grouping-service.ts`, its test file, and the modified `lib/permissions.ts`.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-CONTEXT.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-RESEARCH.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md
<interfaces>
<!-- Contracts the executor needs. Extracted from codebase this session. -->
From lib/services/phishing-detector.ts (existing — DO NOT modify):
export interface DetectableTicket { id: number; ticket_number: string|null; title: string|null; description: string|null; company_id: number|null; contact_id: number|null; created_by_contact_id: number|null; }
export interface DetectPhishingResult { flagged: boolean; reportId?: string; skippedUnchanged?: boolean; ... }
export async function detectPhishingTicket(ticket: DetectableTicket): Promise<DetectPhishingResult>
// Style precedent: computePhishingContentHash / matchesPhishingPatterns are small pure exported helpers alongside one DB-touching orchestrator; console.error + rethrow error handling.
From lib/services/postgres-client.ts (existing):
import { postgresClient } from './postgres-client'; // named+relative import — matches sibling lib/services files
postgresClient.query<T>(sql, params): Promise<{ rows: T[] }>
postgresClient.transaction(async (client) => { ... }) // BEGIN/COMMIT/ROLLBACK wrapper; use client.query inside, NOT postgresClient.query
Schema (migrations/097_phishing_triage_schema.sql, 099_indicators_metadata.sql):
campaigns(id UUID PK, campaign_key TEXT, group_method TEXT, first_seen_at TIMESTAMPTZ, last_seen_at TIMESTAMPTZ, report_count INTEGER DEFAULT 0, status TEXT DEFAULT 'open', created_at, updated_at) -- NO UNIQUE on campaign_key (only a plain index)
reports(id UUID PK, ticket_id BIGINT, ticket_number, company_id, company_name, requester_contact_id BIGINT, title, description, campaign_id UUID nullable FK, ...) -- sender join column is requester_contact_id (NOT contact_id — Pitfall 5)
messages(id UUID PK, report_id UUID FK, message_id TEXT nullable, headers JSONB, urls JSONB, attachments JSONB, body_preview, ...) -- messages.message_id = email Message-ID header (Tier 1 key)
indicators(id UUID PK, message_id UUID FK->messages(id), indicator_type TEXT ('attachment_hash'|'url'|'sender'), value TEXT, metadata JSONB, ...) -- value for 'url' is a RAW URL string, not a domain (Pitfall 4)
contacts(id, email_address VARCHAR, first_name, last_name) -- from migrations/001_initial_schema.sql
From lib/permissions.ts (existing structure — closest analog is the rmm resource):
export const statement = { ... rmm: ["read", "execute"] } as const; // line ~30
export const superAdminRole = ac.newRole({ ... rmm: ["read","execute"] }); // line ~46
export const adminRole = ac.newRole({ ... rmm: ["read","execute"] }); // line ~59
export const userRole = ac.newRole({ ... rmm: ["read"] }); // line ~72
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Pure tier-key helpers (normalizeSubject, extractUrlDomain) + test file scaffold</name>
<files>lib/services/campaign-grouping-service.ts, lib/services/campaign-grouping-service.test.ts</files>
<read_first>
- lib/services/phishing-detector.ts (pure-helper style: computePhishingContentHash, matchesPhishingPatterns — replicate this file shape and error handling)
- lib/services/phishing-detector.test.ts (no-mock pure-function test style: plain describe/it/expect against exported helpers)
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md sections 1 and 2 (exact helper source + test scaffold to replicate)
</read_first>
<behavior>
- normalizeSubject('Re: Your Invoice ') === 'your invoice'
- normalizeSubject('FW: Re: fwd: Urgent Payment') === 'urgent payment' (repeated, mixed-case prefixes stripped)
- normalizeSubject(null) === ''
- extractUrlDomain('https://evil.example.com/path?x=1') === 'evil.example.com'
- extractUrlDomain('not-a-url') === null (returns null, does not throw)
</behavior>
<action>
Create `lib/services/campaign-grouping-service.ts` importing `{ postgresClient } from './postgres-client'` (named+relative, matching sibling lib/services files). Export `normalizeSubject(subject: string | null): string` — strip leading `Re:`/`Fwd:`/`Fw:` prefixes using a case-insensitive regex `/^(re|fwd|fw):\s*/i` applied repeatedly in a while loop, then lowercase and trim (D-03). Export `extractUrlDomain(url: string): string | null` — return `new URL(url).hostname || null` inside a try/catch that returns null on malformed input (Pitfall 4). Create `lib/services/campaign-grouping-service.test.ts` mocking `./postgres-client` via `vi.mock` BEFORE importing the module under test (queryMock + transactionMock spies per PATTERNS.md section 2), and add the pure-helper describe blocks covering the behavior cases above. Follow phishing-detector.ts's file-header doc-comment convention.
</action>
<verify>
<automated>npx vitest run lib/services/campaign-grouping-service.test.ts</automated>
</verify>
<done>Both files exist; normalizeSubject and extractUrlDomain are exported; the pure-helper tests pass; the test file mocks ./postgres-client before import.</done>
<acceptance_criteria>
- campaign-grouping-service.ts contains `export function normalizeSubject(` and `export function extractUrlDomain(`
- campaign-grouping-service.test.ts contains `vi.mock('./postgres-client'`
- `npx vitest run lib/services/campaign-grouping-service.test.ts` exits 0 with the normalizeSubject and extractUrlDomain describe blocks green
- normalizeSubject('FW: Re: fwd: Urgent Payment') returns 'urgent payment'
- extractUrlDomain('not-a-url') returns null and does not throw
</acceptance_criteria>
</task>
<task type="auto" tdd="true">
<name>Task 2: groupReportIntoCampaign — tiered matching + transactional find-or-create</name>
<files>lib/services/campaign-grouping-service.ts, lib/services/campaign-grouping-service.test.ts</files>
<read_first>
- lib/services/campaign-grouping-service.ts (current state from Task 1)
- lib/services/campaign-grouping-service.test.ts (current state from Task 1)
- lib/services/phishing-eml-service.ts (its file-header doc-comment block at lines 1-22 — replicate this "load-bearing limitation stated above the export" style for the D-07 comment; also its vi.mock test approach)
- lib/services/postgres-client.ts (transaction() method, lines ~96-111)
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-RESEARCH.md Pattern 2 (tier query shapes) + Common Pitfalls 1-5
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md section 1 (GroupReportResult interface, function signature, schema facts)
</read_first>
<behavior>
- Tier 1 (Message-ID): when the report has a linked messages row with a non-null message_id that matches another already-grouped report's message, groupReportIntoCampaign links to that existing campaign with groupMethod 'message_id' and does NOT create a new campaign
- Tier 2 (attachment_hash/url-domain + normalized subject + sender within 24h): reached only when Tier 1 finds nothing; url-domain is derived from indicators.value via extractUrlDomain at read time (not queried as an exact domain)
- Tier 3 (sender + normalizeSubject + client + 24h window): reached when Tiers 1-2 find nothing; joins reports.requester_contact_id -> contacts (Pitfall 5)
- No-match: inserts a new campaigns row (campaign_key, group_method, first_seen_at=NOW, last_seen_at=NOW, report_count=1) and sets reports.campaign_id, returning created:true
- Match: UPDATE campaigns SET report_count = report_count + 1, last_seen_at = NOW() and UPDATE reports SET campaign_id, returning created:false — never a second campaign row (CAMP-02)
- skipIfAlreadyGrouped:true returns null immediately when reports.campaign_id IS NOT NULL (D-08); omitting the option always re-runs full tiered matching
</behavior>
<action>
Extend `campaign-grouping-service.ts` with `export interface GroupReportResult { campaignId: string; groupMethod: 'message_id' | 'attachment_or_url' | 'sender_subject_client'; created: boolean; }` and `export async function groupReportIntoCampaign(reportId: string, opts?: { skipIfAlreadyGrouped?: boolean }): Promise<GroupReportResult | null>`. When `opts.skipIfAlreadyGrouped` is set, run a pre-check `SELECT campaign_id FROM reports WHERE id = $1` and return null if already grouped (D-08). Otherwise run the whole find-or-create inside `postgresClient.transaction(async (client) => { ... })` using `client.query` (never `postgresClient.query`) so all reads and writes share one transaction (Pitfall 2 — campaign_key has no UNIQUE constraint). Implement the three tiers in order, each gated on the previous returning nothing: Tier 1 keys on `messages.message_id` (join messages->reports where campaign_id IS NOT NULL); Tier 2 keys on `indicators` rows (indicator_type 'attachment_hash' matched by value, 'url' matched by extractUrlDomain(value) at read time) plus normalizeSubject and sender within a 24h window (D-02); Tier 3 keys on sender (reports.requester_contact_id -> contacts) + normalizeSubject(title) + company_id within 24h (D-02). Compute keys in JS then run targeted parameterized queries (never encode fuzzy tier logic in one WHERE clause — RESEARCH anti-pattern). On match, UPDATE campaigns (report_count+1, last_seen_at=NOW) + UPDATE reports.campaign_id + updated_at=NOW. On no match, INSERT a campaigns row with the computed campaign_key/group_method + link the report. Do NOT set/transition campaigns.status (leave the migration default 'open' — status transitions are Phase 19/20). Do NOT implement multi-campaign merge (D-04 — attach to first/best match). Add a file-level doc comment stating the D-07 limitation explicitly: the automatic webhook/cron path only ever reaches Tier 3 until a report has been through an explicit `/analyze` call, because parseAndStoreMessage (the only writer of messages/indicators) is not wired into the automatic path this phase. Wrap the body in try/catch with `console.error('[CAMPAIGN-GROUPING] ...', reportId, error)` + rethrow. Add mocked-DB tests to the test file covering the behavior cases above (transactionMock invokes its callback with a fake client whose query returns staged rows).
</action>
<verify>
<automated>npx vitest run lib/services/campaign-grouping-service.test.ts && npx tsc --noEmit --pretty</automated>
</verify>
<done>groupReportIntoCampaign + GroupReportResult exported; tiered matching, transactional find-or-create, skipIfAlreadyGrouped short-circuit, and D-07 doc comment all present; mocked-DB tests for CAMP-01/CAMP-02 pass; tsc clean.</done>
<acceptance_criteria>
- campaign-grouping-service.ts contains `export async function groupReportIntoCampaign(` and `export interface GroupReportResult`
- The function body contains `postgresClient.transaction(` and references `report_count + 1` and `last_seen_at`
- A file-level comment names the D-07 Tier-3-only automatic-path limitation (grep for `Tier 3` or `parseAndStoreMessage` in a comment near the top)
- Test asserts: a matching second report increments report_count and does NOT insert a second campaign (transactionMock/queryMock call assertions)
- Test asserts: skipIfAlreadyGrouped:true returns null when the pre-check SELECT reports a non-null campaign_id
- `npx vitest run lib/services/campaign-grouping-service.test.ts` exits 0; `npx tsc --noEmit --pretty` exits 0
</acceptance_criteria>
</task>
<task type="auto">
<name>Task 3: Add phishing permission resource + role grants to lib/permissions.ts</name>
<files>lib/permissions.ts</files>
<read_first>
- lib/permissions.ts (full file — locate the existing rmm entries in statement, superAdminRole, adminRole, userRole; the phishing entries mirror rmm exactly in shape)
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md section 6 (exact diff shape + Assumption A1 resolution)
</read_first>
<action>
Add `phishing: ["read", "analyze", "approve", "remediate"]` to the `statement` object (full vocabulary now, mirroring the `rmm` line — D-05). Add `phishing: ["read", "analyze"]` to both `superAdminRole` and `adminRole` `ac.newRole({...})` calls. Add `phishing: ["read"]` to `userRole`. Do NOT grant `approve`/`remediate` to any role — those stay declared-only until Phase 20 (D-05, deferred). No other file needs editing: `lib/auth-utils.ts` imports `hasPermission`/`Permission`/`statement` from `./permissions`, so `keyof typeof statement` picks up `phishing` automatically for the routes' `requirePermission('phishing', ...)` type-checking (Assumption A1 confirmed in PATTERNS.md).
</action>
<verify>
<automated>npx tsc --noEmit --pretty && grep -c 'phishing:' lib/permissions.ts</automated>
</verify>
<done>phishing resource declared in statement with all four actions; read+analyze granted to admin+super-admin; read granted to user; approve/remediate ungranted; tsc clean.</done>
<acceptance_criteria>
- `grep -c 'phishing:' lib/permissions.ts` returns 4 (statement + 3 role definitions)
- statement line reads `phishing: ["read", "analyze", "approve", "remediate"]`
- superAdminRole and adminRole each contain `phishing: ["read", "analyze"]`
- userRole contains `phishing: ["read"]`
- No role grants `approve` or `remediate` for phishing (grep confirms neither appears in any newRole phishing entry)
- `npx tsc --noEmit --pretty` exits 0
</acceptance_criteria>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| API → Postgres | groupReportIntoCampaign builds SQL from report/indicator/contact data; all values must be parameterized |
| indicator value → grouping logic | indicators.value (attacker-controlled URL strings from parsed phishing emails) is read as a string for domain extraction — must never be fetched |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-18-04 | Tampering/SSRF | extractUrlDomain / Tier 2 matching | mitigate | indicators.value is only ever parsed with `new URL(value).hostname` for a string key — no network fetch is issued during grouping; guarded in try/catch (EVID-04 never-fetch invariant carried forward) |
| T-18-06 | Tampering | Tier query construction | mitigate | All tier queries use `$n` parameterized placeholders inside postgresClient.transaction — no user/indicator value is string-interpolated into SQL text |
| T-18-01 | Elevation of Privilege | phishing permission resource (D-05) | mitigate | approve/remediate declared in statement but ungranted to every role this phase; only read+analyze (admin/super-admin) and read (user) are grantable, so no route can be authorized for approve/remediate until Phase 20 |
No package installs this phase (zero new dependencies per RESEARCH.md) — the supply-chain (T-{phase}-SC) row is not applicable.
</threat_model>
<verification>
- `npx vitest run lib/services/campaign-grouping-service.test.ts` green (CAMP-01, CAMP-02 behaviors)
- `npx tsc --noEmit --pretty` clean
- `grep -c 'phishing:' lib/permissions.ts` returns 4
- Manual read confirms the D-07 Tier-3-only limitation is documented in a comment in campaign-grouping-service.ts
</verification>
<success_criteria>
- groupReportIntoCampaign implements Message-ID → attachment-hash/URL-domain+subject+sender+24h → sender+normalized-subject+client+24h tiered matching (CAMP-01)
- Matching reports increment report_count / update last_seen_at without creating a second campaign (CAMP-02)
- phishing resource + read/analyze/user-read grants exist; approve/remediate ungranted (ACCESS-01 foundation, D-05)
- All automated tests + tsc pass
</success_criteria>
<output>
Create `.planning/phases/18-campaign-grouping-phishing-analysis-api/18-01-SUMMARY.md` when done
</output>

View file

@ -0,0 +1,177 @@
---
phase: 18-campaign-grouping-phishing-analysis-api
plan: 02
type: execute
wave: 2
depends_on: [18-01]
files_modified:
- app/api/phishing/tickets/[ticket_id]/analyze/route.ts
- lib/services/webhook-service.ts
- lib/services/phishing-sweep-service.ts
autonomous: true
requirements: [DETECT-03, CAMP-01, CAMP-02, ACCESS-01]
must_haves:
truths:
- "POST /api/phishing/tickets/{ticket_id}/analyze runs detect -> parse -> group for one ticket and returns { reportId, campaignId, groupMethod, created }"
- "The analyze route rejects a request lacking the phishing:analyze permission with 401/403 (user-role gets read only per D-05)"
- "The analyze route validates ticket_id is numeric and returns 400 for a non-numeric value; 404 when the ticket row is absent; 400 when the ticket does not match phishing patterns"
- "The webhook ticket.created path calls groupReportIntoCampaign(reportId, {skipIfAlreadyGrouped:true}) after detectPhishingTicket flags a report"
- "The cron sweep loop calls groupReportIntoCampaign(reportId, {skipIfAlreadyGrouped:true}) per flagged ticket, inside the same per-ticket try/catch"
artifacts:
- path: "app/api/phishing/tickets/[ticket_id]/analyze/route.ts"
provides: "POST on-demand analyze endpoint (DETECT-03)"
exports: ["POST"]
- path: "lib/services/webhook-service.ts"
provides: "grouping call wired into triggerPhishingDetection (D-01)"
contains: "groupReportIntoCampaign"
- path: "lib/services/phishing-sweep-service.ts"
provides: "grouping call wired into the per-ticket sweep loop (D-01)"
contains: "groupReportIntoCampaign"
key_links:
- from: "app/api/phishing/tickets/[ticket_id]/analyze/route.ts"
to: "detectPhishingTicket -> parseAndStoreMessage -> groupReportIntoCampaign"
via: "orchestration chain (D-08: no skipIfAlreadyGrouped — always re-run)"
pattern: "groupReportIntoCampaign\\(detection\\.reportId\\)"
- from: "app/api/phishing/tickets/[ticket_id]/analyze/route.ts"
to: "requirePermission('phishing', 'analyze')"
via: "call-and-early-return auth gate (D-06)"
pattern: "requirePermission\\('phishing', ?'analyze'\\)"
---
<objective>
Expose the on-demand analysis endpoint and wire the shared grouping function into the two existing automatic trigger points, so campaigns form/accumulate automatically (D-01) and an operator can force full analysis of one ticket on demand (DETECT-03).
Purpose: `POST /api/phishing/tickets/{ticket_id}/analyze` is the first `/api/phishing/*` write route and the intended first caller of Phase 16's `parseAndStoreMessage`. The webhook + cron wiring make grouping happen without an API call (D-01), using `skipIfAlreadyGrouped` to stay cheap (D-08).
Output: one new route file + two modified service files.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-CONTEXT.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-RESEARCH.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-01-SUMMARY.md
<interfaces>
<!-- Contracts the executor needs. From codebase + Plan 01 output. -->
From lib/services/campaign-grouping-service.ts (Plan 01 output):
export async function groupReportIntoCampaign(reportId: string, opts?: { skipIfAlreadyGrouped?: boolean }): Promise<GroupReportResult | null>
export interface GroupReportResult { campaignId: string; groupMethod: 'message_id'|'attachment_or_url'|'sender_subject_client'; created: boolean; }
From lib/services/phishing-detector.ts (existing):
export interface DetectableTicket { id: number; ticket_number: string|null; title: string|null; description: string|null; company_id: number|null; contact_id: number|null; created_by_contact_id: number|null; }
export async function detectPhishingTicket(ticket: DetectableTicket): Promise<{ flagged: boolean; reportId?: string; skippedUnchanged?: boolean; ... }>
From lib/services/phishing-eml-service.ts (existing — currently uncalled outside its test; this route is its intended first caller per Phase 16):
export async function parseAndStoreMessage(input: { reportId: string; ticketId: number }): Promise<{ stored: boolean; reason?: 'no-eml-attachment'|'no-attachment-content'|'oversized-attachment'; messageId?: string }>
// NEVER throws for expected no-op cases (returns { stored:false, reason }); only real Autotask/DB failures throw.
From lib/auth-utils.ts (existing):
export async function requirePermission(resource, action): Promise<{ session?, error? }>
// Usage: const { error } = await requirePermission('phishing','analyze'); if (error) return error;
Route conventions (Next.js 16):
- dynamic params are a Promise: { params }: { params: Promise<{ ticket_id: string }> }; const { ticket_id } = await params;
- route files import: import postgresClient from '@/lib/services/postgres-client'; (default+absolute alias — matches every other route file)
- tickets table (source ticket row): id BIGINT, ticket_number, title, description, company_id, contact_id, created_by_contact_id
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: POST /api/phishing/tickets/[ticket_id]/analyze route</name>
<files>app/api/phishing/tickets/[ticket_id]/analyze/route.ts</files>
<read_first>
- app/api/tickets/[id]/route.ts (Next.js 16 params: Promise<> convention + parseInt(id) + not-found 404 early return)
- app/api/rmm/executions/route.ts (POST handler: requirePermission call-and-early-return + try/catch client-vs-server error status split)
- lib/services/webhook-service.ts (its "read the ticket row back from Postgres" pattern — the analyze route has no Autotask entity payload, so it reconstructs a DetectableTicket from the tickets table the same way)
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md section 3 (exact route shape + the note on parseAndStoreMessage stored:false NOT being an error)
</read_first>
<action>
Create `app/api/phishing/tickets/[ticket_id]/analyze/route.ts` exporting `POST(request: NextRequest, { params }: { params: Promise<{ ticket_id: string }> })`. First line of the handler: `const { error } = await requirePermission('phishing', 'analyze'); if (error) return error;` (D-06). Await params, `const ticketId = Number(ticket_id)`; return `NextResponse.json({ error: 'Invalid ticket_id' }, { status: 400 })` when `!Number.isFinite(ticketId)` (V5 input validation). In a try/catch: SELECT `id, ticket_number, title, description, company_id, contact_id, created_by_contact_id FROM tickets WHERE id = $1` with `[ticketId]`; return 404 `{ error: 'Ticket not found' }` if no row. Build a `DetectableTicket` from the row (Number(id), pass the contact fields through). Call `detectPhishingTicket(ticket)`; if `!detection.flagged || !detection.reportId` return 400 `{ error: 'Ticket does not match known phishing patterns' }`. Call `await parseAndStoreMessage({ reportId: detection.reportId, ticketId })` — do NOT treat a `{ stored:false, reason }` result as an error (grouping still proceeds). Call `await groupReportIntoCampaign(detection.reportId)` WITHOUT `skipIfAlreadyGrouped` (D-08 — always re-run, allowing a Tier-3 grouping to upgrade to Tier-1 now that messages/indicators may exist). Return `NextResponse.json({ reportId: detection.reportId, campaignId: grouped?.campaignId ?? null, groupMethod: grouped?.groupMethod ?? null, created: grouped?.created ?? false })` (camelCase per CLAUDE.md). Catch: `console.error('[PHISHING-ANALYZE] ...', ticketId, err)` and return 500 `{ error, message: err instanceof Error ? err.message : 'Unknown error' }`. Use `import postgresClient from '@/lib/services/postgres-client'` (default+absolute alias).
</action>
<verify>
<automated>npx tsc --noEmit --pretty</automated>
<human-check>Start dev server (npm run dev, port 3100). (a) `curl -X POST http://localhost:3100/api/phishing/tickets/{knownPhishingTicketId}/analyze` with a valid admin session cookie returns 200 with reportId/campaignId/groupMethod/created. (b) Same route with no cookie returns 401. (c) Same route with a user-role session cookie returns 403 (D-05 grants user read only, not analyze). (d) `curl -X POST .../tickets/abc/analyze` returns 400.</human-check>
</verify>
<done>Route compiles; requirePermission('phishing','analyze') gates it; orchestrates detect->parse->group; returns camelCase linkage; ticket_id validated (400), missing ticket (404), non-phishing ticket (400) handled.</done>
<acceptance_criteria>
- File exists at app/api/phishing/tickets/[ticket_id]/analyze/route.ts and exports `POST`
- Contains `requirePermission('phishing', 'analyze')` as the first handler statement with `if (error) return error;`
- Contains `params: Promise<{ ticket_id: string }>` and `await params`
- Calls detectPhishingTicket, parseAndStoreMessage, and groupReportIntoCampaign(detection.reportId) with NO skipIfAlreadyGrouped argument
- Returns 400 for non-finite ticket_id, 404 for missing ticket, 400 for non-phishing ticket
- `npx tsc --noEmit --pretty` exits 0
- Manual: 401 without cookie, 403 with user-role cookie, 200 with admin cookie (recorded in SUMMARY)
</acceptance_criteria>
</task>
<task type="auto">
<name>Task 2: Wire groupReportIntoCampaign into webhook + cron sweep automatic paths</name>
<files>lib/services/webhook-service.ts, lib/services/phishing-sweep-service.ts</files>
<read_first>
- lib/services/webhook-service.ts (the triggerPhishingDetection(payload) private method, ~lines 395-491, and its fire-and-forget .catch() caller ~line 118)
- lib/services/phishing-sweep-service.ts (the `for (const row of candidates.rows)` sweep loop, ~lines 78-84, and its per-ticket try/catch + result.errors accounting)
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md sections 7 and 8 (exact call sites + diffs)
</read_first>
<action>
In `lib/services/webhook-service.ts`: add `import { groupReportIntoCampaign } from './campaign-grouping-service';` near the existing phishing-detector import. Inside `triggerPhishingDetection(payload)`, capture the detection result (`const detection = await detectPhishingTicket(ticket)`) and, when `detection.flagged && detection.reportId`, `await groupReportIntoCampaign(detection.reportId, { skipIfAlreadyGrouped: true })` (D-01 automatic path, D-08 short-circuit). No new try/catch is needed — this method is already fire-and-forget with a `.catch()` at the caller. In `lib/services/phishing-sweep-service.ts`: add the same import. Inside the existing per-ticket loop's try block, after the detect + result-counter logic, when `detection.flagged && detection.reportId`, `await groupReportIntoCampaign(detection.reportId, { skipIfAlreadyGrouped: true })`. Keep the grouping call inside the SAME try/catch as detectPhishingTicket so a grouping failure counts against `result.errors` and does not abort the sweep loop (matching the file's existing per-ticket error isolation). Both automatic paths pass `skipIfAlreadyGrouped: true` (D-08) — only the /analyze route (Task 1) omits it.
</action>
<verify>
<automated>npx tsc --noEmit --pretty && grep -c 'groupReportIntoCampaign' lib/services/webhook-service.ts lib/services/phishing-sweep-service.ts</automated>
</verify>
<done>Both service files import and call groupReportIntoCampaign with skipIfAlreadyGrouped:true on the flagged-report path; sweep call is inside the existing per-ticket try/catch; tsc clean.</done>
<acceptance_criteria>
- webhook-service.ts contains `import { groupReportIntoCampaign } from './campaign-grouping-service'` and a call `groupReportIntoCampaign(detection.reportId, { skipIfAlreadyGrouped: true })`
- phishing-sweep-service.ts contains the same import and a call with `{ skipIfAlreadyGrouped: true }` inside the per-ticket try block (grep confirms the call precedes the catch that increments result.errors)
- Neither automatic-path call omits skipIfAlreadyGrouped
- `npx tsc --noEmit --pretty` exits 0
</acceptance_criteria>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| client → /api/phishing/tickets/{id}/analyze | authenticated staff request; must carry phishing:analyze permission |
| ticket_id path param → SQL | untrusted numeric input crosses into a parameterized query |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-18-01 | Elevation of Privilege | POST /analyze | mitigate | `requirePermission('phishing', 'analyze')` first-line gate returns 403 for user-role (D-05 grants user read only) and 401 unauthenticated — verified by manual curl |
| T-18-05 | Information Disclosure | ticket_id path param | mitigate | `Number(ticket_id)` + `Number.isFinite` guard returns 400 before any query; SELECT uses `$1` parameterized placeholder — a malformed value never reaches SQL text or leaks a 500 |
| T-18-02 | Information Disclosure (IDOR) | analyze by arbitrary ticket_id | accept | single-tenant internal MSP tool (Wulf staff only); consistent with existing tickets/[id] route which does not tenant-scope — no new gap introduced |
No package installs this phase — supply-chain (T-{phase}-SC) row not applicable.
</threat_model>
<verification>
- `npx tsc --noEmit --pretty` clean across all three files
- `grep -c 'groupReportIntoCampaign'` returns >=1 in both webhook-service.ts and phishing-sweep-service.ts
- Manual (pre-verify gate, DETECT-03/ACCESS-01): curl the analyze route — 200 with admin cookie on a known phishing-pattern ticket (response has reportId/campaignId/groupMethod/created), 401 with no cookie, 403 with a user-role cookie, 400 on non-numeric ticket_id
</verification>
<success_criteria>
- POST /api/phishing/tickets/{ticket_id}/analyze orchestrates detect->parse->group and returns camelCase campaign linkage (DETECT-03)
- Automatic webhook + cron sweep paths call the shared grouping function so campaigns accumulate without an API call (D-01, CAMP-01/CAMP-02)
- The analyze route enforces requirePermission('phishing','analyze') (ACCESS-01)
</success_criteria>
<output>
Create `.planning/phases/18-campaign-grouping-phishing-analysis-api/18-02-SUMMARY.md` when done
</output>

View file

@ -0,0 +1,175 @@
---
phase: 18-campaign-grouping-phishing-analysis-api
plan: 03
type: execute
wave: 2
depends_on: [18-01]
files_modified:
- app/api/phishing/campaigns/route.ts
- app/api/phishing/campaigns/[id]/route.ts
autonomous: true
requirements: [CAMP-03, ACCESS-01]
must_haves:
truths:
- "GET /api/phishing/campaigns returns a paginated camelCase list { items, total, limit, offset } with per-campaign fields (id, campaignKey, groupMethod, firstSeenAt, lastSeenAt, reportCount, status, createdAt)"
- "GET /api/phishing/campaigns/{id} returns one campaign with nested linked reports, messages, indicators, and classification history in camelCase"
- "The detail route derives per-report requester email by joining reports.requester_contact_id -> contacts (campaigns has no recipients column)"
- "Both routes call requirePermission('phishing','read') and reject unauthenticated/unauthorized requests with 401/403"
- "The detail route validates the id is a UUID and returns 400 for a malformed id (not a 500), 404 when the campaign is absent"
artifacts:
- path: "app/api/phishing/campaigns/route.ts"
provides: "GET campaign list (CAMP-03)"
exports: ["GET"]
- path: "app/api/phishing/campaigns/[id]/route.ts"
provides: "GET campaign detail with nested children (CAMP-03)"
exports: ["GET"]
key_links:
- from: "app/api/phishing/campaigns/[id]/route.ts"
to: "reports/messages/indicators/classifications tables"
via: "bulk-fetch keyed by ID array (= ANY($1::uuid[])) + Map assembly (device-link-conflicts pattern)"
pattern: "ANY\\(\\$1::uuid\\[\\]\\)"
- from: "app/api/phishing/campaigns/route.ts"
to: "requirePermission('phishing', 'read')"
via: "call-and-early-return auth gate (D-06)"
pattern: "requirePermission\\('phishing', ?'read'\\)"
---
<objective>
Provide the read surface for campaigns: a paginated list and a full nested detail view (CAMP-03), both gated by `requirePermission('phishing','read')` (ACCESS-01). These are read-only routes over existing tables — no grouping logic, no writes.
Purpose: an operator browses campaigns and drills into one campaign's linked reports/messages/indicators/classification history. Depends on Plan 01 only for the `phishing` permission resource (`requirePermission('phishing','read')` must type-check).
Output: two new GET route files.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-CONTEXT.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-RESEARCH.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md
@.planning/phases/18-campaign-grouping-phishing-analysis-api/18-01-SUMMARY.md
<interfaces>
<!-- Contracts the executor needs. From codebase + Plan 01 output. -->
From lib/auth-utils.ts (existing):
export async function requirePermission(resource, action): Promise<{ session?, error? }>
// Usage: const { error } = await requirePermission('phishing','read'); if (error) return error;
// 'phishing' resource + read grant added in Plan 01 (lib/permissions.ts).
Route conventions (Next.js 16):
- route files import: import postgresClient from '@/lib/services/postgres-client'; (default+absolute alias)
- dynamic params are a Promise: { params }: { params: Promise<{ id: string }> }; const { id } = await params;
- API responses are camelCase — handlers transform snake_case rows manually (CLAUDE.md)
Schema (migrations/097 + 099):
campaigns(id UUID, campaign_key TEXT, group_method TEXT, first_seen_at, last_seen_at, report_count INT, status TEXT, created_at, updated_at)
reports(id UUID, ticket_id BIGINT, ticket_number, company_name, title, requester_contact_id BIGINT, campaign_id UUID FK, created_at)
messages(id UUID, report_id UUID, message_id TEXT, headers JSONB, urls JSONB, attachments JSONB, body_preview, created_at) -- NO subject column; subject is headers->>'subject'
indicators(id UUID, message_id UUID FK->messages(id), indicator_type TEXT, value TEXT, metadata JSONB [added migration 099], created_at)
classifications(id UUID, campaign_id UUID, verdict TEXT, confidence NUMERIC, summary TEXT, reasons JSONB, recommended_actions JSONB, requires_approval BOOL, created_at) -- Phase 19 stub; expect empty this phase, but include in response shape
contacts(id, email_address VARCHAR, first_name, last_name)
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: GET /api/phishing/campaigns paginated list</name>
<files>app/api/phishing/campaigns/route.ts</files>
<read_first>
- app/api/admin/device-link-conflicts/route.ts (the recommended analog: requirePermission -> clamp limit/offset -> optional WHERE with parameterized placeholder -> COUNT(*) -> manual camelCase .map())
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md section 4 (exact list route shape + the V5 note that the status filter value is never string-interpolated)
</read_first>
<action>
Create `app/api/phishing/campaigns/route.ts` exporting `GET(request: NextRequest)`. First line: `const { error } = await requirePermission('phishing', 'read'); if (error) return error;` (D-06). Parse and clamp query params: `limit = Math.min(parseInt(searchParams.get('limit') ?? '50',10) || 50, 200)`, `offset = Math.max(parseInt(searchParams.get('offset') ?? '0',10) || 0, 0)`, optional `status`. Build a `params: unknown[] = [limit, offset]` array; if `status` present, push it and set `statusFilter = 'WHERE status = $' + params.length` (parameterized — never string-interpolate the value; V5/T-18-06). Query `SELECT id::text, campaign_key, group_method, first_seen_at::text, last_seen_at::text, report_count, status, created_at::text FROM campaigns ${statusFilter} ORDER BY last_seen_at DESC NULLS LAST LIMIT $1 OFFSET $2`. Run a separate `SELECT COUNT(*)::text AS count FROM campaigns ${statusFilter}` with `status ? [status] : []`. Return `NextResponse.json({ items, total, limit, offset })` where items are manually mapped to camelCase (id, campaignKey, groupMethod, firstSeenAt, lastSeenAt, reportCount, status, createdAt). Use `import postgresClient from '@/lib/services/postgres-client'`. Wrap DB work in try/catch returning 500 `{ error, message }` on failure (match device-link-conflicts error handling).
</action>
<verify>
<automated>npx tsc --noEmit --pretty</automated>
<human-check>Dev server on 3100. `curl http://localhost:3100/api/phishing/campaigns` with admin cookie returns 200 `{ items, total, limit, offset }` in camelCase; no cookie returns 401. `curl '.../campaigns?status=open&limit=5'` respects the filter and clamp.</human-check>
</verify>
<done>List route compiles; gated by requirePermission('phishing','read'); returns paginated camelCase { items, total, limit, offset }; status filter parameterized; limit clamped to 200.</done>
<acceptance_criteria>
- File exists at app/api/phishing/campaigns/route.ts and exports `GET`
- Contains `requirePermission('phishing', 'read')` with `if (error) return error;`
- status filter uses a `$${params.length}` placeholder — no `${status}` string interpolation into SQL text
- Response objects use camelCase keys (campaignKey, groupMethod, firstSeenAt, lastSeenAt, reportCount, createdAt)
- limit is clamped with Math.min(..., 200)
- `npx tsc --noEmit --pretty` exits 0
- Manual: 200 with admin cookie, 401 without (recorded in SUMMARY)
</acceptance_criteria>
</task>
<task type="auto">
<name>Task 2: GET /api/phishing/campaigns/[id] nested detail</name>
<files>app/api/phishing/campaigns/[id]/route.ts</files>
<read_first>
- app/api/admin/device-link-conflicts/route.ts (the exact bulk-fetch + Map + manual-camelCase nested-detail pattern to replicate — Pattern 3)
- app/api/tickets/[id]/route.ts (params: Promise<{ id: string }> convention + not-found 404 early return)
- .planning/phases/18-campaign-grouping-phishing-analysis-api/18-PATTERNS.md section 5 (exact detail route shape, UUID validation, Pitfall 3 recipients-via-join, the messages.subject-is-in-headers-JSONB note)
</read_first>
<action>
Create `app/api/phishing/campaigns/[id]/route.ts` exporting `GET(request: NextRequest, { params }: { params: Promise<{ id: string }> })`. First line: `const { error } = await requirePermission('phishing', 'read'); if (error) return error;` (D-06). Await params; validate `id` against `/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i` and return 400 `{ error: 'Invalid campaign id' }` if it fails (V5/T-18-05 — a malformed UUID would otherwise surface as a Postgres error -> uncaught 500). In try/catch: (1) parent `SELECT ... FROM campaigns WHERE id = $1`; return 404 `{ error: 'Campaign not found' }` if absent. (2) bulk-fetch linked reports: `SELECT r.id::text, r.ticket_id::text, r.ticket_number, r.company_name, r.title, r.created_at::text, c.email_address AS requester_email FROM reports r LEFT JOIN contacts c ON c.id = r.requester_contact_id WHERE r.campaign_id = $1 ORDER BY r.created_at ASC` (Pitfall 3 — recipients derived via this contacts join; Pitfall 5 — join on requester_contact_id). (3) bulk-fetch messages keyed by the report-id array: `... WHERE report_id = ANY($1::uuid[])` (select `headers->>'subject' AS subject` since messages has no subject column). (4) bulk-fetch indicators keyed by the message-id array: `SELECT id::text, message_id::text, indicator_type, value, metadata FROM indicators WHERE message_id = ANY($1::uuid[])`. (5) classifications: `... WHERE campaign_id = $1 ORDER BY created_at DESC` (Phase 19 stub — likely empty, still include in the shape per CAMP-03). Guard each child query with an empty-array check (skip the query when the parent id array is empty). Assemble one camelCase JSON object: campaign fields + `reports[]` (id, ticketId, ticketNumber, companyName, title, createdAt, requesterEmail) + `messages[]` (id, reportId, messageId, subject) + `indicators[]` (id, messageId, indicatorType, value, metadata) + `classifications[]` (id, verdict, confidence, summary, createdAt). Use `import postgresClient from '@/lib/services/postgres-client'`. Catch -> `console.error` + 500 `{ error, message }`.
</action>
<verify>
<automated>npx tsc --noEmit --pretty</automated>
<human-check>Dev server on 3100. `curl http://localhost:3100/api/phishing/campaigns/{realCampaignUuid}` with admin cookie returns 200 with nested camelCase reports/messages/indicators/classifications; a malformed id (e.g. `/campaigns/abc`) returns 400; a well-formed-but-absent UUID returns 404; no cookie returns 401.</human-check>
</verify>
<done>Detail route compiles; gated by requirePermission('phishing','read'); UUID-validated (400) with 404 for absent campaign; returns nested camelCase reports/messages/indicators/classifications assembled via bulk-fetch + Map.</done>
<acceptance_criteria>
- File exists at app/api/phishing/campaigns/[id]/route.ts and exports `GET`
- Contains `requirePermission('phishing', 'read')` with `if (error) return error;`
- Contains a UUID regex validation returning 400 before any query, and a 404 for a missing campaign
- Uses `= ANY($1::uuid[])` bulk-fetch for at least messages and indicators (grep confirms)
- reports query LEFT JOINs contacts on `r.requester_contact_id` and selects `email_address AS requester_email`
- messages subject is pulled via `headers->>'subject'` (messages has no subject column)
- Response nests reports/messages/indicators/classifications in camelCase
- `npx tsc --noEmit --pretty` exits 0
- Manual: 200 (admin) / 400 (malformed id) / 404 (absent) / 401 (no cookie) recorded in SUMMARY
</acceptance_criteria>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| client → /api/phishing/campaigns[/{id}] | authenticated staff request; must carry phishing:read permission |
| status query param + id path param → SQL | untrusted input crosses into DB queries |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-18-01 | Elevation of Privilege | both GET routes | mitigate | `requirePermission('phishing', 'read')` first-line gate returns 401/403; user-role has read (D-05) so browsing works while the fine-grained convention is established |
| T-18-06 | Tampering | list route status filter | mitigate | status value bound via a `$n` placeholder (device-link-conflicts pattern) — never string-interpolated into SQL text |
| T-18-05 | Information Disclosure | detail route id path param | mitigate | UUID regex validation returns 400 before querying — a malformed UUID cannot surface as a leaked 500 Postgres error |
| T-18-02 | Information Disclosure (IDOR) | detail by arbitrary campaign UUID | accept | single-tenant internal MSP tool; consistent with existing route conventions — no per-tenant scoping introduced |
No package installs this phase — supply-chain (T-{phase}-SC) row not applicable.
</threat_model>
<verification>
- `npx tsc --noEmit --pretty` clean for both route files
- `grep` confirms `ANY($1::uuid[])` bulk-fetch in the detail route and a parameterized status filter in the list route
- Manual (pre-verify gate, CAMP-03/ACCESS-01): curl list + detail with admin cookie (camelCase nested shape), 401 without cookie, 400 on malformed detail id, 404 on absent campaign
</verification>
<success_criteria>
- GET /api/phishing/campaigns lists campaigns (paginated camelCase) and GET /api/phishing/campaigns/{id} returns full nested detail (linked reports, messages, indicators, classification history) (CAMP-03)
- Both routes enforce requirePermission('phishing','read') (ACCESS-01)
- Malformed/absent ids handled with 400/404 (no 500 leak)
</success_criteria>
<output>
Create `.planning/phases/18-campaign-grouping-phishing-analysis-api/18-03-SUMMARY.md` when done
</output>