diff --git a/.planning/phases/18-campaign-grouping-phishing-analysis-api/18-01-SUMMARY.md b/.planning/phases/18-campaign-grouping-phishing-analysis-api/18-01-SUMMARY.md new file mode 100644 index 0000000..54cd481 --- /dev/null +++ b/.planning/phases/18-campaign-grouping-phishing-analysis-api/18-01-SUMMARY.md @@ -0,0 +1,112 @@ +--- +phase: 18-campaign-grouping-phishing-analysis-api +plan: 01 +subsystem: api +tags: [postgres, phishing, campaign-grouping, permissions, better-auth, vitest] + +# Dependency graph +requires: + - phase: 15-data-model-detection-ticket-evidence + provides: "campaigns/reports/messages/indicators schema (migrations 097/099), phishing-detector.ts's DetectableTicket shape and orchestration style" + - phase: 16-eml-mime-evidence-parser + provides: "parseAndStoreMessage() as the (currently uncalled) writer of messages/indicators rows that Tier 1/2 depend on" +provides: + - "lib/services/campaign-grouping-service.ts — groupReportIntoCampaign(reportId, opts), normalizeSubject, extractUrlDomain, GroupReportResult" + - "phishing permission resource in lib/permissions.ts (full D-05 vocabulary + partial role grants)" +affects: [18-02, 18-03, 19-classification, 20-remediation] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Shared grouping core called from 3 sites (webhook/cron/analyze), mirrors phishing-detector.ts's shared-detector architecture" + - "Tiered find-or-create inside postgresClient.transaction() — pure-JS key computation + targeted parameterized queries per tier, not one mega-WHERE clause" + - "SQL-content-routed mock client in tests (each query's distinguishing SQL substring maps to staged rows) instead of strict call-order mocking" + +key-files: + created: + - lib/services/campaign-grouping-service.ts + - lib/services/campaign-grouping-service.test.ts + modified: + - lib/permissions.ts + +key-decisions: + - "Every tier query includes a self-exclusion clause (r.id != ) so re-running groupReportIntoCampaign on an already-grouped report can never match its own messages/indicators row against itself and double-increment its own campaign" + - "New campaign creation picks the strongest available tier key (Tier 1 message_id > Tier 2 attachment_or_url > Tier 3 sender_subject_client) so future duplicates of that report have the best chance of matching it" + - "campaigns.status is never set/transitioned by this plan — left at the migration default 'open' per D-04/scope boundary" + - "phishing resource declares the full action vocabulary (read/analyze/approve/remediate) now but only grants read+analyze (admin/super-admin) and read (user) this phase, per D-05" + +patterns-established: + - "Tier queries compute fuzzy keys (normalizeSubject, extractUrlDomain) in JS and query Postgres with targeted parameterized $n placeholders — never encode fuzzy matching in SQL WHERE clauses" + +requirements-completed: [CAMP-01, CAMP-02, ACCESS-01] + +# Metrics +duration: 15min +completed: 2026-07-15 +--- + +# Phase 18 Plan 01: Campaign Grouping Core + Phishing Permission Resource Summary + +**`groupReportIntoCampaign()` — tiered Message-ID → attachment-hash/URL-domain → sender+subject+client campaign matching inside a single Postgres transaction, plus the `phishing` permission resource all future `/api/phishing/*` routes will gate on.** + +## Performance + +- **Duration:** ~15 min +- **Started:** 2026-07-15T19:19Z (approx, first commit) +- **Completed:** 2026-07-15T19:23Z (last task commit) +- **Tasks:** 3/3 completed +- **Files modified:** 3 (2 created, 1 modified) + +## Accomplishments +- `lib/services/campaign-grouping-service.ts` — `groupReportIntoCampaign(reportId, opts)` implements the full CAMP-01 tiered match (Message-ID → attachment-hash/URL-domain+subject+sender+24h → sender+normalized-subject+client+24h) and CAMP-02 accumulation (report_count/last_seen_at bump on match, new campaign on no-match), wrapped in `postgresClient.transaction()` to guard against `campaigns.campaign_key`'s missing UNIQUE constraint. +- `lib/services/campaign-grouping-service.test.ts` — 16 passing tests covering pure helpers (`normalizeSubject`, `extractUrlDomain`) and mocked-DB tiered-matching/find-or-create/self-exclusion/`skipIfAlreadyGrouped` behavior. +- `lib/permissions.ts` — `phishing` resource added with the full D-05 action vocabulary; `read`+`analyze` granted to `super-admin`/`admin`, `read` only to `user`; `approve`/`remediate` declared but ungranted to any role (Phase 20's job). + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Pure tier-key helpers + test file scaffold** - `ea677b7` (test) +2. **Task 2: groupReportIntoCampaign — tiered matching + transactional find-or-create** - `da926bb` (feat) +3. **Task 3: Add phishing permission resource + role grants** - `04ec51f` (feat) + +**Plan metadata:** committed alongside this SUMMARY (see final commit in this plan's history) + +## TDD Gate Compliance + +Plan frontmatter is `type: tdd`. Gate sequence verified in git log: `ea677b7` is a `test(...)` commit (RED gate) followed by `da926bb`/`04ec51f`, both `feat(...)` commits (GREEN gate) — sequence satisfied, no warning needed. No `refactor(...)` commit was needed (no cleanup pass required after GREEN). + +## Files Created/Modified +- `lib/services/campaign-grouping-service.ts` - `groupReportIntoCampaign`, `normalizeSubject`, `extractUrlDomain`, `GroupReportResult`; D-07 doc comment stating the Tier-3-only automatic-path limitation +- `lib/services/campaign-grouping-service.test.ts` - 16 tests: 5 `normalizeSubject`, 3 `extractUrlDomain`, 8 `groupReportIntoCampaign` (Tier 1/2/3 matches, no-match create, `skipIfAlreadyGrouped` short-circuit + pass-through, self-exclusion/no-double-increment) +- `lib/permissions.ts` - `phishing` resource in `statement` (4 actions) + role grants in `superAdminRole`/`adminRole`/`userRole` + +## Decisions Made +- Self-exclusion clause (`r.id != `) added to every one of the 3 tier queries — this was a plan-checker finding baked into the plan's action text, not a deviation, but worth restating: without it, re-running grouping on an already-linked report could match the report's own `messages`/`indicators` row against itself and double-increment its own campaign's `report_count`. +- New-campaign `campaign_key`/`group_method` is chosen from whichever tier's key is computable for the current report (Tier 1 > Tier 2 > Tier 3 > `report:{id}` fallback if no signal exists at all), so a future duplicate of *this* report has the best chance of finding it via the strongest tier reachable. +- Test mocking uses a SQL-content-routed dispatcher (map each query's distinguishing substring to staged rows) rather than strict call-order `mockResolvedValueOnce` chaining — more robust to the branching control flow (tier 2/3 are conditionally skipped once an earlier tier matches). + +## Deviations from Plan + +None - plan executed exactly as written. The self-exclusion requirement and the D-07 doc-comment requirement were both already spelled out explicitly in the plan's task text (not gaps discovered during execution), so no Rule 1-4 deviation applies. + +## Issues Encountered +- Initial version of the self-exclusion test asserted the wrong `groupMethod` for the newly-created campaign (`attachment_or_url` instead of `message_id`) — the test fixture staged an own `message_id`, so `computeTier1Key` correctly won priority over the Tier 2 key in the no-match/create-new-campaign branch. Fixed by correcting the test's expectation, not the implementation (confirmed via re-reading `computeTier1Key > computeTier2Key > computeTier3Key` priority order in the source). + +## User Setup Required + +None - no external service configuration required. + +## Next Phase Readiness +- `groupReportIntoCampaign` and the `phishing` permission resource are ready for Wave 2's 3 new `app/api/phishing/*` routes (list/detail/analyze) and the 2 modified trigger sites (`webhook-service.ts`, `phishing-sweep-service.ts`) to call, per 18-PATTERNS.md's exact call shapes. +- Known limitation carried forward per D-07: the automatic webhook/cron path only reaches Tier 3 (sender+subject+client+24h) until a report has been through an explicit `/analyze` call at least once, since `parseAndStoreMessage` (Phase 16) is still not wired into the automatic path — this plan documents it, does not fix it (out of scope, matches CONTEXT.md's locked decision). +- `approve`/`remediate` phishing actions are declared in `lib/permissions.ts`'s `statement` but ungranted to any role — Phase 20 only needs to add role grants, not touch the `statement` block again. + +--- +*Phase: 18-campaign-grouping-phishing-analysis-api* +*Completed: 2026-07-15* + +## Self-Check: PASSED + +All created/modified files confirmed present on disk; all 3 task commit hashes (`ea677b7`, `da926bb`, `04ec51f`) confirmed present in git log.