diff --git a/.planning/phases/15-data-model-detection-ticket-evidence/15-01-SUMMARY.md b/.planning/phases/15-data-model-detection-ticket-evidence/15-01-SUMMARY.md new file mode 100644 index 0000000..c6cce50 --- /dev/null +++ b/.planning/phases/15-data-model-detection-ticket-evidence/15-01-SUMMARY.md @@ -0,0 +1,92 @@ +--- +phase: 15-data-model-detection-ticket-evidence +plan: 01 +subsystem: database +tags: [postgres, migration, schema, phishing-triage] + +# Dependency graph +requires: [] +provides: + - "migrations/097_phishing_triage_schema.sql applied to dev DB" + - "7-table phishing-triage schema: campaigns, reports, messages, indicators, classifications, remediation_actions, audit_events" + - "reports table fully designed with ticket_id FK, content_hash idempotency key, matched_patterns/evidence JSONB columns, uq_reports_ticket_id unique constraint" +affects: [15-02-phishing-detector, 16-message-parsing, 18-campaign-grouping, 19-classification, 20-remediation] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Schema-only foundation migration lands before any service writes to it (mirrors migration 091 pax8 pattern)" + - "UUID PKs via gen_random_uuid() (no extension needed, Postgres 16 built-in)" + - "Stub tables for future-phase entities created now with hard FKs where insert order is guaranteed within the same migration" + +key-files: + created: [migrations/097_phishing_triage_schema.sql] + modified: [] + +key-decisions: + - "reports.ticket_id is a hard FK to tickets(id) with UNIQUE constraint (uq_reports_ticket_id) so the Plan 02 detector can upsert one report per ticket via ON CONFLICT (ticket_id)" + - "campaigns/messages/indicators/classifications/remediation_actions/audit_events created as stubs now (per STATE.md decision: durable schema lands in Phase 15, before any service that writes to it) so Phases 16-21 never need a second foundation migration" + - "Applied migration directly via docker exec psql (not scripts/apply-migrations.sh) because that script hardcodes the main-repo path /opt/stacks/pulse/migrations, which does not yet contain this file while running inside a worktree" + +patterns-established: + - "Pattern: schema-only migrations for future phases mirror migrations/091_pax8_tables.sql header framing (numbered list of tables, explicit note on what's populated now vs. stubbed)" + +requirements-completed: [DETECT-01, DETECT-02, EVID-01] + +# Metrics +duration: 12min +completed: 2026-07-15 +--- + +# Phase 15 Plan 01: Phishing Triage Schema Foundation Summary + +**Migration 097 lays down the full 7-table phishing-triage schema in Postgres — reports table fully designed for ticket_id, content_hash idempotency, matched_patterns, and EVID-01 evidence; six other tables stubbed for Phases 16-21 — applied and verified in the dev DB.** + +## Performance + +- **Duration:** 12 min +- **Started:** 2026-07-15T11:23:00Z +- **Completed:** 2026-07-15T11:35:46Z +- **Tasks:** 2 completed +- **Files modified:** 1 + +## Accomplishments +- Created `migrations/097_phishing_triage_schema.sql` with all 7 phishing-triage tables (campaigns, reports, messages, indicators, classifications, remediation_actions, audit_events), each `CREATE TABLE IF NOT EXISTS`, snake_case columns, UUID PKs via `gen_random_uuid()`. +- `reports` table fully designed: `ticket_id BIGINT NOT NULL REFERENCES tickets(id)`, `content_hash TEXT NOT NULL` (D-04 idempotency), `matched_patterns JSONB NOT NULL DEFAULT '[]'`, `evidence JSONB NOT NULL DEFAULT '{}'`, `campaign_id UUID REFERENCES campaigns(id)` (nullable), and `CONSTRAINT uq_reports_ticket_id UNIQUE (ticket_id)`. +- Applied migration to the running dev Postgres (`pulse-postgres` container) via `docker exec -i pulse-postgres psql`. +- Verified all 7 tables exist in `information_schema.tables`, confirmed `reports` columns/constraint via `\d reports`, and re-ran the migration to confirm full idempotency (all statements returned `NOTICE: ... already exists, skipping`, zero errors). + +## Task Commits + +1. **Task 1: Write migration 097 — 7-table phishing triage schema** - `84a37e2` (feat) +2. **Task 2: Apply migration 097 to the dev database and verify tables exist** - no commit (DB-only verification step; no file changes produced — the migration file was already committed in Task 1) + +**Plan metadata:** (this SUMMARY.md commit) + +## Files Created/Modified +- `migrations/097_phishing_triage_schema.sql` - 7-table phishing-triage schema: campaigns, reports (fully designed for Plan 02's detector), messages, indicators, classifications, remediation_actions, audit_events (all stubs except reports) + +## Decisions Made +- Hard FK + UNIQUE constraint on `reports.ticket_id` (rather than a soft ref like PAX8's company/product tables) because report inserts always happen one-at-a-time against an already-synced ticket — no batch-insert-order ambiguity to avoid, unlike PAX8's multi-entity sync passes. +- Applied the migration directly via `docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask < migrations/097_phishing_triage_schema.sql` instead of `scripts/apply-migrations.sh`, because that script hardcodes `MIGRATIONS_DIR="/opt/stacks/pulse/migrations"` (the main repo checkout) — inside this worktree the new file doesn't exist at that path yet, so the script would silently fail to find it. Documented here for the orchestrator/user; no change made to the script itself (out of scope for this plan). + +## Deviations from Plan + +None - plan executed exactly as written. Task 2's verification uncovered a worktree-path caveat with `scripts/apply-migrations.sh` (documented above under Decisions Made) but this did not require any code change — the plan's own fallback instruction ("Otherwise pipe the file into the container's psql") was used as designed. + +## Issues Encountered +None. + +## User Setup Required +None - no external service configuration required. Migration applied directly to the existing dev Postgres container; no new env vars or credentials needed. + +## Next Phase Readiness +- All 7 phishing-triage tables exist in the dev database and are idempotently re-appliable. +- `reports` table is fully ready for Plan 02's phishing-detector service to upsert into via `ON CONFLICT (ticket_id)`. +- `campaigns`, `messages`, `indicators`, `classifications`, `remediation_actions`, `audit_events` are schema-ready stubs for Phases 16-21 — no second foundation migration will be needed. +- No blockers. + +--- +*Phase: 15-data-model-detection-ticket-evidence* +*Completed: 2026-07-15*